2024-09-21T07:33:57.764 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T07:33:57.789 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:33:57.879 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913617' kernel: client: branch: testing kdb: 1 sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug client: 20 debug mgr: 20 debug ms: 1 mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: client: pid file: '' global: ms die on skipped message: false mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 1843221895adccfb3b692060251dd6a0183bc410 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: syntax: v1 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGahd5EsZdlwoBlncZWtYPwOxt9Zp9HwVy5Zr22UEOUqw0K1bpHAUz3Sc8trLI3o0FcL+Z//iSxbU5RYU4NebNE= smithi175.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKF2iuxvDzKPXzzeAhlCfSA1UUjoozvjzTym75WLLAdbr7fnlm9hPxsD2Q5ufKcj3JVRLPRpp0m4vAyDMsAcut4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: fail_on_skip: false modules: - tasks.cephfs.test_volumes.TestMisc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T07:33:57.879 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T07:33:57.880 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T07:33:57.880 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T07:33:57.882 INFO:teuthology.task.internal:Checking packages... 2024-09-21T07:33:57.913 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T07:33:57.913 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T07:33:57.914 INFO:teuthology.packaging:ref: None 2024-09-21T07:33:57.914 INFO:teuthology.packaging:tag: None 2024-09-21T07:33:57.914 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:33:57.914 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:33:57.914 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T07:33:58.068 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T07:33:58.069 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T07:33:58.082 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T07:33:58.082 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T07:33:58.139 INFO:teuthology.task.internal:Saving configuration 2024-09-21T07:33:58.157 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T07:33:58.267 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T07:33:58.292 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 07:28:30.361768', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGahd5EsZdlwoBlncZWtYPwOxt9Zp9HwVy5Zr22UEOUqw0K1bpHAUz3Sc8trLI3o0FcL+Z//iSxbU5RYU4NebNE='} 2024-09-21T07:33:58.316 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi175.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 07:28:30.360188', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKF2iuxvDzKPXzzeAhlCfSA1UUjoozvjzTym75WLLAdbr7fnlm9hPxsD2Q5ufKcj3JVRLPRpp0m4vAyDMsAcut4='} 2024-09-21T07:33:58.316 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T07:33:58.322 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2024-09-21T07:33:58.322 INFO:teuthology.task.internal:roles: ubuntu@smithi175.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2024-09-21T07:33:58.322 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T07:33:58.428 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f39cbfa4160>, signals=[15]) 2024-09-21T07:33:58.428 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T07:33:58.434 INFO:teuthology.task.internal:Opening connections... 2024-09-21T07:33:58.434 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:33:58.435 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:33:58.512 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:33:58.513 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:33:58.605 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T07:33:58.612 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-09-21T07:33:58.639 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-09-21T07:33:58.639 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-09-21T07:33:58.697 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-09-21T07:33:58.697 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-09-21T07:33:58.697 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-09-21T07:33:58.697 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-09-21T07:33:58.697 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T07:33:58.698 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T07:33:58.699 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-09-21T07:33:58.716 DEBUG:teuthology.orchestra.run.smithi175:> uname -m 2024-09-21T07:33:58.735 INFO:teuthology.orchestra.run.smithi175.stdout:x86_64 2024-09-21T07:33:58.735 DEBUG:teuthology.orchestra.run.smithi175:> cat /etc/os-release 2024-09-21T07:33:58.793 INFO:teuthology.orchestra.run.smithi175.stdout:NAME="CentOS Stream" 2024-09-21T07:33:58.793 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION="9" 2024-09-21T07:33:58.793 INFO:teuthology.orchestra.run.smithi175.stdout:ID="centos" 2024-09-21T07:33:58.793 INFO:teuthology.orchestra.run.smithi175.stdout:ID_LIKE="rhel fedora" 2024-09-21T07:33:58.793 INFO:teuthology.orchestra.run.smithi175.stdout:VERSION_ID="9" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:PLATFORM_ID="platform:el9" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:ANSI_COLOR="0;31" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:LOGO="fedora-logo-icon" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:HOME_URL="https://centos.org/" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T07:33:58.794 INFO:teuthology.orchestra.run.smithi175.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T07:33:58.794 INFO:teuthology.lock.ops:Updating smithi175.front.sepia.ceph.com on lock server 2024-09-21T07:33:58.816 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T07:33:58.825 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T07:33:58.870 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T07:33:58.871 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-09-21T07:33:58.873 DEBUG:teuthology.orchestra.run.smithi175:> test '!' -e /home/ubuntu/cephtest 2024-09-21T07:33:58.890 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T07:33:58.896 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T07:33:58.896 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-09-21T07:33:58.931 DEBUG:teuthology.orchestra.run.smithi175:> test -z $(ls -A /var/lib/ceph) 2024-09-21T07:33:59.030 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T07:33:59.092 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T07:33:59.106 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T07:33:59.106 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2024-09-21T07:33:59.106 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2024-09-21T07:33:59.107 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T07:33:59.107 DEBUG:teuthology.orchestra.run.smithi175:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T07:33:59.124 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:59.124 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T07:33:59.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:33:59.128 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&ref=testing 2024-09-21T07:33:59.292 DEBUG:teuthology.task.kernel:sha1 for client.1 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:33:59.292 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T07:33:59.293 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-21T07:33:59.294 DEBUG:teuthology.task.kernel:sha1 for client.0 is 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:33:59.294 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "2ca44a2f35ca8f36983aaaa46d49312239ef519a"... 2024-09-21T07:33:59.294 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-09-21T07:33:59.312 INFO:teuthology.orchestra.run.smithi175.stdout:5.14.0-503.el9.x86_64 2024-09-21T07:33:59.312 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:33:59.312 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T07:33:59.313 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum upgrade -y linux-firmware 2024-09-21T07:33:59.314 INFO:teuthology.orchestra.run.smithi088.stdout:5.14.0-503.el9.x86_64 2024-09-21T07:33:59.314 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-503.el9.x86_64 vs 2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:33:59.314 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2024-09-21T07:33:59.315 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum upgrade -y linux-firmware 2024-09-21T07:34:01.896 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.3 MB 00:00 2024-09-21T07:34:03.722 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 27 MB/s | 20 MB 00:00 2024-09-21T07:34:03.817 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 2.9 MB/s | 8.3 MB 00:02 2024-09-21T07:34:05.827 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 22 MB/s | 20 MB 00:00 2024-09-21T07:34:07.947 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 15 MB/s | 6.5 MB 00:00 2024-09-21T07:34:09.473 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 19 kB 00:00 2024-09-21T07:34:10.593 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T07:34:12.402 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-21T07:34:13.969 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 19 kB 00:00 2024-09-21T07:34:15.056 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 23 MB/s | 23 MB 00:00 2024-09-21T07:34:16.100 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 9.8 kB/s | 1.7 kB 00:00 2024-09-21T07:34:18.117 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout: Package Architecture Version Repository Size 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:================================================================================ 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 1 Package 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 388 M 2024-09-21T07:34:18.123 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-09-21T07:34:20.812 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 33 kB/s | 1.7 kB 00:00 2024-09-21T07:34:22.801 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T07:34:22.810 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T07:34:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: Package Architecture Version Repository Size 2024-09-21T07:34:22.810 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T07:34:22.810 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-09-21T07:34:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: linux-firmware noarch 20240905-146.el9 baseos 388 M 2024-09-21T07:34:22.810 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:34:22.811 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T07:34:22.811 INFO:teuthology.orchestra.run.smithi175.stdout:================================================================================ 2024-09-21T07:34:22.811 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 1 Package 2024-09-21T07:34:22.811 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:34:22.811 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 388 M 2024-09-21T07:34:22.811 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T07:34:28.313 INFO:teuthology.orchestra.run.smithi175.stdout:linux-firmware-20240905-146.el9.noarch.rpm 72 MB/s | 388 MB 00:05 2024-09-21T07:34:28.313 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:34:28.313 INFO:teuthology.orchestra.run.smithi175.stdout:Total 71 MB/s | 388 MB 00:05 2024-09-21T07:34:31.417 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T07:34:31.575 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T07:34:34.871 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T07:34:34.872 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T07:34:44.080 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T07:34:44.778 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:34:44.858 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:34:45.650 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:34:45.651 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:34:46.384 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:34:46.384 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:34:46.384 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-09-21T07:34:46.384 INFO:teuthology.orchestra.run.smithi175.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T07:34:46.384 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:34:46.384 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T07:34:46.740 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T07:34:46.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:34:46.888 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T07:34:46.889 DEBUG:teuthology.orchestra.run.smithi175:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T07:34:46.891 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi175.front.sepia.ceph.com to complete... 2024-09-21T07:34:53.209 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21 07:34:53 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T07:34:53.209 INFO:teuthology.orchestra.run.smithi175.stderr:FINISHED --2024-09-21 07:34:53-- 2024-09-21T07:34:53.209 INFO:teuthology.orchestra.run.smithi175.stderr:Total wall clock time: 6.3s 2024-09-21T07:34:53.209 INFO:teuthology.orchestra.run.smithi175.stderr:Downloaded: 1 files, 343M in 6.0s (56.9 MB/s) 2024-09-21T07:34:53.212 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.1... 2024-09-21T07:34:53.212 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T07:34:55.271 INFO:teuthology.orchestra.run.smithi175.stdout:Verifying... ######################################## 2024-09-21T07:34:55.399 INFO:teuthology.orchestra.run.smithi175.stdout:Preparing... ######################################## 2024-09-21T07:34:55.459 INFO:teuthology.orchestra.run.smithi175.stdout:Updating / installing... 2024-09-21T07:34:56.585 INFO:teuthology.orchestra.run.smithi088.stdout:linux-firmware-20240905-146.el9.noarch.rpm 10 MB/s | 388 MB 00:38 2024-09-21T07:34:56.585 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:34:56.585 INFO:teuthology.orchestra.run.smithi088.stdout:Total 10 MB/s | 388 MB 00:38 2024-09-21T07:34:59.693 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-09-21T07:34:59.777 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-09-21T07:34:59.778 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-09-21T07:35:03.055 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-09-21T07:35:03.055 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-09-21T07:35:03.454 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T07:35:12.322 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-09-21T07:35:13.061 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:35:13.133 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:35:13.729 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:35:13.730 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : linux-firmware-20240905-146.el9.noarch 1/2 2024-09-21T07:35:14.488 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : linux-firmware-20240716-145.el9.noarch 2/2 2024-09-21T07:35:14.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:35:14.489 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-09-21T07:35:14.489 INFO:teuthology.orchestra.run.smithi088.stdout: linux-firmware-20240905-146.el9.noarch 2024-09-21T07:35:14.489 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:35:14.489 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-09-21T07:35:14.846 INFO:teuthology.task.kernel:Downloading kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T07:35:14.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=centos%2F9%2Fx86_64&sha1=2ca44a2f35ca8f36983aaaa46d49312239ef519a 2024-09-21T07:35:14.994 INFO:teuthology.task.kernel:fetching, builder baseurl is https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ 2024-09-21T07:35:14.995 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /tmp/kernel.x86_64.rpm && echo kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm | wget -nv -O /tmp/kernel.x86_64.rpm --base=https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/ --input-file=- 2024-09-21T07:35:14.997 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi088.front.sepia.ceph.com to complete... 2024-09-21T07:35:21.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21 07:35:21 URL:https://3.chacra.ceph.com/r/kernel/testing/2ca44a2f35ca8f36983aaaa46d49312239ef519a/centos/9/flavors/default/x86_64/kernel-6.11.0_rc5_g2ca44a2f35ca-1.x86_64.rpm [359633503/359633503] -> "/tmp/kernel.x86_64.rpm" [1] 2024-09-21T07:35:21.295 INFO:teuthology.orchestra.run.smithi088.stderr:FINISHED --2024-09-21 07:35:21-- 2024-09-21T07:35:21.295 INFO:teuthology.orchestra.run.smithi088.stderr:Total wall clock time: 6.3s 2024-09-21T07:35:21.295 INFO:teuthology.orchestra.run.smithi088.stderr:Downloaded: 1 files, 343M in 6.0s (57.2 MB/s) 2024-09-21T07:35:21.297 INFO:teuthology.task.kernel:Installing kernel 2ca44a2f35ca8f36983aaaa46d49312239ef519a on client.0... 2024-09-21T07:35:21.297 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -ivh --oldpackage --replacefiles --replacepkgs /tmp/kernel.x86_64.rpm 2024-09-21T07:35:23.351 INFO:teuthology.orchestra.run.smithi088.stdout:Verifying... ######################################## 2024-09-21T07:35:23.470 INFO:teuthology.orchestra.run.smithi088.stdout:Preparing... ######################################## 2024-09-21T07:35:23.529 INFO:teuthology.orchestra.run.smithi088.stdout:Updating / installing... 2024-09-21T07:35:31.573 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-6.11.0_rc5_g2ca44a2f35ca-1 ######################################## 2024-09-21T07:35:43.363 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi175.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T07:35:43.363 DEBUG:teuthology.orchestra.run.smithi175:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T07:35:43.439 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.439 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.439 INFO:teuthology.orchestra.run.smithi175.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.439 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T07:35:43.440 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T07:35:43.441 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T07:35:43.442 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T07:35:43.443 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T07:35:43.444 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T07:35:43.445 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T07:35:43.446 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T07:35:43.447 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T07:35:43.448 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T07:35:43.449 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T07:35:43.450 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T07:35:43.451 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T07:35:43.452 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T07:35:43.453 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T07:35:43.454 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T07:35:43.455 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T07:35:43.456 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T07:35:43.457 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T07:35:43.458 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T07:35:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T07:35:43.460 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T07:35:43.461 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T07:35:43.462 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T07:35:43.463 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T07:35:43.464 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T07:35:43.465 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T07:35:43.466 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T07:35:43.467 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T07:35:43.468 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T07:35:43.469 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T07:35:43.470 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T07:35:43.471 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T07:35:43.472 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T07:35:43.473 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T07:35:43.474 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T07:35:43.475 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T07:35:43.476 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T07:35:43.477 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T07:35:43.478 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T07:35:43.479 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T07:35:43.480 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T07:35:43.481 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T07:35:43.482 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T07:35:43.483 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T07:35:43.484 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T07:35:43.485 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T07:35:43.486 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T07:35:43.487 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T07:35:43.488 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T07:35:43.489 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T07:35:43.490 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T07:35:43.491 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T07:35:43.492 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T07:35:43.493 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T07:35:43.494 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T07:35:43.495 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T07:35:43.496 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T07:35:43.497 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T07:35:43.498 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T07:35:43.499 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T07:35:43.500 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T07:35:43.501 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T07:35:43.502 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T07:35:43.503 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T07:35:43.504 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T07:35:43.505 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T07:35:43.506 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T07:35:43.507 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T07:35:43.508 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T07:35:43.509 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T07:35:43.510 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T07:35:43.511 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T07:35:43.512 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T07:35:43.513 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T07:35:43.514 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T07:35:43.515 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T07:35:43.516 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T07:35:43.517 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T07:35:43.518 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T07:35:43.519 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T07:35:43.520 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T07:35:43.521 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T07:35:43.522 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T07:35:43.523 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T07:35:43.524 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T07:35:43.525 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T07:35:43.526 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T07:35:43.527 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T07:35:43.528 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T07:35:43.529 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T07:35:43.530 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T07:35:43.531 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T07:35:43.532 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T07:35:43.533 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T07:35:43.534 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T07:35:43.535 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T07:35:43.536 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T07:35:43.537 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T07:35:43.538 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T07:35:43.539 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T07:35:43.540 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T07:35:43.541 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T07:35:43.542 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T07:35:43.543 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T07:35:43.544 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T07:35:43.545 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T07:35:43.546 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T07:35:43.547 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T07:35:43.548 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T07:35:43.549 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T07:35:43.550 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T07:35:43.551 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T07:35:43.552 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T07:35:43.553 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T07:35:43.554 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T07:35:43.555 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T07:35:43.556 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T07:35:43.557 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T07:35:43.558 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T07:35:43.559 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T07:35:43.560 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T07:35:43.561 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T07:35:43.562 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T07:35:43.563 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T07:35:43.564 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T07:35:43.565 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T07:35:43.566 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T07:35:43.567 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T07:35:43.568 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T07:35:43.569 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T07:35:43.570 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T07:35:43.571 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T07:35:43.572 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T07:35:43.573 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T07:35:43.574 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T07:35:43.575 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T07:35:43.576 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T07:35:43.577 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T07:35:43.578 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T07:35:43.579 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T07:35:43.580 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T07:35:43.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T07:35:43.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T07:35:43.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T07:35:43.581 INFO:teuthology.orchestra.run.smithi175.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:35:43.585 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.585 DEBUG:teuthology.orchestra.run.smithi175:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout:for file in vmlinuz System.map config; do 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: fi 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout:done 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:35:43.608 INFO:teuthology.orchestra.run.smithi175.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.609 INFO:teuthology.orchestra.run.smithi175.stdout:fi 2024-09-21T07:35:43.610 DEBUG:teuthology.orchestra.run.smithi175:> sudo rpm -qi grub2-tools 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Name : grub2-tools 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Epoch : 1 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Version : 2.06 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Release : 82.el9 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Architecture: x86_64 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Group : Unspecified 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Size : 8274670 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:License : GPLv3+ 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T07:35:43.683 INFO:teuthology.orchestra.run.smithi175.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:Packager : builder@centos.org 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:Vendor : CentOS 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:Summary : Support tools for GRUB. 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:Description : 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:hardware devices. 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:35:43.684 INFO:teuthology.orchestra.run.smithi175.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T07:35:43.685 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T07:35:43.685 INFO:teuthology.task.kernel:Updating grub on smithi175 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:43.685 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T07:35:44.340 INFO:teuthology.orchestra.run.smithi175.stderr:Generating grub configuration file ... 2024-09-21T07:35:45.730 INFO:teuthology.orchestra.run.smithi175.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T07:35:45.751 INFO:teuthology.orchestra.run.smithi175.stderr:done 2024-09-21T07:35:45.754 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-21T07:35:45.770 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.DfBhU0sc8e 2024-09-21T07:35:45.770 DEBUG:teuthology.orchestra.run.smithi175:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.DfBhU0sc8e 2024-09-21T07:35:45.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.DfBhU0sc8e 2024-09-21T07:35:45.979 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.DfBhU0sc8e is 6KB 2024-09-21T07:35:45.999 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.DfBhU0sc8e 2024-09-21T07:35:46.014 DEBUG:teuthology.orchestra.run.smithi175:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T07:35:46.077 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T07:35:46.077 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T07:35:46.077 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T07:35:46.077 INFO:teuthology.orchestra.run.smithi175.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T07:35:46.078 DEBUG:teuthology.orchestra.run.smithi175:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:35:46.228 DEBUG:teuthology.orchestra.run.smithi175:> sudo shutdown -r now 2024-09-21T07:36:11.947 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi088.front.sepia.ceph.com, path=/tmp/kernel.x86_64.rpm, version=None) 2024-09-21T07:36:11.947 DEBUG:teuthology.orchestra.run.smithi088:> rpm -qlp /tmp/kernel.x86_64.rpm 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/boot/System.map-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/boot/config-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/boot/vmlinuz-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/System.map 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/config 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/blowfish-x86_64.ko 2024-09-21T07:36:12.041 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/camellia-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32-pclmul.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crc32c-intel.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-avx2.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha1-ssse3.ko 2024-09-21T07:36:12.042 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha256-ssse3.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/sha512-ssse3.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/crypto/twofish-x86_64.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-amd.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm-intel.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/arch/x86/kvm/kvm.ko 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto 2024-09-21T07:36:12.043 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ansi_cprng.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/anubis.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/arc4.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_memcpy.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_pq.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_raid6_recov.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_tx.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/async_xor.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/async_tx/raid6test.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authenc.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/authencesn.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blake2b_generic.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_common.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/blowfish_generic.ko 2024-09-21T07:36:12.044 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/camellia_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast5_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast6_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cast_common.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ccm.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cmac.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crc32_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_engine.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/crypto_user.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/cts.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/des_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/dh_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecc.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/ecdh_generic.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/echainiv.ko 2024-09-21T07:36:12.045 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/essiv.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/fcrypt.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/khazad.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/lz4hc.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/md4.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/michael_mic.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcbc.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/pcrypt.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/rmd160.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/seed.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/serpent_generic.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tcrypt.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/tea.ko 2024-09-21T07:36:12.046 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_common.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/twofish_generic.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/vmac.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/wp512.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xcbc.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xor.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/crypto/xxhash_generic.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_ipmi.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/acpi_pad.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/ec_sys.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/platform_profile.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbs.ko 2024-09-21T07:36:12.047 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/sbshc.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/acpi/video.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/acard-ahci.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ahci_platform.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/ata_generic.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/libahci_platform.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_acpi.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ali.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_amd.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_artop.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atiixp.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_atp867x.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd640.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cmd64x.ko 2024-09-21T07:36:12.048 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_cypress.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_efar.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt366.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt37x.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x2n.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_hpt3x3.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it8213.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_it821x.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_jmicron.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_marvell.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_mpiix.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_netcell.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ninja32.ko 2024-09-21T07:36:12.049 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87410.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_ns87415.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_oldpiix.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_opti.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_optidma.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pcmcia.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc2027x.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_pdc202xx_old.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_piccolo.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_rdc.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sch.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_serverworks.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sil680.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sis.ko 2024-09-21T07:36:12.050 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_sl82c105.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_triflex.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pata_via.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/pdc_adma.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_inic162x.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_mv.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_nv.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_promise.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_qstor.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sil24.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sis.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_svw.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_sx4.ko 2024-09-21T07:36:12.051 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_uli.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_via.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ata/sata_vsc.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/auxdisplay/ks0108.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-i2c.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/base/regmap/regmap-mmio.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/bcma/bcma.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/aoe/aoe.ko 2024-09-21T07:36:12.052 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/brd.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/drbd/drbd.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/floppy.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/loop.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/mtip32xx/mtip32xx.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/nbd.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/null_blk/null_blk.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/pktcdvd.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/rbd.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/virtio_blk.ko 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback 2024-09-21T07:36:12.053 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkback/xen-blkback.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/xen-blkfront.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/block/zram/zram.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hangcheck-timer.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/amd-rng.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/intel-rng.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/timeriomem-rng.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/via-rng.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/hw_random/virtio-rng.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_devintf.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2024-09-21T07:36:12.054 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_si.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_ssif.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/lp.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/mwave/mwave.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/ppdev.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tlclk.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_atmel.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_infineon.ko 2024-09-21T07:36:12.055 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_nsc.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/tpm/tpm_tis_core.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/uv_mmtimer.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/char/virtio_console.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/acpi-cpufreq.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/p4-clockmod.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/pcc-cpufreq.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/powernow-k8.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/cpufreq/speedstep-lib.ko 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp 2024-09-21T07:36:12.056 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp-crypto.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/ccp/ccp.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-aes.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/padlock-sha.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/crypto/virtio/virtio_crypto.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dca/dca.ko 2024-09-21T07:36:12.057 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/devfreq/governor_simpleondemand.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/dw/dw_dmac.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/dma/ioat/ioatdma.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/amd64_edac.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/e752x_edac.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/edac_mce_amd.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3000_edac.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i3200_edac.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5100_edac.ko 2024-09-21T07:36:12.058 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i5400_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7300_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i7core_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/i82975x_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/ie31200_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/sb_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/edac/x38_edac.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/edd.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/firmware/iscsi_ibft.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-ich.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/gpio/gpio-viperboard.ko 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid 2024-09-21T07:36:12.059 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-appleir.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-aureal.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-axff.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-dr.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elecom.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-elo.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-emsff.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gaff.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gt683r.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-gyration.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-kbd.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtek-mouse.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-holtekff.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-hyperv.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-icade.ko 2024-09-21T07:36:12.060 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-keytouch.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-kye.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lcpower.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-led.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-lenovo.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-dj.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-logitech-hidpp.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-multitouch.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-ortek.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-penmount.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-petalynx.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-picolcd.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-pl.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-plantronics.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-primax.ko 2024-09-21T07:36:12.061 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-rmi.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-arvo.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-common.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-isku.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kone.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-koneplus.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-konepure.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-kovaplus.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-lua.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-pyra.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-ryos.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat-savu.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-roccat.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-saitek.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-samsung.ko 2024-09-21T07:36:12.062 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sensor-hub.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sjoy.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sony.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-speedlink.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-steelseries.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-sunplus.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-thrustmaster.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tivo.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-tmff.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-topseed.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-twinhan.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-uclogic.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-waltop.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-wiimote.ko 2024-09-21T07:36:12.063 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-xinmo.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zpff.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/hid-zydacron.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/uhid.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hid/wacom.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_balloon.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_utils.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hv/hv_vmbus.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/abituguru3.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/acpi_power_meter.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7414.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ad7418.ko 2024-09-21T07:36:12.064 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adc128d818.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1025.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1026.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1029.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm1031.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adm9240.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ads7828.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7410.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7411.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7462.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7470.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7475.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/adt7x10.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/amc6821.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/applesmc.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asb100.ko 2024-09-21T07:36:12.065 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asc7621.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/asus_atk0110.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/atxp1.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/coretemp.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dell-smm-hwmon.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/dme1737.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds1621.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ds620.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc1403.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/emc6w201.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71805f.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f71882fg.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/f75375s.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fam15h_power.ko 2024-09-21T07:36:12.066 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/fschmd.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g760a.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/g762.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl518sm.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/gl520sm.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/hwmon-vid.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5500_temp.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/i5k_amb.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmaem.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ibmpex.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina209.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ina2xx.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/it87.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k10temp.ko 2024-09-21T07:36:12.067 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/k8temp.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lineage-pem.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm63.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm73.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm75.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm77.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm78.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm80.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm83.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm85.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm87.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm90.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm92.ko 2024-09-21T07:36:12.068 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm93.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95234.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95241.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/lm95245.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc2945.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4151.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4215.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4222.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4245.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4260.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/ltc4261.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max16065.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1619.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max1668.ko 2024-09-21T07:36:12.069 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max197.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6639.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6650.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/max6697.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/mcp3021.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6683.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775-core.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct6775.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/nct7802.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87360.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pc87427.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pcf8591.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/adm1275.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/lm25066.ko 2024-09-21T07:36:12.070 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ltc2978.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max16064.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max34440.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/max8688.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/tps40422.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9000.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/ucd9200.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/pmbus/zl6100.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/powr1220.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5627.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch5636.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sch56xx-common.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht15.ko 2024-09-21T07:36:12.071 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sht21.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/shtc1.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/sis5595.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47b397.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m1.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/smsc47m192.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/thmc50.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp102.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp103.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp401.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/tmp421.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via-cputemp.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/via686a.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt1211.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/vt8231.ko 2024-09-21T07:36:12.072 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627ehf.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83627hf.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83781d.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83791d.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83792d.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83793.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83795.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l785ts.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/hwmon/w83l786ng.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses 2024-09-21T07:36:12.073 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd756.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-amd8111.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-core.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-i801.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-isch.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-ismt.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-nforce2.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-parport.ko 2024-09-21T07:36:12.074 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-piix4.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-scmi.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-simtec.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-sis96x.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-via.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viapro.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/busses/i2c-viperboard.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-dev.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-mux.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-slave-eeprom.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-smbus.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/i2c/i2c-stub.ko 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband 2024-09-21T07:36:12.075 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_cm.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_core.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_umad.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/ib_uverbs.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/iw_cm.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_cm.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/core/rdma_ucm.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5 2024-09-21T07:36:12.076 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2024-09-21T07:36:12.077 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/emu10k1-gp.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/fm801-gp.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/gameport/gameport.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/joydev.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/keyboard/gpio_keys.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/appletouch.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/bcm5974.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/cyapatp.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/elan_i2c.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/sermouse.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_i2c.ko 2024-09-21T07:36:12.078 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/synaptics_usb.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/mouse/vsxxxaa.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/rmi4/rmi_core.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/altera_ps2.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/arc_ps2.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/hyperv-keyboard.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/serio/serio_raw.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/input/sparse-keymap.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-blinkm.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lm3530.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-lp3944.ko 2024-09-21T07:36:12.079 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/leds-ss4200.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-camera.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-gpio.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-timer.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/leds/trigger/ledtrig-transient.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/bcache/bcache.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-bio-prison.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache-smq.ko 2024-09-21T07:36:12.080 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-cache.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-crypt.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-delay.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-flakey.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-log-userspace.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-multipath.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-queue-length.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-raid.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-round-robin.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-service-time.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-switch.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-thin-pool.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/dm-verity.ko 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data 2024-09-21T07:36:12.081 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid0.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid1.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid10.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/md/raid456.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptbase.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptctl.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptfc.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptlan.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptsas.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptscsih.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/message/fusion/mptspi.ko 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd 2024-09-21T07:36:12.082 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_ich.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/lpc_sch.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/mfd-core.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/sm501.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/viperboard.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/vx855.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/mfd/wl1273-core.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds9802als.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/apds990x.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/bh1770glc.ko 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710 2024-09-21T07:36:12.083 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/cb710/cb710.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/at24.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/eeprom/max6875.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/enclosure.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/hpilo.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29003.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/isl29020.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-me.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei-txe.ko 2024-09-21T07:36:12.084 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/mei/mei.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-gru/gru.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xp.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpc.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/sgi-xp/xpnet.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_7xx1.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tifm_core.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/tsl2550.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_balloon.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding 2024-09-21T07:36:12.085 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/bonding/bonding.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/b53/b53_common.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/bcm-sf2.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6060.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/dummy.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/eql.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2024-09-21T07:36:12.086 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/3c59x.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/3com/typhoon.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/8390.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/adaptec/starfire.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/agere/et131x.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/alteon/acenic.ko 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera 2024-09-21T07:36:12.087 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/altera/altera_tse.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/amd8111e.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/amd/pcnet32.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx 2024-09-21T07:36:12.088 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/b44.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/cnic.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/broadcom/tg3.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio 2024-09-21T07:36:12.089 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec 2024-09-21T07:36:12.090 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/dl2k.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dlink/sundance.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/dnet.ko 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet 2024-09-21T07:36:12.091 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ethoc.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/fealnx.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e100.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2024-09-21T07:36:12.092 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igb/igb.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libeth/libeth.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/intel/libie/libie.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/jme.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2024-09-21T07:36:12.093 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/skge.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/marvell/sky2.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2024-09-21T07:36:12.094 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/neterion/s2io.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2024-09-21T07:36:12.095 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rdc/r6040.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139cp.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/8139too.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/atp.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/realtek/r8169.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/rocker/rocker.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sfc/sfc.ko 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan 2024-09-21T07:36:12.096 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/silan/sc92031.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis190.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sis/sis900.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/epic100.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun 2024-09-21T07:36:12.097 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/cassini.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/niu.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sungem.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/sun/sunhme.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/ti/tlan.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-rhine.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/via/via-velocity.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5100.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/wiznet/w5300.ko 2024-09-21T07:36:12.098 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/geneve.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/hyperv/hv_netvsc.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ieee802154/fakelb.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ifb.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ipvlan/ipvlan.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvlan.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/macvtap.ko 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio 2024-09-21T07:36:12.099 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-bitbang.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mdio/mdio-regmap.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/mii.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/net_failover.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/netconsole.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/nlmon.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs-lynx.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/pcs/pcs_xpcs.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/amd.ko 2024-09-21T07:36:12.100 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-lib.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm-phy-ptp.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm7xxx.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/bcm87xx.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/broadcom.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/cicada.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/davicom.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/dp83640.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/et1011c.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/icplus.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/lxt.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/marvell.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/micrel.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/national.ko 2024-09-21T07:36:12.101 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/phylink.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/qsemi.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/realtek.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/smsc.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/ste10Xp.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/phy/vitesse.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/bsd_comp.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_async.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_deflate.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_generic.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_mppe.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/ppp_synctty.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppoe.ko 2024-09-21T07:36:12.102 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pppox.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/ppp/pptp.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slhc.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/slip/slip.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/sungem_phy.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tap.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_activebackup.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_broadcast.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_loadbalance.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_random.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/team/team_mode_roundrobin.ko 2024-09-21T07:36:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/tun.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/veth.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/virtio_net.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vmxnet3/vmxnet3.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/vxlan/vxlan.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netback/xen-netback.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/net/xen-netfront.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ntb/ntb.ko 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host 2024-09-21T07:36:12.104 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme-core.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/nvme/host/nvme.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_cs.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_pc.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/parport/parport_serial.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/pcie/aer_inject.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pci/xen-pcifront.ko 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia 2024-09-21T07:36:12.105 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/i82092.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/pd6729.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pcmcia/yenta_socket.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acer-wmi.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/acerhdf.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/apple-gmux.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-laptop.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-nb-wmi.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/asus-wmi.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/classmate-laptop.ko 2024-09-21T07:36:12.106 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-laptop.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/eeepc-wmi.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-laptop.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/fujitsu-tablet.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/hdaps.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel/intel-rst.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/intel_ips.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/msi-wmi.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/mxm-wmi.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/panasonic-laptop.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-laptop.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/samsung-q10.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/topstar-laptop.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2024-09-21T07:36:12.107 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/toshiba_haps.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi-bmof.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/platform/x86/wmi.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_common.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/powercap/intel_rapl_msr.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-gpio.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps-ldisc.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/clients/pps_parport.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/pps/pps_core.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp.ko 2024-09-21T07:36:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ptp/ptp_kvm.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/atl/amd_atl.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ras/amd/fmpm.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-bq32k.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1286.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1307.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1374.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1511.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1553.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1672.ko 2024-09-21T07:36:12.109 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds1742.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds2404.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-ds3232.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-em3027.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-fm3130.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl12022.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-isl1208.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m41t80.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t35.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-m48t59.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-max6900.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-msm6242.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf2127.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf85063.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8523.ko 2024-09-21T07:36:12.110 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8563.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-pcf8583.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rp5c01.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rs5c372.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rv3029c2.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8025.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-rx8581.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-stk17ta8.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/rtc/rtc-x1205.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-9xxx.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-sas.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/3w-xxxx.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/BusLogic.ko 2024-09-21T07:36:12.111 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/a100u2w.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aacraid/aacraid.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/advansys.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/am53c974.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/arcmsr/arcmsr.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/atp870u.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bfa/bfa.ko 2024-09-21T07:36:12.112 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/bnx2i/bnx2i.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ch.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/csiostor/csiostor.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dc395x.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_alua.ko 2024-09-21T07:36:12.113 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_emc.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_hp_sw.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/device_handler/scsi_dh_rdac.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/dmx3191d.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esas2r/esas2r.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/esp_scsi.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/fcoe.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fcoe/libfcoe.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/fnic/fnic.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hpsa.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hptiop.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/hv_storvsc.ko 2024-09-21T07:36:12.114 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/initio.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ipr.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ips.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/isci/isci.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/iscsi_tcp.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libfc/libfc.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libiscsi_tcp.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/libsas/libsas.ko 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc 2024-09-21T07:36:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/lpfc/lpfc.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvsas/mvsas.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/mvumi.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pm8001/pm80xx.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/pmcraid.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla1280.ko 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx 2024-09-21T07:36:12.116 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/raid_class.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_debug.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_fc.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_iscsi.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_sas.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_spi.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/scsi_transport_srp.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/ses.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/st.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/stex.ko 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2 2024-09-21T07:36:12.117 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/virtio_scsi.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/vmw_pvscsi.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/scsi/wd719x.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ssb/ssb.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/loopback/tcm_loop.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_file.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_iblock.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_mod.ko 2024-09-21T07:36:12.118 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_pscsi.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/target_core_user.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/target/tcm_fc/tcm_fc.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2024-09-21T07:36:12.119 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_power_floor.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_hint.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_wt_req.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/ipwireless/ipwireless.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_gsm.ko 2024-09-21T07:36:12.120 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/n_hdlc.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/nozomi.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/8250/serial_cs.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/arc_uart.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/tty/serial/jsm/jsm.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/core/ufshcd-core.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/ufs/host/ufshcd-pci.ko 2024-09-21T07:36:12.121 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_aec.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_cif.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_pci_generic.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/uio/uio_sercos3.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-acm.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/cdc-wdm.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usblp.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/class/usbtmc.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/host/sl811-hcd.ko 2024-09-21T07:36:12.122 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/mdc800.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/image/microtek.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/adutux.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/appledisplay.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi26.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/emi62.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ezusb.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/idmouse.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/iowarrior.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/isight_firmware.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/ldusb.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/legousbtower.ko 2024-09-21T07:36:12.123 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/trancevibrator.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usb3503.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usblcd.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/usbsevseg.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/uss720.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/misc/yurex.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/phy/phy-generic.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/aircable.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ark3116.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/belkin_sa.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ch341.ko 2024-09-21T07:36:12.124 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cp210x.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cyberjack.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/cypress_m8.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/digi_acceleport.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/empeg.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ftdi_sio.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/garmin_gps.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_edgeport.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/io_ti.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipaq.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ipw.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ir-usb.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/iuu_phoenix.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/keyspan_pda.ko 2024-09-21T07:36:12.125 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kl5kusb105.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/kobil_sct.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mct_u232.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7720.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/mos7840.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/navman.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/omninet.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/opticon.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/option.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/oti6858.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/pl2303.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcaux.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/qcserial.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/quatech2.ko 2024-09-21T07:36:12.126 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/safe_serial.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/sierra.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/spcp8x5.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ssu100.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/symbolserial.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb-serial-simple.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_debug.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/usb_wwan.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/visor.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/whiteheat.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/serial/xsens_mt.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/uas.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-alauda.ko 2024-09-21T07:36:12.127 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-cypress.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-datafab.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-eneub6250.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-freecom.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-isd200.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-jumpshot.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-karma.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-onetouch.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-realtek.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr09.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-sddr55.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/ums-usbat.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/storage/usb-storage.ko 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip 2024-09-21T07:36:12.128 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-core.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/usbip-host.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/usb/usbip/vhci-hcd.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci-core.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/pci/vfio-pci.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vfio/vfio_iommu_type1.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_iotlb.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_net.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/vhost/vhost_scsi.ko 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video 2024-09-21T07:36:12.129 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/apple_bl.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/lcd.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/video/backlight/platform_lcd.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_balloon.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_mmio.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_legacy_dev.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_pci_modern_dev.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/virtio/virtio_ring.ko 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters 2024-09-21T07:36:12.130 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2482.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/masters/ds2490.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2406.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2408.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2413.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2423.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2431.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2433.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2780.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds2781.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_ds28e04.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_smem.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/slaves/w1_therm.ko 2024-09-21T07:36:12.131 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/w1/wire.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim1535_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/alim7101_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/f71808e_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/hpwdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/i6300esb.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_vendor_support.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/iTCO_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ib700wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ibmasr.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/ie6xx_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it8712f_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/it87_wdt.ko 2024-09-21T07:36:12.132 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/machzwd.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/nv_tco.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_pci.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/pcwd_usb.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sch311x_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/softdog.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/sp5100_tco.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/via_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83627hf_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83877f_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/w83977f_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/wdt_pci.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/watchdog/xen_wdt.ko 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen 2024-09-21T07:36:12.133 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-acpi-processor.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-evtchn.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntalloc.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-gntdev.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xen-privcmd.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/drivers/xen/xenfs/xenfs.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/9p/9p.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/affs/affs.ko 2024-09-21T07:36:12.134 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/befs/befs.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/binfmt_misc.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/btrfs/btrfs.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cachefiles/cachefiles.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ceph/ceph.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/coda/coda.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/cramfs/cramfs.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/dlm/dlm.ko 2024-09-21T07:36:12.135 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ecryptfs/ecryptfs.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/f2fs/f2fs.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/fat.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/msdos.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fat/vfat.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/cuse.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/fuse/fuse.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/gfs2/gfs2.ko 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs 2024-09-21T07:36:12.136 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfs/hfs.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/hfsplus/hfsplus.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/isofs/isofs.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/jfs/jfs.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/lockd/lockd.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/minix/minix.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/netfs/netfs.ko 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs 2024-09-21T07:36:12.137 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfs.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv3.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs/nfsv4.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/grace.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfs_common/nfs_acl.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nfsd/nfsd.ko 2024-09-21T07:36:12.138 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nilfs2/nilfs2.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-celtic.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-centeuro.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-croatian.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-cyrillic.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-gaelic.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-greek.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-iceland.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-inuit.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-roman.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-romanian.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/mac-turkish.ko 2024-09-21T07:36:12.139 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1250.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1251.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp1255.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp737.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp775.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp850.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp852.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp855.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp857.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp860.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp861.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp862.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp863.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp864.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp865.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp866.ko 2024-09-21T07:36:12.140 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp869.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp874.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp932.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp936.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp949.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_cp950.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_euc-jp.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-1.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-13.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-14.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-15.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-2.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-3.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-4.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-5.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-6.ko 2024-09-21T07:36:12.141 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-7.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_iso8859-9.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-r.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-ru.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_koi8-u.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_ucs2_utils.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/nls/nls_utf8.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2.ko 2024-09-21T07:36:12.142 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stack_user.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ocfs2/ocfs2_stackglue.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/overlayfs/overlay.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/pstore/ramoops.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/reiserfs/reiserfs.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/romfs/romfs.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/client/cifs.ko 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common 2024-09-21T07:36:12.143 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_arc4.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/smb/common/cifs_md4.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/squashfs/squashfs.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/sysv/sysv.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/udf/udf.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/ufs/ufs.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/fs/xfs/xfs.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/rcu/rcutorture.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/torture.ko 2024-09-21T07:36:12.144 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/kernel/trace/ring_buffer_benchmark.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/asn1_encoder.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc-itu-t.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crc8.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libaescfb.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libarc4.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/crypto/libdes.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/libcrc32c.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lru_cache.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4_compress.ko 2024-09-21T07:36:12.145 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/lz4/lz4hc_compress.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/math/cordic.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/raid6/raid6_pq.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/reed_solomon/reed_solomon.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_bm.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_fsm.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/ts_kmp.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/lib/zstd/zstd_compress.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/mm/hwpoison-inject.ko 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net 2024-09-21T07:36:12.146 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/6lowpan.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_dest.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_fragment.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_hop.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_ipv6.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_mobility.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_routing.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/6lowpan/nhc_udp.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/garp.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/mrp.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/p8022.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/psnap.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/802/stp.ko 2024-09-21T07:36:12.147 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/8021q/8021q.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_fd.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_rdma.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/9p/9pnet_virtio.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/batman-adv/batman-adv.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/br_netfilter.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/bridge.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_802_3.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_among.ko 2024-09-21T07:36:12.148 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arp.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_arpreply.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_dnat.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_ip6.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_limit.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_log.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_mark_m.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_nflog.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_pkttype.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_redirect.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_snat.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_stp.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebt_vlan.ko 2024-09-21T07:36:12.149 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_broute.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_filter.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtable_nat.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/ebtables.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nf_conntrack_bridge.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ceph/libceph.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/failover.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/core/pktgen.ko 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp 2024-09-21T07:36:12.150 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_diag.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv4.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dccp/dccp_ipv6.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dns_resolver/dns_resolver.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/dsa_core.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_brcm.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_dsa.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_none.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/dsa/tag_trailer.ko 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan 2024-09-21T07:36:12.151 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ieee802154/ieee802154_socket.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ah4.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/esp4.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/fou.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/gre.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/inet_diag.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_gre.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_tunnel.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ip_vti.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipcomp.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/ipip.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arp_tables.ko 2024-09-21T07:36:12.152 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arpt_mangle.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/arptable_filter.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ECN.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_ah.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_mangle.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_nat.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_raw.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/iptable_security.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2024-09-21T07:36:12.153 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_bic.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_dctcp.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_diag.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_highspeed.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_htcp.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_hybla.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_illinois.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_lp.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_scalable.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_vegas.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_veno.ko 2024-09-21T07:36:12.154 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_westwood.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tcp_yeah.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/tunnel4.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_diag.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/udp_tunnel.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv4/xfrm4_tunnel.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ah6.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/esp6.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/fou6.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_tunnel.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_udp_tunnel.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ip6_vti.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/ipcomp6.ko 2024-09-21T07:36:12.155 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6_tables.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ah.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_frag.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_mh.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6t_rt.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_filter.ko 2024-09-21T07:36:12.156 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_nat.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_raw.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/ip6table_security.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/sit.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/tunnel6.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/ipv6/xfrm6_tunnel.ko 2024-09-21T07:36:12.157 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/key/af_key.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_core.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_debugfs.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_eth.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ip6.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_netlink.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/l2tp/l2tp_ppp.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/llc/llc.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mac802154/mac802154.ko 2024-09-21T07:36:12.158 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/mpls/mpls_gso.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2024-09-21T07:36:12.159 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipset/ip_set_list_set.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2024-09-21T07:36:12.160 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conncount.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_amanda.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_broadcast.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_ftp.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_h323.ko 2024-09-21T07:36:12.161 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_irc.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_netlink.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_pptp.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sane.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_sip.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_snmp.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_conntrack_tftp.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_dup_netdev.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_flow_table_inet.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_log_syslog.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_amanda.ko 2024-09-21T07:36:12.162 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_ftp.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_irc.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_sip.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_nat_tftp.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_synproxy_core.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nf_tables.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_acct.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_log.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_osf.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nfnetlink_queue.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_chain_nat.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_compat.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_ct.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_dup_netdev.ko 2024-09-21T07:36:12.163 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fib.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_fwd_netdev.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_hash.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_limit.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_log.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_masq.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_nat.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_numgen.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_queue.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_quota.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_redir.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_reject_inet.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_socket.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/nft_tunnel.ko 2024-09-21T07:36:12.164 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_AUDIT.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CHECKSUM.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CLASSIFY.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CONNSECMARK.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_CT.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_DSCP.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HL.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_HMARK.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_IDLETIMER.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LED.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_LOG.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_MASQUERADE.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NETMAP.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFLOG.ko 2024-09-21T07:36:12.165 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_NFQUEUE.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_RATEEST.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_REDIRECT.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_SECMARK.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPMSS.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TEE.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TPROXY.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_TRACE.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_addrtype.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_bpf.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cgroup.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cluster.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_comment.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connbytes.ko 2024-09-21T07:36:12.166 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlabel.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connlimit.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_connmark.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_conntrack.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_cpu.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dccp.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_devgroup.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_dscp.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ecn.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_esp.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hashlimit.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_helper.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_hl.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipcomp.ko 2024-09-21T07:36:12.167 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_iprange.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_ipvs.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_l2tp.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_length.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_limit.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mac.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_mark.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_multiport.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nat.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_nfacct.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_osf.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_owner.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_physdev.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_pkttype.ko 2024-09-21T07:36:12.168 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_policy.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_quota.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_rateest.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_realm.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_recent.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_sctp.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_set.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_socket.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_state.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_statistic.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_string.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_tcpmss.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_time.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netfilter/xt_u32.ko 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink 2024-09-21T07:36:12.169 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/netlink/netlink_diag.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/nsh/nsh.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/openvswitch.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-geneve.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-gre.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/openvswitch/vport-vxlan.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/packet/af_packet_diag.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_rdma.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/rds/rds_tcp.ko 2024-09-21T07:36:12.170 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_csum.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_gact.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_mirred.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_nat.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_pedit.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_police.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_simple.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_skbedit.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/act_vlan.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_basic.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_bpf.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_flow.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_fw.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_route.ko 2024-09-21T07:36:12.171 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/cls_u32.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_cmp.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_ipset.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_meta.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_nbyte.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_text.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/em_u32.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_choke.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_codel.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_drr.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_fq.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_gred.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hfsc.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_hhf.ko 2024-09-21T07:36:12.172 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_htb.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_ingress.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_mqprio_lib.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_multiq.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_netem.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_pie.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_plug.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_prio.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_qfq.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_red.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfb.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_sfq.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_tbf.ko 2024-09-21T07:36:12.173 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sched/sch_teql.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sctp/sctp_diag.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/sunrpc.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/tipc/tipc_diag.ko 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix 2024-09-21T07:36:12.174 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/unix/unix_diag.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_diag.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/vmw_vsock/vsock_loopback.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/net/xfrm/xfrm_ipcomp.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/encrypted-keys/encrypted-keys.ko 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys 2024-09-21T07:36:12.175 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/kernel/security/keys/trusted-keys/trusted.ko 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.alias.bin 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.alias.bin 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.bin 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.builtin.modinfo 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep.bin 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.devname 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.order 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.softdep 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.symbols.bin 2024-09-21T07:36:12.176 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.weakdep 2024-09-21T07:36:12.177 INFO:teuthology.orchestra.run.smithi088.stdout:/lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:36:12.180 DEBUG:teuthology.task.kernel:get_image_version: 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.180 DEBUG:teuthology.orchestra.run.smithi088:> rpm --scripts -qp /tmp/kernel.x86_64.rpm 2024-09-21T07:36:12.204 INFO:teuthology.orchestra.run.smithi088.stdout:postinstall scriptlet (using /bin/sh): 2024-09-21T07:36:12.204 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:36:12.204 INFO:teuthology.orchestra.run.smithi088.stdout: /usr/bin/kernel-install add 6.11.0-rc5-g2ca44a2f35ca /lib/modules/6.11.0-rc5-g2ca44a2f35ca/vmlinuz 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout:for file in vmlinuz System.map config; do 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: if ! cmp --silent "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca"; then 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: cp "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/${file}" "/boot/${file}-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: fi 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout:done 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -d "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" ] && \ 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: ! diff -rq "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" >/dev/null 2>&1; then 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: rm -rf "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: cp -r "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/dtb" "/boot/dtb-6.11.0-rc5-g2ca44a2f35ca" 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout:if [ ! -e "/lib/modules/6.11.0-rc5-g2ca44a2f35ca/modules.dep" ]; then 2024-09-21T07:36:12.205 INFO:teuthology.orchestra.run.smithi088.stdout: /usr/sbin/depmod 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.206 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-09-21T07:36:12.206 INFO:teuthology.orchestra.run.smithi088.stdout:preuninstall scriptlet (using /bin/sh): 2024-09-21T07:36:12.206 INFO:teuthology.orchestra.run.smithi088.stdout:if [ -x /usr/bin/kernel-install ]; then 2024-09-21T07:36:12.206 INFO:teuthology.orchestra.run.smithi088.stdout:kernel-install remove 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.206 INFO:teuthology.orchestra.run.smithi088.stdout:fi 2024-09-21T07:36:12.207 DEBUG:teuthology.orchestra.run.smithi088:> sudo rpm -qi grub2-tools 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Name : grub2-tools 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Epoch : 1 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Version : 2.06 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Release : 82.el9 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Architecture: x86_64 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Group : Unspecified 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Size : 8274670 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:License : GPLv3+ 2024-09-21T07:36:12.282 INFO:teuthology.orchestra.run.smithi088.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Packager : builder@centos.org 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Vendor : CentOS 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Summary : Support tools for GRUB. 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:Description : 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout:hardware devices. 2024-09-21T07:36:12.283 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:36:12.284 INFO:teuthology.orchestra.run.smithi088.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T07:36:12.284 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T07:36:12.284 INFO:teuthology.task.kernel:Updating grub on smithi088 to boot 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:12.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T07:36:12.921 INFO:teuthology.orchestra.run.smithi088.stderr:Generating grub configuration file ... 2024-09-21T07:36:14.261 INFO:teuthology.orchestra.run.smithi088.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T07:36:14.293 INFO:teuthology.orchestra.run.smithi088.stderr:done 2024-09-21T07:36:14.299 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-09-21T07:36:14.315 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.NDRLsrxXlS 2024-09-21T07:36:14.315 DEBUG:teuthology.orchestra.run.smithi088:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.NDRLsrxXlS 2024-09-21T07:36:14.390 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.NDRLsrxXlS 2024-09-21T07:36:14.527 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.NDRLsrxXlS is 6KB 2024-09-21T07:36:14.547 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.NDRLsrxXlS 2024-09-21T07:36:14.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T07:36:14.629 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T07:36:14.630 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T07:36:14.630 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T07:36:14.630 INFO:teuthology.orchestra.run.smithi088.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca.conf 2024-09-21T07:36:14.631 DEBUG:teuthology.orchestra.run.smithi088:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:36:14.802 DEBUG:teuthology.orchestra.run.smithi088:> sudo shutdown -r now 2024-09-21T07:36:16.232 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2024-09-21T07:36:16.233 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T07:36:16.233 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:36:44.825 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T07:36:44.825 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T07:36:44.826 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:37:16.247 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T07:37:19.343 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T07:37:25.254 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T07:37:25.255 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:37:25.592 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-21T07:37:26.319 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T07:37:26.320 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T07:37:26.320 DEBUG:teuthology.orchestra.run.smithi175:> uname -r 2024-09-21T07:37:26.354 INFO:teuthology.orchestra.run.smithi175.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:37:26.355 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:37:26.355 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T07:37:26.355 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T07:37:27.357 DEBUG:teuthology.parallel:result is None 2024-09-21T07:37:28.343 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T07:37:28.344 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:37:31.407 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.88 2024-09-21T07:37:43.408 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T07:37:43.408 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:38:15.206 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-09-21T07:38:15.762 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T07:38:15.762 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.11.0-rc5-g2ca44a2f35ca"... 2024-09-21T07:38:15.762 DEBUG:teuthology.orchestra.run.smithi088:> uname -r 2024-09-21T07:38:15.799 INFO:teuthology.orchestra.run.smithi088.stdout:6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:38:15.799 DEBUG:teuthology.task.kernel:current kernel version is 6.11.0-rc5-g2ca44a2f35ca vs 6.11.0-rc5-g2ca44a2f35ca 2024-09-21T07:38:15.799 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T07:38:15.799 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T07:38:16.800 DEBUG:teuthology.parallel:result is None 2024-09-21T07:38:16.801 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T07:38:16.809 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T07:38:16.809 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T07:38:16.813 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T07:38:16.855 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T07:38:16.865 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T07:38:16.874 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T07:38:16.874 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T07:38:16.889 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T07:38:16.950 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T07:38:16.958 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T07:38:16.958 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T07:38:16.972 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T07:38:17.106 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:38:17.135 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:38:17.145 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:38:17.188 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T07:38:17.191 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T07:38:17.200 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T07:38:17.200 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T07:38:17.204 DEBUG:teuthology.orchestra.run.smithi175:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T07:38:17.261 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T07:38:17.270 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T07:38:17.270 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T07:38:17.274 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T07:38:17.308 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:38:17.411 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:38:17.499 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:38:17.564 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:38:17.638 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:38:17.639 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T07:38:17.712 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:38:17.776 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T07:38:17.854 DEBUG:teuthology.orchestra.run.smithi175:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:38:17.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T07:38:17.986 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:38:17.986 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T07:38:18.041 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-09-21T07:38:18.045 DEBUG:teuthology.orchestra.run.smithi175:> sudo service rsyslog restart 2024-09-21T07:38:18.148 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:38:18.156 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:38:18.671 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T07:38:18.679 INFO:teuthology.task.internal:Starting timer... 2024-09-21T07:38:18.680 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T07:38:18.732 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T07:38:18.741 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-09-21T07:38:18.854 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-09-21T07:38:18.857 DEBUG:teuthology.orchestra.run.smithi175:> sudo service auditd rotate 2024-09-21T07:38:18.954 INFO:teuthology.orchestra.run.smithi175.stdout:Rotating logs: 2024-09-21T07:38:18.957 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T07:38:18.957 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-09-21T07:38:19.009 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-09-21T07:38:19.010 DEBUG:teuthology.orchestra.run.smithi175:> /usr/sbin/getenforce 2024-09-21T07:38:19.046 INFO:teuthology.orchestra.run.smithi175.stdout:Permissive 2024-09-21T07:38:19.046 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi088.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi175.front.sepia.ceph.com': 'permissive'} 2024-09-21T07:38:19.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:38:19.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:38:19.103 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:38:19.162 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:38:19.162 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T07:38:19.162 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-09-21T07:38:19.233 DEBUG:teuthology.orchestra.run.smithi175:> sudo /usr/sbin/setenforce permissive 2024-09-21T07:38:19.297 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T07:38:19.307 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T07:38:19.317 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T07:38:19.318 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi088.front.sepia.ceph.com,smithi175.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T07:43:03.826 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi175.front.sepia.ceph.com')] 2024-09-21T07:43:03.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T07:43:03.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:43:03.919 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-09-21T07:43:04.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-09-21T07:43:04.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T07:43:04.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi175.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T07:43:04.117 DEBUG:teuthology.orchestra.run.smithi175:> true 2024-09-21T07:43:04.220 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi175.front.sepia.ceph.com' 2024-09-21T07:43:04.221 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T07:43:04.239 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T07:43:04.239 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T07:43:04.239 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:43:04.243 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T07:43:04.244 DEBUG:teuthology.orchestra.run.smithi175:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:43:04.300 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T07:43:04.322 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T07:43:04.329 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T07:43:04.353 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T07:43:04.380 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-09-21T07:43:04.403 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-09-21T07:43:04.404 INFO:teuthology.orchestra.run.smithi175.stderr:sudo: ntpd: command not found 2024-09-21T07:43:04.427 INFO:teuthology.orchestra.run.smithi175.stdout:506 Cannot talk to daemon 2024-09-21T07:43:04.431 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T07:43:04.456 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T07:43:04.461 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T07:43:04.486 INFO:teuthology.orchestra.run.smithi175.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T07:43:04.559 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:43:04.565 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:43:04.565 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-09-21T07:43:04.565 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.565 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.565 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.566 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.579 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:43:04.585 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:43:04.586 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-21T07:43:04.586 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.586 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.586 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.586 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:43:04.587 INFO:teuthology.run_tasks:Running task install... 2024-09-21T07:43:04.596 DEBUG:teuthology.task.install:project ceph 2024-09-21T07:43:04.597 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T07:43:04.597 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T07:43:04.597 INFO:teuthology.task.install:Using flavor: default 2024-09-21T07:43:04.603 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T07:43:04.603 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T07:43:04.603 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T07:43:04.604 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:43:04.605 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T07:43:04.606 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:43:04.756 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T07:43:04.756 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T07:43:04.759 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T07:43:04.760 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T07:43:04.863 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T07:43:04.863 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:43:04.864 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T07:43:04.872 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T07:43:04.872 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:43:04.872 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T07:43:04.921 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T07:43:04.921 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T07:43:04.928 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T07:43:04.928 DEBUG:teuthology.orchestra.run.smithi175:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T07:43:05.019 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T07:43:05.025 DEBUG:teuthology.orchestra.run.smithi175:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T07:43:05.096 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T07:43:05.103 DEBUG:teuthology.orchestra.run.smithi175:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T07:43:05.151 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-09-21T07:43:05.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-09-21T07:43:05.157 INFO:teuthology.orchestra.run.smithi175.stdout:check_obsoletes = 1 2024-09-21T07:43:05.160 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum clean all 2024-09-21T07:43:05.432 INFO:teuthology.orchestra.run.smithi088.stdout:45 files removed 2024-09-21T07:43:05.441 INFO:teuthology.orchestra.run.smithi175.stdout:45 files removed 2024-09-21T07:43:05.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T07:43:05.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T07:43:06.139 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 297 kB/s | 84 kB 00:00 2024-09-21T07:43:06.177 INFO:teuthology.orchestra.run.smithi175.stdout:ceph packages for x86_64 271 kB/s | 84 kB 00:00 2024-09-21T07:43:06.448 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 91 kB/s | 24 kB 00:00 2024-09-21T07:43:06.477 INFO:teuthology.orchestra.run.smithi175.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T07:43:06.732 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T07:43:06.762 INFO:teuthology.orchestra.run.smithi175.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T07:43:08.257 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.3 MB 00:01 2024-09-21T07:43:08.386 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.3 MB 00:01 2024-09-21T07:43:12.474 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 6.5 MB/s | 20 MB 00:03 2024-09-21T07:43:13.228 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 20 MB 00:03 2024-09-21T07:43:17.945 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T07:43:18.116 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 4.0 MB/s | 6.5 MB 00:01 2024-09-21T07:43:19.646 INFO:teuthology.orchestra.run.smithi175.stdout:CentOS Stream 9 - Extras packages 48 kB/s | 19 kB 00:00 2024-09-21T07:43:19.692 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 75 kB/s | 19 kB 00:00 2024-09-21T07:43:20.567 INFO:teuthology.orchestra.run.smithi175.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-21T07:43:20.661 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 27 MB/s | 23 MB 00:00 2024-09-21T07:43:26.864 INFO:teuthology.orchestra.run.smithi175.stdout:lab-extras 20 kB/s | 1.7 kB 00:00 2024-09-21T07:43:26.960 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-21T07:43:29.218 INFO:teuthology.orchestra.run.smithi175.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:43:29.219 INFO:teuthology.orchestra.run.smithi175.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:43:29.230 INFO:teuthology.orchestra.run.smithi175.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T07:43:29.231 INFO:teuthology.orchestra.run.smithi175.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T07:43:29.232 INFO:teuthology.orchestra.run.smithi175.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T07:43:29.233 INFO:teuthology.orchestra.run.smithi175.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T07:43:29.235 INFO:teuthology.orchestra.run.smithi175.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T07:43:29.237 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T07:43:29.238 INFO:teuthology.orchestra.run.smithi175.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T07:43:29.303 INFO:teuthology.orchestra.run.smithi175.stdout:Dependencies resolved. 2024-09-21T07:43:29.317 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-21T07:43:29.317 INFO:teuthology.orchestra.run.smithi175.stdout: Package Arch Version Repository Size 2024-09-21T07:43:29.317 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout:Installing: 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T07:43:29.318 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T07:43:29.319 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T07:43:29.320 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrading: 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout:Installing dependencies: 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T07:43:29.321 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T07:43:29.322 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T07:43:29.323 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T07:43:29.324 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T07:43:29.325 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T07:43:29.326 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T07:43:29.327 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T07:43:29.328 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout:Installing weak dependencies: 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction Summary 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout:======================================================================================= 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout:Install 149 Packages 2024-09-21T07:43:29.329 INFO:teuthology.orchestra.run.smithi175.stdout:Upgrade 8 Packages 2024-09-21T07:43:29.330 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:43:29.330 INFO:teuthology.orchestra.run.smithi175.stdout:Total download size: 228 M 2024-09-21T07:43:29.330 INFO:teuthology.orchestra.run.smithi175.stdout:Downloading Packages: 2024-09-21T07:43:29.340 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:43:29.341 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T07:43:29.355 INFO:teuthology.orchestra.run.smithi088.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T07:43:29.356 INFO:teuthology.orchestra.run.smithi088.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T07:43:29.357 INFO:teuthology.orchestra.run.smithi088.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T07:43:29.358 INFO:teuthology.orchestra.run.smithi088.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T07:43:29.360 INFO:teuthology.orchestra.run.smithi088.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T07:43:29.363 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T07:43:29.364 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T07:43:29.430 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-09-21T07:43:29.443 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-09-21T07:43:29.443 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-09-21T07:43:29.443 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T07:43:29.444 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T07:43:29.445 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T07:43:29.446 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T07:43:29.447 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T07:43:29.448 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T07:43:29.449 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T07:43:29.450 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T07:43:29.451 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T07:43:29.452 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T07:43:29.453 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T07:43:29.454 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout:Installing weak dependencies: 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-09-21T07:43:29.455 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-09-21T07:43:29.456 INFO:teuthology.orchestra.run.smithi088.stdout:Install 149 Packages 2024-09-21T07:43:29.456 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 8 Packages 2024-09-21T07:43:29.456 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:43:29.456 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 228 M 2024-09-21T07:43:29.456 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-09-21T07:43:29.791 INFO:teuthology.orchestra.run.smithi175.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 47 kB/s | 6.6 kB 00:00 2024-09-21T07:43:29.832 INFO:teuthology.orchestra.run.smithi088.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 36 kB/s | 6.6 kB 00:00 2024-09-21T07:43:30.024 INFO:teuthology.orchestra.run.smithi175.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.7 MB/s | 873 kB 00:00 2024-09-21T07:43:30.124 INFO:teuthology.orchestra.run.smithi175.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.5 MB/s | 148 kB 00:00 2024-09-21T07:43:30.261 INFO:teuthology.orchestra.run.smithi088.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.0 MB/s | 873 kB 00:00 2024-09-21T07:43:30.361 INFO:teuthology.orchestra.run.smithi088.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 7.7 MB/s | 5.5 MB 00:00 2024-09-21T07:43:30.387 INFO:teuthology.orchestra.run.smithi088.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.2 MB/s | 148 kB 00:00 2024-09-21T07:43:30.483 INFO:teuthology.orchestra.run.smithi175.stdout:(4/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 6.4 MB/s | 2.3 MB 00:00 2024-09-21T07:43:30.554 INFO:teuthology.orchestra.run.smithi088.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 12 MB/s | 2.3 MB 00:00 2024-09-21T07:43:30.658 INFO:teuthology.orchestra.run.smithi175.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 8.7 MB/s | 1.5 MB 00:00 2024-09-21T07:43:30.741 INFO:teuthology.orchestra.run.smithi088.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 4.3 MB/s | 1.5 MB 00:00 2024-09-21T07:43:30.767 INFO:teuthology.orchestra.run.smithi175.stdout:(6/157): ceph-base-19.3.0-5120.g18432218.el9.x8 4.9 MB/s | 5.5 MB 00:01 2024-09-21T07:43:30.846 INFO:teuthology.orchestra.run.smithi088.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 17 MB/s | 5.0 MB 00:00 2024-09-21T07:43:31.158 INFO:teuthology.orchestra.run.smithi175.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 9.9 MB/s | 5.0 MB 00:00 2024-09-21T07:43:31.488 INFO:teuthology.orchestra.run.smithi088.stdout:(8/157): ceph-radosgw-19.3.0-5120.g18432218.el9 16 MB/s | 10 MB 00:00 2024-09-21T07:43:31.538 INFO:teuthology.orchestra.run.smithi088.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 508 kB/s | 25 kB 00:00 2024-09-21T07:43:31.618 INFO:teuthology.orchestra.run.smithi175.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 10 MB/s | 20 MB 00:01 2024-09-21T07:43:31.709 INFO:teuthology.orchestra.run.smithi175.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 275 kB/s | 25 kB 00:00 2024-09-21T07:43:32.301 INFO:teuthology.orchestra.run.smithi175.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 9.1 MB/s | 10 MB 00:01 2024-09-21T07:43:32.393 INFO:teuthology.orchestra.run.smithi175.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 2.4 MB/s | 226 kB 00:00 2024-09-21T07:43:32.459 INFO:teuthology.orchestra.run.smithi175.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 406 kB/s | 27 kB 00:00 2024-09-21T07:43:32.518 INFO:teuthology.orchestra.run.smithi175.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 545 kB/s | 32 kB 00:00 2024-09-21T07:43:32.602 INFO:teuthology.orchestra.run.smithi175.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T07:43:32.613 INFO:teuthology.orchestra.run.smithi088.stdout:(10/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 9.5 MB/s | 18 MB 00:01 2024-09-21T07:43:32.652 INFO:teuthology.orchestra.run.smithi175.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 3.1 MB/s | 158 kB 00:00 2024-09-21T07:43:32.671 INFO:teuthology.orchestra.run.smithi088.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.8 MB/s | 226 kB 00:00 2024-09-21T07:43:32.719 INFO:teuthology.orchestra.run.smithi175.stdout:(16/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T07:43:32.802 INFO:teuthology.orchestra.run.smithi175.stdout:(17/157): libradosstriper1-19.3.0-5120.g1843221 6.0 MB/s | 507 kB 00:00 2024-09-21T07:43:32.922 INFO:teuthology.orchestra.run.smithi088.stdout:(12/157): ceph-common-19.3.0-5120.g18432218.el9 6.1 MB/s | 20 MB 00:03 2024-09-21T07:43:32.939 INFO:teuthology.orchestra.run.smithi088.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 101 kB/s | 27 kB 00:00 2024-09-21T07:43:32.964 INFO:teuthology.orchestra.run.smithi088.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 766 kB/s | 32 kB 00:00 2024-09-21T07:43:33.023 INFO:teuthology.orchestra.run.smithi088.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T07:43:33.056 INFO:teuthology.orchestra.run.smithi088.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 1.7 MB/s | 158 kB 00:00 2024-09-21T07:43:33.090 INFO:teuthology.orchestra.run.smithi088.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 1.9 MB/s | 127 kB 00:00 2024-09-21T07:43:33.240 INFO:teuthology.orchestra.run.smithi088.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 2.7 MB/s | 507 kB 00:00 2024-09-21T07:43:33.344 INFO:teuthology.orchestra.run.smithi175.stdout:(18/157): librgw2-19.3.0-5120.g18432218.el9.x86 9.7 MB/s | 5.3 MB 00:00 2024-09-21T07:43:33.357 INFO:teuthology.orchestra.run.smithi088.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 387 kB/s | 45 kB 00:00 2024-09-21T07:43:33.445 INFO:teuthology.orchestra.run.smithi175.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 453 kB/s | 45 kB 00:00 2024-09-21T07:43:33.512 INFO:teuthology.orchestra.run.smithi175.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 2.2 MB/s | 148 kB 00:00 2024-09-21T07:43:33.587 INFO:teuthology.orchestra.run.smithi175.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T07:43:33.749 INFO:teuthology.orchestra.run.smithi088.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 379 kB/s | 148 kB 00:00 2024-09-21T07:43:33.787 INFO:teuthology.orchestra.run.smithi175.stdout:(22/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 5.9 MB/s | 18 MB 00:03 2024-09-21T07:43:33.813 INFO:teuthology.orchestra.run.smithi175.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 1.4 MB/s | 321 kB 00:00 2024-09-21T07:43:33.824 INFO:teuthology.orchestra.run.smithi088.stdout:(21/157): python3-cephfs-19.3.0-5120.g18432218. 2.1 MB/s | 161 kB 00:00 2024-09-21T07:43:33.855 INFO:teuthology.orchestra.run.smithi175.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 4.5 MB/s | 307 kB 00:00 2024-09-21T07:43:33.880 INFO:teuthology.orchestra.run.smithi175.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.5 MB/s | 99 kB 00:00 2024-09-21T07:43:33.905 INFO:teuthology.orchestra.run.smithi175.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T07:43:33.964 INFO:teuthology.orchestra.run.smithi175.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T07:43:34.005 INFO:teuthology.orchestra.run.smithi175.stdout:(28/157): ceph-grafana-dashboards-19.3.0-5120.g 826 kB/s | 34 kB 00:00 2024-09-21T07:43:34.064 INFO:teuthology.orchestra.run.smithi175.stdout:(29/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T07:43:34.123 INFO:teuthology.orchestra.run.smithi175.stdout:(30/157): rbd-mirror-19.3.0-5120.g18432218.el9. 13 MB/s | 3.1 MB 00:00 2024-09-21T07:43:34.225 INFO:teuthology.orchestra.run.smithi088.stdout:(22/157): ceph-test-19.3.0-5120.g18432218.el9.x 15 MB/s | 40 MB 00:02 2024-09-21T07:43:34.300 INFO:teuthology.orchestra.run.smithi088.stdout:(23/157): librgw2-19.3.0-5120.g18432218.el9.x86 4.4 MB/s | 5.3 MB 00:01 2024-09-21T07:43:34.333 INFO:teuthology.orchestra.run.smithi088.stdout:(24/157): python3-rados-19.3.0-5120.g18432218.e 631 kB/s | 321 kB 00:00 2024-09-21T07:43:34.359 INFO:teuthology.orchestra.run.smithi088.stdout:(25/157): python3-rbd-19.3.0-5120.g18432218.el9 2.2 MB/s | 307 kB 00:00 2024-09-21T07:43:34.384 INFO:teuthology.orchestra.run.smithi088.stdout:(26/157): python3-rgw-19.3.0-5120.g18432218.el9 1.2 MB/s | 99 kB 00:00 2024-09-21T07:43:34.451 INFO:teuthology.orchestra.run.smithi088.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T07:43:34.515 INFO:teuthology.orchestra.run.smithi088.stdout:(28/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 478 kB/s | 86 kB 00:00 2024-09-21T07:43:34.543 INFO:teuthology.orchestra.run.smithi088.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 373 kB/s | 34 kB 00:00 2024-09-21T07:43:34.602 INFO:teuthology.orchestra.run.smithi088.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 1.7 MB/s | 154 kB 00:00 2024-09-21T07:43:34.669 INFO:teuthology.orchestra.run.smithi088.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 10 MB/s | 3.1 MB 00:00 2024-09-21T07:43:34.752 INFO:teuthology.orchestra.run.smithi088.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.4 MB/s | 292 kB 00:00 2024-09-21T07:43:34.764 INFO:teuthology.orchestra.run.smithi175.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 12 MB/s | 7.4 MB 00:00 2024-09-21T07:43:34.836 INFO:teuthology.orchestra.run.smithi088.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 592 kB/s | 49 kB 00:00 2024-09-21T07:43:34.857 INFO:teuthology.orchestra.run.smithi175.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T07:43:34.911 INFO:teuthology.orchestra.run.smithi088.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 220 kB/s | 16 kB 00:00 2024-09-21T07:43:34.940 INFO:teuthology.orchestra.run.smithi175.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 594 kB/s | 49 kB 00:00 2024-09-21T07:43:34.995 INFO:teuthology.orchestra.run.smithi088.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 3.3 MB/s | 285 kB 00:00 2024-09-21T07:43:35.015 INFO:teuthology.orchestra.run.smithi175.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 219 kB/s | 16 kB 00:00 2024-09-21T07:43:35.115 INFO:teuthology.orchestra.run.smithi175.stdout:(35/157): ceph-volume-19.3.0-5120.g18432218.el9 2.8 MB/s | 285 kB 00:00 2024-09-21T07:43:35.162 INFO:teuthology.orchestra.run.smithi088.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 4.6 MB/s | 784 kB 00:00 2024-09-21T07:43:35.254 INFO:teuthology.orchestra.run.smithi088.stdout:(37/157): cephfs-top-19.3.0-5120.g18432218.el9. 201 kB/s | 18 kB 00:00 2024-09-21T07:43:35.346 INFO:teuthology.orchestra.run.smithi088.stdout:(38/157): fuse-2.9.9-16.el9.x86_64.rpm 871 kB/s | 80 kB 00:00 2024-09-21T07:43:35.348 INFO:teuthology.orchestra.run.smithi175.stdout:(36/157): cephadm-19.3.0-5120.g18432218.el9.noa 3.3 MB/s | 784 kB 00:00 2024-09-21T07:43:35.529 INFO:teuthology.orchestra.run.smithi088.stdout:(39/157): ceph-mgr-dashboard-19.3.0-5120.g18432 14 MB/s | 14 MB 00:00 2024-09-21T07:43:35.563 INFO:teuthology.orchestra.run.smithi088.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 3.6 MB/s | 801 kB 00:00 2024-09-21T07:43:35.646 INFO:teuthology.orchestra.run.smithi088.stdout:(41/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 3.0 MB/s | 253 kB 00:00 2024-09-21T07:43:35.672 INFO:teuthology.orchestra.run.smithi088.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2024-09-21T07:43:35.741 INFO:teuthology.orchestra.run.smithi175.stdout:(37/157): ceph-test-19.3.0-5120.g18432218.el9.x 9.9 MB/s | 40 MB 00:04 2024-09-21T07:43:35.755 INFO:teuthology.orchestra.run.smithi088.stdout:(43/157): python3-cryptography-36.0.1-4.el9.x86 11 MB/s | 1.2 MB 00:00 2024-09-21T07:43:35.766 INFO:teuthology.orchestra.run.smithi175.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 44 kB/s | 18 kB 00:00 2024-09-21T07:43:35.781 INFO:teuthology.orchestra.run.smithi088.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-09-21T07:43:35.867 INFO:teuthology.orchestra.run.smithi175.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 640 kB/s | 80 kB 00:00 2024-09-21T07:43:35.872 INFO:teuthology.orchestra.run.smithi088.stdout:(45/157): ceph-mgr-diskprediction-local-19.3.0- 5.8 MB/s | 7.4 MB 00:01 2024-09-21T07:43:35.898 INFO:teuthology.orchestra.run.smithi088.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T07:43:35.926 INFO:teuthology.orchestra.run.smithi088.stdout:(47/157): python3-pycparser-2.20-6.el9.noarch.r 807 kB/s | 135 kB 00:00 2024-09-21T07:43:35.926 INFO:teuthology.orchestra.run.smithi175.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.9 MB/s | 801 kB 00:00 2024-09-21T07:43:35.967 INFO:teuthology.orchestra.run.smithi175.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.9 MB/s | 191 kB 00:00 2024-09-21T07:43:35.998 INFO:teuthology.orchestra.run.smithi088.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 6.7 MB/s | 689 kB 00:00 2024-09-21T07:43:36.041 INFO:teuthology.orchestra.run.smithi088.stdout:(49/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.3 MB/s | 215 kB 00:00 2024-09-21T07:43:36.066 INFO:teuthology.orchestra.run.smithi088.stdout:(50/157): boost-program-options-1.75.0-8.el9.x8 1.6 MB/s | 107 kB 00:00 2024-09-21T07:43:36.108 INFO:teuthology.orchestra.run.smithi088.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 5.0 MB/s | 943 kB 00:00 2024-09-21T07:43:36.125 INFO:teuthology.orchestra.run.smithi088.stdout:(52/157): elfutils-libelf-devel-0.191-4.el9.x86 259 kB/s | 22 kB 00:00 2024-09-21T07:43:36.150 INFO:teuthology.orchestra.run.smithi088.stdout:(53/157): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-09-21T07:43:36.151 INFO:teuthology.orchestra.run.smithi175.stdout:(42/157): ceph-mgr-dashboard-19.3.0-5120.g18432 6.8 MB/s | 14 MB 00:02 2024-09-21T07:43:36.175 INFO:teuthology.orchestra.run.smithi088.stdout:(54/157): flexiblas-3.0.4-8.el9.x86_64.rpm 487 kB/s | 33 kB 00:00 2024-09-21T07:43:36.176 INFO:teuthology.orchestra.run.smithi175.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2024-09-21T07:43:36.200 INFO:teuthology.orchestra.run.smithi088.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T07:43:36.243 INFO:teuthology.orchestra.run.smithi175.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-09-21T07:43:36.259 INFO:teuthology.orchestra.run.smithi088.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 22 MB/s | 3.0 MB 00:00 2024-09-21T07:43:36.268 INFO:teuthology.orchestra.run.smithi175.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-09-21T07:43:36.276 INFO:teuthology.orchestra.run.smithi088.stdout:(57/157): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-21T07:43:36.301 INFO:teuthology.orchestra.run.smithi088.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.7 MB/s | 171 kB 00:00 2024-09-21T07:43:36.327 INFO:teuthology.orchestra.run.smithi175.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-09-21T07:43:36.360 INFO:teuthology.orchestra.run.smithi088.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.6 MB/s | 160 kB 00:00 2024-09-21T07:43:36.369 INFO:teuthology.orchestra.run.smithi175.stdout:(47/157): python3-cryptography-36.0.1-4.el9.x86 3.1 MB/s | 1.2 MB 00:00 2024-09-21T07:43:36.377 INFO:teuthology.orchestra.run.smithi088.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 450 kB/s | 45 kB 00:00 2024-09-21T07:43:36.394 INFO:teuthology.orchestra.run.smithi175.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-21T07:43:36.419 INFO:teuthology.orchestra.run.smithi088.stdout:(61/157): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-09-21T07:43:36.452 INFO:teuthology.orchestra.run.smithi088.stdout:(62/157): libtool-2.4.6-46.el9.x86_64.rpm 6.1 MB/s | 581 kB 00:00 2024-09-21T07:43:36.470 INFO:teuthology.orchestra.run.smithi088.stdout:(63/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 265 kB/s | 24 kB 00:00 2024-09-21T07:43:36.494 INFO:teuthology.orchestra.run.smithi175.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 1.0 MB/s | 107 kB 00:00 2024-09-21T07:43:36.495 INFO:teuthology.orchestra.run.smithi088.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 627 kB/s | 47 kB 00:00 2024-09-21T07:43:36.528 INFO:teuthology.orchestra.run.smithi175.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 3.4 MB/s | 689 kB 00:00 2024-09-21T07:43:36.528 INFO:teuthology.orchestra.run.smithi088.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.8 MB/s | 292 kB 00:00 2024-09-21T07:43:36.553 INFO:teuthology.orchestra.run.smithi175.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 369 kB/s | 22 kB 00:00 2024-09-21T07:43:36.554 INFO:teuthology.orchestra.run.smithi088.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-09-21T07:43:36.579 INFO:teuthology.orchestra.run.smithi088.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 459 kB/s | 38 kB 00:00 2024-09-21T07:43:36.595 INFO:teuthology.orchestra.run.smithi175.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-09-21T07:43:36.612 INFO:teuthology.orchestra.run.smithi175.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 562 kB/s | 33 kB 00:00 2024-09-21T07:43:36.638 INFO:teuthology.orchestra.run.smithi175.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-21T07:43:36.654 INFO:teuthology.orchestra.run.smithi175.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 436 kB/s | 18 kB 00:00 2024-09-21T07:43:36.671 INFO:teuthology.orchestra.run.smithi175.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 336 kB/s | 11 kB 00:00 2024-09-21T07:43:36.704 INFO:teuthology.orchestra.run.smithi088.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 28 MB/s | 4.9 MB 00:00 2024-09-21T07:43:36.730 INFO:teuthology.orchestra.run.smithi175.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-21T07:43:36.755 INFO:teuthology.orchestra.run.smithi175.stdout:(58/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-09-21T07:43:36.771 INFO:teuthology.orchestra.run.smithi088.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 20 MB/s | 4.4 MB 00:00 2024-09-21T07:43:36.772 INFO:teuthology.orchestra.run.smithi175.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-09-21T07:43:36.813 INFO:teuthology.orchestra.run.smithi088.stdout:(70/157): postgresql-13.16-1.el9.x86_64.rpm 6.8 MB/s | 1.6 MB 00:00 2024-09-21T07:43:36.847 INFO:teuthology.orchestra.run.smithi088.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 6.3 MB/s | 910 kB 00:00 2024-09-21T07:43:36.880 INFO:teuthology.orchestra.run.smithi088.stdout:(72/157): postgresql-private-libs-13.16-1.el9.x 1.2 MB/s | 137 kB 00:00 2024-09-21T07:43:36.922 INFO:teuthology.orchestra.run.smithi175.stdout:(60/157): librdkafka-1.6.1-102.el9.x86_64.rpm 3.9 MB/s | 662 kB 00:00 2024-09-21T07:43:36.958 INFO:teuthology.orchestra.run.smithi088.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.1 MB/s | 1.0 MB 00:00 2024-09-21T07:43:36.964 INFO:teuthology.orchestra.run.smithi175.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 3.0 MB/s | 581 kB 00:00 2024-09-21T07:43:36.981 INFO:teuthology.orchestra.run.smithi088.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 873 kB/s | 87 kB 00:00 2024-09-21T07:43:36.981 INFO:teuthology.orchestra.run.smithi175.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 419 kB/s | 24 kB 00:00 2024-09-21T07:43:36.998 INFO:teuthology.orchestra.run.smithi175.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T07:43:37.065 INFO:teuthology.orchestra.run.smithi088.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 23 MB/s | 5.8 MB 00:00 2024-09-21T07:43:37.073 INFO:teuthology.orchestra.run.smithi175.stdout:(64/157): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-09-21T07:43:37.089 INFO:teuthology.orchestra.run.smithi088.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 563 kB/s | 61 kB 00:00 2024-09-21T07:43:37.099 INFO:teuthology.orchestra.run.smithi175.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T07:43:37.115 INFO:teuthology.orchestra.run.smithi175.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 925 kB/s | 38 kB 00:00 2024-09-21T07:43:37.123 INFO:teuthology.orchestra.run.smithi088.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 4.1 MB/s | 245 kB 00:00 2024-09-21T07:43:37.157 INFO:teuthology.orchestra.run.smithi088.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 3.6 MB/s | 248 kB 00:00 2024-09-21T07:43:37.182 INFO:teuthology.orchestra.run.smithi088.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 438 kB/s | 26 kB 00:00 2024-09-21T07:43:37.291 INFO:teuthology.orchestra.run.smithi088.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 18 MB/s | 6.0 MB 00:00 2024-09-21T07:43:37.316 INFO:teuthology.orchestra.run.smithi088.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 121 kB/s | 19 kB 00:00 2024-09-21T07:43:37.341 INFO:teuthology.orchestra.run.smithi088.stdout:(82/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-21T07:43:37.367 INFO:teuthology.orchestra.run.smithi088.stdout:(83/157): python3-markupsafe-1.1.1-12.el9.x86_6 462 kB/s | 35 kB 00:00 2024-09-21T07:43:37.409 INFO:teuthology.orchestra.run.smithi088.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-09-21T07:43:37.450 INFO:teuthology.orchestra.run.smithi088.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.6 MB/s | 222 kB 00:00 2024-09-21T07:43:37.476 INFO:teuthology.orchestra.run.smithi088.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.1 MB/s | 77 kB 00:00 2024-09-21T07:43:37.549 INFO:teuthology.orchestra.run.smithi175.stdout:(67/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 3.1 MB/s | 3.0 MB 00:00 2024-09-21T07:43:37.571 INFO:teuthology.orchestra.run.smithi088.stdout:(87/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-09-21T07:43:37.593 INFO:teuthology.orchestra.run.smithi088.stdout:(88/157): python3-prettytable-0.7.2-27.el9.noar 297 kB/s | 42 kB 00:00 2024-09-21T07:43:37.618 INFO:teuthology.orchestra.run.smithi088.stdout:(89/157): python3-protobuf-3.14.0-13.el9.noarch 1.9 MB/s | 269 kB 00:00 2024-09-21T07:43:37.643 INFO:teuthology.orchestra.run.smithi088.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.2 MB/s | 159 kB 00:00 2024-09-21T07:43:37.669 INFO:teuthology.orchestra.run.smithi088.stdout:(91/157): python3-pyasn1-modules-0.4.8-6.el9.no 3.6 MB/s | 279 kB 00:00 2024-09-21T07:43:37.686 INFO:teuthology.orchestra.run.smithi088.stdout:(92/157): python3-pytz-2021.1-5.el9.noarch.rpm 758 kB/s | 51 kB 00:00 2024-09-21T07:43:37.711 INFO:teuthology.orchestra.run.smithi088.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 797 kB/s | 54 kB 00:00 2024-09-21T07:43:37.737 INFO:teuthology.orchestra.run.smithi088.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 824 kB/s | 42 kB 00:00 2024-09-21T07:43:37.779 INFO:teuthology.orchestra.run.smithi088.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 661 kB/s | 44 kB 00:00 2024-09-21T07:43:37.804 INFO:teuthology.orchestra.run.smithi088.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-09-21T07:43:37.829 INFO:teuthology.orchestra.run.smithi088.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 718 kB/s | 36 kB 00:00 2024-09-21T07:43:37.882 INFO:teuthology.orchestra.run.smithi088.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 609 kB/s | 47 kB 00:00 2024-09-21T07:43:37.922 INFO:teuthology.orchestra.run.smithi088.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 3.2 MB/s | 304 kB 00:00 2024-09-21T07:43:37.938 INFO:teuthology.orchestra.run.smithi088.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.0 MB/s | 58 kB 00:00 2024-09-21T07:43:37.991 INFO:teuthology.orchestra.run.smithi175.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 3.6 MB/s | 1.6 MB 00:00 2024-09-21T07:43:38.122 INFO:teuthology.orchestra.run.smithi088.stdout:(101/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 36 MB/s | 16 MB 00:00 2024-09-21T07:43:38.139 INFO:teuthology.orchestra.run.smithi088.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 318 kB/s | 64 kB 00:00 2024-09-21T07:43:38.191 INFO:teuthology.orchestra.run.smithi175.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 4.1 MB/s | 4.4 MB 00:01 2024-09-21T07:43:38.192 INFO:teuthology.orchestra.run.smithi088.stdout:(103/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 354 kB/s | 96 kB 00:00 2024-09-21T07:43:38.214 INFO:teuthology.orchestra.run.smithi088.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 488 kB/s | 45 kB 00:00 2024-09-21T07:43:38.266 INFO:teuthology.orchestra.run.smithi175.stdout:(70/157): postgresql-private-libs-13.16-1.el9.x 1.8 MB/s | 137 kB 00:00 2024-09-21T07:43:38.282 INFO:teuthology.orchestra.run.smithi088.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-21T07:43:38.307 INFO:teuthology.orchestra.run.smithi088.stdout:(106/157): lua-devel-5.4.4-4.el9.x86_64.rpm 133 kB/s | 22 kB 00:00 2024-09-21T07:43:38.317 INFO:teuthology.orchestra.run.smithi175.stdout:(71/157): postgresql-contrib-13.16-1.el9.x86_64 2.7 MB/s | 910 kB 00:00 2024-09-21T07:43:38.349 INFO:teuthology.orchestra.run.smithi088.stdout:(107/157): gperftools-libs-2.9.1-3.el9.x86_64.r 4.5 MB/s | 308 kB 00:00 2024-09-21T07:43:38.374 INFO:teuthology.orchestra.run.smithi088.stdout:(108/157): grpc-data-1.46.7-10.el9.noarch.rpm 291 kB/s | 19 kB 00:00 2024-09-21T07:43:38.399 INFO:teuthology.orchestra.run.smithi088.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T07:43:38.425 INFO:teuthology.orchestra.run.smithi088.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T07:43:38.450 INFO:teuthology.orchestra.run.smithi088.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.7 MB/s | 67 kB 00:00 2024-09-21T07:43:38.475 INFO:teuthology.orchestra.run.smithi175.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 3.6 MB/s | 4.9 MB 00:01 2024-09-21T07:43:38.512 INFO:teuthology.orchestra.run.smithi088.stdout:(112/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.6 MB/s | 863 kB 00:00 2024-09-21T07:43:38.534 INFO:teuthology.orchestra.run.smithi175.stdout:(73/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T07:43:38.550 INFO:teuthology.orchestra.run.smithi088.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-09-21T07:43:38.601 INFO:teuthology.orchestra.run.smithi088.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2024-09-21T07:43:38.626 INFO:teuthology.orchestra.run.smithi088.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T07:43:38.651 INFO:teuthology.orchestra.run.smithi088.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T07:43:38.676 INFO:teuthology.orchestra.run.smithi088.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 565 kB/s | 14 kB 00:00 2024-09-21T07:43:38.701 INFO:teuthology.orchestra.run.smithi175.stdout:(74/157): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-09-21T07:43:38.718 INFO:teuthology.orchestra.run.smithi088.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.0 MB/s | 839 kB 00:00 2024-09-21T07:43:38.743 INFO:teuthology.orchestra.run.smithi088.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-21T07:43:38.751 INFO:teuthology.orchestra.run.smithi175.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T07:43:38.785 INFO:teuthology.orchestra.run.smithi088.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-09-21T07:43:38.836 INFO:teuthology.orchestra.run.smithi088.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-09-21T07:43:38.843 INFO:teuthology.orchestra.run.smithi175.stdout:(76/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-21T07:43:38.869 INFO:teuthology.orchestra.run.smithi088.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 2.6 MB/s | 220 kB 00:00 2024-09-21T07:43:38.903 INFO:teuthology.orchestra.run.smithi088.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T07:43:38.929 INFO:teuthology.orchestra.run.smithi088.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 425 kB/s | 11 kB 00:00 2024-09-21T07:43:38.954 INFO:teuthology.orchestra.run.smithi088.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 705 kB/s | 18 kB 00:00 2024-09-21T07:43:38.979 INFO:teuthology.orchestra.run.smithi088.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 944 kB/s | 23 kB 00:00 2024-09-21T07:43:38.985 INFO:teuthology.orchestra.run.smithi175.stdout:(77/157): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-09-21T07:43:39.004 INFO:teuthology.orchestra.run.smithi088.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 782 kB/s | 19 kB 00:00 2024-09-21T07:43:39.010 INFO:teuthology.orchestra.run.smithi175.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 1.5 MB/s | 248 kB 00:00 2024-09-21T07:43:39.035 INFO:teuthology.orchestra.run.smithi175.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 516 kB/s | 26 kB 00:00 2024-09-21T07:43:39.038 INFO:teuthology.orchestra.run.smithi088.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 593 kB/s | 20 kB 00:00 2024-09-21T07:43:39.060 INFO:teuthology.orchestra.run.smithi175.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 383 kB/s | 19 kB 00:00 2024-09-21T07:43:39.103 INFO:teuthology.orchestra.run.smithi175.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-09-21T07:43:39.117 INFO:teuthology.orchestra.run.smithi088.stdout:(129/157): python3-kubernetes-26.1.0-2.el9.noar 13 MB/s | 1.0 MB 00:00 2024-09-21T07:43:39.128 INFO:teuthology.orchestra.run.smithi175.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 520 kB/s | 35 kB 00:00 2024-09-21T07:43:39.147 INFO:teuthology.orchestra.run.smithi088.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.5 MB/s | 46 kB 00:00 2024-09-21T07:43:39.181 INFO:teuthology.orchestra.run.smithi088.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T07:43:39.205 INFO:teuthology.orchestra.run.smithi175.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.3 MB/s | 250 kB 00:00 2024-09-21T07:43:39.206 INFO:teuthology.orchestra.run.smithi088.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.2 MB/s | 58 kB 00:00 2024-09-21T07:43:39.248 INFO:teuthology.orchestra.run.smithi088.stdout:(133/157): python3-grpcio-1.46.7-10.el9.x86_64. 5.0 MB/s | 2.0 MB 00:00 2024-09-21T07:43:39.273 INFO:teuthology.orchestra.run.smithi088.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-21T07:43:39.278 INFO:teuthology.orchestra.run.smithi175.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 29 MB/s | 5.1 MB 00:00 2024-09-21T07:43:39.299 INFO:teuthology.orchestra.run.smithi088.stdout:(135/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 3.5 MB/s | 90 kB 00:00 2024-09-21T07:43:39.303 INFO:teuthology.orchestra.run.smithi175.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 2.2 MB/s | 222 kB 00:00 2024-09-21T07:43:39.315 INFO:teuthology.orchestra.run.smithi088.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.7 MB/s | 28 kB 00:00 2024-09-21T07:43:39.329 INFO:teuthology.orchestra.run.smithi175.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T07:43:39.341 INFO:teuthology.orchestra.run.smithi088.stdout:(137/157): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-21T07:43:39.346 INFO:teuthology.orchestra.run.smithi175.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 1.0 MB/s | 42 kB 00:00 2024-09-21T07:43:39.358 INFO:teuthology.orchestra.run.smithi088.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 733 kB/s | 31 kB 00:00 2024-09-21T07:43:39.371 INFO:teuthology.orchestra.run.smithi175.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 6.3 MB/s | 269 kB 00:00 2024-09-21T07:43:39.413 INFO:teuthology.orchestra.run.smithi175.stdout:(89/157): python3-pyasn1-modules-0.4.8-6.el9.no 6.5 MB/s | 279 kB 00:00 2024-09-21T07:43:39.430 INFO:teuthology.orchestra.run.smithi175.stdout:(90/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2024-09-21T07:43:39.441 INFO:teuthology.orchestra.run.smithi088.stdout:(139/157): libarrow-9.0.0-11.el9.x86_64.rpm 4.0 MB/s | 4.4 MB 00:01 2024-09-21T07:43:39.447 INFO:teuthology.orchestra.run.smithi175.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.5 MB/s | 51 kB 00:00 2024-09-21T07:43:39.458 INFO:teuthology.orchestra.run.smithi088.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 590 kB/s | 59 kB 00:00 2024-09-21T07:43:39.464 INFO:teuthology.orchestra.run.smithi175.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 1.6 MB/s | 54 kB 00:00 2024-09-21T07:43:39.475 INFO:teuthology.orchestra.run.smithi088.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 1.1 MB/s | 36 kB 00:00 2024-09-21T07:43:39.509 INFO:teuthology.orchestra.run.smithi088.stdout:(142/157): python3-typing-extensions-4.12.2-2.e 1.5 MB/s | 74 kB 00:00 2024-09-21T07:43:39.514 INFO:teuthology.orchestra.run.smithi175.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-09-21T07:43:39.534 INFO:teuthology.orchestra.run.smithi088.stdout:(143/157): python3-routes-2.5.1-5.el9.noarch.rp 978 kB/s | 188 kB 00:00 2024-09-21T07:43:39.547 INFO:teuthology.orchestra.run.smithi175.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 1.3 MB/s | 44 kB 00:00 2024-09-21T07:43:39.568 INFO:teuthology.orchestra.run.smithi088.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-09-21T07:43:39.584 INFO:teuthology.orchestra.run.smithi088.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 1.2 MB/s | 20 kB 00:00 2024-09-21T07:43:39.610 INFO:teuthology.orchestra.run.smithi088.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.7 MB/s | 230 kB 00:00 2024-09-21T07:43:39.644 INFO:teuthology.orchestra.run.smithi088.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.2 MB/s | 191 kB 00:00 2024-09-21T07:43:39.669 INFO:teuthology.orchestra.run.smithi088.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T07:43:39.739 INFO:teuthology.orchestra.run.smithi088.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T07:43:39.798 INFO:teuthology.orchestra.run.smithi175.stdout:(95/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 46 MB/s | 16 MB 00:00 2024-09-21T07:43:39.823 INFO:teuthology.orchestra.run.smithi175.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 802 kB/s | 221 kB 00:00 2024-09-21T07:43:39.828 INFO:teuthology.orchestra.run.smithi088.stdout:(150/157): NetworkManager-1.51.0-1.el9.x86_64.r 26 MB/s | 2.3 MB 00:00 2024-09-21T07:43:39.840 INFO:teuthology.orchestra.run.smithi175.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 864 kB/s | 36 kB 00:00 2024-09-21T07:43:39.857 INFO:teuthology.orchestra.run.smithi175.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 1.4 MB/s | 47 kB 00:00 2024-09-21T07:43:39.882 INFO:teuthology.orchestra.run.smithi175.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 7.1 MB/s | 304 kB 00:00 2024-09-21T07:43:39.899 INFO:teuthology.orchestra.run.smithi175.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.4 MB/s | 58 kB 00:00 2024-09-21T07:43:39.920 INFO:teuthology.orchestra.run.smithi088.stdout:(151/157): NetworkManager-libnm-1.51.0-1.el9.x8 20 MB/s | 1.8 MB 00:00 2024-09-21T07:43:39.924 INFO:teuthology.orchestra.run.smithi175.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 2.2 MB/s | 96 kB 00:00 2024-09-21T07:43:39.975 INFO:teuthology.orchestra.run.smithi175.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 847 kB/s | 64 kB 00:00 2024-09-21T07:43:39.992 INFO:teuthology.orchestra.run.smithi175.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 671 kB/s | 45 kB 00:00 2024-09-21T07:43:40.003 INFO:teuthology.orchestra.run.smithi088.stdout:(152/157): librados2-19.3.0-5120.g18432218.el9. 9.4 MB/s | 3.4 MB 00:00 2024-09-21T07:43:40.050 INFO:teuthology.orchestra.run.smithi088.stdout:(153/157): NetworkManager-team-1.51.0-1.el9.x86 316 kB/s | 41 kB 00:00 2024-09-21T07:43:40.079 INFO:teuthology.orchestra.run.smithi088.stdout:(154/157): NetworkManager-tui-1.51.0-1.el9.x86_ 3.2 MB/s | 249 kB 00:00 2024-09-21T07:43:40.138 INFO:teuthology.orchestra.run.smithi088.stdout:(155/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.8 MB/s | 3.2 MB 00:00 2024-09-21T07:43:40.188 INFO:teuthology.orchestra.run.smithi088.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.8 MB/s | 1.4 MB 00:00 2024-09-21T07:43:40.238 INFO:teuthology.orchestra.run.smithi088.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 15 MB/s | 2.4 MB 00:00 2024-09-21T07:43:40.243 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:43:40.243 INFO:teuthology.orchestra.run.smithi088.stdout:Total 21 MB/s | 228 MB 00:10 2024-09-21T07:43:40.276 INFO:teuthology.orchestra.run.smithi175.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 74 kB/s | 22 kB 00:00 2024-09-21T07:43:40.334 INFO:teuthology.orchestra.run.smithi175.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 9.3 MB/s | 548 kB 00:00 2024-09-21T07:43:40.368 INFO:teuthology.orchestra.run.smithi175.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.0 MB/s | 308 kB 00:00 2024-09-21T07:43:40.393 INFO:teuthology.orchestra.run.smithi175.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 791 kB/s | 19 kB 00:00 2024-09-21T07:43:40.510 INFO:teuthology.orchestra.run.smithi175.stdout:(108/157): libarrow-9.0.0-11.el9.x86_64.rpm 38 MB/s | 4.4 MB 00:00 2024-09-21T07:43:40.535 INFO:teuthology.orchestra.run.smithi175.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T07:43:40.560 INFO:teuthology.orchestra.run.smithi175.stdout:(110/157): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-21T07:43:40.587 INFO:teuthology.orchestra.run.smithi175.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T07:43:40.611 INFO:teuthology.orchestra.run.smithi175.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 6.0 MB/s | 151 kB 00:00 2024-09-21T07:43:40.687 INFO:teuthology.orchestra.run.smithi175.stdout:(113/157): postgresql-server-13.16-1.el9.x86_64 2.4 MB/s | 5.8 MB 00:02 2024-09-21T07:43:40.753 INFO:teuthology.orchestra.run.smithi175.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 5.8 MB/s | 839 kB 00:00 2024-09-21T07:43:40.779 INFO:teuthology.orchestra.run.smithi175.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T07:43:40.804 INFO:teuthology.orchestra.run.smithi175.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T07:43:40.830 INFO:teuthology.orchestra.run.smithi175.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 570 kB/s | 14 kB 00:00 2024-09-21T07:43:40.905 INFO:teuthology.orchestra.run.smithi175.stdout:(118/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-09-21T07:43:40.938 INFO:teuthology.orchestra.run.smithi175.stdout:(119/157): protobuf-compiler-3.14.0-13.el9.x86_ 913 kB/s | 863 kB 00:00 2024-09-21T07:43:40.963 INFO:teuthology.orchestra.run.smithi175.stdout:(120/157): python3-asyncssh-2.13.2-5.el9.noarch 1.9 MB/s | 548 kB 00:00 2024-09-21T07:43:41.005 INFO:teuthology.orchestra.run.smithi175.stdout:(121/157): python3-google-auth-2.34.0-1.el9.noa 6.8 MB/s | 220 kB 00:00 2024-09-21T07:43:41.031 INFO:teuthology.orchestra.run.smithi175.stdout:(122/157): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-21T07:43:41.064 INFO:teuthology.orchestra.run.smithi175.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 4.3 MB/s | 144 kB 00:00 2024-09-21T07:43:41.089 INFO:teuthology.orchestra.run.smithi175.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 429 kB/s | 11 kB 00:00 2024-09-21T07:43:41.115 INFO:teuthology.orchestra.run.smithi175.stdout:(125/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.7 MB/s | 303 kB 00:00 2024-09-21T07:43:41.140 INFO:teuthology.orchestra.run.smithi175.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 354 kB/s | 18 kB 00:00 2024-09-21T07:43:41.165 INFO:teuthology.orchestra.run.smithi175.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 465 kB/s | 23 kB 00:00 2024-09-21T07:43:41.190 INFO:teuthology.orchestra.run.smithi175.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T07:43:41.216 INFO:teuthology.orchestra.run.smithi175.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 390 kB/s | 20 kB 00:00 2024-09-21T07:43:41.241 INFO:teuthology.orchestra.run.smithi175.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T07:43:41.266 INFO:teuthology.orchestra.run.smithi175.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T07:43:41.291 INFO:teuthology.orchestra.run.smithi175.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T07:43:41.325 INFO:teuthology.orchestra.run.smithi175.stdout:(133/157): python3-pecan-1.4.2-3.el9.noarch.rpm 8.0 MB/s | 272 kB 00:00 2024-09-21T07:43:41.358 INFO:teuthology.orchestra.run.smithi175.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 507 kB/s | 16 kB 00:00 2024-09-21T07:43:41.392 INFO:teuthology.orchestra.run.smithi175.stdout:(135/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-21T07:43:41.417 INFO:teuthology.orchestra.run.smithi175.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.5 MB/s | 90 kB 00:00 2024-09-21T07:43:41.442 INFO:teuthology.orchestra.run.smithi175.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T07:43:41.467 INFO:teuthology.orchestra.run.smithi175.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 621 kB/s | 31 kB 00:00 2024-09-21T07:43:41.509 INFO:teuthology.orchestra.run.smithi175.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-09-21T07:43:41.535 INFO:teuthology.orchestra.run.smithi175.stdout:(140/157): python3-routes-2.5.1-5.el9.noarch.rp 2.8 MB/s | 188 kB 00:00 2024-09-21T07:43:41.560 INFO:teuthology.orchestra.run.smithi175.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-21T07:43:41.610 INFO:teuthology.orchestra.run.smithi175.stdout:(142/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.4 MB/s | 2.0 MB 00:00 2024-09-21T07:43:41.635 INFO:teuthology.orchestra.run.smithi175.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 742 kB/s | 74 kB 00:00 2024-09-21T07:43:41.660 INFO:teuthology.orchestra.run.smithi175.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T07:43:41.694 INFO:teuthology.orchestra.run.smithi175.stdout:(145/157): python3-zc-lockfile-2.0-10.el9.noarc 604 kB/s | 20 kB 00:00 2024-09-21T07:43:41.719 INFO:teuthology.orchestra.run.smithi175.stdout:(146/157): python3-webob-1.8.8-2.el9.noarch.rpm 1.4 MB/s | 230 kB 00:00 2024-09-21T07:43:41.745 INFO:teuthology.orchestra.run.smithi175.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T07:43:41.787 INFO:teuthology.orchestra.run.smithi175.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 2.8 MB/s | 427 kB 00:00 2024-09-21T07:43:41.854 INFO:teuthology.orchestra.run.smithi175.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-21T07:43:42.029 INFO:teuthology.orchestra.run.smithi175.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 12 MB/s | 3.4 MB 00:00 2024-09-21T07:43:42.146 INFO:teuthology.orchestra.run.smithi175.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 8.8 MB/s | 3.2 MB 00:00 2024-09-21T07:43:42.230 INFO:teuthology.orchestra.run.smithi175.stdout:(152/157): NetworkManager-libnm-1.51.0-1.el9.x8 9.0 MB/s | 1.8 MB 00:00 2024-09-21T07:43:42.310 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-09-21T07:43:42.330 INFO:teuthology.orchestra.run.smithi175.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 2.4 MB/s | 249 kB 00:00 2024-09-21T07:43:42.355 INFO:teuthology.orchestra.run.smithi175.stdout:(154/157): NetworkManager-team-1.51.0-1.el9.x86 196 kB/s | 41 kB 00:00 2024-09-21T07:43:42.489 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-09-21T07:43:42.490 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-09-21T07:43:42.680 INFO:teuthology.orchestra.run.smithi175.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 3.9 MB/s | 1.4 MB 00:00 2024-09-21T07:43:42.747 INFO:teuthology.orchestra.run.smithi175.stdout:(156/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.6 MB/s | 2.3 MB 00:00 2024-09-21T07:43:42.856 INFO:teuthology.orchestra.run.smithi175.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 4.8 MB/s | 2.4 MB 00:00 2024-09-21T07:43:42.862 INFO:teuthology.orchestra.run.smithi175.stdout:-------------------------------------------------------------------------------- 2024-09-21T07:43:42.862 INFO:teuthology.orchestra.run.smithi175.stdout:Total 17 MB/s | 228 MB 00:13 2024-09-21T07:43:44.849 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-09-21T07:43:44.849 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-09-21T07:43:44.932 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction check 2024-09-21T07:43:45.117 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction check succeeded. 2024-09-21T07:43:45.117 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction test 2024-09-21T07:43:47.470 INFO:teuthology.orchestra.run.smithi175.stdout:Transaction test succeeded. 2024-09-21T07:43:47.470 INFO:teuthology.orchestra.run.smithi175.stdout:Running transaction 2024-09-21T07:43:52.024 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-09-21T07:43:52.159 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T07:43:52.209 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T07:43:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T07:43:52.344 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T07:43:52.376 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T07:43:52.442 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T07:43:52.825 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T07:43:52.968 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:43:53.117 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:43:53.153 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:43:53.256 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:43:53.300 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:43:53.343 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T07:43:53.558 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T07:43:53.648 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T07:43:53.674 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:43:53.778 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:43:53.809 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:43:53.941 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:43:53.950 INFO:teuthology.orchestra.run.smithi175.stdout: Preparing : 1/1 2024-09-21T07:43:53.968 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:43:54.007 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:43:54.118 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T07:43:54.279 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:43:54.468 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T07:43:54.499 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:43:54.618 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T07:43:54.662 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T07:43:54.691 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T07:43:54.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T07:43:54.755 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T07:43:54.811 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:43:54.934 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:43:55.142 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T07:43:55.217 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:43:55.269 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:43:55.366 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T07:43:55.431 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T07:43:55.446 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:43:55.476 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T07:43:55.478 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:43:55.560 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T07:43:55.582 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:43:55.617 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:43:55.620 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T07:43:55.657 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T07:43:55.659 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T07:43:55.713 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T07:43:55.760 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T07:43:55.794 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T07:43:55.881 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T07:43:55.882 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T07:43:55.937 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T07:43:55.948 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T07:43:55.973 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T07:43:55.974 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:43:56.023 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T07:43:56.094 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:43:56.125 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:43:56.132 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T07:43:56.168 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T07:43:56.199 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T07:43:56.213 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T07:43:56.226 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:43:56.240 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T07:43:56.267 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T07:43:56.332 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T07:43:56.392 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T07:43:56.484 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T07:43:56.516 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:43:56.648 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T07:43:56.713 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T07:43:56.721 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T07:43:56.975 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T07:43:57.111 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:43:57.260 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:43:57.377 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T07:43:57.481 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T07:43:57.532 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T07:43:57.553 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T07:43:57.679 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T07:43:57.689 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T07:43:57.723 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T07:43:57.793 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T07:43:57.883 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T07:43:57.937 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T07:43:57.976 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T07:43:58.016 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T07:43:58.029 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T07:43:58.069 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T07:43:58.072 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T07:43:58.103 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T07:43:58.191 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T07:43:58.245 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T07:43:58.250 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T07:43:58.281 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T07:43:58.327 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T07:43:58.366 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T07:43:58.413 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T07:43:58.454 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T07:43:58.520 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T07:43:58.549 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T07:43:58.549 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T07:43:58.633 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T07:43:58.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T07:43:58.697 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T07:43:58.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T07:43:58.733 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T07:43:58.791 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T07:43:58.792 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T07:43:58.832 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T07:43:58.878 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T07:43:58.928 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T07:43:58.953 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T07:43:59.011 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T07:43:59.116 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T07:43:59.170 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T07:43:59.213 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T07:43:59.331 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T07:43:59.378 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T07:43:59.421 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T07:43:59.450 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T07:43:59.475 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T07:43:59.603 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T07:43:59.638 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T07:43:59.737 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T07:43:59.774 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T07:43:59.787 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T07:43:59.817 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T07:43:59.833 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T07:43:59.864 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T07:43:59.885 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T07:43:59.908 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T07:43:59.921 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T07:43:59.961 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T07:44:00.000 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T07:44:00.038 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T07:44:00.072 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T07:44:00.082 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T07:44:00.105 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T07:44:00.149 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T07:44:00.175 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T07:44:00.215 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T07:44:00.281 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T07:44:00.314 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T07:44:00.353 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T07:44:00.381 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T07:44:00.418 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T07:44:00.438 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T07:44:00.477 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T07:44:00.513 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T07:44:00.540 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T07:44:00.571 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T07:44:00.608 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T07:44:00.621 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T07:44:00.666 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T07:44:00.725 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T07:44:00.897 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T07:44:00.944 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T07:44:00.987 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T07:44:01.103 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T07:44:01.153 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T07:44:01.195 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T07:44:01.224 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T07:44:01.249 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T07:44:01.379 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T07:44:01.499 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T07:44:01.533 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T07:44:01.559 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T07:44:01.593 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T07:44:01.604 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T07:44:01.636 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T07:44:01.669 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T07:44:01.708 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T07:44:01.710 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T07:44:01.749 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T07:44:01.776 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T07:44:01.786 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T07:44:01.821 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T07:44:01.830 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T07:44:01.879 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T07:44:01.923 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T07:44:01.925 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T07:44:01.955 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T07:44:01.962 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T07:44:02.053 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T07:44:02.113 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T07:44:02.150 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T07:44:02.227 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T07:44:02.258 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T07:44:02.271 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T07:44:02.329 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T07:44:02.352 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T07:44:03.166 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T07:44:03.248 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T07:44:03.306 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T07:44:03.331 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T07:44:03.340 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T07:44:03.456 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T07:44:03.515 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T07:44:03.567 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T07:44:03.663 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T07:44:03.700 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T07:44:03.925 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T07:44:03.963 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T07:44:04.056 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T07:44:04.852 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T07:44:04.913 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T07:44:04.993 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T07:44:05.013 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T07:44:05.078 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T07:44:05.115 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T07:44:05.470 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T07:44:05.693 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:44:05.816 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:44:05.849 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T07:44:05.897 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T07:44:06.434 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T07:44:06.471 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:44:06.559 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T07:44:06.562 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:44:06.590 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T07:44:06.650 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T07:44:06.695 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T07:44:06.728 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T07:44:07.092 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T07:44:07.139 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:44:07.251 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T07:44:07.287 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T07:44:07.336 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T07:44:07.898 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T07:44:07.950 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:44:08.041 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T07:44:08.068 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T07:44:09.128 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:44:09.186 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:44:09.863 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:44:09.906 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:44:10.046 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:44:10.046 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T07:44:10.046 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T07:44:10.046 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:10.107 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:44:10.613 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:44:10.657 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:44:11.322 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T07:44:11.368 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:44:11.509 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T07:44:11.509 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T07:44:11.509 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T07:44:11.510 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:11.544 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-09-21T07:44:22.965 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:23.174 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:44:23.209 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:44:23.210 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:23.210 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T07:44:23.210 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /sys 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /proc 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /mnt 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /var/tmp 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /home 2024-09-21T07:44:24.259 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /root 2024-09-21T07:44:24.260 INFO:teuthology.orchestra.run.smithi175.stdout:skipping the directory /tmp 2024-09-21T07:44:24.260 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:24.472 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:44:24.507 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T07:44:24.507 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:24.507 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T07:44:24.507 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:24.962 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:44:25.019 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:44:25.019 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:25.019 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T07:44:25.019 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:44:25.019 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:44:25.019 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:25.270 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:44:25.321 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:44:25.321 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:25.321 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T07:44:25.321 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:44:25.322 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:44:25.322 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:25.851 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:44:25.902 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:44:25.903 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:25.903 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T07:44:25.903 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:44:25.903 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:44:25.903 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:26.038 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T07:44:26.098 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T07:44:26.171 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T07:44:26.251 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:44:26.307 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T07:44:26.307 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:26.307 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T07:44:26.307 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:44:26.308 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T07:44:26.308 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:26.405 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T07:44:26.480 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T07:44:26.551 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:44:26.570 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T07:44:26.601 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T07:44:26.601 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:26.601 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T07:44:26.602 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:44:26.602 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T07:44:26.602 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:26.687 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T07:44:26.863 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T07:44:26.916 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T07:44:26.955 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T07:44:26.981 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T07:44:27.118 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:44:27.169 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T07:44:27.169 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:27.169 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T07:44:27.169 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:44:27.169 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T07:44:27.170 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:27.293 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T07:44:27.353 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T07:44:27.418 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T07:44:27.535 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:44:27.580 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:44:27.631 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T07:44:27.646 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T07:44:27.713 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T07:44:27.714 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T07:44:27.753 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:44:27.800 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T07:44:27.910 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T07:44:28.086 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T07:44:28.129 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T07:44:28.170 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T07:44:28.195 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T07:44:28.666 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:44:28.694 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T07:44:28.728 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T07:44:28.810 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T07:44:28.842 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:44:31.431 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:44:31.500 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:44:32.510 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T07:44:32.581 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:44:32.584 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:44:32.617 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:44:32.769 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:44:32.945 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T07:44:32.978 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:44:33.030 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:44:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T07:44:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:44:33.030 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:44:33.030 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:33.086 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:44:33.113 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:44:33.221 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T07:44:33.288 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T07:44:33.657 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T07:44:33.689 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:44:33.841 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T07:44:34.017 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T07:44:34.049 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:44:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T07:44:34.101 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:34.101 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T07:44:34.101 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:44:34.101 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T07:44:34.101 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:34.158 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:44:34.184 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T07:44:34.292 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T07:44:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T07:44:34.356 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T07:44:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:44:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:44:34.423 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:34.423 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T07:44:34.423 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:44:34.423 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:44:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:44:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:44:34.523 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:34.523 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T07:44:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:44:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:44:34.632 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:34.632 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T07:44:34.632 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:44:34.633 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:44:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:44:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:44:35.075 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:35.075 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T07:44:35.075 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:44:35.075 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:44:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:35.401 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T07:44:35.435 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:44:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T07:44:35.487 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:35.487 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T07:44:35.487 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:44:35.487 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T07:44:35.487 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:35.536 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:44:35.587 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T07:44:35.587 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:35.587 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T07:44:35.587 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:35.645 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:44:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T07:44:35.696 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:35.696 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T07:44:35.696 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:44:35.696 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T07:44:35.696 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:36.046 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:44:36.096 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T07:44:36.097 INFO:teuthology.orchestra.run.smithi175.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T07:44:36.097 INFO:teuthology.orchestra.run.smithi175.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T07:44:36.097 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:44:36.097 INFO:teuthology.orchestra.run.smithi175.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T07:44:36.097 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:39.413 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T07:44:39.462 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T07:44:39.506 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T07:44:39.534 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T07:44:39.599 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T07:44:39.884 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T07:44:39.941 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T07:44:39.970 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T07:44:40.002 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T07:44:40.046 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T07:44:40.426 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T07:44:40.467 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T07:44:40.518 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T07:44:40.547 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T07:44:40.600 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T07:44:40.764 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:44:40.835 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:44:40.876 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:44:40.880 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T07:44:40.937 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T07:44:40.967 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T07:44:40.998 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T07:44:41.067 INFO:teuthology.orchestra.run.smithi175.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T07:44:41.801 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:44:41.848 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:44:41.890 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T07:44:42.153 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T07:44:42.267 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T07:44:42.322 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T07:44:42.463 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T07:44:42.503 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T07:44:42.528 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T07:44:42.573 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T07:44:42.573 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:44:42.705 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:44:42.706 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:44:42.815 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:44:42.847 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T07:44:42.862 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:44:42.862 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:44:43.100 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:44:43.100 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:44:43.177 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T07:44:43.209 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:44:43.272 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T07:44:43.299 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:44:43.322 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T07:44:43.441 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T07:44:43.474 INFO:teuthology.orchestra.run.smithi175.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T07:44:43.507 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T07:44:43.510 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:44:43.510 INFO:teuthology.orchestra.run.smithi088.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T07:44:43.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:43.544 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T07:44:43.544 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:44:43.656 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T07:44:43.656 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:44:43.769 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T07:44:43.793 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T07:44:43.808 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:44:43.808 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:44:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T07:44:44.045 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:44:44.155 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T07:44:44.236 INFO:teuthology.orchestra.run.smithi175.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:44:44.438 INFO:teuthology.orchestra.run.smithi175.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:44:44.438 INFO:teuthology.orchestra.run.smithi175.stdout:Couldn't write '0' to 'kernel/yama/ptrace_scope', ignoring: No such file or directory 2024-09-21T07:44:44.438 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:44:49.902 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T07:44:49.903 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T07:44:49.904 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T07:44:49.904 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T07:44:49.904 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T07:44:49.904 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T07:44:49.915 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T07:44:49.915 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T07:44:49.916 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T07:44:49.917 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T07:44:49.918 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T07:44:49.919 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T07:44:49.920 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T07:44:49.921 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T07:44:49.922 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T07:44:49.923 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T07:44:49.924 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T07:44:49.925 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T07:44:49.926 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T07:44:49.926 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T07:44:49.926 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T07:44:49.926 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T07:44:49.926 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T07:44:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T07:44:50.551 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T07:44:50.552 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T07:44:50.553 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T07:44:50.554 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T07:44:50.555 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T07:44:50.556 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T07:44:50.557 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T07:44:50.558 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T07:44:50.559 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T07:44:50.560 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T07:44:50.561 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T07:44:50.562 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T07:44:50.563 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T07:44:50.564 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T07:44:50.565 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.265 INFO:teuthology.orchestra.run.smithi088.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.266 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.267 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T07:44:52.268 INFO:teuthology.orchestra.run.smithi088.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T07:44:52.269 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T07:44:52.270 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T07:44:52.271 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T07:44:52.272 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T07:44:52.273 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T07:44:52.274 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T07:44:52.275 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.276 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:52.277 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout:Upgraded: 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.708 INFO:teuthology.orchestra.run.smithi175.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout:Installed: 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.709 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.710 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T07:44:52.711 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.712 INFO:teuthology.orchestra.run.smithi175.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T07:44:52.713 INFO:teuthology.orchestra.run.smithi175.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T07:44:52.714 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T07:44:52.715 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T07:44:52.716 INFO:teuthology.orchestra.run.smithi175.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T07:44:52.717 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T07:44:52.718 INFO:teuthology.orchestra.run.smithi175.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T07:44:52.719 INFO:teuthology.orchestra.run.smithi175.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T07:44:52.720 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:52.720 INFO:teuthology.orchestra.run.smithi175.stdout:Complete! 2024-09-21T07:44:52.936 DEBUG:teuthology.parallel:result is None 2024-09-21T07:44:53.288 DEBUG:teuthology.parallel:result is None 2024-09-21T07:44:53.288 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:44:53.430 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T07:44:53.475 INFO:teuthology.orchestra.run.smithi088.stdout:19.3.0-5120.g18432218.el9 2024-09-21T07:44:53.476 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T07:44:53.476 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T07:44:53.478 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T07:44:53.612 DEBUG:teuthology.orchestra.run.smithi175:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T07:44:53.663 INFO:teuthology.orchestra.run.smithi175.stdout:19.3.0-5120.g18432218.el9 2024-09-21T07:44:53.663 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T07:44:53.663 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T07:44:53.665 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T07:44:53.665 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:53.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T07:44:53.725 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:53.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T07:44:53.783 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T07:44:53.783 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:53.783 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T07:44:53.840 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T07:44:53.935 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:53.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T07:44:53.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T07:44:54.088 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T07:44:54.089 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:54.089 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T07:44:54.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T07:44:54.250 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:54.250 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T07:44:54.305 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T07:44:54.409 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T07:44:54.409 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:54.409 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T07:44:54.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T07:44:54.561 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:54.562 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T07:44:54.638 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T07:44:54.741 INFO:teuthology.run_tasks:Running task ceph... 2024-09-21T07:44:54.832 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-09-21T07:44:54.832 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /var/log/ceph 2024-09-21T07:44:54.836 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 777 /var/log/ceph 2024-09-21T07:44:54.890 INFO:tasks.ceph:Disabling ceph logrotate... 2024-09-21T07:44:54.891 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T07:44:54.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/logrotate.d/ceph 2024-09-21T07:44:54.949 INFO:tasks.ceph:Creating extra log directories... 2024-09-21T07:44:54.949 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T07:44:54.953 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-09-21T07:44:55.015 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2024-09-21T07:44:55.015 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2024-09-21T07:44:55.015 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2024-09-21T07:44:55.015 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:55.015 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T07:44:55.049 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:55.218 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:55.273 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:55.273 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2024-09-21T07:44:55.310 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:55.475 DEBUG:teuthology.orchestra.run.smithi175:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2024-09-21T07:44:55.535 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-09-21T07:44:55.536 INFO:tasks.ceph:config {'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-09-21T07:44:55.536 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617', 'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'description': 'fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}}', 'email': 'pdonnell@redhat.com', 'first_in_suite': False, 'job_id': '7913617', 'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi', 'no_nested_subset': False, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-pdonnell-testing-20240920.212106-debug'}, 'ceph': {'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'kclient': {'syntax': 'v1'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-pdonnell-testing-20240920.212106-debug', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, 'owner': 'scheduled_pdonnell@teuthology', 'priority': 90, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7864, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'sleep_before_teardown': 0, 'subset': '67/512', 'suite': 'fs', 'suite_branch': 'wip-pdonnell-testing-20240920.212106-debug', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi088.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGahd5EsZdlwoBlncZWtYPwOxt9Zp9HwVy5Zr22UEOUqw0K1bpHAUz3Sc8trLI3o0FcL+Z//iSxbU5RYU4NebNE=', 'smithi175.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBKF2iuxvDzKPXzzeAhlCfSA1UUjoozvjzTym75WLLAdbr7fnlm9hPxsD2Q5ufKcj3JVRLPRpp0m4vAyDMsAcut4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}}, {'ceph': {'create_rbd_pool': False, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'client': {'pid file': ''}, 'global': {'ms die on skipped message': False}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'fail_on_skip': False, 'modules': ['tasks.cephfs.test_volumes.TestMisc']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a', 'timestamp': '2024-09-20_23:48:22', 'tube': 'smithi', 'user': 'pdonnell', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2902'} 2024-09-21T07:44:55.536 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T07:44:55.540 DEBUG:teuthology.orchestra.run.smithi175:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-09-21T07:44:55.575 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T07:44:55.617 DEBUG:teuthology.orchestra.run.smithi175:> sudo install -d -m0777 -- /var/run/ceph 2024-09-21T07:44:55.672 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:55.672 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:44:55.707 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:44:55.707 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-09-21T07:44:55.782 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:44:55.782 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:55.782 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 632 Links: 1 2024-09-21T07:44:55.782 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:55.783 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:55.783 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 07:44:43.861295054 +0000 2024-09-21T07:44:55.783 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 07:42:37.976672428 +0000 2024-09-21T07:44:55.783 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 07:42:37.976672428 +0000 2024-09-21T07:44:55.783 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 07:42:37.976672428 +0000 2024-09-21T07:44:55.783 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:44:55.876 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:44:55.876 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:44:55.876 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000287215 s, 1.8 MB/s 2024-09-21T07:44:55.878 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:44:55.914 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-09-21T07:44:55.991 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:44:55.991 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:55.991 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 650 Links: 1 2024-09-21T07:44:55.991 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:55.991 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:55.991 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 07:44:43.862295035 +0000 2024-09-21T07:44:55.992 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 07:42:38.444663352 +0000 2024-09-21T07:44:55.992 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 07:42:38.444663352 +0000 2024-09-21T07:44:55.992 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 07:42:38.444663352 +0000 2024-09-21T07:44:55.992 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:44:56.083 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:44:56.083 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:44:56.083 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000321504 s, 1.6 MB/s 2024-09-21T07:44:56.086 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:44:56.122 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-09-21T07:44:56.196 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:44:56.196 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 671 Links: 1 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 07:44:43.862295035 +0000 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 07:42:38.913654256 +0000 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 07:42:38.913654256 +0000 2024-09-21T07:44:56.197 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 07:42:38.913654256 +0000 2024-09-21T07:44:56.197 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:44:56.289 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:44:56.289 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:44:56.289 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000315158 s, 1.6 MB/s 2024-09-21T07:44:56.292 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:44:56.328 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-09-21T07:44:56.402 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:44:56.402 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 6h/6d Inode: 689 Links: 1 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-09-21 07:44:43.863295016 +0000 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-09-21 07:42:39.368645440 +0000 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-09-21 07:42:39.368645440 +0000 2024-09-21T07:44:56.403 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-09-21 07:42:39.368645440 +0000 2024-09-21T07:44:56.403 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:44:56.496 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:44:56.496 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:44:56.496 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000286432 s, 1.8 MB/s 2024-09-21T07:44:56.499 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:44:56.535 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:44:56.535 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:56.535 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T07:44:56.572 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T07:44:56.572 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_1 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 637 Links: 1 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 07:44:44.771319151 +0000 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 07:42:37.944298203 +0000 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 07:42:37.944298203 +0000 2024-09-21T07:44:56.650 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 07:42:37.944298203 +0000 2024-09-21T07:44:56.651 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T07:44:56.744 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T07:44:56.744 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T07:44:56.744 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000277592 s, 1.8 MB/s 2024-09-21T07:44:56.748 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T07:44:56.785 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_2 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 659 Links: 1 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 07:44:44.772319136 +0000 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 07:42:38.407291143 +0000 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 07:42:38.407291143 +0000 2024-09-21T07:44:56.862 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 07:42:38.407291143 +0000 2024-09-21T07:44:56.863 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T07:44:56.955 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T07:44:56.956 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T07:44:56.956 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000322667 s, 1.6 MB/s 2024-09-21T07:44:56.958 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T07:44:56.994 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_3 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 677 Links: 1 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 07:44:44.772319136 +0000 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 07:42:38.899283641 +0000 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 07:42:38.899283641 +0000 2024-09-21T07:44:57.071 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 07:42:38.899283641 +0000 2024-09-21T07:44:57.072 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T07:44:57.163 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T07:44:57.164 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T07:44:57.164 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000277953 s, 1.8 MB/s 2024-09-21T07:44:57.166 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T07:44:57.202 DEBUG:teuthology.orchestra.run.smithi175:> stat /dev/vg_nvme/lv_4 2024-09-21T07:44:57.277 INFO:teuthology.orchestra.run.smithi175.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T07:44:57.277 INFO:teuthology.orchestra.run.smithi175.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T07:44:57.277 INFO:teuthology.orchestra.run.smithi175.stdout:Device: 6h/6d Inode: 695 Links: 1 2024-09-21T07:44:57.277 INFO:teuthology.orchestra.run.smithi175.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T07:44:57.277 INFO:teuthology.orchestra.run.smithi175.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T07:44:57.278 INFO:teuthology.orchestra.run.smithi175.stdout:Access: 2024-09-21 07:44:44.773319120 +0000 2024-09-21T07:44:57.278 INFO:teuthology.orchestra.run.smithi175.stdout:Modify: 2024-09-21 07:42:39.362276581 +0000 2024-09-21T07:44:57.278 INFO:teuthology.orchestra.run.smithi175.stdout:Change: 2024-09-21 07:42:39.362276581 +0000 2024-09-21T07:44:57.278 INFO:teuthology.orchestra.run.smithi175.stdout: Birth: 2024-09-21 07:42:39.362276581 +0000 2024-09-21T07:44:57.278 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T07:44:57.370 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records in 2024-09-21T07:44:57.370 INFO:teuthology.orchestra.run.smithi175.stderr:1+0 records out 2024-09-21T07:44:57.370 INFO:teuthology.orchestra.run.smithi175.stderr:512 bytes copied, 0.000315105 s, 1.6 MB/s 2024-09-21T07:44:57.373 DEBUG:teuthology.orchestra.run.smithi175:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T07:44:57.408 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:44:57.409 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T07:44:57.409 INFO:tasks.ceph:Generating config... 2024-09-21T07:44:57.410 INFO:tasks.ceph:[client] pid file = 2024-09-21T07:44:57.410 INFO:tasks.ceph:[global] ms die on skipped message = False 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mgr] debug client = 20 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mgr] debug finisher = 20 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mgr] debug ms = 1 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mon] debug mon = 20 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mon] debug ms = 1 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mon] debug paxos = 20 2024-09-21T07:44:57.410 INFO:tasks.ceph:[mon] mon down mkfs grace = 300 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] bdev async discard = True 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] debug ms = 1 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] debug osd = 20 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-09-21T07:44:57.411 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-09-21T07:44:57.412 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-09-21T07:44:57.412 INFO:tasks.ceph:Setting up mon.a... 2024-09-21T07:44:57.412 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-09-21T07:44:57.573 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.keyring 2024-09-21T07:44:57.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-09-21T07:44:57.737 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:44:57.792 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.88'), ('mon.b', '172.21.15.175'), ('mon.c', '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]')] 2024-09-21T07:44:57.792 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.88,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon down mkfs grace': 300}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'pid file': ''}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}} 2024-09-21T07:44:57.793 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:57.793 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T07:44:57.827 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.88 --add b 172.21.15.175 --addv c '[v2:172.21.15.175:3301,v1:172.21.15.175:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: generated fsid c1afec4d-fdf7-414b-83b4-ffa85a98fc94 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:setting min_mon_release = quincy 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:epoch 0 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:fsid c1afec4d-fdf7-414b-83b4-ffa85a98fc94 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:last_changed 2024-09-21T07:44:58.006327+0000 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:created 2024-09-21T07:44:58.006327+0000 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:min_mon_release 17 (quincy) 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:election_strategy: 1 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:0: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.a 2024-09-21T07:44:58.007 INFO:teuthology.orchestra.run.smithi088.stdout:1: [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] mon.b 2024-09-21T07:44:58.008 INFO:teuthology.orchestra.run.smithi088.stdout:2: [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] mon.c 2024-09-21T07:44:58.008 INFO:teuthology.orchestra.run.smithi088.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-09-21T07:44:58.015 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-09-21T07:44:58.050 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c1afec4d-fdf7-414b-83b4-ffa85a98fc94... 2024-09-21T07:44:58.052 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:44:58.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-09-21T07:44:58.187 INFO:teuthology.orchestra.run.smithi088.stdout:[global] 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: chdir = "" 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: auth supported = cephx 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: filestore xattr use omap = true 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: mon clock drift allowed = 1.000 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: osd crush chooseleaf type = 0 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: auth debug = true 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on old message = true 2024-09-21T07:44:58.188 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on bug = true 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: mon pg warn max object skew = 0 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default size = 2 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow primary affinity = true 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd allow pg remap = true 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:44:58.189 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on no sortbitwise = false 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: mon warn on too few osds = false 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: mon_allow_pool_size_one = true 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:44:58.190 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: osd default data pool replay window = 5 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: mon allow pool delete = true 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: mon cluster log file level = debug 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: debug asserts on shutdown = true 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: mon health detail to clog = false 2024-09-21T07:44:58.191 INFO:teuthology.orchestra.run.smithi088.stdout: mon host = "172.21.15.88,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: ms die on skipped message = False 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: fsid = c1afec4d-fdf7-414b-83b4-ffa85a98fc94 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout:[osd] 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: osd journal size = 100 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub load threshold = 5.0 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: osd scrub max interval = 600 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:44:58.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: osd recover clone overlap = true 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: osd recovery max chunk = 1048576 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug shutdown = true 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug op order = true 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug verify stray on activate = true 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug trim objects = true 2024-09-21T07:44:58.193 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: osd open classes on start = true 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug pg log writeout = true 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: osd map max advance = 10 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.194 INFO:teuthology.orchestra.run.smithi088.stdout: journal zero on create = true 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: filestore apply finisher threads = 3 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: bdev debug aio = true 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: osd debug misdirected ops = true 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: bdev async discard = True 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: bdev enable discard = True 2024-09-21T07:44:58.195 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore allocator = bitmap 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore block size = 96636764160 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: bluestore fsck on mount = True 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluefs = 1/20 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: debug bluestore = 1/20 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: debug osd = 20 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: debug rocksdb = 4/10 2024-09-21T07:44:58.196 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd full ratio = 0.9 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: osd objectstore = bluestore 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr] 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: debug mgr = 20 2024-09-21T07:44:58.197 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: mgr/telemetry/nag = false 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: debug client = 20 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: debug finisher = 20 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout:[mon] 2024-09-21T07:44:58.198 INFO:teuthology.orchestra.run.smithi088.stdout: debug ms = 1 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: debug mon = 20 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: debug paxos = 20 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: debug auth = 20 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: mon data avail warn = 5 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: mon mgr mkfs grace = 240 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd reporter subtree level = osd 2024-09-21T07:44:58.199 INFO:teuthology.orchestra.run.smithi088.stdout: mon osd prime pg temp = true 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:44:58.200 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: # 1m isn't quite enough 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: mon down mkfs grace = 300 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout:[client] 2024-09-21T07:44:58.201 INFO:teuthology.orchestra.run.smithi088.stdout: rgw cache enabled = true 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable ops log = true 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout: rgw enable usage log = true 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout: pid file = "" 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.a] 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.b] 2024-09-21T07:44:58.202 INFO:teuthology.orchestra.run.smithi088.stdout:[mon.c] 2024-09-21T07:44:58.203 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-09-21T07:44:58.203 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-09-21T07:44:58.203 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-09-21T07:44:58.203 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-09-21T07:44:58.203 INFO:teuthology.orchestra.run.smithi175.stdout:[global] 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: chdir = "" 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: auth supported = cephx 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: filestore xattr use omap = true 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: mon clock drift allowed = 1.000 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: osd crush chooseleaf type = 0 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: auth debug = true 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on old message = true 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on bug = true 2024-09-21T07:44:58.204 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon max pg per osd = 10000 # >= luminous 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon pg warn max object skew = 0 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: # disable pg_autoscaler by default for new pools 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: osd_pool_default_pg_autoscale_mode = off 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default size = 2 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow primary affinity = true 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd allow pg remap = true 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on legacy crush tunables = false 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on crush straw calc version zero = false 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on no sortbitwise = false 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on osd down out interval zero = false 2024-09-21T07:44:58.205 INFO:teuthology.orchestra.run.smithi175.stdout: mon warn on too few osds = false 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_pool_no_redundancy = false 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: mon_allow_pool_size_one = true 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: osd default data pool replay window = 5 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: mon allow pool delete = true 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: mon cluster log file level = debug 2024-09-21T07:44:58.206 INFO:teuthology.orchestra.run.smithi175.stdout: debug asserts on shutdown = true 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: mon health detail to clog = false 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: mon host = "172.21.15.88,172.21.15.175,[v2:172.21.15.175:3301,v1:172.21.15.175:6790]" 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: ms die on skipped message = False 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: fsid = c1afec4d-fdf7-414b-83b4-ffa85a98fc94 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout:[osd] 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: osd journal size = 100 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub load threshold = 5.0 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: osd scrub max interval = 600 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: osd mclock profile = high_recovery_ops 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: osd recover clone overlap = true 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: osd recovery max chunk = 1048576 2024-09-21T07:44:58.207 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug shutdown = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug op order = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug verify stray on activate = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug trim objects = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd open classes on start = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug pg log writeout = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd deep scrub update digest min age = 30 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: osd map max advance = 10 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: journal zero on create = true 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.208 INFO:teuthology.orchestra.run.smithi175.stdout: filestore ondisk finisher threads = 3 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: filestore apply finisher threads = 3 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: bdev debug aio = true 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: osd debug misdirected ops = true 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: bdev async discard = True 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: bdev enable discard = True 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore allocator = bitmap 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore block size = 96636764160 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: bluestore fsck on mount = True 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluefs = 1/20 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: debug bluestore = 1/20 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: debug osd = 20 2024-09-21T07:44:58.209 INFO:teuthology.orchestra.run.smithi175.stdout: debug rocksdb = 4/10 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd backfillfull_ratio = 0.85 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd full ratio = 0.9 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd nearfull ratio = 0.8 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: osd failsafe full ratio = 0.95 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: osd objectstore = bluestore 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr] 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: debug mgr = 20 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:58.210 INFO:teuthology.orchestra.run.smithi175.stdout: mgr/telemetry/nag = false 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: debug client = 20 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: debug finisher = 20 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout:[mon] 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: debug ms = 1 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: debug mon = 20 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: debug paxos = 20 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: debug auth = 20 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: mon data avail warn = 5 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: mon mgr mkfs grace = 240 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min pgs per osd = 4 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd reporter subtree level = osd 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: mon osd prime pg temp = true 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: mon reweight min bytes per osd = 10 2024-09-21T07:44:58.211 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: auth mon ticket ttl = 660 # 11m 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: auth service ticket ttl = 240 # 4m 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: # don't complain about insecure global_id in the test suite 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: # 1m isn't quite enough 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: mon_down_mkfs_grace = 2m 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: mon_warn_on_filestore_osds = false 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: mon down mkfs grace = 300 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout: 2024-09-21T07:44:58.212 INFO:teuthology.orchestra.run.smithi175.stdout:[client] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout: rgw cache enabled = true 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable ops log = true 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout: rgw enable usage log = true 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout: pid file = "" 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.a] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.b] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mon.c] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-09-21T07:44:58.213 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-09-21T07:44:58.220 INFO:tasks.ceph:Creating admin key on mon.a... 2024-09-21T07:44:58.220 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-09-21T07:44:58.384 INFO:tasks.ceph:Copying monmap to all nodes... 2024-09-21T07:44:58.384 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:58.384 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-09-21T07:44:58.419 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:58.419 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-09-21T07:44:58.494 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:44:58.494 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:58.494 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T07:44:58.494 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:44:58.610 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:44:58.611 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:44:58.685 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:44:58.686 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:58.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd of=/etc/ceph/ceph.keyring 2024-09-21T07:44:58.686 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-09-21T07:44:58.765 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:44:58.765 DEBUG:teuthology.orchestra.run.smithi175:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:44:58.841 INFO:tasks.ceph:Setting up mon nodes... 2024-09-21T07:44:58.841 INFO:tasks.ceph:Setting up mgr nodes... 2024-09-21T07:44:58.841 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T07:44:59.005 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-09-21T07:44:59.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T07:44:59.198 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-09-21T07:44:59.211 INFO:tasks.ceph:Setting up mds nodes... 2024-09-21T07:44:59.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T07:44:59.379 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2024-09-21T07:44:59.390 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2024-09-21T07:44:59.445 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T07:44:59.609 INFO:teuthology.orchestra.run.smithi088.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2024-09-21T07:44:59.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2024-09-21T07:44:59.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T07:44:59.840 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2024-09-21T07:44:59.852 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2024-09-21T07:44:59.908 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T07:45:00.072 INFO:teuthology.orchestra.run.smithi175.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2024-09-21T07:45:00.084 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2024-09-21T07:45:00.140 INFO:tasks.ceph_client:Setting up client nodes... 2024-09-21T07:45:00.140 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T07:45:00.285 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-09-21T07:45:00.318 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2024-09-21T07:45:00.459 INFO:teuthology.orchestra.run.smithi088.stdout:creating /etc/ceph/ceph.client.2.keyring 2024-09-21T07:45:00.492 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-21T07:45:00.635 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-09-21T07:45:00.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2024-09-21T07:45:00.810 INFO:teuthology.orchestra.run.smithi175.stdout:creating /etc/ceph/ceph.client.3.keyring 2024-09-21T07:45:00.843 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-09-21T07:45:00.843 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi088.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi175.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-09-21T07:45:00.843 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-09-21T07:45:00.900 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:00.900 INFO:tasks.ceph:role: osd.0 2024-09-21T07:45:00.900 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:45:00.900 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T07:45:00.962 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:00.962 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:00.963 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:01.368 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T07:45:01.373 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T07:45:01.373 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-09-21T07:45:01.512 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-09-21T07:45:01.577 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-09-21T07:45:01.634 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:01.634 INFO:tasks.ceph:role: osd.1 2024-09-21T07:45:01.635 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:45:01.635 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:01.698 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:02.056 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T07:45:02.059 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T07:45:02.060 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-09-21T07:45:02.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-09-21T07:45:02.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-09-21T07:45:02.261 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:02.261 INFO:tasks.ceph:role: osd.2 2024-09-21T07:45:02.261 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:45:02.261 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:02.324 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:02.707 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T07:45:02.710 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T07:45:02.710 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-09-21T07:45:02.810 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-09-21T07:45:02.874 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-09-21T07:45:02.928 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:02.928 INFO:tasks.ceph:role: osd.3 2024-09-21T07:45:02.928 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:45:02.929 DEBUG:teuthology.orchestra.run.smithi088:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:02.991 INFO:teuthology.orchestra.run.smithi088.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:02.992 INFO:teuthology.orchestra.run.smithi088.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:02.992 INFO:teuthology.orchestra.run.smithi088.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:02.992 INFO:teuthology.orchestra.run.smithi088.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:03.410 INFO:teuthology.orchestra.run.smithi088.stdout:Discarding blocks...Done. 2024-09-21T07:45:03.414 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi088.front.sepia.ceph.com -o noatime 2024-09-21T07:45:03.414 DEBUG:teuthology.orchestra.run.smithi088:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-09-21T07:45:03.560 DEBUG:teuthology.orchestra.run.smithi088:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-09-21T07:45:03.622 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:03.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:03.784+0000 7f6da180b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-09-21T07:45:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:03.789+0000 7f6da180b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-09-21T07:45:03.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:03.789+0000 7f6da180b640 -1 bdev(0x564c8afab900 /var/lib/ceph/osd/ceph-0/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:03.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:03.790+0000 7f6da180b640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-09-21T07:45:06.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-09-21T07:45:06.907 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:07.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:07.061+0000 7f677ef4b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-09-21T07:45:07.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:07.061+0000 7f677ef4b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-09-21T07:45:07.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:07.061+0000 7f677ef4b640 -1 bdev(0x55b06825b900 /var/lib/ceph/osd/ceph-1/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:07.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:07.062+0000 7f677ef4b640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-09-21T07:45:10.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-09-21T07:45:10.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:10.305+0000 7f6ef5e1b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-09-21T07:45:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:10.305+0000 7f6ef5e1b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-09-21T07:45:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:10.305+0000 7f6ef5e1b640 -1 bdev(0x561604d07900 /var/lib/ceph/osd/ceph-2/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:10.306+0000 7f6ef5e1b640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-09-21T07:45:13.617 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-09-21T07:45:13.673 DEBUG:teuthology.orchestra.run.smithi088:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:13.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:13.825+0000 7f0f9d3cc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-09-21T07:45:13.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:13.825+0000 7f0f9d3cc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-09-21T07:45:13.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:13.825+0000 7f0f9d3cc640 -1 bdev(0x559ae73d7900 /var/lib/ceph/osd/ceph-3/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:13.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:13.826+0000 7f0f9d3cc640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-09-21T07:45:16.850 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-09-21T07:45:16.905 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-09-21T07:45:16.968 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:16.968 INFO:tasks.ceph:role: osd.4 2024-09-21T07:45:16.968 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:45:16.968 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:17.033 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:17.439 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T07:45:17.443 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T07:45:17.444 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-09-21T07:45:17.582 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-09-21T07:45:17.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-09-21T07:45:17.703 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:17.703 INFO:tasks.ceph:role: osd.5 2024-09-21T07:45:17.703 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:45:17.703 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-09-21T07:45:17.766 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:17.766 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:17.766 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:17.766 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:17.766 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:17.767 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:17.767 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:17.767 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:17.767 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:17.767 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:18.123 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T07:45:18.128 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T07:45:18.128 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-09-21T07:45:18.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-09-21T07:45:18.276 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-09-21T07:45:18.331 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:18.331 INFO:tasks.ceph:role: osd.6 2024-09-21T07:45:18.331 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:45:18.332 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-09-21T07:45:18.394 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:18.394 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:18.394 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:18.395 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:18.775 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T07:45:18.779 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T07:45:18.779 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-09-21T07:45:18.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-09-21T07:45:18.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-09-21T07:45:18.996 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-09-21T07:45:18.996 INFO:tasks.ceph:role: osd.7 2024-09-21T07:45:18.996 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:45:18.996 DEBUG:teuthology.orchestra.run.smithi175:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-09-21T07:45:19.059 INFO:teuthology.orchestra.run.smithi175.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 attr=2, projid32bit=1 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout: = sunit=0 swidth=0 blks 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-09-21T07:45:19.060 INFO:teuthology.orchestra.run.smithi175.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-09-21T07:45:19.061 INFO:teuthology.orchestra.run.smithi175.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-09-21T07:45:19.469 INFO:teuthology.orchestra.run.smithi175.stdout:Discarding blocks...Done. 2024-09-21T07:45:19.474 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi175.front.sepia.ceph.com -o noatime 2024-09-21T07:45:19.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-09-21T07:45:19.606 DEBUG:teuthology.orchestra.run.smithi175:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-09-21T07:45:19.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:19.838 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:19.836+0000 7f9770530640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-09-21T07:45:19.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:19.837+0000 7f9770530640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-09-21T07:45:19.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:19.837+0000 7f9770530640 -1 bdev(0x555b13f51900 /var/lib/ceph/osd/ceph-4/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:19.839 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:19.837+0000 7f9770530640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-09-21T07:45:22.882 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-09-21T07:45:22.941 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:23.094+0000 7f0f2cc19640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-09-21T07:45:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:23.094+0000 7f0f2cc19640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-09-21T07:45:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:23.094+0000 7f0f2cc19640 -1 bdev(0x55ab95a83900 /var/lib/ceph/osd/ceph-5/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:23.096 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:23.095+0000 7f0f2cc19640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-09-21T07:45:25.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:25.544 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:26.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-09-21T07:45:26.196 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:26.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:26.350+0000 7fed5042c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-09-21T07:45:26.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:26.350+0000 7fed5042c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-09-21T07:45:26.352 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:26.350+0000 7fed5042c640 -1 bdev(0x561550bbb900 /var/lib/ceph/osd/ceph-6/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:26.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:26.350+0000 7fed5042c640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-09-21T07:45:29.395 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-09-21T07:45:29.452 DEBUG:teuthology.orchestra.run.smithi175:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:29.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:29.604+0000 7fab961a8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-09-21T07:45:29.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:29.604+0000 7fab961a8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-09-21T07:45:29.606 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:29.604+0000 7fab961a8640 -1 bdev(0x562fdc661900 /var/lib/ceph/osd/ceph-7/block) open stat got: (1) Operation not permitted 2024-09-21T07:45:29.607 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:29.604+0000 7fab961a8640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-09-21T07:45:32.861 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-09-21T07:45:32.917 INFO:tasks.ceph:Reading keys from all nodes... 2024-09-21T07:45:32.918 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:32.918 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-09-21T07:45:32.980 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:32.980 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2024-09-21T07:45:33.078 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:33.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2024-09-21T07:45:33.175 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:33.176 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-09-21T07:45:33.271 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:33.271 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-09-21T07:45:33.365 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:33.365 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-09-21T07:45:33.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:33.461 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-09-21T07:45:33.555 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:33.555 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-09-21T07:45:33.615 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:33.615 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2024-09-21T07:45:33.711 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:33.711 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2024-09-21T07:45:33.808 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:33.808 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-09-21T07:45:33.903 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:33.903 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-09-21T07:45:33.999 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:33.999 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-09-21T07:45:34.094 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:34.094 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-09-21T07:45:34.190 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:34.190 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-09-21T07:45:34.225 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:34.225 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2024-09-21T07:45:34.302 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:34.302 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-09-21T07:45:34.337 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:34.337 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2024-09-21T07:45:34.411 INFO:tasks.ceph:Adding keys to all mons... 2024-09-21T07:45:34.411 DEBUG:teuthology.orchestra.run.smithi088:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T07:45:34.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo tee -a /etc/ceph/ceph.keyring 2024-09-21T07:45:34.463 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.y] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.a] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.c] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.0] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.1] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.2] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.3] 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw== 2024-09-21T07:45:34.464 INFO:teuthology.orchestra.run.smithi088.stdout:[mgr.x] 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA== 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.b] 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ== 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout:[mds.d] 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ== 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.4] 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q== 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.5] 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ== 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.6] 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q== 2024-09-21T07:45:34.465 INFO:teuthology.orchestra.run.smithi088.stdout:[osd.7] 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw== 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout:[client.2] 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA== 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout:[client.1] 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ== 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout:[client.3] 2024-09-21T07:45:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA== 2024-09-21T07:45:34.504 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.y] 2024-09-21T07:45:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw== 2024-09-21T07:45:34.504 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.a] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A== 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.c] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q== 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.0] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg== 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.1] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg== 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.2] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA== 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.3] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw== 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout:[mgr.x] 2024-09-21T07:45:34.505 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.b] 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[mds.d] 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.4] 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.5] 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.6] 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[osd.7] 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw== 2024-09-21T07:45:34.506 INFO:teuthology.orchestra.run.smithi175.stdout:[client.0] 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout:[client.2] 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA== 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout:[client.1] 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ== 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout:[client.3] 2024-09-21T07:45:34.507 INFO:teuthology.orchestra.run.smithi175.stdout: key = AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA== 2024-09-21T07:45:34.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:45:34.512 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:45:34.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:34.715 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:34.882 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:34.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:35.051 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.055 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.217 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.221 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.383 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.387 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.546 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.550 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:35.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:45:35.719 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-09-21T07:45:35.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:35.882 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:36.044 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:36.048 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2024-09-21T07:45:36.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.215 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.376 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.542 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.546 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.708 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.712 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-09-21T07:45:36.873 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:36.878 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.042 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.046 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.209 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.214 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.377 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.380 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-09-21T07:45:37.543 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-09-21T07:45:37.543 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-09-21T07:45:37.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:45:37.783 DEBUG:teuthology.orchestra.run.smithi088:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-09-21T07:45:37.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-09-21T07:45:37.894 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:45:38.070 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-09-21T07:45:38.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-09-21T07:45:38.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-09-21T07:45:38.361 DEBUG:teuthology.orchestra.run.smithi175:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-09-21T07:45:38.418 DEBUG:teuthology.orchestra.run.smithi088:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:38.422 DEBUG:teuthology.orchestra.run.smithi175:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-09-21T07:45:38.456 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-09-21T07:45:38.456 INFO:tasks.ceph.mon.a:Restarting daemon 2024-09-21T07:45:38.456 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-09-21T07:45:38.498 INFO:tasks.ceph.mon.a:Started 2024-09-21T07:45:38.498 INFO:tasks.ceph.mon.b:Restarting daemon 2024-09-21T07:45:38.498 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-09-21T07:45:38.502 INFO:tasks.ceph.mon.b:Started 2024-09-21T07:45:38.502 INFO:tasks.ceph.mon.c:Restarting daemon 2024-09-21T07:45:38.502 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-09-21T07:45:38.506 INFO:tasks.ceph.mon.c:Started 2024-09-21T07:45:38.506 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-09-21T07:45:38.506 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-09-21T07:45:38.506 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-09-21T07:45:38.510 INFO:tasks.ceph.mgr.y:Started 2024-09-21T07:45:38.510 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-09-21T07:45:38.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-09-21T07:45:38.514 INFO:tasks.ceph.mgr.x:Started 2024-09-21T07:45:38.514 INFO:tasks.ceph:setting config [client] client_mount_timeout = 600 2024-09-21T07:45:38.514 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client client_mount_timeout 600 2024-09-21T07:45:38.517 INFO:tasks.ceph:setting config [client] debug_client = 20 2024-09-21T07:45:38.518 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client debug_client 20 2024-09-21T07:45:38.564 INFO:tasks.ceph:setting config [client] debug_ms = 1 2024-09-21T07:45:38.565 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client debug_ms 1 2024-09-21T07:45:38.611 INFO:tasks.ceph:setting config [client] rados_mon_op_timeout = 900 2024-09-21T07:45:38.612 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client rados_mon_op_timeout 900 2024-09-21T07:45:38.657 INFO:tasks.ceph:setting config [client] rados_osd_op_timeout = 900 2024-09-21T07:45:38.658 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set client rados_osd_op_timeout 900 2024-09-21T07:45:38.704 INFO:tasks.ceph:setting config [mds] debug_mds = 20 2024-09-21T07:45:38.704 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds debug_mds 20 2024-09-21T07:45:38.750 INFO:tasks.ceph:setting config [mds] debug_mds_balancer = 20 2024-09-21T07:45:38.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds debug_mds_balancer 20 2024-09-21T07:45:38.797 INFO:tasks.ceph:setting config [mds] debug_ms = 2 2024-09-21T07:45:38.797 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds debug_ms 2 2024-09-21T07:45:38.847 INFO:tasks.ceph:setting config [mds] mds_debug_frag = True 2024-09-21T07:45:38.847 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_debug_frag True 2024-09-21T07:45:38.899 INFO:tasks.ceph:setting config [mds] mds_debug_scatterstat = True 2024-09-21T07:45:38.899 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_debug_scatterstat True 2024-09-21T07:45:38.956 INFO:tasks.ceph:setting config [mds] mds_op_complaint_time = 180 2024-09-21T07:45:38.956 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_op_complaint_time 180 2024-09-21T07:45:39.010 INFO:tasks.ceph:setting config [mds] mds_verify_scatter = True 2024-09-21T07:45:39.010 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds mds_verify_scatter True 2024-09-21T07:45:39.064 INFO:tasks.ceph:setting config [mds] osd_op_complaint_time = 180 2024-09-21T07:45:39.064 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds osd_op_complaint_time 180 2024-09-21T07:45:39.097 INFO:tasks.ceph:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T07:45:39.097 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds rados_mon_op_timeout 900 2024-09-21T07:45:39.150 INFO:tasks.ceph:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T07:45:39.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mds rados_osd_op_timeout 900 2024-09-21T07:45:39.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.148+0000 7fcfc2000640 1 -- 172.21.15.88:0/2704131767 <== mon.1 v2:172.21.15.175:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v3)=0 v3) ==== 109+0+0 (secure 0 0 0) 0x7fcfbc035c10 con 0x7fcfcc14d250 2024-09-21T07:45:39.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.148+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b6816bf30 con 0x7f5b68001980 2024-09-21T07:45:39.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.151+0000 7fcfcb200640 1 -- 172.21.15.88:0/2704131767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc14d250 msgr2=0x7fcfcc1fc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.151+0000 7fcfcb200640 1 --2- 172.21.15.88:0/2704131767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc14d250 0x7fcfcc1fc1f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fcfbc003ed0 tx=0x7fcfcc0afd50 comp rx=0 tx=0).stop 2024-09-21T07:45:39.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.150+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 5 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5b5c00e5f0 con 0x7f5b68001980 2024-09-21T07:45:39.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.150+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 6 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5b5c017e00 con 0x7f5b68001980 2024-09-21T07:45:39.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.156+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b68034f20 con 0x7f5b68001980 2024-09-21T07:45:39.165 INFO:tasks.ceph:setting config [mgr] client_mount_timeout = 30 2024-09-21T07:45:39.165 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr client_mount_timeout 30 2024-09-21T07:45:39.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.162+0000 7f77a3200640 1 -- 172.21.15.88:0/1495406585 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f77780050e0 con 0x7f77a415b670 2024-09-21T07:45:39.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 -- 172.21.15.88:0/2704131767 shutdown_connections 2024-09-21T07:45:39.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 --2- 172.21.15.88:0/2704131767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcfcc1fc730 0x7fcfcc0b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 --2- 172.21.15.88:0/2704131767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcfcc14d250 0x7fcfcc1fc1f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 --2- 172.21.15.88:0/2704131767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcfb0004800 0x7fcfcc14c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 -- 172.21.15.88:0/2704131767 >> 172.21.15.88:0/2704131767 conn(0x7fcfcc0742a0 msgr2=0x7fcfcc14e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 -- 172.21.15.88:0/2704131767 shutdown_connections 2024-09-21T07:45:39.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7fcfcb200640 1 -- 172.21.15.88:0/2704131767 wait complete. 2024-09-21T07:45:39.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.161+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5b5c025020 con 0x7f5b68001980 2024-09-21T07:45:39.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.168+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 8 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f5b5c017530 con 0x7f5b68001980 2024-09-21T07:45:39.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.170+0000 7f779a000640 1 -- 172.21.15.88:0/1495406585 <== mon.1 v2:172.21.15.175:3300/0 9 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f779c004490 con 0x7f77a415b670 2024-09-21T07:45:39.177 INFO:tasks.ceph:setting config [mgr] debug_client = 20 2024-09-21T07:45:39.178 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr debug_client 20 2024-09-21T07:45:39.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.172+0000 7f8d36a00640 1 -- 172.21.15.88:0/1760295289 <== mon.2 v2:172.21.15.175:3301/0 8 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f8d30018420 con 0x7f8d3800b070 2024-09-21T07:45:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.183+0000 7f779a000640 1 -- 172.21.15.88:0/1495406585 <== mon.1 v2:172.21.15.175:3300/0 10 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v4)=0 v4) ==== 114+0+0 (secure 0 0 0) 0x7f77a41524a0 con 0x7f77a415b670 2024-09-21T07:45:39.193 INFO:tasks.ceph:setting config [mgr] debug_mgr = 20 2024-09-21T07:45:39.193 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr debug_mgr 20 2024-09-21T07:45:39.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.190+0000 7f77a3200640 1 -- 172.21.15.88:0/1495406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415b670 msgr2=0x7f77a4149430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.190+0000 7f77a3200640 1 --2- 172.21.15.88:0/1495406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415b670 0x7f77a4149430 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f779c009d60 tx=0x7f779c004a40 comp rx=0 tx=0).stop 2024-09-21T07:45:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.190+0000 7f77a3200640 1 -- 172.21.15.88:0/1495406585 shutdown_connections 2024-09-21T07:45:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.190+0000 7f77a3200640 1 --2- 172.21.15.88:0/1495406585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77a415b670 0x7f77a4149430 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.190+0000 7f77a3200640 1 --2- 172.21.15.88:0/1495406585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77a40ccc20 0x7f77a41f8a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.190+0000 7f77a3200640 1 --2- 172.21.15.88:0/1495406585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77a40b30d0 0x7f77a4154cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.194+0000 7f77a3200640 1 -- 172.21.15.88:0/1495406585 >> 172.21.15.88:0/1495406585 conn(0x7f77a406d620 msgr2=0x7f77a4068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.210+0000 7f77a3200640 1 -- 172.21.15.88:0/1495406585 shutdown_connections 2024-09-21T07:45:39.243 INFO:tasks.ceph:setting config [mgr] debug_ms = 1 2024-09-21T07:45:39.243 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr debug_ms 1 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.210+0000 7f77a3200640 1 -- 172.21.15.88:0/1495406585 wait complete. 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.207+0000 7f907be00640 1 -- 172.21.15.88:0/3203311167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 msgr2=0x7f907c0ccf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.207+0000 7f907be00640 1 --2- 172.21.15.88:0/3203311167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c0ccf60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f9068009860 tx=0x7f906802a460 comp rx=0 tx=0).stop 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.211+0000 7f907be00640 1 -- 172.21.15.88:0/3203311167 shutdown_connections 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.211+0000 7f907be00640 1 --2- 172.21.15.88:0/3203311167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c0ccf60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.211+0000 7f907be00640 1 -- 172.21.15.88:0/3203311167 >> 172.21.15.88:0/3203311167 conn(0x7f907c0742a0 msgr2=0x7f907c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.211+0000 7f907be00640 1 -- 172.21.15.88:0/3203311167 shutdown_connections 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.211+0000 7f907be00640 1 -- 172.21.15.88:0/3203311167 wait complete. 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.221+0000 7f907be00640 1 Processor -- start 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.221+0000 7f907be00640 1 -- start start 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.228+0000 7f907be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f907c098c90 0x7f907c206190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.229+0000 7f907be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c16b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.229+0000 7f907be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c0cf0a0 0x7f907c08b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.229+0000 7f907a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c16b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.229+0000 7f907a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c16b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:37444/0 (socket says 172.21.15.88:37444) 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.229+0000 7f907a000640 1 -- 172.21.15.88:0/258394568 learned_addr learned my addr 172.21.15.88:0/258394568 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.230+0000 7f907b400640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c0cf0a0 0x7f907c08b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.234+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0cfd40 con 0x7f907c0cf0a0 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.234+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f907c0cfbc0 con 0x7f907c098c90 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.234+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f907c0cfec0 con 0x7f907c0ccb50 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.237+0000 7f907b400640 1 -- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 msgr2=0x7f907c16b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.237+0000 7f907b400640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c16b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.237+0000 7f907b400640 1 -- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f907c098c90 msgr2=0x7f907c206190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.232+0000 7f907aa00640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f907c098c90 0x7f907c206190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.241+0000 7f907b400640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f907c098c90 0x7f907c206190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.241+0000 7f907b400640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f907c16bc20 con 0x7f907c0cf0a0 2024-09-21T07:45:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.241+0000 7f907aa00640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f907c098c90 0x7f907c206190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:39.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.242+0000 7f907b400640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c0cf0a0 0x7f907c08b370 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f907c073ef0 tx=0x7f907400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.250+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074016020 con 0x7f907c0cf0a0 2024-09-21T07:45:39.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.250+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7f907400ae10 con 0x7f907c0cf0a0 2024-09-21T07:45:39.274 INFO:tasks.ceph:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T07:45:39.274 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T07:45:39.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.255+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f907c08bf50 con 0x7f907c0cf0a0 2024-09-21T07:45:39.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.255+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f907c2056b0 con 0x7f907c0cf0a0 2024-09-21T07:45:39.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.259+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9074012680 con 0x7f907c0cf0a0 2024-09-21T07:45:39.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.259+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9074019020 con 0x7f907c0cf0a0 2024-09-21T07:45:39.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.259+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f907401bdf0 con 0x7f907c0cf0a0 2024-09-21T07:45:39.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.266+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f907c0d02f0 con 0x7f907c0cf0a0 2024-09-21T07:45:39.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.281+0000 7fd4cd600640 1 -- 172.21.15.88:0/4248113188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8145550 msgr2=0x7fd4c806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.281+0000 7fd4cd600640 1 --2- 172.21.15.88:0/4248113188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8145550 0x7fd4c806fb40 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd4b8009830 tx=0x7fd4b802a5c0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.291 INFO:tasks.ceph:setting config [mon] mon_op_complaint_time = 120 2024-09-21T07:45:39.291 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set mon mon_op_complaint_time 120 2024-09-21T07:45:39.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.281+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9074020020 con 0x7f907c0cf0a0 2024-09-21T07:45:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.284+0000 7fd4cd600640 1 -- 172.21.15.88:0/4248113188 shutdown_connections 2024-09-21T07:45:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.284+0000 7fd4cd600640 1 --2- 172.21.15.88:0/4248113188 >> v2:172.21.15.175:3300/0 conn(0x7fd4c8145a40 0x7fd4c8073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.288+0000 7fd4cd600640 1 --2- 172.21.15.88:0/4248113188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8145550 0x7fd4c806fb40 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.288+0000 7fd4cd600640 1 -- 172.21.15.88:0/4248113188 >> 172.21.15.88:0/4248113188 conn(0x7fd4c806d1c0 msgr2=0x7fd4c806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.289+0000 7fd4cd600640 1 -- 172.21.15.88:0/4248113188 shutdown_connections 2024-09-21T07:45:39.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.289+0000 7fd4cd600640 1 -- 172.21.15.88:0/4248113188 wait complete. 2024-09-21T07:45:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.303+0000 7fd4cd600640 1 Processor -- start 2024-09-21T07:45:39.315 INFO:tasks.ceph:setting config [osd] osd_op_complaint_time = 180 2024-09-21T07:45:39.315 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set osd osd_op_complaint_time 180 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.303+0000 7fd4cd600640 1 -- start start 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.310+0000 7fd4cd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 0x7fd4c813adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.311+0000 7fd4cd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c8145550 0x7fd4c81371e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.311+0000 7fd4c6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 0x7fd4c813adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.311+0000 7fd4c6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 0x7fd4c813adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:37472/0 (socket says 172.21.15.88:37472) 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.311+0000 7fd4c6a00640 1 -- 172.21.15.88:0/1407873963 learned_addr learned my addr 172.21.15.88:0/1407873963 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.311+0000 7fd4cd600640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8145a40 0x7fd4c81387d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.314+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c8160a30 con 0x7fd4c8145a40 2024-09-21T07:45:39.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.314+0000 7fd4c6000640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c8145550 0x7fd4c81371e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.315+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4c80ce830 con 0x7fd4c8145550 2024-09-21T07:45:39.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.315+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4c8160bb0 con 0x7fd4c8073a80 2024-09-21T07:45:39.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.315+0000 7fd4c7400640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8145a40 0x7fd4c81387d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.327 DEBUG:tasks.ceph:set 22 configs 2024-09-21T07:45:39.327 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.327 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.328 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.328 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.323+0000 7fd4c7400640 1 -- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 msgr2=0x7fd4c813adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.323+0000 7fd4c7400640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 0x7fd4c813adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.323+0000 7fd4c7400640 1 -- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c8145550 msgr2=0x7fd4c81371e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.323+0000 7fd4c7400640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c8145550 0x7fd4c81371e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.324+0000 7fd4c7400640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4c815eec0 con 0x7fd4c8145a40 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.324+0000 7fd4c6a00640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 0x7fd4c813adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.324+0000 7fd4c6000640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c8145550 0x7fd4c81371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.325+0000 7fd4c7400640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8145a40 0x7fd4c81387d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000c700 tx=0x7fd4c000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.326+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0003b70 con 0x7fd4c8145a40 2024-09-21T07:45:39.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.326+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7fd4c0003d10 con 0x7fd4c8145a40 2024-09-21T07:45:39.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.330+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4c815f1a0 con 0x7fd4c8145a40 2024-09-21T07:45:39.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.330+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4c815f760 con 0x7fd4c8145a40 2024-09-21T07:45:39.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.330+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4c0005330 con 0x7fd4c8145a40 2024-09-21T07:45:39.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.334+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd4c0012630 con 0x7fd4c8145a40 2024-09-21T07:45:39.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.334+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd4c001b880 con 0x7fd4c8145a40 2024-09-21T07:45:39.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.356+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4a0005080 con 0x7fd4c8145a40 2024-09-21T07:45:39.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.372+0000 7efd76200640 1 -- 172.21.15.88:0/577342811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 msgr2=0x7efd68031670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.372+0000 7efd76200640 1 --2- 172.21.15.88:0/577342811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd68031670 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efd64005fd0 tx=0x7efd6402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.373+0000 7efd76200640 1 -- 172.21.15.88:0/577342811 shutdown_connections 2024-09-21T07:45:39.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.373+0000 7efd76200640 1 --2- 172.21.15.88:0/577342811 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd68031670 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.373+0000 7efd76200640 1 --2- 172.21.15.88:0/577342811 >> v2:172.21.15.88:3300/0 conn(0x7efd68035cd0 0x7efd680e0e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.373+0000 7efd76200640 1 -- 172.21.15.88:0/577342811 >> 172.21.15.88:0/577342811 conn(0x7efd6801d9a0 msgr2=0x7efd68039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.373+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd4c0020060 con 0x7fd4c8145a40 2024-09-21T07:45:39.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.373+0000 7efd76200640 1 -- 172.21.15.88:0/577342811 shutdown_connections 2024-09-21T07:45:39.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.376+0000 7efd76200640 1 -- 172.21.15.88:0/577342811 wait complete. 2024-09-21T07:45:39.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.395+0000 7f8d45a00640 1 -- 172.21.15.88:0/1760295289 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f8d14005100 con 0x7f8d3800b070 2024-09-21T07:45:39.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.398+0000 7efd76200640 1 Processor -- start 2024-09-21T07:45:39.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.398+0000 7efd76200640 1 -- start start 2024-09-21T07:45:39.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.406+0000 7efd76200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd680d07a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.406+0000 7efd76200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd680072e0 0x7efd680d1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.406+0000 7efd76200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd68035cd0 0x7efd680d1590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.406+0000 7efd76200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd68031f70 con 0x7efd68035cd0 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.406+0000 7efd76200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd68031df0 con 0x7efd680072e0 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.406+0000 7efd76200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd680320f0 con 0x7efd68006310 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.407+0000 7efd74e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd680d07a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.407+0000 7efd6fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd680072e0 0x7efd680d1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.407+0000 7efd74e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd680d07a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:37478/0 (socket says 172.21.15.88:37478) 2024-09-21T07:45:39.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.407+0000 7efd74e00640 1 -- 172.21.15.88:0/4176027985 learned_addr learned my addr 172.21.15.88:0/4176027985 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.408+0000 7efd74e00640 1 -- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd680072e0 msgr2=0x7efd680d1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.408+0000 7efd74e00640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd680072e0 0x7efd680d1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.408+0000 7efd74e00640 1 -- 172.21.15.88:0/4176027985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd68035cd0 msgr2=0x7efd680d1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.408+0000 7efd74e00640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd68035cd0 0x7efd680d1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.408+0000 7efd74e00640 1 -- 172.21.15.88:0/4176027985 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd680e3ca0 con 0x7efd68006310 2024-09-21T07:45:39.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.409+0000 7efd74e00640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd680d07a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd7006c440 tx=0x7efd7006a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.410+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700693d0 con 0x7efd68006310 2024-09-21T07:45:39.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.410+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(3 keys) ==== 79+0+0 (secure 0 0 0) 0x7efd70069570 con 0x7efd68006310 2024-09-21T07:45:39.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.410+0000 7efd76200640 1 -- 172.21.15.88:0/4176027985 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd680e3f80 con 0x7efd68006310 2024-09-21T07:45:39.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.411+0000 7efd76200640 1 -- 172.21.15.88:0/4176027985 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7efd680e4540 con 0x7efd68006310 2024-09-21T07:45:39.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.411+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd700693d0 con 0x7efd68006310 2024-09-21T07:45:39.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.417+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efd70069820 con 0x7efd68006310 2024-09-21T07:45:39.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.418+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7efd7009c740 con 0x7efd68006310 2024-09-21T07:45:39.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.419+0000 7f8d36a00640 1 -- 172.21.15.88:0/1760295289 <== mon.2 v2:172.21.15.175:3301/0 9 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f8d30018420 con 0x7f8d3800b070 2024-09-21T07:45:39.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.418+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 9 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f5b5c00e9f0 con 0x7f5b68001980 2024-09-21T07:45:39.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.420+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd44005080 con 0x7efd68006310 2024-09-21T07:45:39.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.423+0000 7f8d36a00640 1 -- 172.21.15.88:0/1760295289 <== mon.2 v2:172.21.15.175:3301/0 10 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v5)=0 v5) ==== 123+0+0 (secure 0 0 0) 0x7f8d300185d0 con 0x7f8d3800b070 2024-09-21T07:45:39.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.422+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 6 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7efd7009c980 con 0x7efd68006310 2024-09-21T07:45:39.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.426+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7efd700a2080 con 0x7efd68006310 2024-09-21T07:45:39.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.427+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7fd4c001bad0 con 0x7fd4c8145a40 2024-09-21T07:45:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.428+0000 7f8d45a00640 1 -- 172.21.15.88:0/1760295289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d3800b070 msgr2=0x7f8d38181c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.431+0000 7f8d45a00640 1 --2- 172.21.15.88:0/1760295289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d3800b070 0x7f8d38181c30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f8d3000e820 tx=0x7f8d3000ece0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.431+0000 7f8d45a00640 1 -- 172.21.15.88:0/1760295289 shutdown_connections 2024-09-21T07:45:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.431+0000 7f8d45a00640 1 --2- 172.21.15.88:0/1760295289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d38052360 0x7f8d38048620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.431+0000 7f8d45a00640 1 --2- 172.21.15.88:0/1760295289 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8d380364c0 0x7f8d3817f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.431+0000 7f8d45a00640 1 --2- 172.21.15.88:0/1760295289 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8d3800b070 0x7f8d38181c30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.431+0000 7f8d45a00640 1 -- 172.21.15.88:0/1760295289 >> 172.21.15.88:0/1760295289 conn(0x7f8d38051420 msgr2=0x7f8d38054d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.432+0000 7f8d45a00640 1 -- 172.21.15.88:0/1760295289 shutdown_connections 2024-09-21T07:45:39.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.432+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 7 ==== config(4 keys) ==== 110+0+0 (secure 0 0 0) 0x7f907401b400 con 0x7f907c0cf0a0 2024-09-21T07:45:39.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.432+0000 7f8d45a00640 1 -- 172.21.15.88:0/1760295289 wait complete. 2024-09-21T07:45:39.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.520+0000 7fe182c00640 1 -- 172.21.15.88:0/1055306875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 msgr2=0x7fe17c15ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.524+0000 7fe17aa00640 1 -- 172.21.15.88:0/1055306875 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe168031600 con 0x7fe17c0ce4b0 2024-09-21T07:45:39.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.524+0000 7fe182c00640 1 --2- 172.21.15.88:0/1055306875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c15ecf0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fe168009830 tx=0x7fe16802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.524+0000 7fe182c00640 1 -- 172.21.15.88:0/1055306875 shutdown_connections 2024-09-21T07:45:39.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.524+0000 7fe182c00640 1 --2- 172.21.15.88:0/1055306875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c15ecf0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.524+0000 7fe182c00640 1 -- 172.21.15.88:0/1055306875 >> 172.21.15.88:0/1055306875 conn(0x7fe17c0d02f0 msgr2=0x7fe17c0d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.527+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f5b6803a930 con 0x7f5b68001980 2024-09-21T07:45:39.533 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.534+0000 7fe182c00640 1 -- 172.21.15.88:0/1055306875 shutdown_connections 2024-09-21T07:45:39.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.535+0000 7fe182c00640 1 -- 172.21.15.88:0/1055306875 wait complete. 2024-09-21T07:45:39.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.542+0000 7fe182c00640 1 Processor -- start 2024-09-21T07:45:39.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.543+0000 7fe182c00640 1 -- start start 2024-09-21T07:45:39.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.550+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 10 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5b5c00e9f0 con 0x7f5b68001980 2024-09-21T07:45:39.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.554+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd7009cb30 con 0x7efd68006310 2024-09-21T07:45:39.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.556+0000 7f5b66a00640 1 -- 172.21.15.88:0/541473646 <== mon.2 v2:172.21.15.175:3301/0 11 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f5b5c013030 con 0x7f5b68001980 2024-09-21T07:45:39.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.559+0000 7fe182c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe17c0cb410 0x7fe17c0c2d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.559+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4c001bce0 con 0x7fd4c8145a40 2024-09-21T07:45:39.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.559+0000 7fe182c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c09bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.560+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f907401b5f0 con 0x7f907c0cf0a0 2024-09-21T07:45:39.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.560+0000 7fe17b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c09bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.560+0000 7fe17b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c09bed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52858/0 (socket says 172.21.15.88:52858) 2024-09-21T07:45:39.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.561+0000 7fe17b400640 1 -- 172.21.15.88:0/3694818943 learned_addr learned my addr 172.21.15.88:0/3694818943 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.561+0000 7fe182c00640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe17c099800 0x7fe17c09c410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.564+0000 7fe182c00640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c0cfe30 con 0x7fe17c0ce4b0 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.564+0000 7fe17be00640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe17c0cb410 0x7fe17c0c2d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.564+0000 7fe182c00640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe17c0b6180 con 0x7fe17c0cb410 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.564+0000 7fe182c00640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe17c0cffb0 con 0x7fe17c099800 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.564+0000 7fe181600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe17c099800 0x7fe17c09c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe181600640 1 -- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe17c0cb410 msgr2=0x7fe17c0c2d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe181600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe17c0cb410 0x7fe17c0c2d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe181600640 1 -- 172.21.15.88:0/3694818943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 msgr2=0x7fe17c09bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe181600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c09bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe181600640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe17c09dcc0 con 0x7fe17c099800 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe17be00640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe17c0cb410 0x7fe17c0c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:39.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.566+0000 7fe17b400640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c09bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:39.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.571+0000 7fe181600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe17c099800 0x7fe17c09c410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c730 tx=0x7fe17400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b68001980 msgr2=0x7f5b68022980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 --2- 172.21.15.88:0/541473646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b68001980 0x7f5b68022980 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00b5c0 tx=0x7f5b5c00ba80 comp rx=0 tx=0).stop 2024-09-21T07:45:39.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 shutdown_connections 2024-09-21T07:45:39.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 --2- 172.21.15.88:0/541473646 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5b68001980 0x7f5b68022980 secure :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5b5c00b5c0 tx=0x7f5b5c00ba80 comp rx=0 tx=0).stop 2024-09-21T07:45:39.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 --2- 172.21.15.88:0/541473646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b680578c0 0x7f5b68023b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.571+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174017070 con 0x7fe17c099800 2024-09-21T07:45:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.572+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe174003bc0 con 0x7fe17c099800 2024-09-21T07:45:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.572+0000 7fe182c00640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe17c09dfa0 con 0x7fe17c099800 2024-09-21T07:45:39.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 --2- 172.21.15.88:0/541473646 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5b68001570 0x7f5b68163770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.567+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 >> 172.21.15.88:0/541473646 conn(0x7f5b68021500 msgr2=0x7f5b6816ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.573+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe174005070 con 0x7fe17c099800 2024-09-21T07:45:39.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.575+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 shutdown_connections 2024-09-21T07:45:39.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.574+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe17401a070 con 0x7fe17c099800 2024-09-21T07:45:39.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.575+0000 7fe182c00640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe17c09d0a0 con 0x7fe17c099800 2024-09-21T07:45:39.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.580+0000 7f5b75a00640 1 -- 172.21.15.88:0/541473646 wait complete. 2024-09-21T07:45:39.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.581+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe150005290 con 0x7fe17c099800 2024-09-21T07:45:39.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.581+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe174005260 con 0x7fe17c099800 2024-09-21T07:45:39.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.586+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe174022070 con 0x7fe17c099800 2024-09-21T07:45:39.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.641+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7f90500052f0 con 0x7f907c0cf0a0 2024-09-21T07:45:39.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.657+0000 7f9073400640 1 -- 172.21.15.88:0/258394568 <== mon.0 v2:172.21.15.88:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v7)=0 v7) ==== 108+0+0 (secure 0 0 0) 0x7f907c16bc20 con 0x7f907c0cf0a0 2024-09-21T07:45:39.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c0cf0a0 msgr2=0x7f907c08b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c0cf0a0 0x7f907c08b370 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f907c073ef0 tx=0x7f907400a430 comp rx=0 tx=0).stop 2024-09-21T07:45:39.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 shutdown_connections 2024-09-21T07:45:39.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f907c0ccb50 0x7f907c16b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f907c0cf0a0 0x7f907c08b370 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 --2- 172.21.15.88:0/258394568 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f907c098c90 0x7f907c206190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.671+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 >> 172.21.15.88:0/258394568 conn(0x7f907c0742a0 msgr2=0x7f907c099320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.677+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 shutdown_connections 2024-09-21T07:45:39.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.689+0000 7f907be00640 1 -- 172.21.15.88:0/258394568 wait complete. 2024-09-21T07:45:39.702 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.726+0000 7fd4cd600640 1 -- 172.21.15.88:0/1407873963 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fd49c0052f0 con 0x7fd4c8145a40 2024-09-21T07:45:39.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.747+0000 7fd4bea00640 1 -- 172.21.15.88:0/1407873963 <== mon.0 v2:172.21.15.88:3300/0 9 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v8)=0 v8) ==== 117+0+0 (secure 0 0 0) 0x7fd4c8073e90 con 0x7fd4c8145a40 2024-09-21T07:45:39.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 -- 172.21.15.88:0/1407873963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8145a40 msgr2=0x7fd4c81387d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8145a40 0x7fd4c81387d0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4c000c700 tx=0x7fd4c000cb00 comp rx=0 tx=0).stop 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 -- 172.21.15.88:0/1407873963 shutdown_connections 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4c8145a40 0x7fd4c81387d0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4c8145550 0x7fd4c81371e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 --2- 172.21.15.88:0/1407873963 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4c8073a80 0x7fd4c813adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.755+0000 7fd4b3e00640 1 -- 172.21.15.88:0/1407873963 >> 172.21.15.88:0/1407873963 conn(0x7fd4c806d1c0 msgr2=0x7fd4c80b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.756+0000 7fd4b3e00640 1 -- 172.21.15.88:0/1407873963 shutdown_connections 2024-09-21T07:45:39.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.756+0000 7fd4b3e00640 1 -- 172.21.15.88:0/1407873963 wait complete. 2024-09-21T07:45:39.777 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd33be00640 1 -- 172.21.15.88:0/420137874 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd330035020 con 0x7fd33c0af490 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd343000640 1 -- 172.21.15.88:0/420137874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0af490 msgr2=0x7fd33c0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd343000640 1 --2- 172.21.15.88:0/420137874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0af490 0x7fd33c0ce0a0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fd330005fd0 tx=0x7fd33002a530 comp rx=0 tx=0).stop 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd343000640 1 -- 172.21.15.88:0/420137874 shutdown_connections 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd343000640 1 --2- 172.21.15.88:0/420137874 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c0af490 0x7fd33c0ce0a0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd343000640 1 --2- 172.21.15.88:0/420137874 >> v2:172.21.15.88:3300/0 conn(0x7fd33c074dc0 0x7fd33c15b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.795+0000 7fd343000640 1 -- 172.21.15.88:0/420137874 >> 172.21.15.88:0/420137874 conn(0x7fd33c0742a0 msgr2=0x7fd33c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.796+0000 7fd343000640 1 -- 172.21.15.88:0/420137874 shutdown_connections 2024-09-21T07:45:39.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.796+0000 7fd343000640 1 -- 172.21.15.88:0/420137874 wait complete. 2024-09-21T07:45:39.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.807+0000 7fd343000640 1 Processor -- start 2024-09-21T07:45:39.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.808+0000 7fd343000640 1 -- start start 2024-09-21T07:45:39.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.815+0000 7fd343000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 0x7fd33c153170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.816+0000 7fd343000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd33c0af490 0x7fd33c1f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.816+0000 7fd343000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c0ce5e0 0x7fd33c0afc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.816+0000 7fd341200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd33c0af490 0x7fd33c1f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.816+0000 7fd341200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd33c0af490 0x7fd33c1f11b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52880/0 (socket says 172.21.15.88:52880) 2024-09-21T07:45:39.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.817+0000 7fd341c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 0x7fd33c153170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.817+0000 7fd341c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 0x7fd33c153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:37492/0 (socket says 172.21.15.88:37492) 2024-09-21T07:45:39.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.817+0000 7fd341200640 1 -- 172.21.15.88:0/250126751 learned_addr learned my addr 172.21.15.88:0/250126751 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.817+0000 7fd343000640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c15fa00 con 0x7fd33c0af490 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.817+0000 7fd343000640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd33c0b4ff0 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.817+0000 7fd343000640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd33c15fb80 con 0x7fd33c074dc0 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.818+0000 7fd342600640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c0ce5e0 0x7fd33c0afc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.819+0000 7fd342600640 1 -- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 msgr2=0x7fd33c153170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.819+0000 7fd342600640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 0x7fd33c153170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.819+0000 7fd342600640 1 -- 172.21.15.88:0/250126751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd33c0af490 msgr2=0x7fd33c1f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.819+0000 7fd342600640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd33c0af490 0x7fd33c1f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.819+0000 7fd342600640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd33c1523b0 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.820+0000 7fd341c00640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 0x7fd33c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T07:45:39.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.821+0000 7fd342600640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c0ce5e0 0x7fd33c0afc70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd334013f30 tx=0x7fd334013f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.822+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334020070 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.822+0000 7fd343000640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd33c152690 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.824+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd3340110d0 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.824+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd334011560 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.827+0000 7fd343000640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd33c152c50 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.831+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3224401943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080ce0a0 msgr2=0x7f0e080ce4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.831+0000 7f0e0ca00640 1 --2- 172.21.15.88:0/3224401943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080ce0a0 0x7f0e080ce4b0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f0df8009860 tx=0x7f0df802a470 comp rx=0 tx=0).stop 2024-09-21T07:45:39.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.831+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd310005290 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.832+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3224401943 shutdown_connections 2024-09-21T07:45:39.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.832+0000 7f0e0ca00640 1 --2- 172.21.15.88:0/3224401943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080ce0a0 0x7f0e080ce4b0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.832+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3224401943 >> 172.21.15.88:0/3224401943 conn(0x7f0e080d02f0 msgr2=0x7f0e080d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.832+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd334023070 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.833+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd3340257e0 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.833+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3224401943 shutdown_connections 2024-09-21T07:45:39.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.837+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd33402a070 con 0x7fd33c0ce5e0 2024-09-21T07:45:39.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.833+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3224401943 wait complete. 2024-09-21T07:45:39.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.842+0000 7f0e0ca00640 1 Processor -- start 2024-09-21T07:45:39.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.844+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7efd44005640 con 0x7efd68006310 2024-09-21T07:45:39.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.842+0000 7f0e0ca00640 1 -- start start 2024-09-21T07:45:39.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.854+0000 7f0e0ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080cad00 0x7f0e080c9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.854+0000 7f0e0ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e080ce0a0 0x7f0e080c9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.855+0000 7f0e0ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 0x7f0e08099350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.858+0000 7f0e07200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 0x7f0e08099350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.858+0000 7f0e0ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e080d0e10 con 0x7f0e0815ecf0 2024-09-21T07:45:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.858+0000 7f0e0ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e080b6290 con 0x7f0e080ce0a0 2024-09-21T07:45:39.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.858+0000 7f0e0ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e080d0f90 con 0x7f0e080cad00 2024-09-21T07:45:39.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.863+0000 7f0e07200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 0x7f0e08099350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52888/0 (socket says 172.21.15.88:52888) 2024-09-21T07:45:39.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.863+0000 7f0e07200640 1 -- 172.21.15.88:0/3683251474 learned_addr learned my addr 172.21.15.88:0/3683251474 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.866+0000 7efd6cc00640 1 -- 172.21.15.88:0/4176027985 <== mon.2 v2:172.21.15.175:3301/0 9 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v9)=0 v9) ==== 106+0+0 (secure 0 0 0) 0x7efd7008f030 con 0x7efd68006310 2024-09-21T07:45:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.864+0000 7f0e07200640 1 -- 172.21.15.88:0/3683251474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080cad00 msgr2=0x7f0e080c9570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:45:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.868+0000 7f0e07200640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080cad00 0x7f0e080c9570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.868+0000 7f0e07200640 1 -- 172.21.15.88:0/3683251474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e080ce0a0 msgr2=0x7f0e080c9ab0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:45:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.868+0000 7f0e07200640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e080ce0a0 0x7f0e080c9ab0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.868+0000 7f0e07200640 1 -- 172.21.15.88:0/3683251474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e08206b80 con 0x7f0e0815ecf0 2024-09-21T07:45:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.869+0000 7f0e07200640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 0x7f0e08099350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c6b0 tx=0x7f0e0000cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.869+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00003ba0 con 0x7f0e0815ecf0 2024-09-21T07:45:39.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.874+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7fe150005850 con 0x7fe17c099800 2024-09-21T07:45:39.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.871+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e00003d60 con 0x7f0e0815ecf0 2024-09-21T07:45:39.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.875+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3683251474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e08206000 con 0x7f0e0815ecf0 2024-09-21T07:45:39.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.876+0000 7f0e0ca00640 1 -- 172.21.15.88:0/3683251474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e082065c0 con 0x7f0e0815ecf0 2024-09-21T07:45:39.878 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 msgr2=0x7efd680d07a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd680d07a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7efd7006c440 tx=0x7efd7006a030 comp rx=0 tx=0).stop 2024-09-21T07:45:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 shutdown_connections 2024-09-21T07:45:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd68035cd0 0x7efd680d1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd680072e0 0x7efd680d1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 --2- 172.21.15.88:0/4176027985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd68006310 0x7efd680d07a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.880+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 >> 172.21.15.88:0/4176027985 conn(0x7efd6801d9a0 msgr2=0x7efd680ddcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.884+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 shutdown_connections 2024-09-21T07:45:39.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.884+0000 7efd62000640 1 -- 172.21.15.88:0/4176027985 wait complete. 2024-09-21T07:45:39.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.885+0000 7fe16fe00640 1 -- 172.21.15.88:0/3694818943 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v10)=0 v10) ==== 115+0+0 (secure 0 0 0) 0x7fe174022070 con 0x7fe17c099800 2024-09-21T07:45:39.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.886+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e00005330 con 0x7f0e0815ecf0 2024-09-21T07:45:39.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe17c099800 msgr2=0x7fe17c09c410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe17c099800 0x7fe17c09c410 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fe17400c730 tx=0x7fe17400cbf0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.886+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f0e00005510 con 0x7f0e0815ecf0 2024-09-21T07:45:39.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 shutdown_connections 2024-09-21T07:45:39.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe17c099800 0x7fe17c09c410 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe17c0ce4b0 0x7fe17c09bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 --2- 172.21.15.88:0/3694818943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe17c0cb410 0x7fe17c0c2d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.890+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 >> 172.21.15.88:0/3694818943 conn(0x7fe17c0d02f0 msgr2=0x7fe17c0cb820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.891+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 shutdown_connections 2024-09-21T07:45:39.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.892+0000 7fe16d600640 1 -- 172.21.15.88:0/3694818943 wait complete. 2024-09-21T07:45:39.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.891+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0de0005290 con 0x7f0e0815ecf0 2024-09-21T07:45:39.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.894+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f0e00012b10 con 0x7f0e0815ecf0 2024-09-21T07:45:39.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.896+0000 7f9e2e800640 1 -- 172.21.15.88:0/2820038007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20036410 msgr2=0x7f9e20036820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.896+0000 7f9e2e800640 1 --2- 172.21.15.88:0/2820038007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20036410 0x7f9e20036820 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f9e1c009830 tx=0x7f9e1c02a410 comp rx=0 tx=0).stop 2024-09-21T07:45:39.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.896+0000 7f9e2e800640 1 -- 172.21.15.88:0/2820038007 shutdown_connections 2024-09-21T07:45:39.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.896+0000 7f9e2e800640 1 --2- 172.21.15.88:0/2820038007 >> v2:172.21.15.175:3300/0 conn(0x7f9e2001c9c0 0x7f9e2001cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.896+0000 7f9e2e800640 1 --2- 172.21.15.88:0/2820038007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20036410 0x7f9e20036820 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.896+0000 7f9e2e800640 1 -- 172.21.15.88:0/2820038007 >> 172.21.15.88:0/2820038007 conn(0x7f9e20051450 msgr2=0x7f9e20051890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.900+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0e00005800 con 0x7f0e0815ecf0 2024-09-21T07:45:39.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.903+0000 7f9e2e800640 1 -- 172.21.15.88:0/2820038007 shutdown_connections 2024-09-21T07:45:39.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.903+0000 7f9e2e800640 1 -- 172.21.15.88:0/2820038007 wait complete. 2024-09-21T07:45:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.915+0000 7f9e2e800640 1 Processor -- start 2024-09-21T07:45:39.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.916+0000 7f9e2e800640 1 -- start start 2024-09-21T07:45:39.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.926+0000 7f9e2e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20009d60 0x7f9e20178ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.926+0000 7f9e2e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2001c9c0 0x7f9e20021e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.926+0000 7f9e2e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e20036410 0x7f9e2017b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:39.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.926+0000 7f9e2e800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9e2001d310 con 0x7f9e20036410 2024-09-21T07:45:39.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.928+0000 7f9e27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e20036410 0x7f9e2017b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.928+0000 7f9e2e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9e200e1620 con 0x7f9e2001c9c0 2024-09-21T07:45:39.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.928+0000 7f9e27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e20036410 0x7f9e2017b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52920/0 (socket says 172.21.15.88:52920) 2024-09-21T07:45:39.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.928+0000 7f9e27e00640 1 -- 172.21.15.88:0/1319968870 learned_addr learned my addr 172.21.15.88:0/1319968870 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:39.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.928+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9e2001d490 con 0x7f9e20009d60 2024-09-21T07:45:39.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.930+0000 7f9e27400640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20009d60 0x7f9e20178ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.930+0000 7f9e26a00640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2001c9c0 0x7f9e20021e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:39.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.930+0000 7f9e27400640 1 -- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2001c9c0 msgr2=0x7f9e20021e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.930+0000 7f9e27400640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2001c9c0 0x7f9e20021e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.930+0000 7f9e27400640 1 -- 172.21.15.88:0/1319968870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e20036410 msgr2=0x7f9e2017b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.930+0000 7f9e27400640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e20036410 0x7f9e2017b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.931+0000 7f9e27400640 1 -- 172.21.15.88:0/1319968870 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e20178050 con 0x7f9e20009d60 2024-09-21T07:45:39.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.931+0000 7f9e26a00640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2001c9c0 0x7f9e20021e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:39.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.936+0000 7f9e27400640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20009d60 0x7f9e20178ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e28052910 tx=0x7f9e2806a200 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:39.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.942+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e280753c0 con 0x7f9e20009d60 2024-09-21T07:45:39.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.942+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9e28069a70 con 0x7f9e20009d60 2024-09-21T07:45:39.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.942+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e28093660 con 0x7f9e20009d60 2024-09-21T07:45:39.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.942+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e2017c070 con 0x7f9e20009d60 2024-09-21T07:45:39.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.942+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e201782d0 con 0x7f9e20009d60 2024-09-21T07:45:39.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.943+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e280938c0 con 0x7f9e20009d60 2024-09-21T07:45:39.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.948+0000 7fcf80a00640 1 -- 172.21.15.88:0/1590424365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 msgr2=0x7fcf7c06ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:39.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.948+0000 7fcf80a00640 1 --2- 172.21.15.88:0/1590424365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c06ef10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0b2e40 tx=0x7fcf7402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.949+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e28092d50 con 0x7f9e20009d60 2024-09-21T07:45:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.954+0000 7fcf80a00640 1 -- 172.21.15.88:0/1590424365 shutdown_connections 2024-09-21T07:45:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.954+0000 7fcf80a00640 1 --2- 172.21.15.88:0/1590424365 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c06ef10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.954+0000 7fcf80a00640 1 --2- 172.21.15.88:0/1590424365 >> v2:172.21.15.175:3300/0 conn(0x7fcf7c0af8f0 0x7fcf7c15b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.954+0000 7fcf80a00640 1 -- 172.21.15.88:0/1590424365 >> 172.21.15.88:0/1590424365 conn(0x7fcf7c06d620 msgr2=0x7fcf7c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.954+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e2001cdd0 con 0x7f9e20009d60 2024-09-21T07:45:39.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.958+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f9e280a2080 con 0x7f9e20009d60 2024-09-21T07:45:39.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.975+0000 7fcf80a00640 1 -- 172.21.15.88:0/1590424365 shutdown_connections 2024-09-21T07:45:39.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.976+0000 7fcf80a00640 1 -- 172.21.15.88:0/1590424365 wait complete. 2024-09-21T07:45:39.983 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.989 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi088') 2024-09-21T07:45:39.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.991+0000 7fcf80a00640 1 Processor -- start 2024-09-21T07:45:39.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:39.991+0000 7fcf80a00640 1 -- start start 2024-09-21T07:45:40.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.005+0000 7fcf80a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c0af4e0 0x7fcf7c1539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.005+0000 7fcf80a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf7c0af8f0 0x7fcf7c1543d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.005+0000 7fcf80a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c0b10b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.006+0000 7fcf80a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c06f450 con 0x7fcf7c15bc20 2024-09-21T07:45:40.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.006+0000 7fcf80a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcf7c0711b0 con 0x7fcf7c0af4e0 2024-09-21T07:45:40.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.006+0000 7fcf80a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcf7c06f5d0 con 0x7fcf7c0af8f0 2024-09-21T07:45:40.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fcf7b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c0af4e0 0x7fcf7c1539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fcf7b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c0af4e0 0x7fcf7c1539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:51770/0 (socket says 172.21.15.88:51770) 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fcf7b400640 1 -- 172.21.15.88:0/973709620 learned_addr learned my addr 172.21.15.88:0/973709620 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fcf72a00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf7c0af8f0 0x7fcf7c1543d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 -- 172.21.15.88:0/2104966797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 msgr2=0x7fd1f4051f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 --2- 172.21.15.88:0/2104966797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4051f20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1fc06ad00 tx=0x7fd1fc0a66b0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 -- 172.21.15.88:0/2104966797 shutdown_connections 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 --2- 172.21.15.88:0/2104966797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4051f20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 --2- 172.21.15.88:0/2104966797 >> v2:172.21.15.88:3300/0 conn(0x7fd1f40e1030 0x7fd1f40532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 -- 172.21.15.88:0/2104966797 >> 172.21.15.88:0/2104966797 conn(0x7fd1f401fc00 msgr2=0x7fd1f4056960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 -- 172.21.15.88:0/2104966797 shutdown_connections 2024-09-21T07:45:40.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fd201800640 1 -- 172.21.15.88:0/2104966797 wait complete. 2024-09-21T07:45:40.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.008+0000 7fcf7be00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c0b10b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.011+0000 7fcf7be00640 1 -- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf7c0af8f0 msgr2=0x7fcf7c1543d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.011+0000 7fcf7be00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf7c0af8f0 0x7fcf7c1543d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.011+0000 7fcf7be00640 1 -- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c0af4e0 msgr2=0x7fcf7c1539d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.011+0000 7fcf7be00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c0af4e0 0x7fcf7c1539d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.011+0000 7fcf7be00640 1 -- 172.21.15.88:0/973709620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf7c1f57d0 con 0x7fcf7c15bc20 2024-09-21T07:45:40.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.013+0000 7fcf7be00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c0b10b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0b2c00 tx=0x7fcf74033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.014+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74031b50 con 0x7fcf7c15bc20 2024-09-21T07:45:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.014+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcf74031e20 con 0x7fcf7c15bc20 2024-09-21T07:45:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.014+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf74005260 con 0x7fcf7c15bc20 2024-09-21T07:45:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.014+0000 7fcf80a00640 1 -- 172.21.15.88:0/973709620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf7c1f4c50 con 0x7fcf7c15bc20 2024-09-21T07:45:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.015+0000 7fcf80a00640 1 -- 172.21.15.88:0/973709620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf7c1f5210 con 0x7fcf7c15bc20 2024-09-21T07:45:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.015+0000 7fd201800640 1 Processor -- start 2024-09-21T07:45:40.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcf740047a0 con 0x7fcf7c15bc20 2024-09-21T07:45:40.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fcf74033470 con 0x7fcf7c15bc20 2024-09-21T07:45:40.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.016+0000 7fd201800640 1 -- start start 2024-09-21T07:45:40.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fd201800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f4053810 0x7fd1f4021200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fd201800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4186300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fd201800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40e1030 0x7fd1f4186840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fd201800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f40527f0 con 0x7fd1f4053810 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fd201800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f4052670 con 0x7fd1f4054d90 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.019+0000 7fd201800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd1f4052970 con 0x7fd1f40e1030 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.021+0000 7fd1faa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f4053810 0x7fd1f4021200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.021+0000 7fd1fa000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4186300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.021+0000 7fd1fa000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4186300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:51792/0 (socket says 172.21.15.88:51792) 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.021+0000 7fd1fa000640 1 -- 172.21.15.88:0/1775611268 learned_addr learned my addr 172.21.15.88:0/1775611268 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.021+0000 7fd1fb400640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40e1030 0x7fd1f4186840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.022+0000 7fd1faa00640 1 -- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40e1030 msgr2=0x7fd1f4186840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.022+0000 7fd1faa00640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40e1030 0x7fd1f4186840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.022+0000 7fd1faa00640 1 -- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 msgr2=0x7fd1f4186300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.023+0000 7fd1faa00640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4186300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.023+0000 7fd1faa00640 1 -- 172.21.15.88:0/1775611268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f4035ee0 con 0x7fd1f4053810 2024-09-21T07:45:40.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.023+0000 7fd1fa000640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4186300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:40.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.024+0000 7fd1faa00640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f4053810 0x7fd1f4021200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d680 tx=0x7fd1ec00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.025+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec017070 con 0x7fd1f4053810 2024-09-21T07:45:40.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.025+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1ec005310 con 0x7fd1f4053810 2024-09-21T07:45:40.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.025+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1ec009820 con 0x7fd1f4053810 2024-09-21T07:45:40.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.025+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f4023f50 con 0x7fd1f4053810 2024-09-21T07:45:40.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.026+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd1ec0099c0 con 0x7fd1f4053810 2024-09-21T07:45:40.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.027+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f4020830 con 0x7fd1f4053810 2024-09-21T07:45:40.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.028+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fd1ec009cb0 con 0x7fd1f4053810 2024-09-21T07:45:40.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.030+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2327081204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 msgr2=0x7fa1e415ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.030+0000 7fea02600640 1 -- 172.21.15.88:0/1705553026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ae4a0 msgr2=0x7fe9fc0cd3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.030+0000 7fea02600640 1 --2- 172.21.15.88:0/1705553026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ae4a0 0x7fe9fc0cd3c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc0ceb30 tx=0x7fe9e802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.030+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2327081204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 0x7fa1e415ee70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4072e40 tx=0x7fa1d402b200 comp rx=0 tx=0).stop 2024-09-21T07:45:40.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.031+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2327081204 shutdown_connections 2024-09-21T07:45:40.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.031+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2327081204 >> v2:172.21.15.175:3300/0 conn(0x7fa1e40b30d0 0x7fa1e40b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.031+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2327081204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 0x7fa1e415ee70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.031+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2327081204 >> 172.21.15.88:0/2327081204 conn(0x7fa1e40742a0 msgr2=0x7fa1e40746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.031+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2327081204 shutdown_connections 2024-09-21T07:45:40.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.032+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2327081204 wait complete. 2024-09-21T07:45:40.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.032+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7fd310005580 con 0x7fd33c0ce5e0 2024-09-21T07:45:40.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.031+0000 7fea02600640 1 -- 172.21.15.88:0/1705553026 shutdown_connections 2024-09-21T07:45:40.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.029+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f4052df0 con 0x7fd1f4053810 2024-09-21T07:45:40.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.033+0000 7fea02600640 1 --2- 172.21.15.88:0/1705553026 >> v2:172.21.15.88:3300/0 conn(0x7fe9fc0cd900 0x7fe9fc1685c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.029+0000 7fcf80a00640 1 -- 172.21.15.88:0/973709620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf54005290 con 0x7fcf7c15bc20 2024-09-21T07:45:40.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.033+0000 7fea02600640 1 --2- 172.21.15.88:0/1705553026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ae4a0 0x7fe9fc0cd3c0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.033+0000 7fea02600640 1 -- 172.21.15.88:0/1705553026 >> 172.21.15.88:0/1705553026 conn(0x7fe9fc0742a0 msgr2=0x7fe9fc0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.034+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fcf7403f030 con 0x7fcf7c15bc20 2024-09-21T07:45:40.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.034+0000 7fea02600640 1 -- 172.21.15.88:0/1705553026 shutdown_connections 2024-09-21T07:45:40.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.034+0000 7fea02600640 1 -- 172.21.15.88:0/1705553026 wait complete. 2024-09-21T07:45:40.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.037+0000 7fa1e9e00640 1 Processor -- start 2024-09-21T07:45:40.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.037+0000 7fa1e9e00640 1 -- start start 2024-09-21T07:45:40.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.040+0000 7fa1e9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 0x7fa1e41e46c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.040+0000 7fa1e9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e40b30d0 0x7fa1e4099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.040+0000 7fa1e9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e415add0 0x7fa1e40b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.040+0000 7fa1e9e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4074f40 con 0x7fa1e40af3f0 2024-09-21T07:45:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.041+0000 7fea02600640 1 Processor -- start 2024-09-21T07:45:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.040+0000 7fa1e9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1e4074dc0 con 0x7fa1e415add0 2024-09-21T07:45:40.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.040+0000 7fa1e9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1e40b3a20 con 0x7fa1e40b30d0 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.042+0000 7fa1e3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e415add0 0x7fa1e40b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.042+0000 7fa1e3400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e415add0 0x7fa1e40b0b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:51802/0 (socket says 172.21.15.88:51802) 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.042+0000 7fa1e3400640 1 -- 172.21.15.88:0/2934262871 learned_addr learned my addr 172.21.15.88:0/2934262871 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.042+0000 7fa1e2000640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e40b30d0 0x7fa1e4099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.042+0000 7fa1e2000640 1 -- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e415add0 msgr2=0x7fa1e40b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fa1e2000640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e415add0 0x7fa1e40b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fa1e2000640 1 -- 172.21.15.88:0/2934262871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 msgr2=0x7fa1e41e46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fa1e2000640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 0x7fa1e41e46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fa1e2000640 1 -- 172.21.15.88:0/2934262871 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1e41e60e0 con 0x7fa1e40b30d0 2024-09-21T07:45:40.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fd1ec0047a0 con 0x7fd1f4053810 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.041+0000 7fea02600640 1 -- start start 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fea02600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0ae4a0 0x7fe9fc1549e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fea02600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0b0ff0 0x7fe9fc158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fea02600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0cd900 0x7fe9fc1f97d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fea02600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc168c80 con 0x7fe9fc0b0ff0 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.045+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0b0ff0 0x7fe9fc158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.045+0000 7fe9fbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0b0ff0 0x7fe9fc158220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52994/0 (socket says 172.21.15.88:52994) 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.045+0000 7fe9fbe00640 1 -- 172.21.15.88:0/3097115162 learned_addr learned my addr 172.21.15.88:0/3097115162 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.045+0000 7fea01200640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0ae4a0 0x7fe9fc1549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.044+0000 7fea01c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0cd900 0x7fe9fc1f97d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.045+0000 7fea02600640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc168b00 con 0x7fe9fc0cd900 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.045+0000 7fea02600640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc168e00 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.046+0000 7fea01200640 1 -- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0cd900 msgr2=0x7fe9fc1f97d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.046+0000 7fea01200640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0cd900 0x7fe9fc1f97d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.046+0000 7fea01200640 1 -- 172.21.15.88:0/3097115162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0b0ff0 msgr2=0x7fe9fc158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.046+0000 7fea01200640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0b0ff0 0x7fe9fc158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.046+0000 7fea01200640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc0b3200 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.047+0000 7fea01200640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0ae4a0 0x7fe9fc1549e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c3c0 tx=0x7fe9ec00c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.047+0000 7fea01c00640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0cd900 0x7fe9fc1f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:40.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.048+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec017070 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.048+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9ec0053c0 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.048+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9ec003850 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.050+0000 7fea02600640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc158f80 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.052+0000 7fea02600640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc158ca0 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.052+0000 7fd339600640 1 -- 172.21.15.88:0/250126751 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v11)=0 v11) ==== 122+0+0 (secure 0 0 0) 0x7fd334011700 con 0x7fd33c0ce5e0 2024-09-21T07:45:40.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c0ce5e0 msgr2=0x7fd33c0afc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fd32ea00640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c0ce5e0 0x7fd33c0afc70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd334013f30 tx=0x7fd334013f60 comp rx=0 tx=0).stop 2024-09-21T07:45:40.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 shutdown_connections 2024-09-21T07:45:40.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fd32ea00640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd33c0ce5e0 0x7fd33c0afc70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fd32ea00640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd33c0af490 0x7fd33c1f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fe9ec01a070 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.054+0000 7fa1e2000640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e40b30d0 0x7fa1e4099910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4031000 tx=0x7fa1d4003820 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.055+0000 7fd32ea00640 1 --2- 172.21.15.88:0/250126751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd33c074dc0 0x7fd33c153170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.055+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 >> 172.21.15.88:0/250126751 conn(0x7fd33c0742a0 msgr2=0x7fd33c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.055+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc0cccb0 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.055+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 shutdown_connections 2024-09-21T07:45:40.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.056+0000 7fd32ea00640 1 -- 172.21.15.88:0/250126751 wait complete. 2024-09-21T07:45:40.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.055+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4003e20 con 0x7fa1e40b30d0 2024-09-21T07:45:40.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.056+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1d4005110 con 0x7fa1e40b30d0 2024-09-21T07:45:40.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.056+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1d4003e20 con 0x7fa1e40b30d0 2024-09-21T07:45:40.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.059+0000 7f54c6000640 1 -- 172.21.15.88:0/1240917634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d5150 msgr2=0x7f54c00b9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.059+0000 7f54c6000640 1 --2- 172.21.15.88:0/1240917634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d5150 0x7f54c00b9b10 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f54b0009c90 tx=0x7f54b002aa50 comp rx=0 tx=0).stop 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.059+0000 7f54c6000640 1 -- 172.21.15.88:0/1240917634 shutdown_connections 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.059+0000 7f54c6000640 1 --2- 172.21.15.88:0/1240917634 >> v2:172.21.15.175:3300/0 conn(0x7f54c00d0730 0x7f54c00b6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.059+0000 7f54c6000640 1 --2- 172.21.15.88:0/1240917634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d5150 0x7f54c00b9b10 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.059+0000 7f54c6000640 1 -- 172.21.15.88:0/1240917634 >> 172.21.15.88:0/1240917634 conn(0x7f54c008bfe0 msgr2=0x7f54c00d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.060+0000 7f54c6000640 1 -- 172.21.15.88:0/1240917634 shutdown_connections 2024-09-21T07:45:40.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.060+0000 7f54c6000640 1 -- 172.21.15.88:0/1240917634 wait complete. 2024-09-21T07:45:40.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.061+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1e41456c0 con 0x7fa1e40b30d0 2024-09-21T07:45:40.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.061+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa1d4044060 con 0x7fa1e40b30d0 2024-09-21T07:45:40.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.063+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1e4145c80 con 0x7fa1e40b30d0 2024-09-21T07:45:40.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.063+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fe9ec01c780 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.064+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa1d400a690 con 0x7fa1e40b30d0 2024-09-21T07:45:40.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.064+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fe9ec011df0 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.071+0000 7f54c6000640 1 Processor -- start 2024-09-21T07:45:40.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.072+0000 7f54c6000640 1 -- start start 2024-09-21T07:45:40.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.071+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1e40716f0 con 0x7fa1e40b30d0 2024-09-21T07:45:40.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.079+0000 7f54c6000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c006dea0 0x7f54c00cad40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.080+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fa1e40716f0 con 0x7fa1e40b30d0 2024-09-21T07:45:40.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f5a2ca00640 1 -- 172.21.15.88:0/1223055340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a280af980 msgr2=0x7f5a28073300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/1223055340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a280af980 0x7f5a28073300 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a28072050 tx=0x7f5a1402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f54c6000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d0730 0x7f54c01fa6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f54bf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c006dea0 0x7f54c00cad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f54bf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c006dea0 0x7f54c00cad40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:51834/0 (socket says 172.21.15.88:51834) 2024-09-21T07:45:40.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f54bf400640 1 -- 172.21.15.88:0/2362264438 learned_addr learned my addr 172.21.15.88:0/2362264438 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.084+0000 7f54b6a00640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d0730 0x7f54c01fa6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.085+0000 7f5a2ca00640 1 -- 172.21.15.88:0/1223055340 shutdown_connections 2024-09-21T07:45:40.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.085+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/1223055340 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a2806ff60 0x7f5a280b3480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.085+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/1223055340 >> v2:172.21.15.88:3300/0 conn(0x7f5a28073840 0x7f5a2806fa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.085+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/1223055340 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a280af980 0x7f5a28073300 secure :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f5a28072050 tx=0x7f5a1402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.085+0000 7f5a2ca00640 1 -- 172.21.15.88:0/1223055340 >> 172.21.15.88:0/1223055340 conn(0x7f5a2806cda0 msgr2=0x7f5a2806d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.089+0000 7f5a2ca00640 1 -- 172.21.15.88:0/1223055340 shutdown_connections 2024-09-21T07:45:40.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.085+0000 7f54c6000640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00d5150 0x7f54c00cd7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.092+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54c006e2b0 con 0x7f54c00d5150 2024-09-21T07:45:40.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.092+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f54c00d31e0 con 0x7f54c006dea0 2024-09-21T07:45:40.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.092+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f54c006e430 con 0x7f54c00d0730 2024-09-21T07:45:40.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.093+0000 7f54bfe00640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00d5150 0x7f54c00cd7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.094+0000 7f54bfe00640 1 -- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d0730 msgr2=0x7f54c01fa6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.094+0000 7f54bfe00640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d0730 0x7f54c01fa6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.094+0000 7f54bfe00640 1 -- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c006dea0 msgr2=0x7f54c00cad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.094+0000 7f54bfe00640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c006dea0 0x7f54c00cad40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.094+0000 7f54bfe00640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54c01e8ab0 con 0x7f54c00d5150 2024-09-21T07:45:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.098+0000 7f54bfe00640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00d5150 0x7f54c00cd7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e640 tx=0x7f54b800eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.099+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8004220 con 0x7f54c00d5150 2024-09-21T07:45:40.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.099+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54c01e8d90 con 0x7f54c00d5150 2024-09-21T07:45:40.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.099+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54c01e9350 con 0x7f54c00d5150 2024-09-21T07:45:40.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.102+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54c01ea170 con 0x7f54c00d5150 2024-09-21T07:45:40.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.106+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f54b8005020 con 0x7f54c00d5150 2024-09-21T07:45:40.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.106+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54b8004220 con 0x7f54c00d5150 2024-09-21T07:45:40.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.106+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f54b800c810 con 0x7f54c00d5150 2024-09-21T07:45:40.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.106+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f54b8003930 con 0x7f54c00d5150 2024-09-21T07:45:40.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.103+0000 7f5a2ca00640 1 -- 172.21.15.88:0/1223055340 wait complete. 2024-09-21T07:45:40.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.114+0000 7f5a2ca00640 1 Processor -- start 2024-09-21T07:45:40.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.114+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f54b8005640 con 0x7f54c00d5150 2024-09-21T07:45:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.117+0000 7f5a2ca00640 1 -- start start 2024-09-21T07:45:40.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.123+0000 7fee6a800640 1 -- 172.21.15.88:0/4044782429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee480079a0 msgr2=0x7fee48009e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.125 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.123+0000 7fee6a800640 1 --2- 172.21.15.88:0/4044782429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee480079a0 0x7fee48009e80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fee640b3530 tx=0x7fee48003810 comp rx=0 tx=0).stop 2024-09-21T07:45:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.124+0000 7fee6a800640 1 -- 172.21.15.88:0/4044782429 shutdown_connections 2024-09-21T07:45:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.124+0000 7fee6a800640 1 --2- 172.21.15.88:0/4044782429 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee480079a0 0x7fee48009e80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.124+0000 7fee6a800640 1 -- 172.21.15.88:0/4044782429 >> 172.21.15.88:0/4044782429 conn(0x7fee6406d1c0 msgr2=0x7fee6406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.125+0000 7fee6a800640 1 -- 172.21.15.88:0/4044782429 shutdown_connections 2024-09-21T07:45:40.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.125+0000 7fee6a800640 1 -- 172.21.15.88:0/4044782429 wait complete. 2024-09-21T07:45:40.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.136+0000 7fee6a800640 1 Processor -- start 2024-09-21T07:45:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.136+0000 7fee6a800640 1 -- start start 2024-09-21T07:45:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.138+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2806ff60 0x7f5a28140f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.142+0000 7f9486200640 1 -- 172.21.15.88:0/4163284297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9478053de0 msgr2=0x7f9478052530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.142+0000 7f9486200640 1 --2- 172.21.15.88:0/4163284297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9478053de0 0x7f9478052530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f9480068dc0 tx=0x7f94800a79d0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.138+0000 7f5a25e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2806ff60 0x7f5a28140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.143+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a28073840 0x7f5a280abc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.143+0000 7f5a2ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28141ce0 0x7f5a281fa010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b3b40 con 0x7f5a2806ff60 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a280b39c0 con 0x7f5a28141ce0 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a2ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a280b3cc0 con 0x7f5a28073840 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a25400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a28073840 0x7f5a280abc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a25e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2806ff60 0x7f5a28140f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53036/0 (socket says 172.21.15.88:53036) 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a25400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a28073840 0x7f5a280abc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:37598/0 (socket says 172.21.15.88:37598) 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a25400640 1 -- 172.21.15.88:0/505217056 learned_addr learned my addr 172.21.15.88:0/505217056 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.144+0000 7f5a26800640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28141ce0 0x7f5a281fa010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f9486200640 1 -- 172.21.15.88:0/4163284297 shutdown_connections 2024-09-21T07:45:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f5a26800640 1 -- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a28073840 msgr2=0x7f5a280abc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f5a26800640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a28073840 0x7f5a280abc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f5a26800640 1 -- 172.21.15.88:0/505217056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2806ff60 msgr2=0x7f5a28140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f5a26800640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2806ff60 0x7f5a28140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f5a26800640 1 -- 172.21.15.88:0/505217056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a28144b10 con 0x7f5a28141ce0 2024-09-21T07:45:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f9486200640 1 --2- 172.21.15.88:0/4163284297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9478053de0 0x7f9478052530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f9486200640 1 --2- 172.21.15.88:0/4163284297 >> v2:172.21.15.88:3300/0 conn(0x7f94780349d0 0x7f94780e2820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f9486200640 1 -- 172.21.15.88:0/4163284297 >> 172.21.15.88:0/4163284297 conn(0x7f947801db70 msgr2=0x7f9478039830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.145+0000 7f5a26800640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28141ce0 0x7f5a281fa010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000bdf0 tx=0x7f5a2000bee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.146+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20019070 con 0x7f5a28141ce0 2024-09-21T07:45:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.146+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a20014440 con 0x7f5a28141ce0 2024-09-21T07:45:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.146+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a20014890 con 0x7f5a28141ce0 2024-09-21T07:45:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.148+0000 7f9486200640 1 -- 172.21.15.88:0/4163284297 shutdown_connections 2024-09-21T07:45:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.148+0000 7f9486200640 1 -- 172.21.15.88:0/4163284297 wait complete. 2024-09-21T07:45:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.147+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a28145010 con 0x7f5a28141ce0 2024-09-21T07:45:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.147+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281455d0 con 0x7f5a28141ce0 2024-09-21T07:45:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.152+0000 7fee6a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee480079a0 0x7fee6413f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.152+0000 7fee6a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee64145450 0x7fee64145860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.152+0000 7fee63400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee480079a0 0x7fee6413f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.152+0000 7fee63400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee480079a0 0x7fee6413f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:51858/0 (socket says 172.21.15.88:51858) 2024-09-21T07:45:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.152+0000 7fee63400640 1 -- 172.21.15.88:0/2622798852 learned_addr learned my addr 172.21.15.88:0/2622798852 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.153+0000 7fee6a800640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee6414c2b0 0x7fee6414c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.153+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee64151ad0 con 0x7fee64145450 2024-09-21T07:45:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.153+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee64151950 con 0x7fee480079a0 2024-09-21T07:45:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.153+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee64151c50 con 0x7fee6414c2b0 2024-09-21T07:45:40.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.153+0000 7fee5aa00640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee64145450 0x7fee64145860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.150+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a28144d80 con 0x7f5a28141ce0 2024-09-21T07:45:40.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.155+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5a2001c070 con 0x7f5a28141ce0 2024-09-21T07:45:40.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.155+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5a2001d6f0 con 0x7f5a28141ce0 2024-09-21T07:45:40.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.154+0000 7fee63e00640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee6414c2b0 0x7fee6414c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.159+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5a20023070 con 0x7f5a28141ce0 2024-09-21T07:45:40.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.157+0000 7fee5aa00640 1 -- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee6414c2b0 msgr2=0x7fee6414c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.157+0000 7fee5aa00640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee6414c2b0 0x7fee6414c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.157+0000 7fee5aa00640 1 -- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee480079a0 msgr2=0x7fee6413f860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.157+0000 7fee5aa00640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee480079a0 0x7fee6413f860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.158+0000 7fee5aa00640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee6414cc00 con 0x7fee64145450 2024-09-21T07:45:40.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.163+0000 7fee5aa00640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee64145450 0x7fee64145860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee4800d8d0 tx=0x7fee4c000f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.162+0000 7f9486200640 1 Processor -- start 2024-09-21T07:45:40.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.164+0000 7f9486200640 1 -- start start 2024-09-21T07:45:40.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.163+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c031040 con 0x7fee64145450 2024-09-21T07:45:40.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.164+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee4c045030 con 0x7fee64145450 2024-09-21T07:45:40.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.164+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee4c004be0 con 0x7fee64145450 2024-09-21T07:45:40.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.167+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee641f8bd0 con 0x7fee64145450 2024-09-21T07:45:40.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.167+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee641f9190 con 0x7fee64145450 2024-09-21T07:45:40.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.167+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fee4c034070 con 0x7fee64145450 2024-09-21T07:45:40.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.168+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee4c02ccd0 con 0x7fee64145450 2024-09-21T07:45:40.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.168+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee64073cc0 con 0x7fee64145450 2024-09-21T07:45:40.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.172+0000 7f9486200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 0x7f94780e0540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.172+0000 7f9484c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 0x7f94780e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9486200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94780543a0 0x7f94780dc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9484c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 0x7f94780e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53052/0 (socket says 172.21.15.88:53052) 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9484c00640 1 -- 172.21.15.88:0/2866826575 learned_addr learned my addr 172.21.15.88:0/2866826575 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9486200640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9478187600 0x7f9478006000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f947ea00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94780543a0 0x7f94780dc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9486200640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9478052a70 con 0x7f9478053de0 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9486200640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f947803a150 con 0x7f9478187600 2024-09-21T07:45:40.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f9486200640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9478052bf0 con 0x7f94780543a0 2024-09-21T07:45:40.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.176+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fee4c04f030 con 0x7fee64145450 2024-09-21T07:45:40.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.173+0000 7f947f400640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9478187600 0x7f9478006000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.177+0000 7f947ea00640 1 -- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9478187600 msgr2=0x7f9478006000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.177+0000 7f947ea00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9478187600 0x7f9478006000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.177+0000 7f947ea00640 1 -- 172.21.15.88:0/2866826575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 msgr2=0x7f94780e0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.177+0000 7f947ea00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 0x7f94780e0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.177+0000 7f947ea00640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f947817da70 con 0x7f94780543a0 2024-09-21T07:45:40.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.178+0000 7f9484c00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 0x7f94780e0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:40.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.178+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f0de0005580 con 0x7f0e0815ecf0 2024-09-21T07:45:40.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.179+0000 7f947ea00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94780543a0 0x7f94780dc8a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c970 tx=0x7f946c00ce30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.180+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f9e2001de40 con 0x7f9e20009d60 2024-09-21T07:45:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.181+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c007bb0 con 0x7f94780543a0 2024-09-21T07:45:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.181+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f946c007e40 con 0x7f94780543a0 2024-09-21T07:45:40.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.181+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f946c00f690 con 0x7f94780543a0 2024-09-21T07:45:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.181+0000 7f9486200640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9478186b20 con 0x7f94780543a0 2024-09-21T07:45:40.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.181+0000 7f9486200640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f94781870e0 con 0x7f94780543a0 2024-09-21T07:45:40.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.183+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f946c00f830 con 0x7f94780543a0 2024-09-21T07:45:40.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.183+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f946c0213f0 con 0x7f94780543a0 2024-09-21T07:45:40.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.185+0000 7f0dfe000640 1 -- 172.21.15.88:0/3683251474 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v12)=0 v12) ==== 119+0+0 (secure 0 0 0) 0x7f0e0000a640 con 0x7f0e0815ecf0 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.186+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 msgr2=0x7f0e08099350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.187+0000 7f0df3400640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 0x7f0e08099350 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f0e0000c6b0 tx=0x7f0e0000cb70 comp rx=0 tx=0).stop 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.187+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 shutdown_connections 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.187+0000 7f0df3400640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0815ecf0 0x7f0e08099350 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.187+0000 7f0df3400640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e080ce0a0 0x7f0e080c9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.187+0000 7f0df3400640 1 --2- 172.21.15.88:0/3683251474 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e080cad00 0x7f0e080c9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.187+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 >> 172.21.15.88:0/3683251474 conn(0x7f0e080d02f0 msgr2=0x7f0e080b4880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.190+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 shutdown_connections 2024-09-21T07:45:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.191+0000 7f0df3400640 1 -- 172.21.15.88:0/3683251474 wait complete. 2024-09-21T07:45:40.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.191+0000 7fcf80a00640 1 -- 172.21.15.88:0/973709620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fcf54005580 con 0x7fcf7c15bc20 2024-09-21T07:45:40.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.192+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94780e2820 con 0x7f94780543a0 2024-09-21T07:45:40.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.197+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f946c026020 con 0x7f94780543a0 2024-09-21T07:45:40.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.198+0000 7f9e1b400640 1 -- 172.21.15.88:0/1319968870 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v13)=0 v13) ==== 121+0+0 (secure 0 0 0) 0x7f9e20178050 con 0x7f9e20009d60 2024-09-21T07:45:40.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.201+0000 7f42a2800640 1 -- 172.21.15.88:0/3195593191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0af490 msgr2=0x7f429c0ce0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.201+0000 7f42a2800640 1 --2- 172.21.15.88:0/3195593191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0af490 0x7f429c0ce0a0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f4288009830 tx=0x7f428802a510 comp rx=0 tx=0).stop 2024-09-21T07:45:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20009d60 msgr2=0x7f9e20178ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20009d60 0x7f9e20178ad0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f9e28052910 tx=0x7f9e2806a200 comp rx=0 tx=0).stop 2024-09-21T07:45:40.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f42a2800640 1 -- 172.21.15.88:0/3195593191 shutdown_connections 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 shutdown_connections 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e20036410 0x7f9e2017b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9e2001c9c0 0x7f9e20021e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 --2- 172.21.15.88:0/1319968870 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9e20009d60 0x7f9e20178ad0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 >> 172.21.15.88:0/1319968870 conn(0x7f9e20051450 msgr2=0x7f9e200de400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f42a2800640 1 --2- 172.21.15.88:0/3195593191 >> v2:172.21.15.88:3300/0 conn(0x7f429c0ce5e0 0x7f429c0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f42a2800640 1 --2- 172.21.15.88:0/3195593191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0af490 0x7f429c0ce0a0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.207+0000 7f42a2800640 1 -- 172.21.15.88:0/3195593191 >> 172.21.15.88:0/3195593191 conn(0x7f429c0742a0 msgr2=0x7f429c0746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.207+0000 7f42a2800640 1 -- 172.21.15.88:0/3195593191 shutdown_connections 2024-09-21T07:45:40.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.206+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 shutdown_connections 2024-09-21T07:45:40.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.207+0000 7f9e2e800640 1 -- 172.21.15.88:0/1319968870 wait complete. 2024-09-21T07:45:40.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.208+0000 7f42a2800640 1 -- 172.21.15.88:0/3195593191 wait complete. 2024-09-21T07:45:40.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.208+0000 7fcf73e00640 1 -- 172.21.15.88:0/973709620 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v14)=0 v14) ==== 121+0+0 (secure 0 0 0) 0x7fcf7403f030 con 0x7fcf7c15bc20 2024-09-21T07:45:40.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.218+0000 7f42a2800640 1 Processor -- start 2024-09-21T07:45:40.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.218+0000 7fcf70c00640 1 -- 172.21.15.88:0/973709620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 msgr2=0x7fcf7c0b10b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.218+0000 7fcf70c00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c0b10b0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fcf7c0b2c00 tx=0x7fcf74033ba0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.218+0000 7f42a2800640 1 -- start start 2024-09-21T07:45:40.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.225+0000 7fcf70c00640 1 -- 172.21.15.88:0/973709620 shutdown_connections 2024-09-21T07:45:40.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.225+0000 7fcf70c00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcf7c0af8f0 0x7fcf7c1543d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.225+0000 7fcf70c00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf7c15bc20 0x7fcf7c0b10b0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.225+0000 7fcf70c00640 1 --2- 172.21.15.88:0/973709620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcf7c0af4e0 0x7fcf7c1539d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.225+0000 7fcf70c00640 1 -- 172.21.15.88:0/973709620 >> 172.21.15.88:0/973709620 conn(0x7fcf7c06d620 msgr2=0x7fcf7c0ce440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.229+0000 7fcf70c00640 1 -- 172.21.15.88:0/973709620 shutdown_connections 2024-09-21T07:45:40.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.229+0000 7fcf70c00640 1 -- 172.21.15.88:0/973709620 wait complete. 2024-09-21T07:45:40.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.229+0000 7f42a2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 0x7f429c1fa1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 0x7f429c1fa1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 0x7f429c1fa1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53076/0 (socket says 172.21.15.88:53076) 2024-09-21T07:45:40.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f429c0af490 0x7f429c150280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0ce5e0 0x7f429c1544d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a2800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0b4780 con 0x7f429c074dc0 2024-09-21T07:45:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f429c0b4600 con 0x7f429c0af490 2024-09-21T07:45:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.230+0000 7f42a2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f429c0b4900 con 0x7f429c0ce5e0 2024-09-21T07:45:40.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.231+0000 7f42a1400640 1 -- 172.21.15.88:0/908371387 learned_addr learned my addr 172.21.15.88:0/908371387 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.231+0000 7f42a0a00640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f429c0af490 0x7f429c150280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.241+0000 7f42a1e00640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0ce5e0 0x7f429c1544d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.241+0000 7f42a1400640 1 -- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0ce5e0 msgr2=0x7f429c1544d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.241+0000 7f42a1400640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0ce5e0 0x7f429c1544d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.242+0000 7f42a1400640 1 -- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f429c0af490 msgr2=0x7f429c150280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.242+0000 7f42a1400640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f429c0af490 0x7f429c150280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.242+0000 7f42a1400640 1 -- 172.21.15.88:0/908371387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429c1faa50 con 0x7f429c074dc0 2024-09-21T07:45:40.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.243+0000 7f42a1400640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 0x7f429c1fa1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f429000c3f0 tx=0x7f429000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.243+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290011790 con 0x7f429c074dc0 2024-09-21T07:45:40.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.243+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4290017070 con 0x7f429c074dc0 2024-09-21T07:45:40.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.243+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4290011ca0 con 0x7f429c074dc0 2024-09-21T07:45:40.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.243+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429c1fad30 con 0x7f429c074dc0 2024-09-21T07:45:40.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.244+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f429c16b7f0 con 0x7f429c074dc0 2024-09-21T07:45:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.243+0000 7f5dc4a00640 1 -- 172.21.15.88:0/966022214 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db8031600 con 0x7f5dc80b98b0 2024-09-21T07:45:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.244+0000 7f5dc7200640 1 -- 172.21.15.88:0/966022214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b98b0 msgr2=0x7f5dc80b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.244+0000 7f5dc7200640 1 --2- 172.21.15.88:0/966022214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b98b0 0x7f5dc80b4770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f5db80098e0 tx=0x7f5db802a610 comp rx=0 tx=0).stop 2024-09-21T07:45:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.245+0000 7f5dc7200640 1 -- 172.21.15.88:0/966022214 shutdown_connections 2024-09-21T07:45:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.245+0000 7f5dc7200640 1 --2- 172.21.15.88:0/966022214 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80cf870 0x7f5dc80d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.245+0000 7f5dc7200640 1 --2- 172.21.15.88:0/966022214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b98b0 0x7f5dc80b4770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.245+0000 7f5dc7200640 1 -- 172.21.15.88:0/966022214 >> 172.21.15.88:0/966022214 conn(0x7f5dc808bfe0 msgr2=0x7f5dc80d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.246+0000 7f5dc7200640 1 -- 172.21.15.88:0/966022214 shutdown_connections 2024-09-21T07:45:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.246+0000 7f5dc7200640 1 -- 172.21.15.88:0/966022214 wait complete. 2024-09-21T07:45:40.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.248+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fd1f4050d90 con 0x7fd1f4053810 2024-09-21T07:45:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.246+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f429001c460 con 0x7f429c074dc0 2024-09-21T07:45:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.246+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f42900127c0 con 0x7f429c074dc0 2024-09-21T07:45:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.250+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f429c16ba50 con 0x7f429c074dc0 2024-09-21T07:45:40.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.253+0000 7f5dc7200640 1 Processor -- start 2024-09-21T07:45:40.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.253+0000 7f5dc7200640 1 -- start start 2024-09-21T07:45:40.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.258+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f429c0ce9f0 con 0x7f429c074dc0 2024-09-21T07:45:40.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.259+0000 7fd1f2a00640 1 -- 172.21.15.88:0/1775611268 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v15)=0 v15) ==== 121+0+0 (secure 0 0 0) 0x7fd1ec0047a0 con 0x7fd1f4053810 2024-09-21T07:45:40.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.260+0000 7f5dc7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b4cb0 0x7f5dc81f6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f4053810 msgr2=0x7fd1f4021200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f4053810 0x7fd1f4021200 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec00d680 tx=0x7fd1ec00db40 comp rx=0 tx=0).stop 2024-09-21T07:45:40.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 shutdown_connections 2024-09-21T07:45:40.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd1f40e1030 0x7fd1f4186840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd1f4054d90 0x7fd1f4186300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 --2- 172.21.15.88:0/1775611268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f4053810 0x7fd1f4021200 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 >> 172.21.15.88:0/1775611268 conn(0x7fd1f401fc00 msgr2=0x7fd1f40078d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 shutdown_connections 2024-09-21T07:45:40.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7fd201800640 1 -- 172.21.15.88:0/1775611268 wait complete. 2024-09-21T07:45:40.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.261+0000 7f5dc7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 0x7f5dc806e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.262+0000 7f5dc7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc80cf870 0x7f5dc80a1150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.262+0000 7f5dc5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 0x7f5dc806e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.262+0000 7f5dc5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 0x7f5dc806e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:37654/0 (socket says 172.21.15.88:37654) 2024-09-21T07:45:40.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.262+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc80cf870 0x7f5dc80a1150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.263+0000 7f5dc6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc80cf870 0x7f5dc80a1150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:51902/0 (socket says 172.21.15.88:51902) 2024-09-21T07:45:40.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.263+0000 7f5dc5400640 1 -- 172.21.15.88:0/3700967751 learned_addr learned my addr 172.21.15.88:0/3700967751 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.263+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80d14c0 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.263+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5dc80d1340 con 0x7f5dc80cf870 2024-09-21T07:45:40.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.263+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5dc80d1640 con 0x7f5dc80b98b0 2024-09-21T07:45:40.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.263+0000 7f5dc5e00640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b4cb0 0x7f5dc81f6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.265+0000 7f5dc5e00640 1 -- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 msgr2=0x7f5dc806e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.265+0000 7f5dc5e00640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 0x7f5dc806e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.265+0000 7f5dc5e00640 1 -- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc80cf870 msgr2=0x7f5dc80a1150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.265+0000 7f5dc5e00640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc80cf870 0x7f5dc80a1150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.265+0000 7f5dc5e00640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dc809ec60 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.269+0000 7f5dc5e00640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b4cb0 0x7f5dc81f6330 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5db8031000 tx=0x7f5db80309c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.270+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db803f070 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.270+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dc809ef40 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.270+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dc80a0950 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.272+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dc806fca0 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.273+0000 7f5dc5400640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 0x7f5dc806e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:45:40.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.273+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5db8004030 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.273+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5db803f070 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.273+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5db8042070 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.275+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fa1e40b34e0 con 0x7fa1e40b30d0 2024-09-21T07:45:40.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.275+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5db8005390 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.282 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.282 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.282+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5db8049030 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.292+0000 7fa1db400640 1 -- 172.21.15.88:0/2934262871 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v16)=0 v16) ==== 108+0+0 (secure 0 0 0) 0x7fa1e40716f0 con 0x7fa1e40b30d0 2024-09-21T07:45:40.294 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.293+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e40b30d0 msgr2=0x7fa1e4099910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.293+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e40b30d0 0x7fa1e4099910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4031000 tx=0x7fa1d4003820 comp rx=0 tx=0).stop 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 shutdown_connections 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1e40b30d0 0x7fa1e4099910 secure :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4031000 tx=0x7fa1d4003820 comp rx=0 tx=0).stop 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1e415add0 0x7fa1e40b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 --2- 172.21.15.88:0/2934262871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1e40af3f0 0x7fa1e41e46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 >> 172.21.15.88:0/2934262871 conn(0x7fa1e40742a0 msgr2=0x7fa1e4068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 shutdown_connections 2024-09-21T07:45:40.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.294+0000 7fa1e9e00640 1 -- 172.21.15.88:0/2934262871 wait complete. 2024-09-21T07:45:40.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.299+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fe9fc0aef40 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.308+0000 7fe9f8c00640 1 -- 172.21.15.88:0/3097115162 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v17)=0 v17) ==== 120+0+0 (secure 0 0 0) 0x7fe9ec011590 con 0x7fe9fc0ae4a0 2024-09-21T07:45:40.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.313+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0ae4a0 msgr2=0x7fe9fc1549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.313+0000 7fe9f2000640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0ae4a0 0x7fe9fc1549e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec00c3c0 tx=0x7fe9ec00c880 comp rx=0 tx=0).stop 2024-09-21T07:45:40.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.316+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 shutdown_connections 2024-09-21T07:45:40.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.316+0000 7fe9f2000640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0cd900 0x7fe9fc1f97d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.316+0000 7fe9f2000640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0b0ff0 0x7fe9fc158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.316+0000 7fe9f2000640 1 --2- 172.21.15.88:0/3097115162 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0ae4a0 0x7fe9fc1549e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.316+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 >> 172.21.15.88:0/3097115162 conn(0x7fe9fc0742a0 msgr2=0x7fe9fc0cfcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.316+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 shutdown_connections 2024-09-21T07:45:40.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.317+0000 7fe9f2000640 1 -- 172.21.15.88:0/3097115162 wait complete. 2024-09-21T07:45:40.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.325+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f54c00b6810 con 0x7f54c00d5150 2024-09-21T07:45:40.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.328+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fee640b00f0 con 0x7fee64145450 2024-09-21T07:45:40.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.330+0000 7f54b7e00640 1 -- 172.21.15.88:0/2362264438 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v18)=0 v18) ==== 111+0+0 (secure 0 0 0) 0x7f54b80047a0 con 0x7f54c00d5150 2024-09-21T07:45:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.334+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00d5150 msgr2=0x7f54c00cd7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.334+0000 7f54c6000640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00d5150 0x7f54c00cd7d0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f54b800e640 tx=0x7f54b800eb00 comp rx=0 tx=0).stop 2024-09-21T07:45:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 shutdown_connections 2024-09-21T07:45:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f54c00d0730 0x7f54c01fa6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54c00d5150 0x7f54c00cd7d0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 --2- 172.21.15.88:0/2362264438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54c006dea0 0x7f54c00cad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 >> 172.21.15.88:0/2362264438 conn(0x7f54c008bfe0 msgr2=0x7f54c00b8f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 shutdown_connections 2024-09-21T07:45:40.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.335+0000 7f54c6000640 1 -- 172.21.15.88:0/2362264438 wait complete. 2024-09-21T07:45:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.338+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f5a280b3480 con 0x7f5a28141ce0 2024-09-21T07:45:40.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.338+0000 7fee5be00640 1 -- 172.21.15.88:0/2622798852 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v19)=0 v19) ==== 120+0+0 (secure 0 0 0) 0x7fee6414cc00 con 0x7fee64145450 2024-09-21T07:45:40.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee64145450 msgr2=0x7fee64145860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee64145450 0x7fee64145860 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fee4800d8d0 tx=0x7fee4c000f30 comp rx=0 tx=0).stop 2024-09-21T07:45:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 shutdown_connections 2024-09-21T07:45:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee6414c2b0 0x7fee6414c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee64145450 0x7fee64145860 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 --2- 172.21.15.88:0/2622798852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee480079a0 0x7fee6413f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.340+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 >> 172.21.15.88:0/2622798852 conn(0x7fee6406d1c0 msgr2=0x7fee64068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.341+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 shutdown_connections 2024-09-21T07:45:40.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.341+0000 7fee6a800640 1 -- 172.21.15.88:0/2622798852 wait complete. 2024-09-21T07:45:40.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.348+0000 7f5a1e000640 1 -- 172.21.15.88:0/505217056 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v20)=0 v20) ==== 119+0+0 (secure 0 0 0) 0x7f5a2001e440 con 0x7f5a28141ce0 2024-09-21T07:45:40.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28141ce0 msgr2=0x7f5a281fa010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28141ce0 0x7f5a281fa010 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f5a2000bdf0 tx=0x7f5a2000bee0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 shutdown_connections 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a28073840 0x7f5a280abc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a28141ce0 0x7f5a281fa010 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 --2- 172.21.15.88:0/505217056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2806ff60 0x7f5a28140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.350+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 >> 172.21.15.88:0/505217056 conn(0x7f5a2806cda0 msgr2=0x7f5a281617b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.351+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 shutdown_connections 2024-09-21T07:45:40.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.351+0000 7f5a2ca00640 1 -- 172.21.15.88:0/505217056 wait complete. 2024-09-21T07:45:40.371 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.371 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.381+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f94780227d0 con 0x7f94780543a0 2024-09-21T07:45:40.386 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.386 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.386 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.387+0000 7f9473400640 1 -- 172.21.15.88:0/2866826575 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v21)=0 v21) ==== 106+0+0 (secure 0 0 0) 0x7f946c014030 con 0x7f94780543a0 2024-09-21T07:45:40.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94780543a0 msgr2=0x7f94780dc8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94780543a0 0x7f94780dc8a0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f946c00c970 tx=0x7f946c00ce30 comp rx=0 tx=0).stop 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 shutdown_connections 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94780543a0 0x7f94780dc8a0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9478187600 0x7f9478006000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 --2- 172.21.15.88:0/2866826575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9478053de0 0x7f94780e0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 >> 172.21.15.88:0/2866826575 conn(0x7f947801db70 msgr2=0x7f9478039830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.388+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 shutdown_connections 2024-09-21T07:45:40.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.389+0000 7f9470c00640 1 -- 172.21.15.88:0/2866826575 wait complete. 2024-09-21T07:45:40.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.406+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7f5dc81eb0c0 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.407+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7f429c168cb0 con 0x7f429c074dc0 2024-09-21T07:45:40.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.411+0000 7f5dbe000640 1 -- 172.21.15.88:0/3700967751 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v22)=0 v22) ==== 121+0+0 (secure 0 0 0) 0x7f5db8005590 con 0x7f5dc80b4cb0 2024-09-21T07:45:40.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b4cb0 msgr2=0x7f5dc81f6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b4cb0 0x7f5dc81f6330 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5db8031000 tx=0x7f5db80309c0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 shutdown_connections 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5dc80b98b0 0x7f5dc806e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5dc80cf870 0x7f5dc80a1150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 --2- 172.21.15.88:0/3700967751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dc80b4cb0 0x7f5dc81f6330 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.412+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 >> 172.21.15.88:0/3700967751 conn(0x7f5dc808bfe0 msgr2=0x7f5dc80d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.413+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 shutdown_connections 2024-09-21T07:45:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.413+0000 7f5dc7200640 1 -- 172.21.15.88:0/3700967751 wait complete. 2024-09-21T07:45:40.416 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.417+0000 7f4299600640 1 -- 172.21.15.88:0/908371387 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v23)=0 v23) ==== 125+0+0 (secure 0 0 0) 0x7f4290011e40 con 0x7f429c074dc0 2024-09-21T07:45:40.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 msgr2=0x7f429c1fa1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 0x7f429c1fa1f0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f429000c3f0 tx=0x7f429000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 shutdown_connections 2024-09-21T07:45:40.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f429c0ce5e0 0x7f429c1544d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f429c0af490 0x7f429c150280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 --2- 172.21.15.88:0/908371387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f429c074dc0 0x7f429c1fa1f0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 >> 172.21.15.88:0/908371387 conn(0x7f429c0742a0 msgr2=0x7f429c06ef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 shutdown_connections 2024-09-21T07:45:40.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.418+0000 7f42a2800640 1 -- 172.21.15.88:0/908371387 wait complete. 2024-09-21T07:45:40.439 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.439 DEBUG:tasks.ceph:waiting for RemoteProcess(client=, args=['sudo', 'ceph', '--cluster', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi088') 2024-09-21T07:45:40.440 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config dump 2024-09-21T07:45:40.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 -- 172.21.15.88:0/2496710909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980b3160 msgr2=0x7f7c980d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 --2- 172.21.15.88:0/2496710909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980b3160 0x7f7c980d24a0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7c84009830 tx=0x7f7c8402a520 comp rx=0 tx=0).stop 2024-09-21T07:45:40.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 -- 172.21.15.88:0/2496710909 shutdown_connections 2024-09-21T07:45:40.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 --2- 172.21.15.88:0/2496710909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980b3160 0x7f7c980d24a0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 -- 172.21.15.88:0/2496710909 >> 172.21.15.88:0/2496710909 conn(0x7f7c9808bfe0 msgr2=0x7f7c9809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 -- 172.21.15.88:0/2496710909 shutdown_connections 2024-09-21T07:45:40.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.695+0000 7f7c9d400640 1 -- 172.21.15.88:0/2496710909 wait complete. 2024-09-21T07:45:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.697+0000 7f7c9d400640 1 Processor -- start 2024-09-21T07:45:40.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.697+0000 7f7c9d400640 1 -- start start 2024-09-21T07:45:40.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c980b3160 0x7f7c98159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c980b9e00 0x7f7c980b2120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 0x7f7c980b2660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:40.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9d400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980d4700 con 0x7f7c980d29e0 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c980d4580 con 0x7f7c980b3160 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c980d4880 con 0x7f7c980b9e00 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 0x7f7c980b2660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c97e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c980b3160 0x7f7c98159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 0x7f7c980b2660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53110/0 (socket says 172.21.15.88:53110) 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c97400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c980b9e00 0x7f7c980b2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:40.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 -- 172.21.15.88:0/669131518 learned_addr learned my addr 172.21.15.88:0/669131518 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 -- 172.21.15.88:0/669131518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c980b9e00 msgr2=0x7f7c980b2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c980b9e00 0x7f7c980b2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 -- 172.21.15.88:0/669131518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c980b3160 msgr2=0x7f7c98159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.699+0000 7f7c9ca00640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c980b3160 0x7f7c98159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c9ca00640 1 -- 172.21.15.88:0/669131518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c98158e10 con 0x7f7c980d29e0 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c9ca00640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 0x7f7c980b2660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000cb10 tx=0x7f7c9000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:40.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90016020 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c9000ad60 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c90012680 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c981f92a0 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.700+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c981f9860 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.701+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f7c90019020 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.701+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f7c9001bd60 con 0x7f7c980d29e0 2024-09-21T07:45:40.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.701+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c980d24a0 con 0x7f7c980d29e0 2024-09-21T07:45:40.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.704+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f7c90020030 con 0x7f7c980d29e0 2024-09-21T07:45:40.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.767+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f7c980b57a0 con 0x7f7c980d29e0 2024-09-21T07:45:40.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.768+0000 7f7c8fe00640 1 -- 172.21.15.88:0/669131518 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v23) ==== 59+0+1472 (secure 0 0 0) 0x7f7c98158e10 con 0x7f7c980d29e0 2024-09-21T07:45:40.769 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:45:40.769 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:45:40.770 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:45:40.771 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 msgr2=0x7f7c980b2660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 0x7f7c980b2660 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f7c9000cb10 tx=0x7f7c9000a060 comp rx=0 tx=0).stop 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 shutdown_connections 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c980b9e00 0x7f7c980b2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c980d29e0 0x7f7c980b2660 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 --2- 172.21.15.88:0/669131518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c980b3160 0x7f7c98159270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:40.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 >> 172.21.15.88:0/669131518 conn(0x7f7c9808bfe0 msgr2=0x7f7c9806de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 shutdown_connections 2024-09-21T07:45:40.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:40.769+0000 7f7c9d400640 1 -- 172.21.15.88:0/669131518 wait complete. 2024-09-21T07:45:40.791 INFO:tasks.ceph:Setting crush tunables to default 2024-09-21T07:45:40.791 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph osd crush tunables default 2024-09-21T07:45:41.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 -- 172.21.15.88:0/4257761293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44680d2370 msgr2=0x7f4468162fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 --2- 172.21.15.88:0/4257761293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44680d2370 0x7f4468162fd0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4454009830 tx=0x7f445402a510 comp rx=0 tx=0).stop 2024-09-21T07:45:41.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 -- 172.21.15.88:0/4257761293 shutdown_connections 2024-09-21T07:45:41.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 --2- 172.21.15.88:0/4257761293 >> v2:172.21.15.175:3300/0 conn(0x7f4468166be0 0x7f4468169000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 --2- 172.21.15.88:0/4257761293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44680d2370 0x7f4468162fd0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 -- 172.21.15.88:0/4257761293 >> 172.21.15.88:0/4257761293 conn(0x7f44680d4680 msgr2=0x7f44680d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:41.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 -- 172.21.15.88:0/4257761293 shutdown_connections 2024-09-21T07:45:41.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.005+0000 7f446ec00640 1 -- 172.21.15.88:0/4257761293 wait complete. 2024-09-21T07:45:41.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.007+0000 7f446ec00640 1 Processor -- start 2024-09-21T07:45:41.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.007+0000 7f446ec00640 1 -- start start 2024-09-21T07:45:41.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.008+0000 7f446ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44680b3ae0 0x7f44680b7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.008+0000 7f446ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 0x7f446814c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.008+0000 7f446ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4468166be0 0x7f446814cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.008+0000 7f446ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44680d2bf0 con 0x7f44680d2370 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.008+0000 7f446ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4468163720 con 0x7f44680b3ae0 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.008+0000 7f446ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4468169540 con 0x7f4468166be0 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 0x7f446814c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 0x7f446814c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53124/0 (socket says 172.21.15.88:53124) 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 -- 172.21.15.88:0/3445390520 learned_addr learned my addr 172.21.15.88:0/3445390520 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:41.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f446cc00640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4468166be0 0x7f446814cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467e00640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44680b3ae0 0x7f44680b7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 -- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4468166be0 msgr2=0x7f446814cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4468166be0 0x7f446814cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 -- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44680b3ae0 msgr2=0x7f44680b7d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44680b3ae0 0x7f44680b7d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 -- 172.21.15.88:0/3445390520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f446815fa20 con 0x7f44680d2370 2024-09-21T07:45:41.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.009+0000 7f4467400640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 0x7f446814c7c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4454009950 tx=0x7f4454033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:41.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454005c90 con 0x7f44680d2370 2024-09-21T07:45:41.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4454031e50 con 0x7f44680d2370 2024-09-21T07:45:41.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4454005260 con 0x7f44680d2370 2024-09-21T07:45:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4468163940 con 0x7f44680d2370 2024-09-21T07:45:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f446815f150 con 0x7f44680d2370 2024-09-21T07:45:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f44540047a0 con 0x7f44680d2370 2024-09-21T07:45:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.010+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4454033330 con 0x7f44680d2370 2024-09-21T07:45:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.011+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f446808d090 con 0x7f44680d2370 2024-09-21T07:45:41.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.013+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f446808d090 con 0x7f44680d2370 2024-09-21T07:45:41.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.079+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7f44681628c0 con 0x7f44680d2370 2024-09-21T07:45:41.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.081+0000 7f445fe00640 1 -- 172.21.15.88:0/3445390520 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7f445403f030 con 0x7f44680d2370 2024-09-21T07:45:41.082 INFO:teuthology.orchestra.run.smithi088.stderr:adjusted tunables profile to default 2024-09-21T07:45:41.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 msgr2=0x7f446814c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:41.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 0x7f446814c7c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4454009950 tx=0x7f4454033c20 comp rx=0 tx=0).stop 2024-09-21T07:45:41.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 shutdown_connections 2024-09-21T07:45:41.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4468166be0 0x7f446814cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44680d2370 0x7f446814c7c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 --2- 172.21.15.88:0/3445390520 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44680b3ae0 0x7f44680b7d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:41.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 >> 172.21.15.88:0/3445390520 conn(0x7f44680d4680 msgr2=0x7f4468068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:41.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 shutdown_connections 2024-09-21T07:45:41.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:41.082+0000 7f446ec00640 1 -- 172.21.15.88:0/3445390520 wait complete. 2024-09-21T07:45:41.103 INFO:tasks.ceph:check_enable_crimson: False 2024-09-21T07:45:41.103 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-09-21T07:45:41.103 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:41.104 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-09-21T07:45:41.158 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:41.158 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-09-21T07:45:41.254 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:41.254 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-09-21T07:45:41.351 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:45:41.351 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-09-21T07:45:41.447 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:41.447 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-09-21T07:45:41.504 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:41.504 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-09-21T07:45:41.599 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:41.599 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-09-21T07:45:41.696 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:45:41.696 DEBUG:teuthology.orchestra.run.smithi175:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-09-21T07:45:41.792 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new c4f0fa6c-e896-4720-9487-4bd90e732c8f 0 2024-09-21T07:45:42.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 -- 172.21.15.175:0/1754000207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40cf160 msgr2=0x7f5df40cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.051 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 --2- 172.21.15.175:0/1754000207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40cf160 0x7f5df40cf570 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009860 tx=0x7f5de002a4f0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 -- 172.21.15.175:0/1754000207 shutdown_connections 2024-09-21T07:45:42.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 --2- 172.21.15.175:0/1754000207 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40cf160 0x7f5df40cf570 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 -- 172.21.15.175:0/1754000207 >> 172.21.15.175:0/1754000207 conn(0x7f5df408bfe0 msgr2=0x7f5df40d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 -- 172.21.15.175:0/1754000207 shutdown_connections 2024-09-21T07:45:42.052 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.049+0000 7f5dfb000640 1 -- 172.21.15.175:0/1754000207 wait complete. 2024-09-21T07:45:42.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.051+0000 7f5dfb000640 1 Processor -- start 2024-09-21T07:45:42.053 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.051+0000 7f5dfb000640 1 -- start start 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 0x7f5df4166350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfb000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5df40cf160 0x7f5df40a1960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40d0bf0 0x7f5df415c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfb000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40d1710 con 0x7f5df40cf160 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5df40d1f50 con 0x7f5df40b98b0 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5df40d1890 con 0x7f5df40d0bf0 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5df9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 0x7f5df4166350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5dfa600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40d0bf0 0x7f5df415c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5df9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 0x7f5df4166350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35348/0 (socket says 172.21.15.175:35348) 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5df9c00640 1 -- 172.21.15.175:0/2673663613 learned_addr learned my addr 172.21.15.175:0/2673663613 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5df9200640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5df40cf160 0x7f5df40a1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.056 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5df9c00640 1 -- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40d0bf0 msgr2=0x7f5df415c230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.053+0000 7f5df9c00640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40d0bf0 0x7f5df415c230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df9c00640 1 -- 172.21.15.175:0/2673663613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5df40cf160 msgr2=0x7f5df40a1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df9c00640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5df40cf160 0x7f5df40a1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df9c00640 1 -- 172.21.15.175:0/2673663613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5df41f5ea0 con 0x7f5df40b98b0 2024-09-21T07:45:42.057 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df9c00640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 0x7f5df4166350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c3c0 tx=0x7f5de800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8017070 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5de80035f0 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5df41f6180 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de8003a40 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5df41f5b70 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.054+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5de801a070 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.055+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7f5de8012740 con 0x7f5df40b98b0 2024-09-21T07:45:42.058 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.055+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5df40cf570 con 0x7f5df40b98b0 2024-09-21T07:45:42.059 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.058+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5de8021020 con 0x7f5df40b98b0 2024-09-21T07:45:42.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.130+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "c4f0fa6c-e896-4720-9487-4bd90e732c8f", "id": 0} v 0) -- 0x7f5df40b81e0 con 0x7f5df40b98b0 2024-09-21T07:45:42.138 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.136+0000 7f5df1600640 1 -- 172.21.15.175:0/2673663613 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c4f0fa6c-e896-4720-9487-4bd90e732c8f", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7f5de8012940 con 0x7f5df40b98b0 2024-09-21T07:45:42.138 INFO:teuthology.orchestra.run.smithi175.stdout:0 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 msgr2=0x7f5df4166350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 0x7f5df4166350 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5de800c3c0 tx=0x7f5de800c880 comp rx=0 tx=0).stop 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 shutdown_connections 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5df40d0bf0 0x7f5df415c230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5df40cf160 0x7f5df40a1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 --2- 172.21.15.175:0/2673663613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5df40b98b0 0x7f5df4166350 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 >> 172.21.15.175:0/2673663613 conn(0x7f5df408bfe0 msgr2=0x7f5df40d21d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 shutdown_connections 2024-09-21T07:45:42.140 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.137+0000 7f5dfb000640 1 -- 172.21.15.175:0/2673663613 wait complete. 2024-09-21T07:45:42.159 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 1a85d167-4a6b-49c0-a642-3935989cfde3 1 2024-09-21T07:45:42.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.376+0000 7f01a7000640 1 -- 172.21.15.175:0/1062654668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 msgr2=0x7f01a00d4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.376+0000 7f01a7000640 1 --2- 172.21.15.175:0/1062654668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00d4560 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f01a006dd40 tx=0x7f018c02a530 comp rx=0 tx=0).stop 2024-09-21T07:45:42.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.377+0000 7f01a7000640 1 -- 172.21.15.175:0/1062654668 shutdown_connections 2024-09-21T07:45:42.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.377+0000 7f01a7000640 1 --2- 172.21.15.175:0/1062654668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00d4560 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.377+0000 7f01a7000640 1 --2- 172.21.15.175:0/1062654668 >> v2:172.21.15.88:3300/0 conn(0x7f01a00d24f0 0x7f01a00d29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.377+0000 7f01a7000640 1 -- 172.21.15.175:0/1062654668 >> 172.21.15.175:0/1062654668 conn(0x7f01a008dd50 msgr2=0x7f01a009d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.377+0000 7f01a7000640 1 -- 172.21.15.175:0/1062654668 shutdown_connections 2024-09-21T07:45:42.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.377+0000 7f01a7000640 1 -- 172.21.15.175:0/1062654668 wait complete. 2024-09-21T07:45:42.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.379+0000 7f01a7000640 1 Processor -- start 2024-09-21T07:45:42.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.379+0000 7f01a7000640 1 -- start start 2024-09-21T07:45:42.383 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 0x7f01a0152aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a7000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0162100 0x7f01a00b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a7000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00d4f10 con 0x7f01a0162100 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00d4d90 con 0x7f01a00d2f00 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01a00cff40 con 0x7f01a00d24f0 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 0x7f01a0152aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 0x7f01a0152aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:53430/0 (socket says 172.21.15.175:53430) 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 -- 172.21.15.175:0/3147853455 learned_addr learned my addr 172.21.15.175:0/3147853455 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a6600640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0162100 0x7f01a00b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 -- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 msgr2=0x7f01a00b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 -- 172.21.15.175:0/3147853455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0162100 msgr2=0x7f01a00b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0162100 0x7f01a00b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5c00640 1 -- 172.21.15.175:0/3147853455 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a009ea60 con 0x7f01a00d24f0 2024-09-21T07:45:42.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.381+0000 7f01a5200640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:42.385 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f01a5c00640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 0x7f01a0152aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f019400e7c0 tx=0x7f019400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019400ccc0 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0194010430 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0194010840 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a009ed40 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a009fd00 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f01940109e0 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.382+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f01940217a0 con 0x7f01a00d24f0 2024-09-21T07:45:42.386 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.383+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a00b55a0 con 0x7f01a00d24f0 2024-09-21T07:45:42.387 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.385+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f0194018c40 con 0x7f01a00d24f0 2024-09-21T07:45:42.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.459+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "1a85d167-4a6b-49c0-a642-3935989cfde3", "id": 1} v 0) -- 0x7f01a00b52c0 con 0x7f01a00d24f0 2024-09-21T07:45:42.466 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.464+0000 7f019d600640 1 -- 172.21.15.175:0/3147853455 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "1a85d167-4a6b-49c0-a642-3935989cfde3", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f0194018e20 con 0x7f01a00d24f0 2024-09-21T07:45:42.466 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 msgr2=0x7f01a0152aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 0x7f01a0152aa0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f019400e7c0 tx=0x7f019400ec80 comp rx=0 tx=0).stop 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 shutdown_connections 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0162100 0x7f01a00b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a00d2f00 0x7f01a00b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 --2- 172.21.15.175:0/3147853455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01a00d24f0 0x7f01a0152aa0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 >> 172.21.15.175:0/3147853455 conn(0x7f01a008dd50 msgr2=0x7f01a00b4380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.465+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 shutdown_connections 2024-09-21T07:45:42.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.466+0000 7f01a7000640 1 -- 172.21.15.175:0/3147853455 wait complete. 2024-09-21T07:45:42.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 7ecabd95-42fd-4a37-9c85-7a603ad4f623 2 2024-09-21T07:45:42.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 -- 172.21.15.175:0/4063831584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b98b0 msgr2=0x7f29600b4770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.711 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 --2- 172.21.15.175:0/4063831584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b98b0 0x7f29600b4770 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f296008cac0 tx=0x7f295002a540 comp rx=0 tx=0).stop 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 -- 172.21.15.175:0/4063831584 shutdown_connections 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 --2- 172.21.15.175:0/4063831584 >> v2:172.21.15.88:3300/0 conn(0x7f29600cf6a0 0x7f29600d0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 --2- 172.21.15.175:0/4063831584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600b4cb0 0x7f29600cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 --2- 172.21.15.175:0/4063831584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b98b0 0x7f29600b4770 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 -- 172.21.15.175:0/4063831584 >> 172.21.15.175:0/4063831584 conn(0x7f296008bfe0 msgr2=0x7f29600d4600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 -- 172.21.15.175:0/4063831584 shutdown_connections 2024-09-21T07:45:42.712 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.709+0000 7f2965600640 1 -- 172.21.15.175:0/4063831584 wait complete. 2024-09-21T07:45:42.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.711+0000 7f2965600640 1 Processor -- start 2024-09-21T07:45:42.713 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.712+0000 7f2965600640 1 -- start start 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2965600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b4cb0 0x7f29601e1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2965600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29600b98b0 0x7f29601db230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2965600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600cf6a0 0x7f29601db770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2965600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29600d14c0 con 0x7f29600b98b0 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2965600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29600d1340 con 0x7f29600b4cb0 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2965600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29600d1640 con 0x7f29600cf6a0 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f295fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b4cb0 0x7f29601e1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2964c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600cf6a0 0x7f29601db770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f295fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b4cb0 0x7f29601e1700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35382/0 (socket says 172.21.15.175:35382) 2024-09-21T07:45:42.716 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f295fe00640 1 -- 172.21.15.175:0/612005103 learned_addr learned my addr 172.21.15.175:0/612005103 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.713+0000 7f2957e00640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29600b98b0 0x7f29601db230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2964c00640 1 -- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b4cb0 msgr2=0x7f29601e1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2964c00640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b4cb0 0x7f29601e1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2964c00640 1 -- 172.21.15.175:0/612005103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29600b98b0 msgr2=0x7f29601db230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2964c00640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29600b98b0 0x7f29601db230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2964c00640 1 -- 172.21.15.175:0/612005103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29601de550 con 0x7f29600cf6a0 2024-09-21T07:45:42.717 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2964c00640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600cf6a0 0x7f29601db770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f295800e9e0 tx=0x7f295800eea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f295800cdb0 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2958004b20 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2958005020 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.714+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29601e3370 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.715+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f29601e3930 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.715+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f2958005550 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.715+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f2958018dd0 con 0x7f29600cf6a0 2024-09-21T07:45:42.718 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.716+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29600b4770 con 0x7f29600cf6a0 2024-09-21T07:45:42.720 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.718+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f29580058a0 con 0x7f29600cf6a0 2024-09-21T07:45:42.793 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.791+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7ecabd95-42fd-4a37-9c85-7a603ad4f623", "id": 2} v 0) -- 0x7f29601e3090 con 0x7f29600cf6a0 2024-09-21T07:45:42.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.796+0000 7f295cc00640 1 -- 172.21.15.175:0/612005103 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7ecabd95-42fd-4a37-9c85-7a603ad4f623", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f2958018470 con 0x7f29600cf6a0 2024-09-21T07:45:42.798 INFO:teuthology.orchestra.run.smithi175.stdout:2 2024-09-21T07:45:42.799 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600cf6a0 msgr2=0x7f29601db770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600cf6a0 0x7f29601db770 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f295800e9e0 tx=0x7f295800eea0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 shutdown_connections 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29600cf6a0 0x7f29601db770 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29600b98b0 0x7f29601db230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 --2- 172.21.15.175:0/612005103 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29600b4cb0 0x7f29601e1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 >> 172.21.15.175:0/612005103 conn(0x7f296008bfe0 msgr2=0x7f29600b88a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 shutdown_connections 2024-09-21T07:45:42.800 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:42.797+0000 7f2965600640 1 -- 172.21.15.175:0/612005103 wait complete. 2024-09-21T07:45:42.820 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 9f106eba-bb27-41e1-8bc9-78e321c28923 3 2024-09-21T07:45:43.041 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 -- 172.21.15.175:0/3678655968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0d4100 msgr2=0x7fdb9c0b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 --2- 172.21.15.175:0/3678655968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0d4100 0x7fdb9c0b5fa0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c0098e0 tx=0x7fdb8c02a540 comp rx=0 tx=0).stop 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 -- 172.21.15.175:0/3678655968 shutdown_connections 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 --2- 172.21.15.175:0/3678655968 >> v2:172.21.15.175:3300/0 conn(0x7fdb9c0b98e0 0x7fdb9c0b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 --2- 172.21.15.175:0/3678655968 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0d4100 0x7fdb9c0b5fa0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 -- 172.21.15.175:0/3678655968 >> 172.21.15.175:0/3678655968 conn(0x7fdb9c0d4520 msgr2=0x7fdb9c0d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 -- 172.21.15.175:0/3678655968 shutdown_connections 2024-09-21T07:45:43.042 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.039+0000 7fdba1200640 1 -- 172.21.15.175:0/3678655968 wait complete. 2024-09-21T07:45:43.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.041+0000 7fdba1200640 1 Processor -- start 2024-09-21T07:45:43.043 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.041+0000 7fdba1200640 1 -- start start 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdba1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0b98e0 0x7fdb9c0c9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdba1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 0x7fdb9c1f5a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdba1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb9c1f5f40 0x7fdb9c15dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdba1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06f760 con 0x7fdb9c0b98e0 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdba1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb9c06f5e0 con 0x7fdb9c1f5f40 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdba1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb9c06f8e0 con 0x7fdb9c0ced80 2024-09-21T07:45:43.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdb9aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 0x7fdb9c1f5a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdb9be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb9c1f5f40 0x7fdb9c15dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdb9aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 0x7fdb9c1f5a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:53460/0 (socket says 172.21.15.175:53460) 2024-09-21T07:45:43.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdb9aa00640 1 -- 172.21.15.175:0/3460115875 learned_addr learned my addr 172.21.15.175:0/3460115875 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:43.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.043+0000 7fdb9b400640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0b98e0 0x7fdb9c0c9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9aa00640 1 -- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb9c1f5f40 msgr2=0x7fdb9c15dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9aa00640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb9c1f5f40 0x7fdb9c15dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9aa00640 1 -- 172.21.15.175:0/3460115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0b98e0 msgr2=0x7fdb9c0c9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9aa00640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0b98e0 0x7fdb9c0c9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9aa00640 1 -- 172.21.15.175:0/3460115875 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9c0cb800 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9b400640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0b98e0 0x7fdb9c0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9be00640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb9c1f5f40 0x7fdb9c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb9aa00640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 0x7fdb9c1f5a00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800e9f0 tx=0x7fdb8800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800cd80 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb88004020 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb88004470 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9c0cbae0 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.044+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb9c0cc0a0 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.045+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fdb88003980 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.045+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fdb88009490 con 0x7fdb9c0ced80 2024-09-21T07:45:43.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.045+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb9c0cfa60 con 0x7fdb9c0ced80 2024-09-21T07:45:43.049 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.048+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fdb88003c70 con 0x7fdb9c0ced80 2024-09-21T07:45:43.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.121+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "9f106eba-bb27-41e1-8bc9-78e321c28923", "id": 3} v 0) -- 0x7fdb9c0cb3b0 con 0x7fdb9c0ced80 2024-09-21T07:45:43.127 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.125+0000 7fdb93400640 1 -- 172.21.15.175:0/3460115875 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "9f106eba-bb27-41e1-8bc9-78e321c28923", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fdb88018bb0 con 0x7fdb9c0ced80 2024-09-21T07:45:43.127 INFO:teuthology.orchestra.run.smithi175.stdout:3 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 msgr2=0x7fdb9c1f5a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 0x7fdb9c1f5a00 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800e9f0 tx=0x7fdb8800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 shutdown_connections 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb9c0ced80 0x7fdb9c1f5a00 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb9c1f5f40 0x7fdb9c15dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 --2- 172.21.15.175:0/3460115875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9c0b98e0 0x7fdb9c0c9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 >> 172.21.15.175:0/3460115875 conn(0x7fdb9c0d4520 msgr2=0x7fdb9c0d1790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 shutdown_connections 2024-09-21T07:45:43.129 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.127+0000 7fdba1200640 1 -- 172.21.15.175:0/3460115875 wait complete. 2024-09-21T07:45:43.151 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 7cc1d1f3-890e-4a86-9cbd-9376658e53b3 4 2024-09-21T07:45:43.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 -- 172.21.15.175:0/3188742386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00d4100 msgr2=0x7f75b00b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 --2- 172.21.15.175:0/3188742386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00d4100 0x7f75b00b5fa0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f75b008c940 tx=0x7f75a002a4b0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 -- 172.21.15.175:0/3188742386 shutdown_connections 2024-09-21T07:45:43.372 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 --2- 172.21.15.175:0/3188742386 >> v2:172.21.15.88:3300/0 conn(0x7f75b00ced80 0x7f75b00cf190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 --2- 172.21.15.175:0/3188742386 >> v2:172.21.15.175:3300/0 conn(0x7f75b00b98e0 0x7f75b00b64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 --2- 172.21.15.175:0/3188742386 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00d4100 0x7f75b00b5fa0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 -- 172.21.15.175:0/3188742386 >> 172.21.15.175:0/3188742386 conn(0x7f75b00d4520 msgr2=0x7f75b00d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 -- 172.21.15.175:0/3188742386 shutdown_connections 2024-09-21T07:45:43.373 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.370+0000 7f75af200640 1 -- 172.21.15.175:0/3188742386 wait complete. 2024-09-21T07:45:43.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.372+0000 7f75af200640 1 Processor -- start 2024-09-21T07:45:43.374 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.372+0000 7f75af200640 1 -- start start 2024-09-21T07:45:43.376 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75af200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b00b98e0 0x7f75b015f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75af200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 0x7f75b00ccbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75af200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 0x7f75b00cd120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75af200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00cf850 con 0x7f75b00b98e0 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75af200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f75b00cf6d0 con 0x7f75b00a1960 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75af200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f75b00cf9d0 con 0x7f75b00ced80 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 0x7f75b00ccbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ae800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 0x7f75b00cd120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 0x7f75b00ccbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:53470/0 (socket says 172.21.15.175:53470) 2024-09-21T07:45:43.377 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 -- 172.21.15.175:0/3514161964 learned_addr learned my addr 172.21.15.175:0/3514161964 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ae800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 0x7f75b00cd120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35418/0 (socket says 172.21.15.175:35418) 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ade00640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b00b98e0 0x7f75b015f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 -- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 msgr2=0x7f75b00cd120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 0x7f75b00cd120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 -- 172.21.15.175:0/3514161964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b00b98e0 msgr2=0x7f75b015f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.374+0000 7f75ad400640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b00b98e0 0x7f75b015f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75ad400640 1 -- 172.21.15.175:0/3514161964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75b00cdf60 con 0x7f75b00ced80 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75ae800640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 0x7f75b00cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:43.378 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75ad400640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 0x7f75b00ccbe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e9f0 tx=0x7f759c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:43.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c00cd80 con 0x7f75b00ced80 2024-09-21T07:45:43.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f759c004020 con 0x7f75b00ced80 2024-09-21T07:45:43.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75b00ce240 con 0x7f75b00ced80 2024-09-21T07:45:43.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f759c004470 con 0x7f75b00ced80 2024-09-21T07:45:43.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.375+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f75b00c8180 con 0x7f75b00ced80 2024-09-21T07:45:43.379 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.376+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f759c003980 con 0x7f75b00ced80 2024-09-21T07:45:43.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.376+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f759c003c70 con 0x7f75b00ced80 2024-09-21T07:45:43.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.376+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75b00b9700 con 0x7f75b00ced80 2024-09-21T07:45:43.380 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.379+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f75b00b9700 con 0x7f75b00ced80 2024-09-21T07:45:43.453 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.451+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "7cc1d1f3-890e-4a86-9cbd-9376658e53b3", "id": 4} v 0) -- 0x7f75b00b6df0 con 0x7f75b00ced80 2024-09-21T07:45:43.459 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.457+0000 7f75a6000640 1 -- 172.21.15.175:0/3514161964 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7cc1d1f3-890e-4a86-9cbd-9376658e53b3", "id": 4}]=0 v7) ==== 112+0+2 (secure 0 0 0) 0x7f759c021440 con 0x7f75b00ced80 2024-09-21T07:45:43.459 INFO:teuthology.orchestra.run.smithi175.stdout:4 2024-09-21T07:45:43.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 msgr2=0x7f75b00ccbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 0x7f75b00ccbe0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f759c00e9f0 tx=0x7f759c00eeb0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 shutdown_connections 2024-09-21T07:45:43.460 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f75b00ced80 0x7f75b00ccbe0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f75b00a1960 0x7f75b00cd120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 --2- 172.21.15.175:0/3514161964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75b00b98e0 0x7f75b015f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 >> 172.21.15.175:0/3514161964 conn(0x7f75b00d4520 msgr2=0x7f75b00b1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 shutdown_connections 2024-09-21T07:45:43.461 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.458+0000 7f75af200640 1 -- 172.21.15.175:0/3514161964 wait complete. 2024-09-21T07:45:43.480 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new ab03d263-cece-4d04-aeff-abb758a02353 5 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 -- 172.21.15.175:0/889457629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51400d4100 msgr2=0x7f51400b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 --2- 172.21.15.175:0/889457629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51400d4100 0x7f51400b5fa0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f51340098e0 tx=0x7f513402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 -- 172.21.15.175:0/889457629 shutdown_connections 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 --2- 172.21.15.175:0/889457629 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51400d4100 0x7f51400b5fa0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 -- 172.21.15.175:0/889457629 >> 172.21.15.175:0/889457629 conn(0x7f51400d4520 msgr2=0x7f51400d4930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 -- 172.21.15.175:0/889457629 shutdown_connections 2024-09-21T07:45:43.700 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.697+0000 7f5145e00640 1 -- 172.21.15.175:0/889457629 wait complete. 2024-09-21T07:45:43.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.700+0000 7f5145e00640 1 Processor -- start 2024-09-21T07:45:43.702 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.700+0000 7f5145e00640 1 -- start start 2024-09-21T07:45:43.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5145e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 0x7f51400ca940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5145e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51400b98e0 0x7f51400c8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5145e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51400c8b80 0x7f51401ea860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:43.704 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5145e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f51400b64e0 con 0x7f51400b98e0 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5145e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51400b3bb0 con 0x7f514006f5e0 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5145e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f51400b6660 con 0x7f51400c8b80 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.701+0000 7f5144a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 0x7f51400ca940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5145400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51400c8b80 0x7f51401ea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 0x7f51400ca940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35446/0 (socket says 172.21.15.175:35446) 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 -- 172.21.15.175:0/11372007 learned_addr learned my addr 172.21.15.175:0/11372007 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f513fe00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51400b98e0 0x7f51400c8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 -- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51400c8b80 msgr2=0x7f51401ea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51400c8b80 0x7f51401ea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 -- 172.21.15.175:0/11372007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51400b98e0 msgr2=0x7f51400c8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51400b98e0 0x7f51400c8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 -- 172.21.15.175:0/11372007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51400b7490 con 0x7f514006f5e0 2024-09-21T07:45:43.705 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5145400640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51400c8b80 0x7f51401ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:43.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.702+0000 7f5144a00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 0x7f51400ca940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5134009a00 tx=0x7f5134005040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:43.706 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f513403f070 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5134004050 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51340044a0 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51401f65b0 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51401f59a0 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5134042070 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.703+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1300+0+0 (secure 0 0 0) 0x7f5134031750 con 0x7f514006f5e0 2024-09-21T07:45:43.707 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.704+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51400d4100 con 0x7f514006f5e0 2024-09-21T07:45:43.708 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.706+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f5134004a70 con 0x7f514006f5e0 2024-09-21T07:45:43.782 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.780+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "ab03d263-cece-4d04-aeff-abb758a02353", "id": 5} v 0) -- 0x7f51400b48e0 con 0x7f514006f5e0 2024-09-21T07:45:43.786 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.784+0000 7f513cc00640 1 -- 172.21.15.175:0/11372007 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "ab03d263-cece-4d04-aeff-abb758a02353", "id": 5}]=0 v8) ==== 112+0+2 (secure 0 0 0) 0x7f5134004c50 con 0x7f514006f5e0 2024-09-21T07:45:43.787 INFO:teuthology.orchestra.run.smithi175.stdout:5 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.785+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 msgr2=0x7f51400ca940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.785+0000 7f5145e00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 0x7f51400ca940 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5134009a00 tx=0x7f5134005040 comp rx=0 tx=0).stop 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 shutdown_connections 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f51400c8b80 0x7f51401ea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f51400b98e0 0x7f51400c8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 --2- 172.21.15.175:0/11372007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f514006f5e0 0x7f51400ca940 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 >> 172.21.15.175:0/11372007 conn(0x7f51400d4520 msgr2=0x7f51401e7a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 shutdown_connections 2024-09-21T07:45:43.788 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:43.786+0000 7f5145e00640 1 -- 172.21.15.175:0/11372007 wait complete. 2024-09-21T07:45:43.810 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new dd7287f4-2961-40e0-ac15-cfd409cce208 6 2024-09-21T07:45:44.034 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 -- 172.21.15.175:0/592811475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63400cd000 msgr2=0x7f63400af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 --2- 172.21.15.175:0/592811475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63400cd000 0x7f63400af0d0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6338005fd0 tx=0x7f633802aa90 comp rx=0 tx=0).stop 2024-09-21T07:45:44.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 -- 172.21.15.175:0/592811475 shutdown_connections 2024-09-21T07:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 --2- 172.21.15.175:0/592811475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63400cd000 0x7f63400af0d0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 --2- 172.21.15.175:0/592811475 >> v2:172.21.15.88:3300/0 conn(0x7f6340156ca0 0x7f63400ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.036 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 -- 172.21.15.175:0/592811475 >> 172.21.15.175:0/592811475 conn(0x7f634006d620 msgr2=0x7f63400b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 -- 172.21.15.175:0/592811475 shutdown_connections 2024-09-21T07:45:44.037 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.031+0000 7f6344a00640 1 -- 172.21.15.175:0/592811475 wait complete. 2024-09-21T07:45:44.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.033+0000 7f6344a00640 1 Processor -- start 2024-09-21T07:45:44.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.033+0000 7f6344a00640 1 -- start start 2024-09-21T07:45:44.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f6344a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63400b3f30 0x7f6340095650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f6344a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63400cd000 0x7f634015e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f6344a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 0x7f63400976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f6344a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63400af9a0 con 0x7f63400cd000 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f6344a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63400af820 con 0x7f63400b3f30 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f6344a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63400afb20 con 0x7f6340156ca0 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 0x7f63400976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63400b3f30 0x7f6340095650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.046 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 0x7f63400976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.175:53562/0 (socket says 172.21.15.175:53562) 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633fe00640 1 -- 172.21.15.175:0/647406028 learned_addr learned my addr 172.21.15.175:0/647406028 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633ea00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63400cd000 0x7f634015e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633fe00640 1 -- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63400b3f30 msgr2=0x7f6340095650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.035+0000 7f633fe00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63400b3f30 0x7f6340095650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f633fe00640 1 -- 172.21.15.175:0/647406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63400cd000 msgr2=0x7f634015e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f633fe00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63400cd000 0x7f634015e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f633fe00640 1 -- 172.21.15.175:0/647406028 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634015e100 con 0x7f6340156ca0 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f633f400640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63400b3f30 0x7f6340095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f633fe00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 0x7f63400976b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6338009d90 tx=0x7f6338003f10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6338044070 con 0x7f6340156ca0 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f633802e7f0 con 0x7f6340156ca0 2024-09-21T07:45:44.047 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63380044b0 con 0x7f6340156ca0 2024-09-21T07:45:44.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6340145ed0 con 0x7f6340156ca0 2024-09-21T07:45:44.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.036+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f6340095b90 con 0x7f6340156ca0 2024-09-21T07:45:44.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.037+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f6338047070 con 0x7f6340156ca0 2024-09-21T07:45:44.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.037+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6340098560 con 0x7f6340156ca0 2024-09-21T07:45:44.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.038+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(8..8 src has 1..8) ==== 1415+0+0 (secure 0 0 0) 0x7f6338073090 con 0x7f6340156ca0 2024-09-21T07:45:44.048 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.040+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f6338032030 con 0x7f6340156ca0 2024-09-21T07:45:44.114 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.111+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd new", "uuid": "dd7287f4-2961-40e0-ac15-cfd409cce208", "id": 6} v 0) -- 0x7f63400af0d0 con 0x7f6340156ca0 2024-09-21T07:45:44.121 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.119+0000 7f6337400640 1 -- 172.21.15.175:0/647406028 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "dd7287f4-2961-40e0-ac15-cfd409cce208", "id": 6}]=0 v9) ==== 112+0+2 (secure 0 0 0) 0x7f633802d3c0 con 0x7f6340156ca0 2024-09-21T07:45:44.121 INFO:teuthology.orchestra.run.smithi175.stdout:6 2024-09-21T07:45:44.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 msgr2=0x7f63400976b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 0x7f63400976b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6338009d90 tx=0x7f6338003f10 comp rx=0 tx=0).stop 2024-09-21T07:45:44.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 shutdown_connections 2024-09-21T07:45:44.123 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6340156ca0 0x7f63400976b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63400cd000 0x7f634015e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 --2- 172.21.15.175:0/647406028 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63400b3f30 0x7f6340095650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 >> 172.21.15.175:0/647406028 conn(0x7f634006d620 msgr2=0x7f634015b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 shutdown_connections 2024-09-21T07:45:44.124 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.120+0000 7f6344a00640 1 -- 172.21.15.175:0/647406028 wait complete. 2024-09-21T07:45:44.142 DEBUG:teuthology.orchestra.run.smithi175:> sudo ceph --cluster ceph osd new 0e813d39-967a-403e-8dea-c31d2392cd7a 7 2024-09-21T07:45:44.354 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 -- 172.21.15.175:0/232285930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450073c50 msgr2=0x7fb4500cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.355 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 --2- 172.21.15.175:0/232285930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450073c50 0x7fb4500cd610 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb4501464e0 tx=0x7fb44802ab50 comp rx=0 tx=0).stop 2024-09-21T07:45:44.357 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 -- 172.21.15.175:0/232285930 shutdown_connections 2024-09-21T07:45:44.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 --2- 172.21.15.175:0/232285930 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450073c50 0x7fb4500cd610 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 --2- 172.21.15.175:0/232285930 >> v2:172.21.15.88:3300/0 conn(0x7fb450145550 0x7fb45006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 -- 172.21.15.175:0/232285930 >> 172.21.15.175:0/232285930 conn(0x7fb45006d1c0 msgr2=0x7fb45006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 -- 172.21.15.175:0/232285930 shutdown_connections 2024-09-21T07:45:44.358 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.351+0000 7fb456400640 1 -- 172.21.15.175:0/232285930 wait complete. 2024-09-21T07:45:44.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.354+0000 7fb456400640 1 Processor -- start 2024-09-21T07:45:44.363 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.354+0000 7fb456400640 1 -- start start 2024-09-21T07:45:44.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb456400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 0x7fb4501596b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb456400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb450145a40 0x7fb4500b2030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb456400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45012b5b0 0x7fb4500b2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb456400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4500ce180 con 0x7fb450145a40 2024-09-21T07:45:44.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb456400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4500ce000 con 0x7fb450145550 2024-09-21T07:45:44.364 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb456400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4500ce300 con 0x7fb45012b5b0 2024-09-21T07:45:44.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb44f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 0x7fb4501596b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb44fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45012b5b0 0x7fb4500b2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb44f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 0x7fb4501596b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.175:35544/0 (socket says 172.21.15.175:35544) 2024-09-21T07:45:44.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb44f400640 1 -- 172.21.15.175:0/3103953546 learned_addr learned my addr 172.21.15.175:0/3103953546 (peer_addr_for_me v2:172.21.15.175:0/0) 2024-09-21T07:45:44.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.355+0000 7fb44ea00640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb450145a40 0x7fb4500b2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.365 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb44f400640 1 -- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45012b5b0 msgr2=0x7fb4500b2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb44f400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45012b5b0 0x7fb4500b2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb44f400640 1 -- 172.21.15.175:0/3103953546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb450145a40 msgr2=0x7fb4500b2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb44f400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb450145a40 0x7fb4500b2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb44f400640 1 -- 172.21.15.175:0/3103953546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4500b2ab0 con 0x7fb450145550 2024-09-21T07:45:44.366 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb44f400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 0x7fb4501596b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb44000d770 tx=0x7fb44000dc30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:44.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb440017070 con 0x7fb450145550 2024-09-21T07:45:44.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb440009d20 con 0x7fb450145550 2024-09-21T07:45:44.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4500b1270 con 0x7fb450145550 2024-09-21T07:45:44.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.356+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4400106c0 con 0x7fb450145550 2024-09-21T07:45:44.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.357+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4500b1830 con 0x7fb450145550 2024-09-21T07:45:44.367 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.358+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4500ce820 con 0x7fb450145550 2024-09-21T07:45:44.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.358+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7fb44001a070 con 0x7fb450145550 2024-09-21T07:45:44.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.358+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1530+0+0 (secure 0 0 0) 0x7fb440052590 con 0x7fb450145550 2024-09-21T07:45:44.368 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.360+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7fb44001bdc0 con 0x7fb450145550 2024-09-21T07:45:44.433 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.430+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd new", "uuid": "0e813d39-967a-403e-8dea-c31d2392cd7a", "id": 7} v 0) -- 0x7fb450160bf0 con 0x7fb450145550 2024-09-21T07:45:44.443 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.440+0000 7fb447400640 1 -- 172.21.15.175:0/3103953546 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0e813d39-967a-403e-8dea-c31d2392cd7a", "id": 7}]=0 v10) ==== 112+0+2 (secure 0 0 0) 0x7fb440010d30 con 0x7fb450145550 2024-09-21T07:45:44.444 INFO:teuthology.orchestra.run.smithi175.stdout:7 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 msgr2=0x7fb4501596b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 0x7fb4501596b0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb44000d770 tx=0x7fb44000dc30 comp rx=0 tx=0).stop 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 shutdown_connections 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb45012b5b0 0x7fb4500b2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb450145a40 0x7fb4500b2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 --2- 172.21.15.175:0/3103953546 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb450145550 0x7fb4501596b0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.447 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 >> 172.21.15.175:0/3103953546 conn(0x7fb45006d1c0 msgr2=0x7fb450068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 shutdown_connections 2024-09-21T07:45:44.448 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:45:44.442+0000 7fb456400640 1 -- 172.21.15.175:0/3103953546 wait complete. 2024-09-21T07:45:44.467 INFO:tasks.ceph.osd.0:Restarting daemon 2024-09-21T07:45:44.467 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-09-21T07:45:44.471 INFO:tasks.ceph.osd.0:Started 2024-09-21T07:45:44.471 INFO:tasks.ceph.osd.1:Restarting daemon 2024-09-21T07:45:44.471 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-09-21T07:45:44.475 INFO:tasks.ceph.osd.1:Started 2024-09-21T07:45:44.475 INFO:tasks.ceph.osd.2:Restarting daemon 2024-09-21T07:45:44.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-09-21T07:45:44.479 INFO:tasks.ceph.osd.2:Started 2024-09-21T07:45:44.479 INFO:tasks.ceph.osd.3:Restarting daemon 2024-09-21T07:45:44.479 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-09-21T07:45:44.483 INFO:tasks.ceph.osd.3:Started 2024-09-21T07:45:44.483 INFO:tasks.ceph.osd.4:Restarting daemon 2024-09-21T07:45:44.484 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-09-21T07:45:44.488 INFO:tasks.ceph.osd.4:Started 2024-09-21T07:45:44.488 INFO:tasks.ceph.osd.5:Restarting daemon 2024-09-21T07:45:44.488 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-09-21T07:45:44.491 INFO:tasks.ceph.osd.5:Started 2024-09-21T07:45:44.491 INFO:tasks.ceph.osd.6:Restarting daemon 2024-09-21T07:45:44.491 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-09-21T07:45:44.495 INFO:tasks.ceph.osd.6:Started 2024-09-21T07:45:44.495 INFO:tasks.ceph.osd.7:Restarting daemon 2024-09-21T07:45:44.495 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-09-21T07:45:44.500 INFO:tasks.ceph.osd.7:Started 2024-09-21T07:45:44.500 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.829+0000 7f1c1d600640 1 -- 172.21.15.88:0/2390972976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 msgr2=0x7f1c180b58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.829+0000 7f1c1d600640 1 --2- 172.21.15.88:0/2390972976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c180b58b0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1c08009900 tx=0x7f1c0802a610 comp rx=0 tx=0).stop 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.829+0000 7f1c1d600640 1 -- 172.21.15.88:0/2390972976 shutdown_connections 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.829+0000 7f1c1d600640 1 --2- 172.21.15.88:0/2390972976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180d0e70 0x7f1c180d1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.829+0000 7f1c1d600640 1 --2- 172.21.15.88:0/2390972976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c180b58b0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.829+0000 7f1c1d600640 1 -- 172.21.15.88:0/2390972976 >> 172.21.15.88:0/2390972976 conn(0x7f1c1809d820 msgr2=0x7f1c1808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.830+0000 7f1c1d600640 1 -- 172.21.15.88:0/2390972976 shutdown_connections 2024-09-21T07:45:44.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.830+0000 7f1c1d600640 1 -- 172.21.15.88:0/2390972976 wait complete. 2024-09-21T07:45:44.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.832+0000 7f1c1d600640 1 Processor -- start 2024-09-21T07:45:44.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.832+0000 7f1c1d600640 1 -- start start 2024-09-21T07:45:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c1d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180b3e70 0x7f1c1815e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c1d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c1815c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c1d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c180d0e70 0x7f1c180c94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c1d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c180cfcb0 con 0x7f1c180b98d0 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c1d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c180cfb30 con 0x7f1c180d0e70 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c1d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c180cfe30 con 0x7f1c180b3e70 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.833+0000 7f1c17400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c1815c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180b3e70 0x7f1c1815e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c1815c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53152/0 (socket says 172.21.15.88:53152) 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17400640 1 -- 172.21.15.88:0/904739419 learned_addr learned my addr 172.21.15.88:0/904739419 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c1cc00640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c180d0e70 0x7f1c180c94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 -- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c180d0e70 msgr2=0x7f1c180c94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c180d0e70 0x7f1c180c94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 -- 172.21.15.88:0/904739419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 msgr2=0x7f1c1815c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c1815c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 -- 172.21.15.88:0/904739419 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c1815cda0 con 0x7f1c180b3e70 2024-09-21T07:45:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17400640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c1815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:44.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.834+0000 7f1c17e00640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180b3e70 0x7f1c1815e030 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c180d2290 tx=0x7f1c08005770 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:44.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.835+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c0803e030 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.835+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c08003a70 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.835+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c181f6d50 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.835+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c08035040 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.835+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c181f96b0 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.836+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c181f6f20 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.836+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 2) ==== 50742+0+0 (secure 0 0 0) 0x7f1c08041030 con 0x7f1c180b3e70 2024-09-21T07:45:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.836+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7f1c08072950 con 0x7f1c180b3e70 2024-09-21T07:45:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.838+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+75856 (secure 0 0 0) 0x7f1c1815cda0 con 0x7f1c180b3e70 2024-09-21T07:45:44.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.883+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mgrmap(e 3) ==== 101279+0+0 (secure 0 0 0) 0x7f1c08004cc0 con 0x7f1c180b3e70 2024-09-21T07:45:44.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.883+0000 7f1c0fe00640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bf8078ef0 0x7f1bf807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:44.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.884+0000 7f1c17400640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bf8078ef0 0x7f1bf807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.885+0000 7f1c17400640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bf8078ef0 0x7f1bf807b310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1c181f6d00 tx=0x7f1c04009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:44.888 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:45:44.886+0000 7ff2c7e00640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-09-21T07:45:44.888 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:45:44.886+0000 7ff2c7e00640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-09-21T07:45:44.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.904+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1c180d02c0 con 0x7f1c180b3e70 2024-09-21T07:45:44.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.905+0000 7f1c0fe00640 1 -- 172.21.15.88:0/904739419 <== mon.2 v2:172.21.15.175:3301/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7f1c08058df0 con 0x7f1c180b3e70 2024-09-21T07:45:44.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:45:44.907 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":10,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:45:44.433149+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bf8078ef0 msgr2=0x7f1bf807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bf8078ef0 0x7f1bf807b310 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f1c181f6d00 tx=0x7f1c04009040 comp rx=0 tx=0).stop 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180b3e70 msgr2=0x7f1c1815e030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180b3e70 0x7f1c1815e030 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1c180d2290 tx=0x7f1c08005770 comp rx=0 tx=0).stop 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 shutdown_connections 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bf8078ef0 0x7f1bf807b310 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c180d0e70 0x7f1c180c94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c180b98d0 0x7f1c1815c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 --2- 172.21.15.88:0/904739419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c180b3e70 0x7f1c1815e030 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:44.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.906+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 >> 172.21.15.88:0/904739419 conn(0x7f1c1809d820 msgr2=0x7f1c18068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:44.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.907+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 shutdown_connections 2024-09-21T07:45:44.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:45:44.907+0000 7f1c1d600640 1 -- 172.21.15.88:0/904739419 wait complete. 2024-09-21T07:45:44.929 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-09-21T07:45:44.929 INFO:tasks.ceph:Waiting for OSDs to come up 2024-09-21T07:45:45.735 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:45.875 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T07:45:45.873+0000 7f47d3c18640 -1 Falling back to public interface 2024-09-21T07:45:45.946 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 -- 172.21.15.88:0/2179237181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 msgr2=0x7ff4f406f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:45.946 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/2179237181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f406f2e0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4070140 tx=0x7ff4ec02aa30 comp rx=0 tx=0).stop 2024-09-21T07:45:45.946 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 -- 172.21.15.88:0/2179237181 shutdown_connections 2024-09-21T07:45:45.946 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/2179237181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f406f2e0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:45.947 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/2179237181 >> v2:172.21.15.175:3300/0 conn(0x7ff4f40ca6d0 0x7ff4f415c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:45.947 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 -- 172.21.15.88:0/2179237181 >> 172.21.15.88:0/2179237181 conn(0x7ff4f40b1eb0 msgr2=0x7ff4f40b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:45.947 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 -- 172.21.15.88:0/2179237181 shutdown_connections 2024-09-21T07:45:45.947 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.945+0000 7ff4f9c00640 1 -- 172.21.15.88:0/2179237181 wait complete. 2024-09-21T07:45:45.948 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.947+0000 7ff4f9c00640 1 Processor -- start 2024-09-21T07:45:45.948 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.947+0000 7ff4f9c00640 1 -- start start 2024-09-21T07:45:45.951 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f415cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:45.951 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f40c6bd0 0x7ff4f40afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:45.951 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4f40ca6d0 0x7ff4f4153b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:45.951 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4073020 con 0x7ff4f406eed0 2024-09-21T07:45:45.951 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4f4072ea0 con 0x7ff4f40c6bd0 2024-09-21T07:45:45.951 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4f40731a0 con 0x7ff4f40ca6d0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f415cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f415cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58664/0 (socket says 172.21.15.88:58664) 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 -- 172.21.15.88:0/1621927860 learned_addr learned my addr 172.21.15.88:0/1621927860 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4eb400640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f40c6bd0 0x7ff4f40afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9200640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4f40ca6d0 0x7ff4f4153b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 -- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4f40ca6d0 msgr2=0x7ff4f4153b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4f40ca6d0 0x7ff4f4153b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 -- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f40c6bd0 msgr2=0x7ff4f40afe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f40c6bd0 0x7ff4f40afe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f3e00640 1 -- 172.21.15.88:0/1621927860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4f415d2f0 con 0x7ff4f406eed0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.949+0000 7ff4f9200640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4f40ca6d0 0x7ff4f4153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.950+0000 7ff4f3e00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f415cdb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c3a0 tx=0x7ff4e400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.950+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4017070 con 0x7ff4f406eed0 2024-09-21T07:45:45.952 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.950+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff4e40053c0 con 0x7ff4f406eed0 2024-09-21T07:45:45.953 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.950+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4f40b4270 con 0x7ff4f406eed0 2024-09-21T07:45:45.953 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.950+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff4e4003850 con 0x7ff4f406eed0 2024-09-21T07:45:45.953 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.950+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4f40b4830 con 0x7ff4f406eed0 2024-09-21T07:45:45.953 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.951+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4f406f2e0 con 0x7ff4f406eed0 2024-09-21T07:45:45.953 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.952+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7ff4e401a070 con 0x7ff4f406eed0 2024-09-21T07:45:45.954 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.953+0000 7ff4f0c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff4d0078b30 0x7ff4d007b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:45.954 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.953+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 1645+0+0 (secure 0 0 0) 0x7ff4e4098fd0 con 0x7ff4f406eed0 2024-09-21T07:45:45.955 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.954+0000 7ff4eb400640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff4d0078b30 0x7ff4d007b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:45.955 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.955+0000 7ff4eb400640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff4d0078b30 0x7ff4d007b040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40c7840 tx=0x7ff4dc00b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:45.959 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:45.958+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff4e4061de0 con 0x7ff4f406eed0 2024-09-21T07:45:46.093 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T07:45:46.091+0000 7fdccb218640 -1 Falling back to public interface 2024-09-21T07:45:46.100 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T07:45:46.098+0000 7ff4d1233640 -1 Falling back to public interface 2024-09-21T07:45:46.105 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T07:45:46.103+0000 7efc52218640 -1 Falling back to public interface 2024-09-21T07:45:46.116 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T07:45:46.115+0000 7f039d018640 -1 Falling back to public interface 2024-09-21T07:45:46.145 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.144+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff4f40cf650 con 0x7ff4f406eed0 2024-09-21T07:45:46.145 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.144+0000 7ff4f0c00640 1 -- 172.21.15.88:0/1621927860 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) ==== 74+0+7058 (secure 0 0 0) 0x7ff4e4066810 con 0x7ff4f406eed0 2024-09-21T07:45:46.146 INFO:teuthology.misc.health.smithi088.stdout: 2024-09-21T07:45:46.146 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":10,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:45:44.433149+0000","last_up_change":"0.000000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:46.148 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff4d0078b30 msgr2=0x7ff4d007b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:46.148 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff4d0078b30 0x7ff4d007b040 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff4f40c7840 tx=0x7ff4dc00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:45:46.148 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 msgr2=0x7ff4f415cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f415cdb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff4e400c3a0 tx=0x7ff4e400c860 comp rx=0 tx=0).stop 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 shutdown_connections 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff4d0078b30 0x7ff4d007b040 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4f40ca6d0 0x7ff4f4153b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4f40c6bd0 0x7ff4f40afe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 --2- 172.21.15.88:0/1621927860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4f406eed0 0x7ff4f415cdb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 >> 172.21.15.88:0/1621927860 conn(0x7ff4f40b1eb0 msgr2=0x7ff4f40c5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 shutdown_connections 2024-09-21T07:45:46.149 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:46.147+0000 7ff4f9c00640 1 -- 172.21.15.88:0/1621927860 wait complete. 2024-09-21T07:45:46.166 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-09-21T07:45:46.199 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T07:45:46.198+0000 7f5542a19640 -1 Falling back to public interface 2024-09-21T07:45:46.199 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T07:45:46.198+0000 7f0531618640 -1 Falling back to public interface 2024-09-21T07:45:46.452 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T07:45:46.451+0000 7f0e6641e640 -1 Falling back to public interface 2024-09-21T07:45:49.625 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T07:45:49.623+0000 7f47d3c18640 -1 osd.7 0 log_to_monitors true 2024-09-21T07:45:50.019 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T07:45:50.017+0000 7fdccb218640 -1 osd.4 0 log_to_monitors true 2024-09-21T07:45:50.058 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T07:45:50.058+0000 7f039d018640 -1 osd.1 0 log_to_monitors true 2024-09-21T07:45:50.111 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T07:45:50.109+0000 7ff4d1233640 -1 osd.6 0 log_to_monitors true 2024-09-21T07:45:50.159 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T07:45:50.158+0000 7f5542a19640 -1 osd.0 0 log_to_monitors true 2024-09-21T07:45:50.185 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T07:45:50.184+0000 7f0531618640 -1 osd.2 0 log_to_monitors true 2024-09-21T07:45:50.276 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T07:45:50.274+0000 7efc52218640 -1 osd.5 0 log_to_monitors true 2024-09-21T07:45:50.590 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T07:45:50.589+0000 7f0e6641e640 -1 osd.3 0 log_to_monitors true 2024-09-21T07:45:51.828 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T07:45:51.826+0000 7f47cf000640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T07:45:51.998 INFO:tasks.ceph.osd.7.smithi175.stderr:2024-09-21T07:45:51.996+0000 7f47c4600640 -1 osd.7 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:52.971 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:45:53.252 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.249+0000 7f3452400640 1 -- 172.21.15.88:0/1476340609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444005c60 msgr2=0x7f3444006660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:53.252 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.249+0000 7f3452400640 1 --2- 172.21.15.88:0/1476340609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444005c60 0x7f3444006660 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3440007350 tx=0x7f344002b1a0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.252 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.250+0000 7f3452400640 1 -- 172.21.15.88:0/1476340609 shutdown_connections 2024-09-21T07:45:53.252 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.250+0000 7f3452400640 1 --2- 172.21.15.88:0/1476340609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444005c60 0x7f3444006660 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.252 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.250+0000 7f3452400640 1 --2- 172.21.15.88:0/1476340609 >> v2:172.21.15.88:3300/0 conn(0x7f3444036b40 0x7f34440df180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.253 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.250+0000 7f3452400640 1 -- 172.21.15.88:0/1476340609 >> 172.21.15.88:0/1476340609 conn(0x7f3444051420 msgr2=0x7f3444051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:53.253 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.250+0000 7f3452400640 1 -- 172.21.15.88:0/1476340609 shutdown_connections 2024-09-21T07:45:53.253 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.250+0000 7f3452400640 1 -- 172.21.15.88:0/1476340609 wait complete. 2024-09-21T07:45:53.254 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.252+0000 7f3452400640 1 Processor -- start 2024-09-21T07:45:53.254 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.252+0000 7f3452400640 1 -- start start 2024-09-21T07:45:53.257 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.255+0000 7f3452400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 0x7f34440ddf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:53.257 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.255+0000 7f3452400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444009ba0 0x7f34440304c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:53.258 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.255+0000 7f3451000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 0x7f34440ddf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:53.258 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.255+0000 7f3452400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3444030a00 0x7f3444030e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:53.258 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.255+0000 7f3451000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 0x7f34440ddf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58716/0 (socket says 172.21.15.88:58716) 2024-09-21T07:45:53.258 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451000640 1 -- 172.21.15.88:0/1669563846 learned_addr learned my addr 172.21.15.88:0/1669563846 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:45:53.258 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3444006ba0 con 0x7f3444005c60 2024-09-21T07:45:53.258 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f344400be40 con 0x7f3444030a00 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3444006d20 con 0x7f3444009ba0 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f344be00640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444009ba0 0x7f34440304c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451a00640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3444030a00 0x7f3444030e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451a00640 1 -- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444009ba0 msgr2=0x7f34440304c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451a00640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444009ba0 0x7f34440304c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451a00640 1 -- 172.21.15.88:0/1669563846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 msgr2=0x7f34440ddf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451a00640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 0x7f34440ddf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451a00640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3444031350 con 0x7f3444030a00 2024-09-21T07:45:53.259 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.256+0000 7f3451000640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 0x7f34440ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:45:53.260 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.257+0000 7f3451a00640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3444030a00 0x7f3444030e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34440e3700 tx=0x7f3440003b90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:53.260 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.257+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344002bae0 con 0x7f3444030a00 2024-09-21T07:45:53.260 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.258+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3440007c50 con 0x7f3444030a00 2024-09-21T07:45:53.260 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.258+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f344000a330 con 0x7f3444030a00 2024-09-21T07:45:53.260 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.258+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34440deb00 con 0x7f3444030a00 2024-09-21T07:45:53.260 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.258+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f344416f7b0 con 0x7f3444030a00 2024-09-21T07:45:53.261 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.260+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f344416f520 con 0x7f3444030a00 2024-09-21T07:45:53.266 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.262+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 4) ==== 101385+0+0 (secure 0 0 0) 0x7f3440044060 con 0x7f3444030a00 2024-09-21T07:45:53.266 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.264+0000 7f3448c00640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f342c078b00 0x7f342c07b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:45:53.266 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.264+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2511+0+0 (secure 0 0 0) 0x7f34400b9ae0 con 0x7f3444030a00 2024-09-21T07:45:53.267 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.265+0000 7f3451000640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f342c078b00 0x7f342c07b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:45:53.268 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.266+0000 7f3451000640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f342c078b00 0x7f342c07b010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f344c06ad60 tx=0x7f344c076d30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:45:53.275 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.272+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3440082d80 con 0x7f3444030a00 2024-09-21T07:45:53.527 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T07:45:53.525+0000 7fdcc5c00640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T07:45:53.537 INFO:tasks.ceph.osd.4.smithi175.stderr:2024-09-21T07:45:53.535+0000 7fdcbbc00640 -1 osd.4 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:53.571 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T07:45:53.570+0000 7efc4ca00640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T07:45:53.581 INFO:tasks.ceph.osd.5.smithi175.stderr:2024-09-21T07:45:53.579+0000 7efc42a00640 -1 osd.5 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:53.609 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T07:45:53.607+0000 7ff4cc000640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T07:45:53.621 INFO:tasks.ceph.osd.6.smithi175.stderr:2024-09-21T07:45:53.620+0000 7ff4c2000640 -1 osd.6 14 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:53.659 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.656+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f344401d080 con 0x7f3444030a00 2024-09-21T07:45:53.659 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.657+0000 7f3448c00640 1 -- 172.21.15.88:0/1669563846 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v14) ==== 74+0+7567 (secure 0 0 0) 0x7f3440007e20 con 0x7f3444030a00 2024-09-21T07:45:53.662 INFO:teuthology.misc.health.smithi088.stdout: 2024-09-21T07:45:53.663 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":14,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:45:52.978937+0000","last_up_change":"2024-09-21T07:45:52.978937+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:45:53.663 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.661+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f342c078b00 msgr2=0x7f342c07b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:53.664 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.661+0000 7f3452400640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f342c078b00 0x7f342c07b010 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f344c06ad60 tx=0x7f344c076d30 comp rx=0 tx=0).stop 2024-09-21T07:45:53.664 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.661+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3444030a00 msgr2=0x7f3444030e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.661+0000 7f3452400640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3444030a00 0x7f3444030e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f34440e3700 tx=0x7f3440003b90 comp rx=0 tx=0).stop 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 shutdown_connections 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f342c078b00 0x7f342c07b010 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3444009ba0 0x7f34440304c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3444030a00 0x7f3444030e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 --2- 172.21.15.88:0/1669563846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3444005c60 0x7f34440ddf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 >> 172.21.15.88:0/1669563846 conn(0x7f3444051420 msgr2=0x7f3444038410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 shutdown_connections 2024-09-21T07:45:53.665 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:45:53.662+0000 7f3452400640 1 -- 172.21.15.88:0/1669563846 wait complete. 2024-09-21T07:45:53.704 DEBUG:teuthology.misc:1 of 8 OSDs are up 2024-09-21T07:45:54.089 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T07:45:54.087+0000 7f553d800640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T07:45:54.102 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T07:45:54.100+0000 7f0397a00640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T07:45:54.104 INFO:tasks.ceph.osd.0.smithi088.stderr:2024-09-21T07:45:54.102+0000 7f5533800640 -1 osd.0 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:54.109 INFO:tasks.ceph.osd.1.smithi088.stderr:2024-09-21T07:45:54.108+0000 7f038da00640 -1 osd.1 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:54.112 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T07:45:54.111+0000 7f0e60c00640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T07:45:54.114 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T07:45:54.113+0000 7f052be00640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T07:45:54.121 INFO:tasks.ceph.osd.3.smithi088.stderr:2024-09-21T07:45:54.119+0000 7f0e56c00640 -1 osd.3 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:54.131 INFO:tasks.ceph.osd.2.smithi088.stderr:2024-09-21T07:45:54.129+0000 7f0521e00640 -1 osd.2 15 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T07:45:55.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:45:55.669 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:00.511 DEBUG:teuthology.orchestra.run.smithi088:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 -- 172.21.15.88:0/2587868835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 msgr2=0x7f876c0c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 --2- 172.21.15.88:0/2587868835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 0x7f876c0c6b20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f875c005fd0 tx=0x7f875c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 -- 172.21.15.88:0/2587868835 shutdown_connections 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 --2- 172.21.15.88:0/2587868835 >> v2:172.21.15.175:3300/0 conn(0x7f876c06ffe0 0x7f876c0703f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 --2- 172.21.15.88:0/2587868835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 0x7f876c0c6b20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 -- 172.21.15.88:0/2587868835 >> 172.21.15.88:0/2587868835 conn(0x7f876c0b1eb0 msgr2=0x7f876c0b22c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 -- 172.21.15.88:0/2587868835 shutdown_connections 2024-09-21T07:46:00.721 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.718+0000 7f8772600640 1 -- 172.21.15.88:0/2587868835 wait complete. 2024-09-21T07:46:00.723 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.721+0000 7f8772600640 1 Processor -- start 2024-09-21T07:46:00.723 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.721+0000 7f8772600640 1 -- start start 2024-09-21T07:46:00.725 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8772600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 0x7f876c15a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:00.725 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8772600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f876c0c5530 0x7f876c15d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:00.725 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8772600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 0x7f876c15d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:00.725 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8772600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f876c073420 con 0x7f876c06ffe0 2024-09-21T07:46:00.725 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8772600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f876c0732a0 con 0x7f876c0c5530 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8772600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f876c0735a0 con 0x7f876c15b770 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8771200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 0x7f876c15a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8771200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 0x7f876c15a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46196/0 (socket says 172.21.15.88:46196) 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.722+0000 7f8771200640 1 -- 172.21.15.88:0/1148002784 learned_addr learned my addr 172.21.15.88:0/1148002784 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771c00640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 0x7f876c15d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f876be00640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f876c0c5530 0x7f876c15d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771200640 1 -- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 msgr2=0x7f876c15d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771200640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 0x7f876c15d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771200640 1 -- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f876c0c5530 msgr2=0x7f876c15d180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771200640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f876c0c5530 0x7f876c15d180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771200640 1 -- 172.21.15.88:0/1148002784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f876c1e9480 con 0x7f876c06ffe0 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f876be00640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f876c0c5530 0x7f876c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:00.726 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8771200640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 0x7f876c15a0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f875c034040 tx=0x7f875c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:00.727 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c03e030 con 0x7f876c06ffe0 2024-09-21T07:46:00.727 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.723+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f875c003ae0 con 0x7f876c06ffe0 2024-09-21T07:46:00.727 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.724+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f876c15dc00 con 0x7f876c06ffe0 2024-09-21T07:46:00.727 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.724+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f875c035040 con 0x7f876c06ffe0 2024-09-21T07:46:00.727 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.724+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f876c1e91a0 con 0x7f876c06ffe0 2024-09-21T07:46:00.727 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.725+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f876c073a50 con 0x7f876c06ffe0 2024-09-21T07:46:00.728 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.726+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f875c041030 con 0x7f876c06ffe0 2024-09-21T07:46:00.728 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.726+0000 7f8768c00640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f874c078b90 0x7f874c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:00.728 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.727+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f875c0baee0 con 0x7f876c06ffe0 2024-09-21T07:46:00.729 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.727+0000 7f876be00640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f874c078b90 0x7f874c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:00.729 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.728+0000 7f876be00640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f874c078b90 0x7f874c07b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f876c0ae410 tx=0x7f8758009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:00.734 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.732+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f875c083660 con 0x7f876c06ffe0 2024-09-21T07:46:00.919 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.917+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f876c0b2460 con 0x7f876c06ffe0 2024-09-21T07:46:00.920 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.918+0000 7f8768c00640 1 -- 172.21.15.88:0/1148002784 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) ==== 74+0+12908 (secure 0 0 0) 0x7f875c082f60 con 0x7f876c06ffe0 2024-09-21T07:46:00.921 INFO:teuthology.misc.health.smithi088.stdout: 2024-09-21T07:46:00.921 INFO:teuthology.misc.health.smithi088.stdout:{"epoch":19,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:45:58.089197+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:00.923 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f874c078b90 msgr2=0x7f874c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:00.923 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f874c078b90 0x7f874c07b0a0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f876c0ae410 tx=0x7f8758009040 comp rx=0 tx=0).stop 2024-09-21T07:46:00.923 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 msgr2=0x7f876c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 0x7f876c15a0f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f875c034040 tx=0x7f875c0035f0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 shutdown_connections 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f874c078b90 0x7f874c07b0a0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f876c15b770 0x7f876c15d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f876c0c5530 0x7f876c15d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 --2- 172.21.15.88:0/1148002784 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f876c06ffe0 0x7f876c15a0f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 >> 172.21.15.88:0/1148002784 conn(0x7f876c0b1eb0 msgr2=0x7f876c0cfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 shutdown_connections 2024-09-21T07:46:00.924 INFO:teuthology.misc.health.smithi088.stderr:2024-09-21T07:46:00.921+0000 7f8772600640 1 -- 172.21.15.88:0/1148002784 wait complete. 2024-09-21T07:46:00.942 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-09-21T07:46:00.942 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-09-21T07:46:00.942 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:46:00.942 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:46:00.946 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:46:00.946 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:46:00.946 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:46:00.950 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:46:00.950 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:46:00.950 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:46:00.955 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:46:00.955 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:46:00.955 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:46:00.959 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:46:00.959 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2024-09-21T07:46:00.959 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:01.144 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:46:01.145 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:46:01.161 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:46:01.161 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 -- 172.21.15.88:0/3129389575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 msgr2=0x7f1c900d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 --2- 172.21.15.88:0/3129389575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900d24a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1c9008d440 tx=0x7f1c7c02a560 comp rx=0 tx=0).stop 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 -- 172.21.15.88:0/3129389575 shutdown_connections 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 --2- 172.21.15.88:0/3129389575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900d24a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 --2- 172.21.15.88:0/3129389575 >> v2:172.21.15.175:3300/0 conn(0x7f1c900b9e00 0x7f1c90162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 -- 172.21.15.88:0/3129389575 >> 172.21.15.88:0/3129389575 conn(0x7f1c9008bfe0 msgr2=0x7f1c9009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:01.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 -- 172.21.15.88:0/3129389575 shutdown_connections 2024-09-21T07:46:01.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.189+0000 7f1c95600640 1 -- 172.21.15.88:0/3129389575 wait complete. 2024-09-21T07:46:01.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.191+0000 7f1c95600640 1 Processor -- start 2024-09-21T07:46:01.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.192+0000 7f1c95600640 1 -- start start 2024-09-21T07:46:01.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c95600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900a0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c95600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900b9e00 0x7f1c900c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c95600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900d29e0 0x7f1c900c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c95600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900d44b0 con 0x7f1c900b3160 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c95600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1c900d4330 con 0x7f1c900d29e0 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c95600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1c900d4630 con 0x7f1c900b9e00 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900a0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900a0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46252/0 (socket says 172.21.15.88:46252) 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c8fe00640 1 -- 172.21.15.88:0/3297675633 learned_addr learned my addr 172.21.15.88:0/3297675633 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c8f400640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900b9e00 0x7f1c900c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.193+0000 7f1c94c00640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900d29e0 0x7f1c900c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c8fe00640 1 -- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900b9e00 msgr2=0x7f1c900c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:01.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c8fe00640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900b9e00 0x7f1c900c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c8fe00640 1 -- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900d29e0 msgr2=0x7f1c900c85f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c8fe00640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900d29e0 0x7f1c900c85f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c8fe00640 1 -- 172.21.15.88:0/3297675633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c9015f620 con 0x7f1c900b3160 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c8fe00640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900a0080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000eb30 tx=0x7f1c8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80010430 con 0x7f1c900b3160 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1c800105d0 con 0x7f1c900b3160 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c80010a20 con 0x7f1c900b3160 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c900a2340 con 0x7f1c900b3160 2024-09-21T07:46:01.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.194+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c9009f130 con 0x7f1c900b3160 2024-09-21T07:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.195+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c900d4b50 con 0x7f1c900b3160 2024-09-21T07:46:01.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.197+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1c80010bc0 con 0x7f1c900b3160 2024-09-21T07:46:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.198+0000 7f1c87e00640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1c70078bc0 0x7f1c7007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.198+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f1c8009b7d0 con 0x7f1c900b3160 2024-09-21T07:46:01.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.198+0000 7f1c8f400640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1c70078bc0 0x7f1c7007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:01.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.199+0000 7f1c8f400640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1c70078bc0 0x7f1c7007b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c033000 tx=0x7f1c7c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:01.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.203+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1c80063870 con 0x7f1c900b3160 2024-09-21T07:46:01.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.423+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1c900b4de0 con 0x7f1c900b3160 2024-09-21T07:46:01.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.424+0000 7f1c87e00640 1 -- 172.21.15.88:0/3297675633 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1434 (secure 0 0 0) 0x7f1c800682a0 con 0x7f1c900b3160 2024-09-21T07:46:01.426 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:01.426 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":2,"btime":"2024-09-21T07:46:01:152583+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T07:46:01.426 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 2 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1c70078bc0 msgr2=0x7f1c7007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1c70078bc0 0x7f1c7007b0d0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c7c033000 tx=0x7f1c7c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 msgr2=0x7f1c900a0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900a0080 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1c8000eb30 tx=0x7f1c8000c2d0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 shutdown_connections 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1c70078bc0 0x7f1c7007b0d0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1c900b9e00 0x7f1c900c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1c900d29e0 0x7f1c900c85f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 --2- 172.21.15.88:0/3297675633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1c900b3160 0x7f1c900a0080 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 >> 172.21.15.88:0/3297675633 conn(0x7f1c9008bfe0 msgr2=0x7f1c9006de80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.426+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 shutdown_connections 2024-09-21T07:46:01.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:01.427+0000 7f1c95600640 1 -- 172.21.15.88:0/3297675633 wait complete. 2024-09-21T07:46:03.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:03.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 -- 172.21.15.88:0/689658151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e015d760 msgr2=0x7f12e01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:03.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 --2- 172.21.15.88:0/689658151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e015d760 0x7f12e01535c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f12d0005fd0 tx=0x7f12d002a540 comp rx=0 tx=0).stop 2024-09-21T07:46:03.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 -- 172.21.15.88:0/689658151 shutdown_connections 2024-09-21T07:46:03.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 --2- 172.21.15.88:0/689658151 >> v2:172.21.15.175:3300/0 conn(0x7f12e00ad0e0 0x7f12e00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 --2- 172.21.15.88:0/689658151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e015d760 0x7f12e01535c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 -- 172.21.15.88:0/689658151 >> 172.21.15.88:0/689658151 conn(0x7f12e006d730 msgr2=0x7f12e00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 -- 172.21.15.88:0/689658151 shutdown_connections 2024-09-21T07:46:03.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.678+0000 7f12e7a00640 1 -- 172.21.15.88:0/689658151 wait complete. 2024-09-21T07:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.680+0000 7f12e7a00640 1 Processor -- start 2024-09-21T07:46:03.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.680+0000 7f12e7a00640 1 -- start start 2024-09-21T07:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e00ad0e0 0x7f12e0156250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e015d760 0x7f12e014f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 0x7f12e015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12e006f220 con 0x7f12e0160310 2024-09-21T07:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12e006f0a0 con 0x7f12e015d760 2024-09-21T07:46:03.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f12e00b4c20 con 0x7f12e00ad0e0 2024-09-21T07:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 0x7f12e015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e00ad0e0 0x7f12e0156250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 0x7f12e015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46260/0 (socket says 172.21.15.88:46260) 2024-09-21T07:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 -- 172.21.15.88:0/3415224813 learned_addr learned my addr 172.21.15.88:0/3415224813 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12d7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e015d760 0x7f12e014f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:03.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 -- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e00ad0e0 msgr2=0x7f12e0156250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e00ad0e0 0x7f12e0156250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 -- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e015d760 msgr2=0x7f12e014f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e015d760 0x7f12e014f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.682+0000 7f12e5a00640 1 -- 172.21.15.88:0/3415224813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12e01f66b0 con 0x7f12e0160310 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.683+0000 7f12e5a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 0x7f12e015b7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c730 tx=0x7f12d800cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.683+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8003b70 con 0x7f12e0160310 2024-09-21T07:46:03.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.683+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f12d8003d10 con 0x7f12e0160310 2024-09-21T07:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.683+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12e01f9000 con 0x7f12e0160310 2024-09-21T07:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.683+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12d8005330 con 0x7f12e0160310 2024-09-21T07:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.683+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12e01f95c0 con 0x7f12e0160310 2024-09-21T07:46:03.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.684+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12e01f8d70 con 0x7f12e0160310 2024-09-21T07:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.685+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f12d80054d0 con 0x7f12e0160310 2024-09-21T07:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.686+0000 7f12dd600640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12bc078b90 0x7f12bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.686+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f12d809a5d0 con 0x7f12e0160310 2024-09-21T07:46:03.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.687+0000 7f12e5000640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12bc078b90 0x7f12bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.687+0000 7f12e5000640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12bc078b90 0x7f12bc07b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12d0033000 tx=0x7f12d0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:03.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.691+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f12e00ada30 con 0x7f12e0160310 2024-09-21T07:46:03.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.915+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12e0160fa0 con 0x7f12e0160310 2024-09-21T07:46:03.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.915+0000 7f12dd600640 1 -- 172.21.15.88:0/3415224813 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+3909 (secure 0 0 0) 0x7f12d8062790 con 0x7f12e0160310 2024-09-21T07:46:03.918 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:03.918 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":3,"btime":"2024-09-21T07:46:02:156202+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4423,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T07:46:03.918 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 3 2024-09-21T07:46:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12bc078b90 msgr2=0x7f12bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12bc078b90 0x7f12bc07b0a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f12d0033000 tx=0x7f12d0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 msgr2=0x7f12e015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 0x7f12e015b7f0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f12d800c730 tx=0x7f12d800cb30 comp rx=0 tx=0).stop 2024-09-21T07:46:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 shutdown_connections 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12bc078b90 0x7f12bc07b0a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12e0160310 0x7f12e015b7f0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12e015d760 0x7f12e014f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 --2- 172.21.15.88:0/3415224813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12e00ad0e0 0x7f12e0156250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 >> 172.21.15.88:0/3415224813 conn(0x7f12e006d730 msgr2=0x7f12e0160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 shutdown_connections 2024-09-21T07:46:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:03.918+0000 7f12e7a00640 1 -- 172.21.15.88:0/3415224813 wait complete. 2024-09-21T07:46:03.942 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:46:04.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 -- 172.21.15.88:0/2779732915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80b3160 msgr2=0x7f03f80d24a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 --2- 172.21.15.88:0/2779732915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80b3160 0x7f03f80d24a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f03e0009830 tx=0x7f03e002a530 comp rx=0 tx=0).stop 2024-09-21T07:46:04.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 -- 172.21.15.88:0/2779732915 shutdown_connections 2024-09-21T07:46:04.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 --2- 172.21.15.88:0/2779732915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80b3160 0x7f03f80d24a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 -- 172.21.15.88:0/2779732915 >> 172.21.15.88:0/2779732915 conn(0x7f03f808bfe0 msgr2=0x7f03f809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:04.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 -- 172.21.15.88:0/2779732915 shutdown_connections 2024-09-21T07:46:04.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.161+0000 7f03fd200640 1 -- 172.21.15.88:0/2779732915 wait complete. 2024-09-21T07:46:04.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.163+0000 7f03fd200640 1 Processor -- start 2024-09-21T07:46:04.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.163+0000 7f03fd200640 1 -- start start 2024-09-21T07:46:04.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.164+0000 7f03fd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f80b3160 0x7f03f81f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.164+0000 7f03fd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03f80b9e00 0x7f03f80cdc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.164+0000 7f03fd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 0x7f03f80cf350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03fd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03f80d4700 con 0x7f03f80d29e0 2024-09-21T07:46:04.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03fd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03f80d4580 con 0x7f03f80b3160 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03fd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03f80d4880 con 0x7f03f80b9e00 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 0x7f03f80cf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 0x7f03f80cf350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46276/0 (socket says 172.21.15.88:46276) 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 -- 172.21.15.88:0/1442298946 learned_addr learned my addr 172.21.15.88:0/1442298946 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03eea00640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03f80b9e00 0x7f03f80cdc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7400640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f80b3160 0x7f03f81f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 -- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03f80b9e00 msgr2=0x7f03f80cdc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03f80b9e00 0x7f03f80cdc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 -- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f80b3160 msgr2=0x7f03f81f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f80b3160 0x7f03f81f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7e00640 1 -- 172.21.15.88:0/1442298946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f81f5890 con 0x7f03f80d29e0 2024-09-21T07:46:04.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.165+0000 7f03f7400640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f80b3160 0x7f03f81f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.166+0000 7f03f7e00640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 0x7f03f80cf350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c730 tx=0x7f03f000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.166+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f0003b70 con 0x7f03f80d29e0 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.166+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03f0003d10 con 0x7f03f80d29e0 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.166+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03f00052d0 con 0x7f03f80d29e0 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.166+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f81f5b70 con 0x7f03f80d29e0 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.166+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f81f6130 con 0x7f03f80d29e0 2024-09-21T07:46:04.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.167+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f81f64a0 con 0x7f03f80d29e0 2024-09-21T07:46:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.168+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f03f0005470 con 0x7f03f80d29e0 2024-09-21T07:46:04.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.169+0000 7f03efe00640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f03d4078bc0 0x7f03d407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.169+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f03f009a680 con 0x7f03f80d29e0 2024-09-21T07:46:04.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.170+0000 7f03f7400640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f03d4078bc0 0x7f03d407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.170+0000 7f03f7400640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f03d4078bc0 0x7f03d407b0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03e8007fd0 tx=0x7f03e8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:04.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.175+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03f0062ea0 con 0x7f03f80d29e0 2024-09-21T07:46:04.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.363+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f03f80d24a0 con 0x7f03f80d29e0 2024-09-21T07:46:04.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.363+0000 7f03efe00640 1 -- 172.21.15.88:0/1442298946 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) ==== 84+0+65 (secure 0 0 0) 0x7f03f00627a0 con 0x7f03f80d29e0 2024-09-21T07:46:04.366 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:04.366 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:46:04.366 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:04.366 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:46:04.367 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:46:04.367 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:04.367 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:46:04.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f03d4078bc0 msgr2=0x7f03d407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f03d4078bc0 0x7f03d407b0d0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f03e8007fd0 tx=0x7f03e8009040 comp rx=0 tx=0).stop 2024-09-21T07:46:04.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 msgr2=0x7f03f80cf350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 0x7f03f80cf350 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f03f000c730 tx=0x7f03f000cb30 comp rx=0 tx=0).stop 2024-09-21T07:46:04.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 shutdown_connections 2024-09-21T07:46:04.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f03d4078bc0 0x7f03d407b0d0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03f80b9e00 0x7f03f80cdc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f80d29e0 0x7f03f80cf350 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 --2- 172.21.15.88:0/1442298946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f80b3160 0x7f03f81f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 >> 172.21.15.88:0/1442298946 conn(0x7f03f808bfe0 msgr2=0x7f03f8161d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 shutdown_connections 2024-09-21T07:46:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.366+0000 7f03fd200640 1 -- 172.21.15.88:0/1442298946 wait complete. 2024-09-21T07:46:04.389 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:46:04.389 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 -- 172.21.15.88:0/1196513304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0af100 msgr2=0x7f881c0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 --2- 172.21.15.88:0/1196513304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0af100 0x7f881c0af510 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8814005fd0 tx=0x7f881402aa90 comp rx=0 tx=0).stop 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 -- 172.21.15.88:0/1196513304 shutdown_connections 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 --2- 172.21.15.88:0/1196513304 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0af100 0x7f881c0af510 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 --2- 172.21.15.88:0/1196513304 >> v2:172.21.15.175:3300/0 conn(0x7f881c06d4b0 0x7f881c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 -- 172.21.15.88:0/1196513304 >> 172.21.15.88:0/1196513304 conn(0x7f881c0b2540 msgr2=0x7f881c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 -- 172.21.15.88:0/1196513304 shutdown_connections 2024-09-21T07:46:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.612+0000 7f8821a00640 1 -- 172.21.15.88:0/1196513304 wait complete. 2024-09-21T07:46:04.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.615+0000 7f8821a00640 1 Processor -- start 2024-09-21T07:46:04.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.615+0000 7f8821a00640 1 -- start start 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f8821a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 0x7f881c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f8821a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0cc830 0x7f881c1533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f8821a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c15f3e0 0x7f881c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f8821a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0b3650 con 0x7f881c0af100 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f8821a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f881c0b34d0 con 0x7f881c15f3e0 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f8821a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f881c0b37d0 con 0x7f881c0cc830 2024-09-21T07:46:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f881aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 0x7f881c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f881aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 0x7f881c0cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46278/0 (socket says 172.21.15.88:46278) 2024-09-21T07:46:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.616+0000 7f881aa00640 1 -- 172.21.15.88:0/2294270810 learned_addr learned my addr 172.21.15.88:0/2294270810 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881b400640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c15f3e0 0x7f881c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f8812000640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0cc830 0x7f881c1533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881aa00640 1 -- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0cc830 msgr2=0x7f881c1533f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881aa00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0cc830 0x7f881c1533f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881aa00640 1 -- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c15f3e0 msgr2=0x7f881c153f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881aa00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c15f3e0 0x7f881c153f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881aa00640 1 -- 172.21.15.88:0/2294270810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f881c15e7e0 con 0x7f881c0af100 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f881aa00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 0x7f881c0cfb90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c3f0 tx=0x7f880c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c017070 con 0x7f881c0af100 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f880c0035f0 con 0x7f881c0af100 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f881c15eac0 con 0x7f881c0af100 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.617+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f880c003a40 con 0x7f881c0af100 2024-09-21T07:46:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.618+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f881c1549f0 con 0x7f881c0af100 2024-09-21T07:46:04.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.618+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f881c0cba00 con 0x7f881c0af100 2024-09-21T07:46:04.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.620+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f880c01a070 con 0x7f881c0af100 2024-09-21T07:46:04.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.620+0000 7f8813400640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f87f8078b90 0x7f87f807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:04.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.621+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 4857+0+0 (secure 0 0 0) 0x7f880c099c30 con 0x7f881c0af100 2024-09-21T07:46:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.621+0000 7f8812000640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f87f8078b90 0x7f87f807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:04.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.622+0000 7f8812000640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f87f8078b90 0x7f87f807b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f881c0b0c40 tx=0x7f8804005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.625+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f881c0cba00 con 0x7f881c0af100 2024-09-21T07:46:04.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:04.812+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f881c154c50 con 0x7f881c0af100 2024-09-21T07:46:05.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.174+0000 7f8813400640 1 -- 172.21.15.88:0/2294270810 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) ==== 138+0+0 (secure 0 0 0) 0x7f880c061df0 con 0x7f881c0af100 2024-09-21T07:46:05.177 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.177+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f87f8078b90 msgr2=0x7f87f807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.177+0000 7f8821a00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f87f8078b90 0x7f87f807b0a0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f881c0b0c40 tx=0x7f8804005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.177+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 msgr2=0x7f881c0cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.177+0000 7f8821a00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 0x7f881c0cfb90 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f880c00c3f0 tx=0x7f880c00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 shutdown_connections 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f87f8078b90 0x7f87f807b0a0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f881c0cc830 0x7f881c1533f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f881c15f3e0 0x7f881c153f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 --2- 172.21.15.88:0/2294270810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f881c0af100 0x7f881c0cfb90 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 >> 172.21.15.88:0/2294270810 conn(0x7f881c0b2540 msgr2=0x7f881c0ae7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:05.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 shutdown_connections 2024-09-21T07:46:05.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.178+0000 7f8821a00640 1 -- 172.21.15.88:0/2294270810 wait complete. 2024-09-21T07:46:05.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:46:05.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b3ca00640 1 -- 172.21.15.88:0/3541435229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 msgr2=0x7f0b1c004c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:05.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/3541435229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b1c004c70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f0b24001800 tx=0x7f0b1c005830 comp rx=0 tx=0).stop 2024-09-21T07:46:05.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b37e00640 1 -- 172.21.15.88:0/3541435229 reap_dead start 2024-09-21T07:46:05.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b3ca00640 1 -- 172.21.15.88:0/3541435229 shutdown_connections 2024-09-21T07:46:05.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b3ca00640 1 -- 172.21.15.88:0/3541435229 >> 172.21.15.88:0/3541435229 conn(0x7f0b380b1dd0 msgr2=0x7f0b380b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:05.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b3ca00640 1 -- 172.21.15.88:0/3541435229 shutdown_connections 2024-09-21T07:46:05.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.435+0000 7f0b3ca00640 1 -- 172.21.15.88:0/3541435229 wait complete. 2024-09-21T07:46:05.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.438+0000 7f0b3ca00640 1 Processor -- start 2024-09-21T07:46:05.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.438+0000 7f0b3ca00640 1 -- start start 2024-09-21T07:46:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b38157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b3815c3b0 0x7f0b3806f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b3ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b3815c7c0 0x7f0b3806f970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b3815d790 con 0x7f0b1c004800 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b38160990 con 0x7f0b3815c7c0 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b3ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b3815d910 con 0x7f0b3815c3b0 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b37400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b38157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b37400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b38157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46298/0 (socket says 172.21.15.88:46298) 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.439+0000 7f0b37400640 1 -- 172.21.15.88:0/1996575118 learned_addr learned my addr 172.21.15.88:0/1996575118 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37e00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b3815c7c0 0x7f0b3806f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b36a00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b3815c3b0 0x7f0b3806f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37400640 1 -- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b3815c3b0 msgr2=0x7f0b3806f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37400640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b3815c3b0 0x7f0b3806f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37400640 1 -- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b3815c7c0 msgr2=0x7f0b3806f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37400640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b3815c7c0 0x7f0b3806f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37400640 1 -- 172.21.15.88:0/1996575118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b38145f90 con 0x7f0b1c004800 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b36a00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b3815c3b0 0x7f0b3806f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b37400640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b38157370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b28011ab0 tx=0x7f0b28011eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2801e070 con 0x7f0b1c004800 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.440+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b28004460 con 0x7f0b1c004800 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.441+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b381578b0 con 0x7f0b1c004800 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.441+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b2800fc00 con 0x7f0b1c004800 2024-09-21T07:46:05.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.441+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b38146530 con 0x7f0b1c004800 2024-09-21T07:46:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.442+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b380b47b0 con 0x7f0b1c004800 2024-09-21T07:46:05.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.443+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0b28021070 con 0x7f0b1c004800 2024-09-21T07:46:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.444+0000 7f0b2f400640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0b18078bc0 0x7f0b1807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.444+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(20..20 src has 1..20) ==== 5227+0+0 (secure 0 0 0) 0x7f0b2809b420 con 0x7f0b1c004800 2024-09-21T07:46:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.445+0000 7f0b36a00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0b18078bc0 0x7f0b1807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.445+0000 7f0b36a00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0b18078bc0 0x7f0b1807b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b24032000 tx=0x7f0b24001360 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:05.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.449+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b280633e0 con 0x7f0b1c004800 2024-09-21T07:46:05.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:05.636+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f0b380d0eb0 con 0x7f0b1c004800 2024-09-21T07:46:06.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.181+0000 7f0b2f400640 1 -- 172.21.15.88:0/1996575118 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) ==== 170+0+0 (secure 0 0 0) 0x7f0b28067e10 con 0x7f0b1c004800 2024-09-21T07:46:06.183 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.183+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0b18078bc0 msgr2=0x7f0b1807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.183+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0b18078bc0 0x7f0b1807b0d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f0b24032000 tx=0x7f0b24001360 comp rx=0 tx=0).stop 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.183+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 msgr2=0x7f0b38157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.183+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b38157370 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f0b28011ab0 tx=0x7f0b28011eb0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 shutdown_connections 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0b18078bc0 0x7f0b1807b0d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b3815c3b0 0x7f0b3806f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b3815c7c0 0x7f0b3806f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 --2- 172.21.15.88:0/1996575118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b1c004800 0x7f0b38157370 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 >> 172.21.15.88:0/1996575118 conn(0x7f0b380b1dd0 msgr2=0x7f0b38072ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 shutdown_connections 2024-09-21T07:46:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.184+0000 7f0b3ca00640 1 -- 172.21.15.88:0/1996575118 wait complete. 2024-09-21T07:46:06.233 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:46:06.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 -- 172.21.15.88:0/391976847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 msgr2=0x7f4f4006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 --2- 172.21.15.88:0/391976847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4006fb40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009830 tx=0x7f4f3002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 -- 172.21.15.88:0/391976847 shutdown_connections 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 --2- 172.21.15.88:0/391976847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4006fb40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 -- 172.21.15.88:0/391976847 >> 172.21.15.88:0/391976847 conn(0x7f4f4006d1c0 msgr2=0x7f4f4006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 -- 172.21.15.88:0/391976847 shutdown_connections 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.453+0000 7f4f44c00640 1 -- 172.21.15.88:0/391976847 wait complete. 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.455+0000 7f4f44c00640 1 Processor -- start 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.455+0000 7f4f44c00640 1 -- start start 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f44c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4013a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f44c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f40145a40 0x7f4f40139c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f44c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f400b22c0 0x7f4f400b26d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:06.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f44c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400ce3a0 con 0x7f4f40145550 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f44c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f400ce220 con 0x7f4f40145a40 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f44c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f400ce520 con 0x7f4f400b22c0 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4013a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4013a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49796/0 (socket says 172.21.15.88:49796) 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 -- 172.21.15.88:0/2704149217 learned_addr learned my addr 172.21.15.88:0/2704149217 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3d400640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f40145a40 0x7f4f40139c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3e800640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f400b22c0 0x7f4f400b26d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 -- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f400b22c0 msgr2=0x7f4f400b26d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f400b22c0 0x7f4f400b26d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 -- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f40145a40 msgr2=0x7f4f40139c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f40145a40 0x7f4f40139c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.457+0000 7f4f3de00640 1 -- 172.21.15.88:0/2704149217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f400b2c10 con 0x7f4f40145550 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.458+0000 7f4f3de00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4013a490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009950 tx=0x7f4f30033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:06.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.458+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30033e10 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.458+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f30031c50 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.458+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4013a9d0 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.458+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f30005280 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.458+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4013b1c0 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.460+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f40069a50 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.461+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4f30045020 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.462+0000 7f4f36000640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f20078b90 0x7f4f2007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.462+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(21..21 src has 1..21) ==== 5609+0+0 (secure 0 0 0) 0x7f4f300bb810 con 0x7f4f40145550 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.462+0000 7f4f3d400640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f20078b90 0x7f4f2007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.463+0000 7f4f3d400640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f20078b90 0x7f4f2007b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c006fd0 tx=0x7f4f2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:06.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.467+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f40069a50 con 0x7f4f40145550 2024-09-21T07:46:06.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:06.689+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f4f4006fb40 con 0x7f4f40145550 2024-09-21T07:46:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.183+0000 7f4f36000640 1 -- 172.21.15.88:0/2704149217 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) ==== 172+0+0 (secure 0 0 0) 0x7f4f400b2c10 con 0x7f4f40145550 2024-09-21T07:46:07.185 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T07:46:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.187+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f20078b90 msgr2=0x7f4f2007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.187+0000 7f4f44c00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f20078b90 0x7f4f2007b0a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4f2c006fd0 tx=0x7f4f2c008040 comp rx=0 tx=0).stop 2024-09-21T07:46:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.187+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 msgr2=0x7f4f4013a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.187+0000 7f4f44c00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4013a490 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f4f30009950 tx=0x7f4f30033c20 comp rx=0 tx=0).stop 2024-09-21T07:46:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 shutdown_connections 2024-09-21T07:46:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f20078b90 0x7f4f2007b0a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f400b22c0 0x7f4f400b26d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f40145a40 0x7f4f40139c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 --2- 172.21.15.88:0/2704149217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f40145550 0x7f4f4013a490 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 >> 172.21.15.88:0/2704149217 conn(0x7f4f4006d1c0 msgr2=0x7f4f400b3580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:07.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 shutdown_connections 2024-09-21T07:46:07.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.188+0000 7f4f44c00640 1 -- 172.21.15.88:0/2704149217 wait complete. 2024-09-21T07:46:07.232 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:07.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 -- 172.21.15.88:0/2900196624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 msgr2=0x7f98ac071b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 --2- 172.21.15.88:0/2900196624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac071b00 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9894005fd0 tx=0x7f989402a580 comp rx=0 tx=0).stop 2024-09-21T07:46:07.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 -- 172.21.15.88:0/2900196624 shutdown_connections 2024-09-21T07:46:07.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 --2- 172.21.15.88:0/2900196624 >> v2:172.21.15.175:3300/0 conn(0x7f98ac160350 0x7f98ac160760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 --2- 172.21.15.88:0/2900196624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac071b00 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 --2- 172.21.15.88:0/2900196624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac15d730 0x7f98ac06f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 -- 172.21.15.88:0/2900196624 >> 172.21.15.88:0/2900196624 conn(0x7f98ac0d0b50 msgr2=0x7f98ac0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:07.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 -- 172.21.15.88:0/2900196624 shutdown_connections 2024-09-21T07:46:07.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.454+0000 7f98b2600640 1 -- 172.21.15.88:0/2900196624 wait complete. 2024-09-21T07:46:07.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.456+0000 7f98b2600640 1 Processor -- start 2024-09-21T07:46:07.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.456+0000 7f98b2600640 1 -- start start 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98b2600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac13e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98b2600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac15d730 0x7f98ac145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98b2600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98ac160350 0x7f98ac145780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98b2600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac160e20 con 0x7f98ac0716f0 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98b2600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98ac160ca0 con 0x7f98ac160350 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98b2600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98ac160fa0 con 0x7f98ac15d730 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac13e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac13e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49810/0 (socket says 172.21.15.88:49810) 2024-09-21T07:46:07.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 -- 172.21.15.88:0/2613982338 learned_addr learned my addr 172.21.15.88:0/2613982338 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98a2a00640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac15d730 0x7f98ac145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98abe00640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98ac160350 0x7f98ac145780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 -- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac15d730 msgr2=0x7f98ac145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac15d730 0x7f98ac145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 -- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98ac160350 msgr2=0x7f98ac145780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98ac160350 0x7f98ac145780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.458+0000 7f98ab400640 1 -- 172.21.15.88:0/2613982338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98ac0cd3f0 con 0x7f98ac0716f0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.459+0000 7f98ab400640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac13e450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f989c00d690 tx=0x7f989c00db50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.459+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c017070 con 0x7f98ac0716f0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.459+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f989c005310 con 0x7f98ac0716f0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.459+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98ac1f3f30 con 0x7f98ac0716f0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.459+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f989c009820 con 0x7f98ac0716f0 2024-09-21T07:46:07.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.459+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98ac1f41d0 con 0x7f98ac0716f0 2024-09-21T07:46:07.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.460+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98ac0b2540 con 0x7f98ac0716f0 2024-09-21T07:46:07.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.461+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f989c0099c0 con 0x7f98ac0716f0 2024-09-21T07:46:07.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.462+0000 7f98a3e00640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9888078bc0 0x7f988807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.462+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f989c09ad90 con 0x7f98ac0716f0 2024-09-21T07:46:07.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.463+0000 7f98a2a00640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9888078bc0 0x7f988807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.463+0000 7f98a2a00640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9888078bc0 0x7f988807b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9894033000 tx=0x7f9894004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:07.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.467+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98ac1f3f30 con 0x7f98ac0716f0 2024-09-21T07:46:07.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.652+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f98ac06f8c0 con 0x7f98ac0716f0 2024-09-21T07:46:07.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.654+0000 7f98a3e00640 1 -- 172.21.15.88:0/2613982338 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f989c062b70 con 0x7f98ac0716f0 2024-09-21T07:46:07.656 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:07.656 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:06.691589+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:07.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9888078bc0 msgr2=0x7f988807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9888078bc0 0x7f988807b0d0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9894033000 tx=0x7f9894004730 comp rx=0 tx=0).stop 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 msgr2=0x7f98ac13e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac13e450 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f989c00d690 tx=0x7f989c00db50 comp rx=0 tx=0).stop 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 shutdown_connections 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9888078bc0 0x7f988807b0d0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98ac15d730 0x7f98ac145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98ac160350 0x7f98ac145780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 --2- 172.21.15.88:0/2613982338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98ac0716f0 0x7f98ac13e450 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 >> 172.21.15.88:0/2613982338 conn(0x7f98ac0d0b50 msgr2=0x7f98ac15f000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 shutdown_connections 2024-09-21T07:46:07.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.656+0000 7f98b2600640 1 -- 172.21.15.88:0/2613982338 wait complete. 2024-09-21T07:46:07.681 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 -- 172.21.15.88:0/1771085724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100aebc0 msgr2=0x7f7710152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 --2- 172.21.15.88:0/1771085724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100aebc0 0x7f7710152920 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7710160420 tx=0x7f770002a560 comp rx=0 tx=0).stop 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 -- 172.21.15.88:0/1771085724 shutdown_connections 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 --2- 172.21.15.88:0/1771085724 >> v2:172.21.15.175:3300/0 conn(0x7f771015fae0 0x7f77100b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 --2- 172.21.15.88:0/1771085724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100aebc0 0x7f7710152920 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 -- 172.21.15.88:0/1771085724 >> 172.21.15.88:0/1771085724 conn(0x7f771006d730 msgr2=0x7f77100b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 -- 172.21.15.88:0/1771085724 shutdown_connections 2024-09-21T07:46:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.898+0000 7f7716600640 1 -- 172.21.15.88:0/1771085724 wait complete. 2024-09-21T07:46:07.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.900+0000 7f7716600640 1 Processor -- start 2024-09-21T07:46:07.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.901+0000 7f7716600640 1 -- start start 2024-09-21T07:46:07.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f7716600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77100aebc0 0x7f7710150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f7716600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 0x7f77101550d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f7716600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7710151760 0x7f7710155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f7716600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77100b40d0 con 0x7f77100b2750 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f7716600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77100b3f50 con 0x7f77100aebc0 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f7716600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77100b4250 con 0x7f7710151760 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f770ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 0x7f77101550d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f770ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 0x7f77101550d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49828/0 (socket says 172.21.15.88:49828) 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f770ea00640 1 -- 172.21.15.88:0/1206865352 learned_addr learned my addr 172.21.15.88:0/1206865352 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f770f400640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77100aebc0 0x7f7710150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.902+0000 7f770fe00640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7710151760 0x7f7710155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770f400640 1 -- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7710151760 msgr2=0x7f7710155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770f400640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7710151760 0x7f7710155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770f400640 1 -- 172.21.15.88:0/1206865352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 msgr2=0x7f77101550d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:07.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770f400640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 0x7f77101550d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:07.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770f400640 1 -- 172.21.15.88:0/1206865352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7710150f50 con 0x7f77100aebc0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770ea00640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 0x7f77101550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f770f400640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77100aebc0 0x7f7710150a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7700031000 tx=0x7f77000038a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.903+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f770003e030 con 0x7f77100aebc0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.904+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7700003d90 con 0x7f77100aebc0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.904+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f771015ff20 con 0x7f77100aebc0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.904+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7700035040 con 0x7f77100aebc0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.904+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77100cc8c0 con 0x7f77100aebc0 2024-09-21T07:46:07.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.905+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f771015d320 con 0x7f77100aebc0 2024-09-21T07:46:07.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.906+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7700041030 con 0x7f77100aebc0 2024-09-21T07:46:07.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.907+0000 7f7707400640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f76f0078bc0 0x7f76f007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:07.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.907+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(22..22 src has 1..22) ==== 5709+0+0 (secure 0 0 0) 0x7f77000bb840 con 0x7f77100aebc0 2024-09-21T07:46:07.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.907+0000 7f770ea00640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f76f0078bc0 0x7f76f007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:07.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.908+0000 7f770ea00640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f76f0078bc0 0x7f76f007b0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000f60 tx=0x7f76fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:07.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:07.912+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f771015d320 con 0x7f77100aebc0 2024-09-21T07:46:08.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.098+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f77100cc410 con 0x7f77100aebc0 2024-09-21T07:46:08.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.100+0000 7f7707400640 1 -- 172.21.15.88:0/1206865352 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) ==== 74+0+16694 (secure 0 0 0) 0x7f77000837a0 con 0x7f77100aebc0 2024-09-21T07:46:08.102 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:08.102 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":22,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:06.691589+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f76f0078bc0 msgr2=0x7f76f007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f76f0078bc0 0x7f76f007b0d0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f76fc000f60 tx=0x7f76fc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77100aebc0 msgr2=0x7f7710150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77100aebc0 0x7f7710150a10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7700031000 tx=0x7f77000038a0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 shutdown_connections 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f76f0078bc0 0x7f76f007b0d0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7710151760 0x7f7710155610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77100b2750 0x7f77101550d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 --2- 172.21.15.88:0/1206865352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77100aebc0 0x7f7710150a10 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 >> 172.21.15.88:0/1206865352 conn(0x7f771006d730 msgr2=0x7f7710068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.102+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 shutdown_connections 2024-09-21T07:46:08.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.103+0000 7f7716600640 1 -- 172.21.15.88:0/1206865352 wait complete. 2024-09-21T07:46:08.126 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.348+0000 7f6d30a00640 1 -- 172.21.15.88:0/3330847919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c15b6e0 msgr2=0x7f6d2c15baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.348+0000 7f6d30a00640 1 --2- 172.21.15.88:0/3330847919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c15b6e0 0x7f6d2c15baf0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f6d1800a940 tx=0x7f6d1802b7b0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.349+0000 7f6d30a00640 1 -- 172.21.15.88:0/3330847919 shutdown_connections 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.349+0000 7f6d30a00640 1 --2- 172.21.15.88:0/3330847919 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c15b6e0 0x7f6d2c15baf0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.349+0000 7f6d30a00640 1 --2- 172.21.15.88:0/3330847919 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d2c070fa0 0x7f6d2c0af4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.349+0000 7f6d30a00640 1 -- 172.21.15.88:0/3330847919 >> 172.21.15.88:0/3330847919 conn(0x7f6d2c06d620 msgr2=0x7f6d2c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.349+0000 7f6d30a00640 1 -- 172.21.15.88:0/3330847919 shutdown_connections 2024-09-21T07:46:08.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.349+0000 7f6d30a00640 1 -- 172.21.15.88:0/3330847919 wait complete. 2024-09-21T07:46:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.351+0000 7f6d30a00640 1 Processor -- start 2024-09-21T07:46:08.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.351+0000 7f6d30a00640 1 -- start start 2024-09-21T07:46:08.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.352+0000 7f6d30a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 0x7f6d2c1f6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.352+0000 7f6d30a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c070fa0 0x7f6d2c157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.352+0000 7f6d30a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d2c15b6e0 0x7f6d2c1f6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.352+0000 7f6d30a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c0afb60 con 0x7f6d2c06ef10 2024-09-21T07:46:08.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.352+0000 7f6d30a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d2c070cf0 con 0x7f6d2c070fa0 2024-09-21T07:46:08.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.352+0000 7f6d30a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d2c06f320 con 0x7f6d2c15b6e0 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 0x7f6d2c1f6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 0x7f6d2c1f6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49832/0 (socket says 172.21.15.88:49832) 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 -- 172.21.15.88:0/1470105253 learned_addr learned my addr 172.21.15.88:0/1470105253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d29e00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c070fa0 0x7f6d2c157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2b200640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d2c15b6e0 0x7f6d2c1f6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 -- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d2c15b6e0 msgr2=0x7f6d2c1f6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d2c15b6e0 0x7f6d2c1f6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 -- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c070fa0 msgr2=0x7f6d2c157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c070fa0 0x7f6d2c157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 -- 172.21.15.88:0/1470105253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d2c1e4e90 con 0x7f6d2c06ef10 2024-09-21T07:46:08.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.353+0000 7f6d2a800640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 0x7f6d2c1f6800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00e9c0 tx=0x7f6d1c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.354+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c00cd80 con 0x7f6d2c06ef10 2024-09-21T07:46:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.354+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6d1c004020 con 0x7f6d2c06ef10 2024-09-21T07:46:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.354+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d1c004450 con 0x7f6d2c06ef10 2024-09-21T07:46:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.354+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d2c1f5d20 con 0x7f6d2c06ef10 2024-09-21T07:46:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.354+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d2c1f62e0 con 0x7f6d2c06ef10 2024-09-21T07:46:08.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.355+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d2c06f750 con 0x7f6d2c06ef10 2024-09-21T07:46:08.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.356+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6d1c0035f0 con 0x7f6d2c06ef10 2024-09-21T07:46:08.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.357+0000 7f6d22a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6d0c078b90 0x7f6d0c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:08.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.357+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6d1c09b640 con 0x7f6d2c06ef10 2024-09-21T07:46:08.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.358+0000 7f6d29e00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6d0c078b90 0x7f6d0c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:08.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.358+0000 7f6d29e00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6d0c078b90 0x7f6d0c07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0b41f0 tx=0x7f6d18004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:08.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.362+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6d1c063390 con 0x7f6d2c06ef10 2024-09-21T07:46:08.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:08.586+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f6d2c074140 con 0x7f6d2c06ef10 2024-09-21T07:46:09.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.211+0000 7f6d22a00640 1 -- 172.21.15.88:0/1470105253 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) ==== 118+0+0 (secure 0 0 0) 0x7f6d1c067dc0 con 0x7f6d2c06ef10 2024-09-21T07:46:09.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6d0c078b90 msgr2=0x7f6d0c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6d0c078b90 0x7f6d0c07b0a0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c0b41f0 tx=0x7f6d18004730 comp rx=0 tx=0).stop 2024-09-21T07:46:09.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 msgr2=0x7f6d2c1f6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 0x7f6d2c1f6800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d1c00e9c0 tx=0x7f6d1c00ee80 comp rx=0 tx=0).stop 2024-09-21T07:46:09.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 shutdown_connections 2024-09-21T07:46:09.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6d0c078b90 0x7f6d0c07b0a0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d2c15b6e0 0x7f6d2c1f6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d2c070fa0 0x7f6d2c157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 --2- 172.21.15.88:0/1470105253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d2c06ef10 0x7f6d2c1f6800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 >> 172.21.15.88:0/1470105253 conn(0x7f6d2c06d620 msgr2=0x7f6d2c0b1d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:09.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 shutdown_connections 2024-09-21T07:46:09.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.213+0000 7f6d30a00640 1 -- 172.21.15.88:0/1470105253 wait complete. 2024-09-21T07:46:09.236 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:46:09.236 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:46:09.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.456+0000 7f19a6e00640 1 -- 172.21.15.88:0/2118675845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0145450 msgr2=0x7f19a006f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.456+0000 7f19a6e00640 1 --2- 172.21.15.88:0/2118675845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0145450 0x7f19a006f180 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f19a00b00a0 tx=0x7f199002a560 comp rx=0 tx=0).stop 2024-09-21T07:46:09.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 -- 172.21.15.88:0/2118675845 shutdown_connections 2024-09-21T07:46:09.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 --2- 172.21.15.88:0/2118675845 >> v2:172.21.15.175:3300/0 conn(0x7f19a0160320 0x7f19a0160730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 --2- 172.21.15.88:0/2118675845 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a00735e0 0x7f19a00739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 --2- 172.21.15.88:0/2118675845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0145450 0x7f19a006f180 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 -- 172.21.15.88:0/2118675845 >> 172.21.15.88:0/2118675845 conn(0x7f19a006d1c0 msgr2=0x7f19a006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:09.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 -- 172.21.15.88:0/2118675845 shutdown_connections 2024-09-21T07:46:09.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.457+0000 7f19a6e00640 1 -- 172.21.15.88:0/2118675845 wait complete. 2024-09-21T07:46:09.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.459+0000 7f19a6e00640 1 Processor -- start 2024-09-21T07:46:09.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.459+0000 7f19a6e00640 1 -- start start 2024-09-21T07:46:09.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.460+0000 7f19a6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19a00735e0 0x7f19a01f9c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.460+0000 7f19a6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a0145450 0x7f19a00b47a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.460+0000 7f19a6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 0x7f19a0139980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.460+0000 7f19a6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0160c70 con 0x7f19a0160320 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.460+0000 7f19a6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0071020 con 0x7f19a00735e0 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.460+0000 7f19a6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f19a0160df0 con 0x7f19a0145450 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 0x7f19a0139980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 0x7f19a0139980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49842/0 (socket says 172.21.15.88:49842) 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 -- 172.21.15.88:0/974532197 learned_addr learned my addr 172.21.15.88:0/974532197 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f199fe00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19a00735e0 0x7f19a01f9c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f199f400640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a0145450 0x7f19a00b47a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 -- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a0145450 msgr2=0x7f19a00b47a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a0145450 0x7f19a00b47a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 -- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19a00735e0 msgr2=0x7f19a01f9c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19a00735e0 0x7f19a01f9c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 -- 172.21.15.88:0/974532197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a00b4ce0 con 0x7f19a0160320 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f199f400640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a0145450 0x7f19a00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.461+0000 7f19a4e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 0x7f19a0139980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f199800ea70 tx=0x7f199800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.462+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199800ce40 con 0x7f19a0160320 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.462+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1998004c10 con 0x7f19a0160320 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.462+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a01f9140 con 0x7f19a0160320 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.462+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1998005020 con 0x7f19a0160320 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.462+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a00b5440 con 0x7f19a0160320 2024-09-21T07:46:09.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.463+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a00cee40 con 0x7f19a0160320 2024-09-21T07:46:09.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.464+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f19980051c0 con 0x7f19a0160320 2024-09-21T07:46:09.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.465+0000 7f1997e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1980078b90 0x7f198007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:09.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.466+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f199809bc20 con 0x7f19a0160320 2024-09-21T07:46:09.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.466+0000 7f199fe00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1980078b90 0x7f198007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:09.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.467+0000 7f199fe00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1980078b90 0x7f198007b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1990033000 tx=0x7f1990004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:09.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.470+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19a00cee40 con 0x7f19a0160320 2024-09-21T07:46:09.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:09.693+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f19a00b4eb0 con 0x7f19a0160320 2024-09-21T07:46:10.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.218+0000 7f1997e00640 1 -- 172.21.15.88:0/974532197 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) ==== 122+0+0 (secure 0 0 0) 0x7f1998063fc0 con 0x7f19a0160320 2024-09-21T07:46:10.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1980078b90 msgr2=0x7f198007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1980078b90 0x7f198007b0a0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f1990033000 tx=0x7f1990004730 comp rx=0 tx=0).stop 2024-09-21T07:46:10.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 msgr2=0x7f19a0139980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 0x7f19a0139980 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f199800ea70 tx=0x7f199800ef30 comp rx=0 tx=0).stop 2024-09-21T07:46:10.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 shutdown_connections 2024-09-21T07:46:10.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1980078b90 0x7f198007b0a0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19a0145450 0x7f19a00b47a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19a0160320 0x7f19a0139980 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 --2- 172.21.15.88:0/974532197 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19a00735e0 0x7f19a01f9c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 >> 172.21.15.88:0/974532197 conn(0x7f19a006d1c0 msgr2=0x7f19a00b12b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 shutdown_connections 2024-09-21T07:46:10.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.221+0000 7f19a6e00640 1 -- 172.21.15.88:0/974532197 wait complete. 2024-09-21T07:46:10.247 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:46:10.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:10.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 -- 172.21.15.88:0/4220332607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc0ad9c0 msgr2=0x7f2bdc0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4220332607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc0ad9c0 0x7f2bdc0af170 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f2bdc06e790 tx=0x7f2bd002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 -- 172.21.15.88:0/4220332607 shutdown_connections 2024-09-21T07:46:10.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4220332607 >> v2:172.21.15.175:3300/0 conn(0x7f2bdc0b2f60 0x7f2bdc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4220332607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc0ad9c0 0x7f2bdc0af170 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 -- 172.21.15.88:0/4220332607 >> 172.21.15.88:0/4220332607 conn(0x7f2bdc0b1e40 msgr2=0x7f2bdc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 -- 172.21.15.88:0/4220332607 shutdown_connections 2024-09-21T07:46:10.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.464+0000 7f2be2a00640 1 -- 172.21.15.88:0/4220332607 wait complete. 2024-09-21T07:46:10.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.466+0000 7f2be2a00640 1 Processor -- start 2024-09-21T07:46:10.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.466+0000 7f2be2a00640 1 -- start start 2024-09-21T07:46:10.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bdc0ad9c0 0x7f2bdc149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc0b2f60 0x7f2bdc09dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 0x7f2bdc0ce9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc06f290 con 0x7f2bdc15d670 2024-09-21T07:46:10.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bdc070d50 con 0x7f2bdc0ad9c0 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bdc06f410 con 0x7f2bdc0b2f60 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 0x7f2bdc0ce9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 0x7f2bdc0ce9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49862/0 (socket says 172.21.15.88:49862) 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 -- 172.21.15.88:0/4184070622 learned_addr learned my addr 172.21.15.88:0/4184070622 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be1600640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bdc0ad9c0 0x7f2bdc149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be0c00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc0b2f60 0x7f2bdc09dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 -- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc0b2f60 msgr2=0x7f2bdc09dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc0b2f60 0x7f2bdc09dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 -- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bdc0ad9c0 msgr2=0x7f2bdc149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bdc0ad9c0 0x7f2bdc149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be2000640 1 -- 172.21.15.88:0/4184070622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bdc14d3d0 con 0x7f2bdc15d670 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.468+0000 7f2be0c00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc0b2f60 0x7f2bdc09dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.469+0000 7f2be2000640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 0x7f2bdc0ce9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400be60 tx=0x7f2bd400bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.469+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4019070 con 0x7f2bdc15d670 2024-09-21T07:46:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.469+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd4014440 con 0x7f2bdc15d670 2024-09-21T07:46:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.469+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bdc14a0e0 con 0x7f2bdc15d670 2024-09-21T07:46:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.469+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd40148d0 con 0x7f2bdc15d670 2024-09-21T07:46:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.469+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bdc158f30 con 0x7f2bdc15d670 2024-09-21T07:46:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.470+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bdc158ca0 con 0x7f2bdc15d670 2024-09-21T07:46:10.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.471+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2bd401c070 con 0x7f2bdc15d670 2024-09-21T07:46:10.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.472+0000 7f2bd9600640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bbc078b90 0x7f2bbc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.472+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2bd409ad80 con 0x7f2bdc15d670 2024-09-21T07:46:10.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.473+0000 7f2be1600640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bbc078b90 0x7f2bbc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.473+0000 7f2be1600640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bbc078b90 0x7f2bbc07b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0033000 tx=0x7f2bd0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.477+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2bdc14a0e0 con 0x7f2bdc15d670 2024-09-21T07:46:10.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.699+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bdc0b00b0 con 0x7f2bdc15d670 2024-09-21T07:46:10.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.700+0000 7f2bd9600640 1 -- 172.21.15.88:0/4184070622 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f2bdc14d3d0 con 0x7f2bdc15d670 2024-09-21T07:46:10.702 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:10.702 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:10.703 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bbc078b90 msgr2=0x7f2bbc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bbc078b90 0x7f2bbc07b0a0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2bd0033000 tx=0x7f2bd0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 msgr2=0x7f2bdc0ce9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 0x7f2bdc0ce9b0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2bd400be60 tx=0x7f2bd400bf50 comp rx=0 tx=0).stop 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 shutdown_connections 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bbc078b90 0x7f2bbc07b0a0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bdc0b2f60 0x7f2bdc09dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bdc15d670 0x7f2bdc0ce9b0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 --2- 172.21.15.88:0/4184070622 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bdc0ad9c0 0x7f2bdc149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 >> 172.21.15.88:0/4184070622 conn(0x7f2bdc0b1e40 msgr2=0x7f2bdc0b05d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 shutdown_connections 2024-09-21T07:46:10.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.703+0000 7f2be2a00640 1 -- 172.21.15.88:0/4184070622 wait complete. 2024-09-21T07:46:10.729 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.944+0000 7fdf04800640 1 -- 172.21.15.88:0/3847864904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc15d760 msgr2=0x7fdefc1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.944+0000 7fdf04800640 1 --2- 172.21.15.88:0/3847864904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc15d760 0x7fdefc1535c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdef0009830 tx=0x7fdef002a530 comp rx=0 tx=0).stop 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.945+0000 7fdf04800640 1 -- 172.21.15.88:0/3847864904 shutdown_connections 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.945+0000 7fdf04800640 1 --2- 172.21.15.88:0/3847864904 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc15d760 0x7fdefc1535c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.945+0000 7fdf04800640 1 -- 172.21.15.88:0/3847864904 >> 172.21.15.88:0/3847864904 conn(0x7fdefc06d730 msgr2=0x7fdefc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.945+0000 7fdf04800640 1 -- 172.21.15.88:0/3847864904 shutdown_connections 2024-09-21T07:46:10.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.945+0000 7fdf04800640 1 -- 172.21.15.88:0/3847864904 wait complete. 2024-09-21T07:46:10.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.947+0000 7fdf04800640 1 Processor -- start 2024-09-21T07:46:10.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.947+0000 7fdf04800640 1 -- start start 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.948+0000 7fdf04800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc0ad0e0 0x7fdefc150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.948+0000 7fdf04800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 0x7fdefc1f9000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.948+0000 7fdf04800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc15d760 0x7fdefc1f9540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.948+0000 7fdf04800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc06f240 con 0x7fdefc0b3550 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.948+0000 7fdf04800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdefc0b4e30 con 0x7fdefc0ad0e0 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.948+0000 7fdf04800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdefc0b4070 con 0x7fdefc15d760 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 0x7fdefc1f9000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 0x7fdefc1f9000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49872/0 (socket says 172.21.15.88:49872) 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 -- 172.21.15.88:0/2659061624 learned_addr learned my addr 172.21.15.88:0/2659061624 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01e00640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc0ad0e0 0x7fdefc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf02800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc15d760 0x7fdefc1f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 -- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc15d760 msgr2=0x7fdefc1f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc15d760 0x7fdefc1f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 -- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc0ad0e0 msgr2=0x7fdefc150180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:10.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc0ad0e0 0x7fdefc150180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 -- 172.21.15.88:0/2659061624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdefc1f9e60 con 0x7fdefc0b3550 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01e00640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc0ad0e0 0x7fdefc150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.949+0000 7fdf01400640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 0x7fdefc1f9000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00c430 tx=0x7fdeec00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.950+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec017070 con 0x7fdefc0b3550 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.950+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdeec0053c0 con 0x7fdefc0b3550 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.950+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdefc150d60 con 0x7fdefc0b3550 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.950+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeec003850 con 0x7fdefc0b3550 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.950+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdefc14d610 con 0x7fdefc0b3550 2024-09-21T07:46:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.951+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdefc150ad0 con 0x7fdefc0b3550 2024-09-21T07:46:10.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.952+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdeec01a070 con 0x7fdefc0b3550 2024-09-21T07:46:10.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.953+0000 7fdef6000640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdee0078b90 0x7fdee007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:10.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.953+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdeec09a080 con 0x7fdefc0b3550 2024-09-21T07:46:10.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.954+0000 7fdf01e00640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdee0078b90 0x7fdee007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:10.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.954+0000 7fdf01e00640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdee0078b90 0x7fdee007b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdef0034000 tx=0x7fdef0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:10.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:10.958+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdefc1f9e60 con 0x7fdefc0b3550 2024-09-21T07:46:11.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.144+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdefc1535c0 con 0x7fdefc0b3550 2024-09-21T07:46:11.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.145+0000 7fdef6000640 1 -- 172.21.15.88:0/2659061624 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fdeec061e60 con 0x7fdefc0b3550 2024-09-21T07:46:11.147 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:11.148 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:11.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdee0078b90 msgr2=0x7fdee007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdee0078b90 0x7fdee007b0a0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdef0034000 tx=0x7fdef0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:11.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 msgr2=0x7fdefc1f9000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 0x7fdefc1f9000 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00c430 tx=0x7fdeec00c8f0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 shutdown_connections 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdee0078b90 0x7fdee007b0a0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdefc15d760 0x7fdefc1f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdefc0b3550 0x7fdefc1f9000 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 --2- 172.21.15.88:0/2659061624 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdefc0ad0e0 0x7fdefc150180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 >> 172.21.15.88:0/2659061624 conn(0x7fdefc06d730 msgr2=0x7fdefc15dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.147+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 shutdown_connections 2024-09-21T07:46:11.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.148+0000 7fdf04800640 1 -- 172.21.15.88:0/2659061624 wait complete. 2024-09-21T07:46:11.171 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:46:11.171 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 -- 172.21.15.88:0/1149713610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58540ad0e0 msgr2=0x7f58540ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 --2- 172.21.15.88:0/1149713610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58540ad0e0 0x7f58540ad4f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f583c009830 tx=0x7f583c02a410 comp rx=0 tx=0).stop 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 -- 172.21.15.88:0/1149713610 shutdown_connections 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 --2- 172.21.15.88:0/1149713610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 0x7f58540b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 --2- 172.21.15.88:0/1149713610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58540ad0e0 0x7f58540ad4f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 -- 172.21.15.88:0/1149713610 >> 172.21.15.88:0/1149713610 conn(0x7f585406d730 msgr2=0x7f58540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 -- 172.21.15.88:0/1149713610 shutdown_connections 2024-09-21T07:46:11.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.387+0000 7f5858e00640 1 -- 172.21.15.88:0/1149713610 wait complete. 2024-09-21T07:46:11.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.389+0000 7f5858e00640 1 Processor -- start 2024-09-21T07:46:11.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.390+0000 7f5858e00640 1 -- start start 2024-09-21T07:46:11.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5858e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 0x7f5854146630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5858e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 0x7f58540b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5858e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585415d760 0x7f58540b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5858e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58540b4da0 con 0x7f58540ad0e0 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5858e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58540b4c20 con 0x7f585415d760 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5858e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58540b3ea0 con 0x7f58540b3550 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5851e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 0x7f5854146630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5851e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 0x7f5854146630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49878/0 (socket says 172.21.15.88:49878) 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.391+0000 7f5851e00640 1 -- 172.21.15.88:0/343622201 learned_addr learned my addr 172.21.15.88:0/343622201 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5849400640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 0x7f58540b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5852800640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585415d760 0x7f58540b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5851e00640 1 -- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 msgr2=0x7f58540b0da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5851e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 0x7f58540b0da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5851e00640 1 -- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585415d760 msgr2=0x7f58540b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5851e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585415d760 0x7f58540b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5851e00640 1 -- 172.21.15.88:0/343622201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5854158870 con 0x7f58540ad0e0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5849400640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 0x7f58540b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5852800640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585415d760 0x7f58540b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5851e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 0x7f5854146630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f584400c500 tx=0x7f584400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844016020 con 0x7f58540ad0e0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5844012940 con 0x7f58540ad0e0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844012d90 con 0x7f58540ad0e0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5854158b50 con 0x7f58540ad0e0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.392+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58541f3b20 con 0x7f58540ad0e0 2024-09-21T07:46:11.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.393+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58540b4380 con 0x7f58540ad0e0 2024-09-21T07:46:11.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.394+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5844019020 con 0x7f58540ad0e0 2024-09-21T07:46:11.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.396+0000 7f584aa00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5830078b90 0x7f583007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.396+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f584409ad60 con 0x7f58540ad0e0 2024-09-21T07:46:11.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.397+0000 7f5849400640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5830078b90 0x7f583007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.397+0000 7f5849400640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5830078b90 0x7f583007b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f583c034000 tx=0x7f583c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.401+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5844020020 con 0x7f58540ad0e0 2024-09-21T07:46:11.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.583+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5854158620 con 0x7f5830078b90 2024-09-21T07:46:11.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.589+0000 7f584aa00640 1 -- 172.21.15.88:0/343622201 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+245562 (secure 0 0 0) 0x7f5854158620 con 0x7f5830078b90 2024-09-21T07:46:11.594 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:11.595 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:46:11.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5830078b90 msgr2=0x7f583007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5830078b90 0x7f583007b0a0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f583c034000 tx=0x7f583c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 msgr2=0x7f5854146630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 0x7f5854146630 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f584400c500 tx=0x7f584400c9c0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 shutdown_connections 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5830078b90 0x7f583007b0a0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58540b3550 0x7f58540b0da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585415d760 0x7f58540b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 --2- 172.21.15.88:0/343622201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58540ad0e0 0x7f5854146630 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 >> 172.21.15.88:0/343622201 conn(0x7f585406d730 msgr2=0x7f58540cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 shutdown_connections 2024-09-21T07:46:11.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.593+0000 7f5858e00640 1 -- 172.21.15.88:0/343622201 wait complete. 2024-09-21T07:46:11.617 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-21T07:46:09.854844+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591144,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":129,"num_write_kb":1638,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":95,"ondisk_log_size":95,"up":124,"acting":124,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":71,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219288,"kb_used_data":2808,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755432,"statfs":{"total":773094113280,"available":772869562368,"internally_reserved":0,"allocated":2875392,"data_stored":1726582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":776,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.677077"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208860+0000","last_change":"2024-09-21T07:46:07.219201+0000","last_active":"2024-09-21T07:46:08.208860+0000","last_peered":"2024-09-21T07:46:08.208860+0000","last_clean":"2024-09-21T07:46:08.208860+0000","last_became_active":"2024-09-21T07:46:07.218520+0000","last_became_peered":"2024-09-21T07:46:07.218520+0000","last_unstale":"2024-09-21T07:46:08.208860+0000","last_undegraded":"2024-09-21T07:46:08.208860+0000","last_fullsized":"2024-09-21T07:46:08.208860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:38.468021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208382+0000","last_change":"2024-09-21T07:46:07.225678+0000","last_active":"2024-09-21T07:46:08.208382+0000","last_peered":"2024-09-21T07:46:08.208382+0000","last_clean":"2024-09-21T07:46:08.208382+0000","last_became_active":"2024-09-21T07:46:07.225167+0000","last_became_peered":"2024-09-21T07:46:07.225167+0000","last_unstale":"2024-09-21T07:46:08.208382+0000","last_undegraded":"2024-09-21T07:46:08.208382+0000","last_fullsized":"2024-09-21T07:46:08.208382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208796+0000","last_change":"2024-09-21T07:46:07.217547+0000","last_active":"2024-09-21T07:46:08.208796+0000","last_peered":"2024-09-21T07:46:08.208796+0000","last_clean":"2024-09-21T07:46:08.208796+0000","last_became_active":"2024-09-21T07:46:07.215538+0000","last_became_peered":"2024-09-21T07:46:07.215538+0000","last_unstale":"2024-09-21T07:46:08.208796+0000","last_undegraded":"2024-09-21T07:46:08.208796+0000","last_fullsized":"2024-09-21T07:46:08.208796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:00.117867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521098+0000","last_change":"2024-09-21T07:46:07.238701+0000","last_active":"2024-09-21T07:46:08.521098+0000","last_peered":"2024-09-21T07:46:08.521098+0000","last_clean":"2024-09-21T07:46:08.521098+0000","last_became_active":"2024-09-21T07:46:07.238406+0000","last_became_peered":"2024-09-21T07:46:07.238406+0000","last_unstale":"2024-09-21T07:46:08.521098+0000","last_undegraded":"2024-09-21T07:46:08.521098+0000","last_fullsized":"2024-09-21T07:46:08.521098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521245+0000","last_change":"2024-09-21T07:46:07.211337+0000","last_active":"2024-09-21T07:46:08.521245+0000","last_peered":"2024-09-21T07:46:08.521245+0000","last_clean":"2024-09-21T07:46:08.521245+0000","last_became_active":"2024-09-21T07:46:07.210983+0000","last_became_peered":"2024-09-21T07:46:07.210983+0000","last_unstale":"2024-09-21T07:46:08.521245+0000","last_undegraded":"2024-09-21T07:46:08.521245+0000","last_fullsized":"2024-09-21T07:46:08.521245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:43.997921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209926+0000","last_change":"2024-09-21T07:46:07.225928+0000","last_active":"2024-09-21T07:46:08.209926+0000","last_peered":"2024-09-21T07:46:08.209926+0000","last_clean":"2024-09-21T07:46:08.209926+0000","last_became_active":"2024-09-21T07:46:07.225685+0000","last_became_peered":"2024-09-21T07:46:07.225685+0000","last_unstale":"2024-09-21T07:46:08.209926+0000","last_undegraded":"2024-09-21T07:46:08.209926+0000","last_fullsized":"2024-09-21T07:46:08.209926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.222961+0000","last_change":"2024-09-21T07:46:07.221477+0000","last_active":"2024-09-21T07:46:07.222961+0000","last_peered":"2024-09-21T07:46:07.222961+0000","last_clean":"2024-09-21T07:46:07.222961+0000","last_became_active":"2024-09-21T07:46:07.217967+0000","last_became_peered":"2024-09-21T07:46:07.217967+0000","last_unstale":"2024-09-21T07:46:07.222961+0000","last_undegraded":"2024-09-21T07:46:07.222961+0000","last_fullsized":"2024-09-21T07:46:07.222961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:34.913461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207714+0000","last_change":"2024-09-21T07:46:07.220220+0000","last_active":"2024-09-21T07:46:08.207714+0000","last_peered":"2024-09-21T07:46:08.207714+0000","last_clean":"2024-09-21T07:46:08.207714+0000","last_became_active":"2024-09-21T07:46:07.219890+0000","last_became_peered":"2024-09-21T07:46:07.219890+0000","last_unstale":"2024-09-21T07:46:08.207714+0000","last_undegraded":"2024-09-21T07:46:08.207714+0000","last_fullsized":"2024-09-21T07:46:08.207714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.210100+0000","last_change":"2024-09-21T07:46:07.215158+0000","last_active":"2024-09-21T07:46:08.210100+0000","last_peered":"2024-09-21T07:46:08.210100+0000","last_clean":"2024-09-21T07:46:08.210100+0000","last_became_active":"2024-09-21T07:46:07.214842+0000","last_became_peered":"2024-09-21T07:46:07.214842+0000","last_unstale":"2024-09-21T07:46:08.210100+0000","last_undegraded":"2024-09-21T07:46:08.210100+0000","last_fullsized":"2024-09-21T07:46:08.210100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208288+0000","last_change":"2024-09-21T07:46:07.218857+0000","last_active":"2024-09-21T07:46:08.208288+0000","last_peered":"2024-09-21T07:46:08.208288+0000","last_clean":"2024-09-21T07:46:08.208288+0000","last_became_active":"2024-09-21T07:46:07.218490+0000","last_became_peered":"2024-09-21T07:46:07.218490+0000","last_unstale":"2024-09-21T07:46:08.208288+0000","last_undegraded":"2024-09-21T07:46:08.208288+0000","last_fullsized":"2024-09-21T07:46:08.208288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521582+0000","last_change":"2024-09-21T07:46:07.213848+0000","last_active":"2024-09-21T07:46:08.521582+0000","last_peered":"2024-09-21T07:46:08.521582+0000","last_clean":"2024-09-21T07:46:08.521582+0000","last_became_active":"2024-09-21T07:46:07.212350+0000","last_became_peered":"2024-09-21T07:46:07.212350+0000","last_unstale":"2024-09-21T07:46:08.521582+0000","last_undegraded":"2024-09-21T07:46:08.521582+0000","last_fullsized":"2024-09-21T07:46:08.521582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521460+0000","last_change":"2024-09-21T07:46:07.231247+0000","last_active":"2024-09-21T07:46:08.521460+0000","last_peered":"2024-09-21T07:46:08.521460+0000","last_clean":"2024-09-21T07:46:08.521460+0000","last_became_active":"2024-09-21T07:46:07.230913+0000","last_became_peered":"2024-09-21T07:46:07.230913+0000","last_unstale":"2024-09-21T07:46:08.521460+0000","last_undegraded":"2024-09-21T07:46:08.521460+0000","last_fullsized":"2024-09-21T07:46:08.521460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208726+0000","last_change":"2024-09-21T07:46:07.212029+0000","last_active":"2024-09-21T07:46:08.208726+0000","last_peered":"2024-09-21T07:46:08.208726+0000","last_clean":"2024-09-21T07:46:08.208726+0000","last_became_active":"2024-09-21T07:46:07.211701+0000","last_became_peered":"2024-09-21T07:46:07.211701+0000","last_unstale":"2024-09-21T07:46:08.208726+0000","last_undegraded":"2024-09-21T07:46:08.208726+0000","last_fullsized":"2024-09-21T07:46:08.208726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.216348+0000","last_change":"2024-09-21T07:46:07.213929+0000","last_active":"2024-09-21T07:46:07.216348+0000","last_peered":"2024-09-21T07:46:07.216348+0000","last_clean":"2024-09-21T07:46:07.216348+0000","last_became_active":"2024-09-21T07:46:07.213456+0000","last_became_peered":"2024-09-21T07:46:07.213456+0000","last_unstale":"2024-09-21T07:46:07.216348+0000","last_undegraded":"2024-09-21T07:46:07.216348+0000","last_fullsized":"2024-09-21T07:46:07.216348+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:50.786316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208870+0000","last_change":"2024-09-21T07:46:06.210978+0000","last_active":"2024-09-21T07:46:08.208870+0000","last_peered":"2024-09-21T07:46:08.208870+0000","last_clean":"2024-09-21T07:46:08.208870+0000","last_became_active":"2024-09-21T07:46:06.210615+0000","last_became_peered":"2024-09-21T07:46:06.210615+0000","last_unstale":"2024-09-21T07:46:08.208870+0000","last_undegraded":"2024-09-21T07:46:08.208870+0000","last_fullsized":"2024-09-21T07:46:08.208870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:12.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207933+0000","last_change":"2024-09-21T07:46:07.236139+0000","last_active":"2024-09-21T07:46:08.207933+0000","last_peered":"2024-09-21T07:46:08.207933+0000","last_clean":"2024-09-21T07:46:08.207933+0000","last_became_active":"2024-09-21T07:46:07.235357+0000","last_became_peered":"2024-09-21T07:46:07.235357+0000","last_unstale":"2024-09-21T07:46:08.207933+0000","last_undegraded":"2024-09-21T07:46:08.207933+0000","last_fullsized":"2024-09-21T07:46:08.207933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.185906+0000","last_change":"2024-09-21T07:46:05.185906+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.185906+0000","last_undegraded":"2024-09-21T07:46:05.185906+0000","last_fullsized":"2024-09-21T07:46:05.185906+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.520888+0000","last_change":"2024-09-21T07:46:06.213787+0000","last_active":"2024-09-21T07:46:08.520888+0000","last_peered":"2024-09-21T07:46:08.520888+0000","last_clean":"2024-09-21T07:46:08.520888+0000","last_became_active":"2024-09-21T07:46:06.213454+0000","last_became_peered":"2024-09-21T07:46:06.213454+0000","last_unstale":"2024-09-21T07:46:08.520888+0000","last_undegraded":"2024-09-21T07:46:08.520888+0000","last_fullsized":"2024-09-21T07:46:08.520888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208744+0000","last_change":"2024-09-21T07:46:07.217173+0000","last_active":"2024-09-21T07:46:08.208744+0000","last_peered":"2024-09-21T07:46:08.208744+0000","last_clean":"2024-09-21T07:46:08.208744+0000","last_became_active":"2024-09-21T07:46:07.216857+0000","last_became_peered":"2024-09-21T07:46:07.216857+0000","last_unstale":"2024-09-21T07:46:08.208744+0000","last_undegraded":"2024-09-21T07:46:08.208744+0000","last_fullsized":"2024-09-21T07:46:08.208744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.185244+0000","last_change":"2024-09-21T07:46:05.185244+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.185244+0000","last_undegraded":"2024-09-21T07:46:05.185244+0000","last_fullsized":"2024-09-21T07:46:05.185244+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207314+0000","last_change":"2024-09-21T07:46:07.217122+0000","last_active":"2024-09-21T07:46:08.207314+0000","last_peered":"2024-09-21T07:46:08.207314+0000","last_clean":"2024-09-21T07:46:08.207314+0000","last_became_active":"2024-09-21T07:46:07.216831+0000","last_became_peered":"2024-09-21T07:46:07.216831+0000","last_unstale":"2024-09-21T07:46:08.207314+0000","last_undegraded":"2024-09-21T07:46:08.207314+0000","last_fullsized":"2024-09-21T07:46:08.207314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.183242+0000","last_change":"2024-09-21T07:46:05.183242+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.183242+0000","last_undegraded":"2024-09-21T07:46:05.183242+0000","last_fullsized":"2024-09-21T07:46:05.183242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.187704+0000","last_change":"2024-09-21T07:46:05.187704+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.187704+0000","last_undegraded":"2024-09-21T07:46:05.187704+0000","last_fullsized":"2024-09-21T07:46:05.187704+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208537+0000","last_change":"2024-09-21T07:46:07.210270+0000","last_active":"2024-09-21T07:46:08.208537+0000","last_peered":"2024-09-21T07:46:08.208537+0000","last_clean":"2024-09-21T07:46:08.208537+0000","last_became_active":"2024-09-21T07:46:07.202021+0000","last_became_peered":"2024-09-21T07:46:07.202021+0000","last_unstale":"2024-09-21T07:46:08.208537+0000","last_undegraded":"2024-09-21T07:46:08.208537+0000","last_fullsized":"2024-09-21T07:46:08.208537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.223921+0000","last_change":"2024-09-21T07:46:06.212266+0000","last_active":"2024-09-21T07:46:07.223921+0000","last_peered":"2024-09-21T07:46:07.223921+0000","last_clean":"2024-09-21T07:46:07.223921+0000","last_became_active":"2024-09-21T07:46:06.210122+0000","last_became_peered":"2024-09-21T07:46:06.210122+0000","last_unstale":"2024-09-21T07:46:07.223921+0000","last_undegraded":"2024-09-21T07:46:07.223921+0000","last_fullsized":"2024-09-21T07:46:07.223921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:13.095998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.190396+0000","last_change":"2024-09-21T07:46:05.190396+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.190396+0000","last_undegraded":"2024-09-21T07:46:05.190396+0000","last_fullsized":"2024-09-21T07:46:05.190396+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.238063+0000","last_change":"2024-09-21T07:46:06.212633+0000","last_active":"2024-09-21T07:46:07.238063+0000","last_peered":"2024-09-21T07:46:07.238063+0000","last_clean":"2024-09-21T07:46:07.238063+0000","last_became_active":"2024-09-21T07:46:06.212240+0000","last_became_peered":"2024-09-21T07:46:06.212240+0000","last_unstale":"2024-09-21T07:46:07.238063+0000","last_undegraded":"2024-09-21T07:46:07.238063+0000","last_fullsized":"2024-09-21T07:46:07.238063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:36.198191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521167+0000","last_change":"2024-09-21T07:46:07.216368+0000","last_active":"2024-09-21T07:46:08.521167+0000","last_peered":"2024-09-21T07:46:08.521167+0000","last_clean":"2024-09-21T07:46:08.521167+0000","last_became_active":"2024-09-21T07:46:07.216061+0000","last_became_peered":"2024-09-21T07:46:07.216061+0000","last_unstale":"2024-09-21T07:46:08.521167+0000","last_undegraded":"2024-09-21T07:46:08.521167+0000","last_fullsized":"2024-09-21T07:46:08.521167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.185257+0000","last_change":"2024-09-21T07:46:05.185257+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.185257+0000","last_undegraded":"2024-09-21T07:46:05.185257+0000","last_fullsized":"2024-09-21T07:46:05.185257+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.187493+0000","last_change":"2024-09-21T07:46:05.187493+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.187493+0000","last_undegraded":"2024-09-21T07:46:05.187493+0000","last_fullsized":"2024-09-21T07:46:05.187493+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207016+0000","last_change":"2024-09-21T07:46:06.202414+0000","last_active":"2024-09-21T07:46:08.207016+0000","last_peered":"2024-09-21T07:46:08.207016+0000","last_clean":"2024-09-21T07:46:08.207016+0000","last_became_active":"2024-09-21T07:46:06.201528+0000","last_became_peered":"2024-09-21T07:46:06.201528+0000","last_unstale":"2024-09-21T07:46:08.207016+0000","last_undegraded":"2024-09-21T07:46:08.207016+0000","last_fullsized":"2024-09-21T07:46:08.207016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.f","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.188726+0000","last_change":"2024-09-21T07:46:05.188726+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.188726+0000","last_undegraded":"2024-09-21T07:46:05.188726+0000","last_fullsized":"2024-09-21T07:46:05.188726+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208692+0000","last_change":"2024-09-21T07:46:06.208887+0000","last_active":"2024-09-21T07:46:08.208692+0000","last_peered":"2024-09-21T07:46:08.208692+0000","last_clean":"2024-09-21T07:46:08.208692+0000","last_became_active":"2024-09-21T07:46:06.208508+0000","last_became_peered":"2024-09-21T07:46:06.208508+0000","last_unstale":"2024-09-21T07:46:08.208692+0000","last_undegraded":"2024-09-21T07:46:08.208692+0000","last_fullsized":"2024-09-21T07:46:08.208692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208511+0000","last_change":"2024-09-21T07:46:06.209460+0000","last_active":"2024-09-21T07:46:08.208511+0000","last_peered":"2024-09-21T07:46:08.208511+0000","last_clean":"2024-09-21T07:46:08.208511+0000","last_became_active":"2024-09-21T07:46:06.207853+0000","last_became_peered":"2024-09-21T07:46:06.207853+0000","last_unstale":"2024-09-21T07:46:08.208511+0000","last_undegraded":"2024-09-21T07:46:08.208511+0000","last_fullsized":"2024-09-21T07:46:08.208511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521589+0000","last_change":"2024-09-21T07:46:06.216893+0000","last_active":"2024-09-21T07:46:08.521589+0000","last_peered":"2024-09-21T07:46:08.521589+0000","last_clean":"2024-09-21T07:46:08.521589+0000","last_became_active":"2024-09-21T07:46:06.216657+0000","last_became_peered":"2024-09-21T07:46:06.216657+0000","last_unstale":"2024-09-21T07:46:08.521589+0000","last_undegraded":"2024-09-21T07:46:08.521589+0000","last_fullsized":"2024-09-21T07:46:08.521589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":105,"reported_epoch":20,"state":"active+clean","last_fresh":"2024-09-21T07:46:05.181421+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:05.181421+0000","last_peered":"2024-09-21T07:46:05.181421+0000","last_clean":"2024-09-21T07:46:05.181421+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:05.181421+0000","last_undegraded":"2024-09-21T07:46:05.181421+0000","last_fullsized":"2024-09-21T07:46:05.181421+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.184431+0000","last_change":"2024-09-21T07:46:05.184431+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.184431+0000","last_undegraded":"2024-09-21T07:46:05.184431+0000","last_fullsized":"2024-09-21T07:46:05.184431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207407+0000","last_change":"2024-09-21T07:46:07.222509+0000","last_active":"2024-09-21T07:46:08.207407+0000","last_peered":"2024-09-21T07:46:08.207407+0000","last_clean":"2024-09-21T07:46:08.207407+0000","last_became_active":"2024-09-21T07:46:07.221105+0000","last_became_peered":"2024-09-21T07:46:07.221105+0000","last_unstale":"2024-09-21T07:46:08.207407+0000","last_undegraded":"2024-09-21T07:46:08.207407+0000","last_fullsized":"2024-09-21T07:46:08.207407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.184428+0000","last_change":"2024-09-21T07:46:05.184428+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.184428+0000","last_undegraded":"2024-09-21T07:46:05.184428+0000","last_fullsized":"2024-09-21T07:46:05.184428+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521036+0000","last_change":"2024-09-21T07:46:07.218927+0000","last_active":"2024-09-21T07:46:08.521036+0000","last_peered":"2024-09-21T07:46:08.521036+0000","last_clean":"2024-09-21T07:46:08.521036+0000","last_became_active":"2024-09-21T07:46:07.218594+0000","last_became_peered":"2024-09-21T07:46:07.218594+0000","last_unstale":"2024-09-21T07:46:08.521036+0000","last_undegraded":"2024-09-21T07:46:08.521036+0000","last_fullsized":"2024-09-21T07:46:08.521036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.201232+0000","last_change":"2024-09-21T07:46:06.208141+0000","last_active":"2024-09-21T07:46:07.201232+0000","last_peered":"2024-09-21T07:46:07.201232+0000","last_clean":"2024-09-21T07:46:07.201232+0000","last_became_active":"2024-09-21T07:46:06.207799+0000","last_became_peered":"2024-09-21T07:46:06.207799+0000","last_unstale":"2024-09-21T07:46:07.201232+0000","last_undegraded":"2024-09-21T07:46:07.201232+0000","last_fullsized":"2024-09-21T07:46:07.201232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:02.343379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208515+0000","last_change":"2024-09-21T07:46:07.208889+0000","last_active":"2024-09-21T07:46:08.208515+0000","last_peered":"2024-09-21T07:46:08.208515+0000","last_clean":"2024-09-21T07:46:08.208515+0000","last_became_active":"2024-09-21T07:46:07.208365+0000","last_became_peered":"2024-09-21T07:46:07.208365+0000","last_unstale":"2024-09-21T07:46:08.208515+0000","last_undegraded":"2024-09-21T07:46:08.208515+0000","last_fullsized":"2024-09-21T07:46:08.208515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.184424+0000","last_change":"2024-09-21T07:46:05.184424+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.184424+0000","last_undegraded":"2024-09-21T07:46:05.184424+0000","last_fullsized":"2024-09-21T07:46:05.184424+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209588+0000","last_change":"2024-09-21T07:46:06.213705+0000","last_active":"2024-09-21T07:46:08.209588+0000","last_peered":"2024-09-21T07:46:08.209588+0000","last_clean":"2024-09-21T07:46:08.209588+0000","last_became_active":"2024-09-21T07:46:06.213351+0000","last_became_peered":"2024-09-21T07:46:06.213351+0000","last_unstale":"2024-09-21T07:46:08.209588+0000","last_undegraded":"2024-09-21T07:46:08.209588+0000","last_fullsized":"2024-09-21T07:46:08.209588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.184431+0000","last_change":"2024-09-21T07:46:05.184431+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.184431+0000","last_undegraded":"2024-09-21T07:46:05.184431+0000","last_fullsized":"2024-09-21T07:46:05.184431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521474+0000","last_change":"2024-09-21T07:46:07.225723+0000","last_active":"2024-09-21T07:46:08.521474+0000","last_peered":"2024-09-21T07:46:08.521474+0000","last_clean":"2024-09-21T07:46:08.521474+0000","last_became_active":"2024-09-21T07:46:07.225424+0000","last_became_peered":"2024-09-21T07:46:07.225424+0000","last_unstale":"2024-09-21T07:46:08.521474+0000","last_undegraded":"2024-09-21T07:46:08.521474+0000","last_fullsized":"2024-09-21T07:46:08.521474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.196020+0000","last_change":"2024-09-21T07:46:06.203823+0000","last_active":"2024-09-21T07:46:09.196020+0000","last_peered":"2024-09-21T07:46:09.196020+0000","last_clean":"2024-09-21T07:46:09.196020+0000","last_became_active":"2024-09-21T07:46:06.203500+0000","last_became_peered":"2024-09-21T07:46:06.203500+0000","last_unstale":"2024-09-21T07:46:09.196020+0000","last_undegraded":"2024-09-21T07:46:09.196020+0000","last_fullsized":"2024-09-21T07:46:09.196020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208233+0000","last_change":"2024-09-21T07:46:07.222640+0000","last_active":"2024-09-21T07:46:08.208233+0000","last_peered":"2024-09-21T07:46:08.208233+0000","last_clean":"2024-09-21T07:46:08.208233+0000","last_became_active":"2024-09-21T07:46:07.220945+0000","last_became_peered":"2024-09-21T07:46:07.220945+0000","last_unstale":"2024-09-21T07:46:08.208233+0000","last_undegraded":"2024-09-21T07:46:08.208233+0000","last_fullsized":"2024-09-21T07:46:08.208233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:38:12.923471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.186518+0000","last_change":"2024-09-21T07:46:05.186518+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.186518+0000","last_undegraded":"2024-09-21T07:46:05.186518+0000","last_fullsized":"2024-09-21T07:46:05.186518+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208617+0000","last_change":"2024-09-21T07:46:07.220815+0000","last_active":"2024-09-21T07:46:08.208617+0000","last_peered":"2024-09-21T07:46:08.208617+0000","last_clean":"2024-09-21T07:46:08.208617+0000","last_became_active":"2024-09-21T07:46:07.220441+0000","last_became_peered":"2024-09-21T07:46:07.220441+0000","last_unstale":"2024-09-21T07:46:08.208617+0000","last_undegraded":"2024-09-21T07:46:08.208617+0000","last_fullsized":"2024-09-21T07:46:08.208617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.190435+0000","last_change":"2024-09-21T07:46:05.190435+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.190435+0000","last_undegraded":"2024-09-21T07:46:05.190435+0000","last_fullsized":"2024-09-21T07:46:05.190435+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207301+0000","last_change":"2024-09-21T07:46:06.206432+0000","last_active":"2024-09-21T07:46:08.207301+0000","last_peered":"2024-09-21T07:46:08.207301+0000","last_clean":"2024-09-21T07:46:08.207301+0000","last_became_active":"2024-09-21T07:46:06.203791+0000","last_became_peered":"2024-09-21T07:46:06.203791+0000","last_unstale":"2024-09-21T07:46:08.207301+0000","last_undegraded":"2024-09-21T07:46:08.207301+0000","last_fullsized":"2024-09-21T07:46:08.207301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207491+0000","last_change":"2024-09-21T07:46:07.214468+0000","last_active":"2024-09-21T07:46:08.207491+0000","last_peered":"2024-09-21T07:46:08.207491+0000","last_clean":"2024-09-21T07:46:08.207491+0000","last_became_active":"2024-09-21T07:46:07.213517+0000","last_became_peered":"2024-09-21T07:46:07.213517+0000","last_unstale":"2024-09-21T07:46:08.207491+0000","last_undegraded":"2024-09-21T07:46:08.207491+0000","last_fullsized":"2024-09-21T07:46:08.207491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521331+0000","last_change":"2024-09-21T07:46:06.211420+0000","last_active":"2024-09-21T07:46:08.521331+0000","last_peered":"2024-09-21T07:46:08.521331+0000","last_clean":"2024-09-21T07:46:08.521331+0000","last_became_active":"2024-09-21T07:46:06.208177+0000","last_became_peered":"2024-09-21T07:46:06.208177+0000","last_unstale":"2024-09-21T07:46:08.521331+0000","last_undegraded":"2024-09-21T07:46:08.521331+0000","last_fullsized":"2024-09-21T07:46:08.521331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.216520+0000","last_change":"2024-09-21T07:46:07.213281+0000","last_active":"2024-09-21T07:46:07.216520+0000","last_peered":"2024-09-21T07:46:07.216520+0000","last_clean":"2024-09-21T07:46:07.216520+0000","last_became_active":"2024-09-21T07:46:07.212947+0000","last_became_peered":"2024-09-21T07:46:07.212947+0000","last_unstale":"2024-09-21T07:46:07.216520+0000","last_undegraded":"2024-09-21T07:46:07.216520+0000","last_fullsized":"2024-09-21T07:46:07.216520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:17.897897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.185165+0000","last_change":"2024-09-21T07:46:05.185165+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.185165+0000","last_undegraded":"2024-09-21T07:46:05.185165+0000","last_fullsized":"2024-09-21T07:46:05.185165+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.190163+0000","last_change":"2024-09-21T07:46:05.190163+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.190163+0000","last_undegraded":"2024-09-21T07:46:05.190163+0000","last_fullsized":"2024-09-21T07:46:05.190163+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:46:06.176676+0000","last_change":"2024-09-21T07:46:06.176676+0000","last_active":"2024-09-21T07:46:06.176676+0000","last_peered":"2024-09-21T07:46:06.176676+0000","last_clean":"2024-09-21T07:46:06.176676+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:06.176676+0000","last_undegraded":"2024-09-21T07:46:06.176676+0000","last_fullsized":"2024-09-21T07:46:06.176676+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":1,"reported_epoch":20,"state":"creating+peering","last_fresh":"2024-09-21T07:46:05.188036+0000","last_change":"2024-09-21T07:46:05.188036+0000","last_active":"2024-09-21T07:46:05.169767+0000","last_peered":"2024-09-21T07:46:05.169767+0000","last_clean":"2024-09-21T07:46:05.169767+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:05.188036+0000","last_undegraded":"2024-09-21T07:46:05.188036+0000","last_fullsized":"2024-09-21T07:46:05.188036+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208316+0000","last_change":"2024-09-21T07:46:07.210700+0000","last_active":"2024-09-21T07:46:08.208316+0000","last_peered":"2024-09-21T07:46:08.208316+0000","last_clean":"2024-09-21T07:46:08.208316+0000","last_became_active":"2024-09-21T07:46:07.203963+0000","last_became_peered":"2024-09-21T07:46:07.203963+0000","last_unstale":"2024-09-21T07:46:08.208316+0000","last_undegraded":"2024-09-21T07:46:08.208316+0000","last_fullsized":"2024-09-21T07:46:08.208316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209121+0000","last_change":"2024-09-21T07:46:06.209967+0000","last_active":"2024-09-21T07:46:08.209121+0000","last_peered":"2024-09-21T07:46:08.209121+0000","last_clean":"2024-09-21T07:46:08.209121+0000","last_became_active":"2024-09-21T07:46:06.207030+0000","last_became_peered":"2024-09-21T07:46:06.207030+0000","last_unstale":"2024-09-21T07:46:08.209121+0000","last_undegraded":"2024-09-21T07:46:08.209121+0000","last_fullsized":"2024-09-21T07:46:08.209121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:27.264294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209044+0000","last_change":"2024-09-21T07:46:07.219277+0000","last_active":"2024-09-21T07:46:08.209044+0000","last_peered":"2024-09-21T07:46:08.209044+0000","last_clean":"2024-09-21T07:46:08.209044+0000","last_became_active":"2024-09-21T07:46:07.218947+0000","last_became_peered":"2024-09-21T07:46:07.218947+0000","last_unstale":"2024-09-21T07:46:08.209044+0000","last_undegraded":"2024-09-21T07:46:08.209044+0000","last_fullsized":"2024-09-21T07:46:08.209044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208464+0000","last_change":"2024-09-21T07:46:06.209178+0000","last_active":"2024-09-21T07:46:08.208464+0000","last_peered":"2024-09-21T07:46:08.208464+0000","last_clean":"2024-09-21T07:46:08.208464+0000","last_became_active":"2024-09-21T07:46:06.201083+0000","last_became_peered":"2024-09-21T07:46:06.201083+0000","last_unstale":"2024-09-21T07:46:08.208464+0000","last_undegraded":"2024-09-21T07:46:08.208464+0000","last_fullsized":"2024-09-21T07:46:08.208464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":58,"acting":58,"num_store_stats":4},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":776,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":28,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476743,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27212,"kb_used_data":176,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344628,"statfs":{"total":96636764160,"available":96608899072,"internally_reserved":0,"allocated":180224,"data_stored":54668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476740,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28028,"kb_used_data":756,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343812,"statfs":{"total":96636764160,"available":96608063488,"internally_reserved":0,"allocated":774144,"data_stored":645036,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476740,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27216,"kb_used_data":188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344624,"statfs":{"total":96636764160,"available":96608894976,"internally_reserved":0,"allocated":192512,"data_stored":60485,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":73659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:46:11.621 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:46:11.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc518e00640 1 -- 172.21.15.88:0/728587124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4f8004d40 msgr2=0x7fc4f8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc518e00640 1 --2- 172.21.15.88:0/728587124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4f8004d40 0x7fc4f8005190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc50c006fd0 tx=0x7fc4f8005830 comp rx=0 tx=0).stop 2024-09-21T07:46:11.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc513e00640 1 -- 172.21.15.88:0/728587124 reap_dead start 2024-09-21T07:46:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc518e00640 1 -- 172.21.15.88:0/728587124 shutdown_connections 2024-09-21T07:46:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc518e00640 1 -- 172.21.15.88:0/728587124 >> 172.21.15.88:0/728587124 conn(0x7fc5140b1e40 msgr2=0x7fc5140b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc518e00640 1 -- 172.21.15.88:0/728587124 shutdown_connections 2024-09-21T07:46:11.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.877+0000 7fc518e00640 1 -- 172.21.15.88:0/728587124 wait complete. 2024-09-21T07:46:11.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.879+0000 7fc518e00640 1 Processor -- start 2024-09-21T07:46:11.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.879+0000 7fc518e00640 1 -- start start 2024-09-21T07:46:11.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc518e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 0x7fc51414ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc518e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51414b290 0x7fc51409bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc518e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51409c050 0x7fc5141f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc518e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5140b2f60 con 0x7fc514159560 2024-09-21T07:46:11.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc518e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc51406fbb0 con 0x7fc51409c050 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc518e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5140b30e0 con 0x7fc51414b290 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 0x7fc51414ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 0x7fc51414ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49894/0 (socket says 172.21.15.88:49894) 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 -- 172.21.15.88:0/2144687885 learned_addr learned my addr 172.21.15.88:0/2144687885 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc512a00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51414b290 0x7fc51409bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51409c050 0x7fc5141f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 -- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51414b290 msgr2=0x7fc51409bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51414b290 0x7fc51409bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 -- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51409c050 msgr2=0x7fc5141f66c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51409c050 0x7fc5141f66c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513400640 1 -- 172.21.15.88:0/2144687885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc51414b6a0 con 0x7fc514159560 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc513e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51409c050 0x7fc5141f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.881+0000 7fc512a00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51414b290 0x7fc51409bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:11.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.882+0000 7fc513400640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 0x7fc51414ad50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc504003ed0 tx=0x7fc504001730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.882+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504015070 con 0x7fc514159560 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.882+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc504011920 con 0x7fc514159560 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.882+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc504011d70 con 0x7fc514159560 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.882+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc51409c880 con 0x7fc514159560 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.882+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc51409ce40 con 0x7fc514159560 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.883+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc51415d670 con 0x7fc514159560 2024-09-21T07:46:11.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.884+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc504018070 con 0x7fc514159560 2024-09-21T07:46:11.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.885+0000 7fc50b400640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc4f4078b90 0x7fc4f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:11.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.885+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc50409b640 con 0x7fc514159560 2024-09-21T07:46:11.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.886+0000 7fc512a00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc4f4078b90 0x7fc4f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:11.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.886+0000 7fc512a00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc4f4078b90 0x7fc4f407b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc514071370 tx=0x7fc500005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:11.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:11.889+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc51414b6a0 con 0x7fc514159560 2024-09-21T07:46:12.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.073+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc5140700e0 con 0x7fc4f4078b90 2024-09-21T07:46:12.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.079+0000 7fc50b400640 1 -- 172.21.15.88:0/2144687885 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251931 (secure 0 0 0) 0x7fc5140700e0 con 0x7fc4f4078b90 2024-09-21T07:46:12.083 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:12.084 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc4f4078b90 msgr2=0x7fc4f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc4f4078b90 0x7fc4f407b0a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fc514071370 tx=0x7fc500005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 msgr2=0x7fc51414ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 0x7fc51414ad50 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc504003ed0 tx=0x7fc504001730 comp rx=0 tx=0).stop 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 shutdown_connections 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc4f4078b90 0x7fc4f407b0a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc51414b290 0x7fc51409bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc51409c050 0x7fc5141f66c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 --2- 172.21.15.88:0/2144687885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc514159560 0x7fc51414ad50 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.082+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 >> 172.21.15.88:0/2144687885 conn(0x7fc5140b1e40 msgr2=0x7fc5140afc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.083+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 shutdown_connections 2024-09-21T07:46:12.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.083+0000 7fc518e00640 1 -- 172.21.15.88:0/2144687885 wait complete. 2024-09-21T07:46:12.108 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-21T07:46:11.855349+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":159,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219240,"kb_used_data":3004,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755480,"statfs":{"total":773094113280,"available":772869611520,"internally_reserved":0,"allocated":3076096,"data_stored":1817936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001368"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199749+0000","last_change":"2024-09-21T07:46:07.234192+0000","last_active":"2024-09-21T07:46:09.199749+0000","last_peered":"2024-09-21T07:46:09.199749+0000","last_clean":"2024-09-21T07:46:09.199749+0000","last_became_active":"2024-09-21T07:46:07.231852+0000","last_became_peered":"2024-09-21T07:46:07.231852+0000","last_unstale":"2024-09-21T07:46:09.199749+0000","last_undegraded":"2024-09-21T07:46:09.199749+0000","last_fullsized":"2024-09-21T07:46:09.199749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208860+0000","last_change":"2024-09-21T07:46:07.219201+0000","last_active":"2024-09-21T07:46:08.208860+0000","last_peered":"2024-09-21T07:46:08.208860+0000","last_clean":"2024-09-21T07:46:08.208860+0000","last_became_active":"2024-09-21T07:46:07.218520+0000","last_became_peered":"2024-09-21T07:46:07.218520+0000","last_unstale":"2024-09-21T07:46:08.208860+0000","last_undegraded":"2024-09-21T07:46:08.208860+0000","last_fullsized":"2024-09-21T07:46:08.208860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:38.468021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208382+0000","last_change":"2024-09-21T07:46:07.225678+0000","last_active":"2024-09-21T07:46:08.208382+0000","last_peered":"2024-09-21T07:46:08.208382+0000","last_clean":"2024-09-21T07:46:08.208382+0000","last_became_active":"2024-09-21T07:46:07.225167+0000","last_became_peered":"2024-09-21T07:46:07.225167+0000","last_unstale":"2024-09-21T07:46:08.208382+0000","last_undegraded":"2024-09-21T07:46:08.208382+0000","last_fullsized":"2024-09-21T07:46:08.208382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205764+0000","last_change":"2024-09-21T07:46:07.229463+0000","last_active":"2024-09-21T07:46:08.205764+0000","last_peered":"2024-09-21T07:46:08.205764+0000","last_clean":"2024-09-21T07:46:08.205764+0000","last_became_active":"2024-09-21T07:46:07.229040+0000","last_became_peered":"2024-09-21T07:46:07.229040+0000","last_unstale":"2024-09-21T07:46:08.205764+0000","last_undegraded":"2024-09-21T07:46:08.205764+0000","last_fullsized":"2024-09-21T07:46:08.205764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:25.366313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199458+0000","last_change":"2024-09-21T07:46:07.234579+0000","last_active":"2024-09-21T07:46:09.199458+0000","last_peered":"2024-09-21T07:46:09.199458+0000","last_clean":"2024-09-21T07:46:09.199458+0000","last_became_active":"2024-09-21T07:46:07.233770+0000","last_became_peered":"2024-09-21T07:46:07.233770+0000","last_unstale":"2024-09-21T07:46:09.199458+0000","last_undegraded":"2024-09-21T07:46:09.199458+0000","last_fullsized":"2024-09-21T07:46:09.199458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205760+0000","last_change":"2024-09-21T07:46:07.222135+0000","last_active":"2024-09-21T07:46:08.205760+0000","last_peered":"2024-09-21T07:46:08.205760+0000","last_clean":"2024-09-21T07:46:08.205760+0000","last_became_active":"2024-09-21T07:46:07.221847+0000","last_became_peered":"2024-09-21T07:46:07.221847+0000","last_unstale":"2024-09-21T07:46:08.205760+0000","last_undegraded":"2024-09-21T07:46:08.205760+0000","last_fullsized":"2024-09-21T07:46:08.205760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:09.010971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199222+0000","last_change":"2024-09-21T07:46:07.231015+0000","last_active":"2024-09-21T07:46:09.199222+0000","last_peered":"2024-09-21T07:46:09.199222+0000","last_clean":"2024-09-21T07:46:09.199222+0000","last_became_active":"2024-09-21T07:46:07.230719+0000","last_became_peered":"2024-09-21T07:46:07.230719+0000","last_unstale":"2024-09-21T07:46:09.199222+0000","last_undegraded":"2024-09-21T07:46:09.199222+0000","last_fullsized":"2024-09-21T07:46:09.199222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208796+0000","last_change":"2024-09-21T07:46:07.217547+0000","last_active":"2024-09-21T07:46:08.208796+0000","last_peered":"2024-09-21T07:46:08.208796+0000","last_clean":"2024-09-21T07:46:08.208796+0000","last_became_active":"2024-09-21T07:46:07.215538+0000","last_became_peered":"2024-09-21T07:46:07.215538+0000","last_unstale":"2024-09-21T07:46:08.208796+0000","last_undegraded":"2024-09-21T07:46:08.208796+0000","last_fullsized":"2024-09-21T07:46:08.208796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:00.117867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500477+0000","last_change":"2024-09-21T07:46:07.201245+0000","last_active":"2024-09-21T07:46:09.500477+0000","last_peered":"2024-09-21T07:46:09.500477+0000","last_clean":"2024-09-21T07:46:09.500477+0000","last_became_active":"2024-09-21T07:46:07.200940+0000","last_became_peered":"2024-09-21T07:46:07.200940+0000","last_unstale":"2024-09-21T07:46:09.500477+0000","last_undegraded":"2024-09-21T07:46:09.500477+0000","last_fullsized":"2024-09-21T07:46:09.500477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:19.780073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199520+0000","last_change":"2024-09-21T07:46:07.226919+0000","last_active":"2024-09-21T07:46:09.199520+0000","last_peered":"2024-09-21T07:46:09.199520+0000","last_clean":"2024-09-21T07:46:09.199520+0000","last_became_active":"2024-09-21T07:46:07.226619+0000","last_became_peered":"2024-09-21T07:46:07.226619+0000","last_unstale":"2024-09-21T07:46:09.199520+0000","last_undegraded":"2024-09-21T07:46:09.199520+0000","last_fullsized":"2024-09-21T07:46:09.199520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:47.470275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199269+0000","last_change":"2024-09-21T07:46:07.225286+0000","last_active":"2024-09-21T07:46:09.199269+0000","last_peered":"2024-09-21T07:46:09.199269+0000","last_clean":"2024-09-21T07:46:09.199269+0000","last_became_active":"2024-09-21T07:46:07.223018+0000","last_became_peered":"2024-09-21T07:46:07.223018+0000","last_unstale":"2024-09-21T07:46:09.199269+0000","last_undegraded":"2024-09-21T07:46:09.199269+0000","last_fullsized":"2024-09-21T07:46:09.199269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199545+0000","last_change":"2024-09-21T07:46:07.229988+0000","last_active":"2024-09-21T07:46:09.199545+0000","last_peered":"2024-09-21T07:46:09.199545+0000","last_clean":"2024-09-21T07:46:09.199545+0000","last_became_active":"2024-09-21T07:46:07.229716+0000","last_became_peered":"2024-09-21T07:46:07.229716+0000","last_unstale":"2024-09-21T07:46:09.199545+0000","last_undegraded":"2024-09-21T07:46:09.199545+0000","last_fullsized":"2024-09-21T07:46:09.199545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521098+0000","last_change":"2024-09-21T07:46:07.238701+0000","last_active":"2024-09-21T07:46:08.521098+0000","last_peered":"2024-09-21T07:46:08.521098+0000","last_clean":"2024-09-21T07:46:08.521098+0000","last_became_active":"2024-09-21T07:46:07.238406+0000","last_became_peered":"2024-09-21T07:46:07.238406+0000","last_unstale":"2024-09-21T07:46:08.521098+0000","last_undegraded":"2024-09-21T07:46:08.521098+0000","last_fullsized":"2024-09-21T07:46:08.521098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521245+0000","last_change":"2024-09-21T07:46:07.211337+0000","last_active":"2024-09-21T07:46:08.521245+0000","last_peered":"2024-09-21T07:46:08.521245+0000","last_clean":"2024-09-21T07:46:08.521245+0000","last_became_active":"2024-09-21T07:46:07.210983+0000","last_became_peered":"2024-09-21T07:46:07.210983+0000","last_unstale":"2024-09-21T07:46:08.521245+0000","last_undegraded":"2024-09-21T07:46:08.521245+0000","last_fullsized":"2024-09-21T07:46:08.521245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:43.997921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209926+0000","last_change":"2024-09-21T07:46:07.225928+0000","last_active":"2024-09-21T07:46:08.209926+0000","last_peered":"2024-09-21T07:46:08.209926+0000","last_clean":"2024-09-21T07:46:08.209926+0000","last_became_active":"2024-09-21T07:46:07.225685+0000","last_became_peered":"2024-09-21T07:46:07.225685+0000","last_unstale":"2024-09-21T07:46:08.209926+0000","last_undegraded":"2024-09-21T07:46:08.209926+0000","last_fullsized":"2024-09-21T07:46:08.209926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399827+0000","last_change":"2024-09-21T07:46:07.211743+0000","last_active":"2024-09-21T07:46:08.399827+0000","last_peered":"2024-09-21T07:46:08.399827+0000","last_clean":"2024-09-21T07:46:08.399827+0000","last_became_active":"2024-09-21T07:46:07.209957+0000","last_became_peered":"2024-09-21T07:46:07.209957+0000","last_unstale":"2024-09-21T07:46:08.399827+0000","last_undegraded":"2024-09-21T07:46:08.399827+0000","last_fullsized":"2024-09-21T07:46:08.399827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:18.660926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400124+0000","last_change":"2024-09-21T07:46:07.228920+0000","last_active":"2024-09-21T07:46:08.400124+0000","last_peered":"2024-09-21T07:46:08.400124+0000","last_clean":"2024-09-21T07:46:08.400124+0000","last_became_active":"2024-09-21T07:46:07.227861+0000","last_became_peered":"2024-09-21T07:46:07.227861+0000","last_unstale":"2024-09-21T07:46:08.400124+0000","last_undegraded":"2024-09-21T07:46:08.400124+0000","last_fullsized":"2024-09-21T07:46:08.400124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:46:34.920768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.222961+0000","last_change":"2024-09-21T07:46:07.221477+0000","last_active":"2024-09-21T07:46:07.222961+0000","last_peered":"2024-09-21T07:46:07.222961+0000","last_clean":"2024-09-21T07:46:07.222961+0000","last_became_active":"2024-09-21T07:46:07.217967+0000","last_became_peered":"2024-09-21T07:46:07.217967+0000","last_unstale":"2024-09-21T07:46:07.222961+0000","last_undegraded":"2024-09-21T07:46:07.222961+0000","last_fullsized":"2024-09-21T07:46:07.222961+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:34.913461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399789+0000","last_change":"2024-09-21T07:46:07.221262+0000","last_active":"2024-09-21T07:46:08.399789+0000","last_peered":"2024-09-21T07:46:08.399789+0000","last_clean":"2024-09-21T07:46:08.399789+0000","last_became_active":"2024-09-21T07:46:07.221018+0000","last_became_peered":"2024-09-21T07:46:07.221018+0000","last_unstale":"2024-09-21T07:46:08.399789+0000","last_undegraded":"2024-09-21T07:46:08.399789+0000","last_fullsized":"2024-09-21T07:46:08.399789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:46.384301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199556+0000","last_change":"2024-09-21T07:46:07.228997+0000","last_active":"2024-09-21T07:46:09.199556+0000","last_peered":"2024-09-21T07:46:09.199556+0000","last_clean":"2024-09-21T07:46:09.199556+0000","last_became_active":"2024-09-21T07:46:07.227078+0000","last_became_peered":"2024-09-21T07:46:07.227078+0000","last_unstale":"2024-09-21T07:46:09.199556+0000","last_undegraded":"2024-09-21T07:46:09.199556+0000","last_fullsized":"2024-09-21T07:46:09.199556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207714+0000","last_change":"2024-09-21T07:46:07.220220+0000","last_active":"2024-09-21T07:46:08.207714+0000","last_peered":"2024-09-21T07:46:08.207714+0000","last_clean":"2024-09-21T07:46:08.207714+0000","last_became_active":"2024-09-21T07:46:07.219890+0000","last_became_peered":"2024-09-21T07:46:07.219890+0000","last_unstale":"2024-09-21T07:46:08.207714+0000","last_undegraded":"2024-09-21T07:46:08.207714+0000","last_fullsized":"2024-09-21T07:46:08.207714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499999+0000","last_change":"2024-09-21T07:46:07.227381+0000","last_active":"2024-09-21T07:46:09.499999+0000","last_peered":"2024-09-21T07:46:09.499999+0000","last_clean":"2024-09-21T07:46:09.499999+0000","last_became_active":"2024-09-21T07:46:07.225810+0000","last_became_peered":"2024-09-21T07:46:07.225810+0000","last_unstale":"2024-09-21T07:46:09.499999+0000","last_undegraded":"2024-09-21T07:46:09.499999+0000","last_fullsized":"2024-09-21T07:46:09.499999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:23.838546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.210100+0000","last_change":"2024-09-21T07:46:07.215158+0000","last_active":"2024-09-21T07:46:08.210100+0000","last_peered":"2024-09-21T07:46:08.210100+0000","last_clean":"2024-09-21T07:46:08.210100+0000","last_became_active":"2024-09-21T07:46:07.214842+0000","last_became_peered":"2024-09-21T07:46:07.214842+0000","last_unstale":"2024-09-21T07:46:08.210100+0000","last_undegraded":"2024-09-21T07:46:08.210100+0000","last_fullsized":"2024-09-21T07:46:08.210100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399861+0000","last_change":"2024-09-21T07:46:07.209932+0000","last_active":"2024-09-21T07:46:08.399861+0000","last_peered":"2024-09-21T07:46:08.399861+0000","last_clean":"2024-09-21T07:46:08.399861+0000","last_became_active":"2024-09-21T07:46:07.208985+0000","last_became_peered":"2024-09-21T07:46:07.208985+0000","last_unstale":"2024-09-21T07:46:08.399861+0000","last_undegraded":"2024-09-21T07:46:08.399861+0000","last_fullsized":"2024-09-21T07:46:08.399861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:32.406624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208288+0000","last_change":"2024-09-21T07:46:07.218857+0000","last_active":"2024-09-21T07:46:08.208288+0000","last_peered":"2024-09-21T07:46:08.208288+0000","last_clean":"2024-09-21T07:46:08.208288+0000","last_became_active":"2024-09-21T07:46:07.218490+0000","last_became_peered":"2024-09-21T07:46:07.218490+0000","last_unstale":"2024-09-21T07:46:08.208288+0000","last_undegraded":"2024-09-21T07:46:08.208288+0000","last_fullsized":"2024-09-21T07:46:08.208288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206165+0000","last_change":"2024-09-21T07:46:07.218802+0000","last_active":"2024-09-21T07:46:08.206165+0000","last_peered":"2024-09-21T07:46:08.206165+0000","last_clean":"2024-09-21T07:46:08.206165+0000","last_became_active":"2024-09-21T07:46:07.216480+0000","last_became_peered":"2024-09-21T07:46:07.216480+0000","last_unstale":"2024-09-21T07:46:08.206165+0000","last_undegraded":"2024-09-21T07:46:08.206165+0000","last_fullsized":"2024-09-21T07:46:08.206165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521582+0000","last_change":"2024-09-21T07:46:07.213848+0000","last_active":"2024-09-21T07:46:08.521582+0000","last_peered":"2024-09-21T07:46:08.521582+0000","last_clean":"2024-09-21T07:46:08.521582+0000","last_became_active":"2024-09-21T07:46:07.212350+0000","last_became_peered":"2024-09-21T07:46:07.212350+0000","last_unstale":"2024-09-21T07:46:08.521582+0000","last_undegraded":"2024-09-21T07:46:08.521582+0000","last_fullsized":"2024-09-21T07:46:08.521582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521460+0000","last_change":"2024-09-21T07:46:07.231247+0000","last_active":"2024-09-21T07:46:08.521460+0000","last_peered":"2024-09-21T07:46:08.521460+0000","last_clean":"2024-09-21T07:46:08.521460+0000","last_became_active":"2024-09-21T07:46:07.230913+0000","last_became_peered":"2024-09-21T07:46:07.230913+0000","last_unstale":"2024-09-21T07:46:08.521460+0000","last_undegraded":"2024-09-21T07:46:08.521460+0000","last_fullsized":"2024-09-21T07:46:08.521460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208726+0000","last_change":"2024-09-21T07:46:07.212029+0000","last_active":"2024-09-21T07:46:08.208726+0000","last_peered":"2024-09-21T07:46:08.208726+0000","last_clean":"2024-09-21T07:46:08.208726+0000","last_became_active":"2024-09-21T07:46:07.211701+0000","last_became_peered":"2024-09-21T07:46:07.211701+0000","last_unstale":"2024-09-21T07:46:08.208726+0000","last_undegraded":"2024-09-21T07:46:08.208726+0000","last_fullsized":"2024-09-21T07:46:08.208726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399383+0000","last_change":"2024-09-21T07:46:07.209431+0000","last_active":"2024-09-21T07:46:08.399383+0000","last_peered":"2024-09-21T07:46:08.399383+0000","last_clean":"2024-09-21T07:46:08.399383+0000","last_became_active":"2024-09-21T07:46:07.208558+0000","last_became_peered":"2024-09-21T07:46:07.208558+0000","last_unstale":"2024-09-21T07:46:08.399383+0000","last_undegraded":"2024-09-21T07:46:08.399383+0000","last_fullsized":"2024-09-21T07:46:08.399383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.216348+0000","last_change":"2024-09-21T07:46:07.213929+0000","last_active":"2024-09-21T07:46:07.216348+0000","last_peered":"2024-09-21T07:46:07.216348+0000","last_clean":"2024-09-21T07:46:07.216348+0000","last_became_active":"2024-09-21T07:46:07.213456+0000","last_became_peered":"2024-09-21T07:46:07.213456+0000","last_unstale":"2024-09-21T07:46:07.216348+0000","last_undegraded":"2024-09-21T07:46:07.216348+0000","last_fullsized":"2024-09-21T07:46:07.216348+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:38:50.786316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205701+0000","last_change":"2024-09-21T07:46:07.218008+0000","last_active":"2024-09-21T07:46:08.205701+0000","last_peered":"2024-09-21T07:46:08.205701+0000","last_clean":"2024-09-21T07:46:08.205701+0000","last_became_active":"2024-09-21T07:46:07.217294+0000","last_became_peered":"2024-09-21T07:46:07.217294+0000","last_unstale":"2024-09-21T07:46:08.205701+0000","last_undegraded":"2024-09-21T07:46:08.205701+0000","last_fullsized":"2024-09-21T07:46:08.205701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:48.673151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500036+0000","last_change":"2024-09-21T07:46:07.203504+0000","last_active":"2024-09-21T07:46:09.500036+0000","last_peered":"2024-09-21T07:46:09.500036+0000","last_clean":"2024-09-21T07:46:09.500036+0000","last_became_active":"2024-09-21T07:46:07.203135+0000","last_became_peered":"2024-09-21T07:46:07.203135+0000","last_unstale":"2024-09-21T07:46:09.500036+0000","last_undegraded":"2024-09-21T07:46:09.500036+0000","last_fullsized":"2024-09-21T07:46:09.500036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:11.674361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208870+0000","last_change":"2024-09-21T07:46:06.210978+0000","last_active":"2024-09-21T07:46:08.208870+0000","last_peered":"2024-09-21T07:46:08.208870+0000","last_clean":"2024-09-21T07:46:08.208870+0000","last_became_active":"2024-09-21T07:46:06.210615+0000","last_became_peered":"2024-09-21T07:46:06.210615+0000","last_unstale":"2024-09-21T07:46:08.208870+0000","last_undegraded":"2024-09-21T07:46:08.208870+0000","last_fullsized":"2024-09-21T07:46:08.208870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:12.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207933+0000","last_change":"2024-09-21T07:46:07.236139+0000","last_active":"2024-09-21T07:46:08.207933+0000","last_peered":"2024-09-21T07:46:08.207933+0000","last_clean":"2024-09-21T07:46:08.207933+0000","last_became_active":"2024-09-21T07:46:07.235357+0000","last_became_peered":"2024-09-21T07:46:07.235357+0000","last_unstale":"2024-09-21T07:46:08.207933+0000","last_undegraded":"2024-09-21T07:46:08.207933+0000","last_fullsized":"2024-09-21T07:46:08.207933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399662+0000","last_change":"2024-09-21T07:46:06.218149+0000","last_active":"2024-09-21T07:46:08.399662+0000","last_peered":"2024-09-21T07:46:08.399662+0000","last_clean":"2024-09-21T07:46:08.399662+0000","last_became_active":"2024-09-21T07:46:06.217846+0000","last_became_peered":"2024-09-21T07:46:06.217846+0000","last_unstale":"2024-09-21T07:46:08.399662+0000","last_undegraded":"2024-09-21T07:46:08.399662+0000","last_fullsized":"2024-09-21T07:46:08.399662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.520888+0000","last_change":"2024-09-21T07:46:06.213787+0000","last_active":"2024-09-21T07:46:08.520888+0000","last_peered":"2024-09-21T07:46:08.520888+0000","last_clean":"2024-09-21T07:46:08.520888+0000","last_became_active":"2024-09-21T07:46:06.213454+0000","last_became_peered":"2024-09-21T07:46:06.213454+0000","last_unstale":"2024-09-21T07:46:08.520888+0000","last_undegraded":"2024-09-21T07:46:08.520888+0000","last_fullsized":"2024-09-21T07:46:08.520888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208744+0000","last_change":"2024-09-21T07:46:07.217173+0000","last_active":"2024-09-21T07:46:08.208744+0000","last_peered":"2024-09-21T07:46:08.208744+0000","last_clean":"2024-09-21T07:46:08.208744+0000","last_became_active":"2024-09-21T07:46:07.216857+0000","last_became_peered":"2024-09-21T07:46:07.216857+0000","last_unstale":"2024-09-21T07:46:08.208744+0000","last_undegraded":"2024-09-21T07:46:08.208744+0000","last_fullsized":"2024-09-21T07:46:08.208744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399914+0000","last_change":"2024-09-21T07:46:06.210041+0000","last_active":"2024-09-21T07:46:08.399914+0000","last_peered":"2024-09-21T07:46:08.399914+0000","last_clean":"2024-09-21T07:46:08.399914+0000","last_became_active":"2024-09-21T07:46:06.209744+0000","last_became_peered":"2024-09-21T07:46:06.209744+0000","last_unstale":"2024-09-21T07:46:08.399914+0000","last_undegraded":"2024-09-21T07:46:08.399914+0000","last_fullsized":"2024-09-21T07:46:08.399914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207314+0000","last_change":"2024-09-21T07:46:07.217122+0000","last_active":"2024-09-21T07:46:08.207314+0000","last_peered":"2024-09-21T07:46:08.207314+0000","last_clean":"2024-09-21T07:46:08.207314+0000","last_became_active":"2024-09-21T07:46:07.216831+0000","last_became_peered":"2024-09-21T07:46:07.216831+0000","last_unstale":"2024-09-21T07:46:08.207314+0000","last_undegraded":"2024-09-21T07:46:08.207314+0000","last_fullsized":"2024-09-21T07:46:08.207314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500434+0000","last_change":"2024-09-21T07:46:06.202767+0000","last_active":"2024-09-21T07:46:09.500434+0000","last_peered":"2024-09-21T07:46:09.500434+0000","last_clean":"2024-09-21T07:46:09.500434+0000","last_became_active":"2024-09-21T07:46:06.200291+0000","last_became_peered":"2024-09-21T07:46:06.200291+0000","last_unstale":"2024-09-21T07:46:09.500434+0000","last_undegraded":"2024-09-21T07:46:09.500434+0000","last_fullsized":"2024-09-21T07:46:09.500434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206404+0000","last_change":"2024-09-21T07:46:07.216776+0000","last_active":"2024-09-21T07:46:08.206404+0000","last_peered":"2024-09-21T07:46:08.206404+0000","last_clean":"2024-09-21T07:46:08.206404+0000","last_became_active":"2024-09-21T07:46:07.214952+0000","last_became_peered":"2024-09-21T07:46:07.214952+0000","last_unstale":"2024-09-21T07:46:08.206404+0000","last_undegraded":"2024-09-21T07:46:08.206404+0000","last_fullsized":"2024-09-21T07:46:08.206404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199670+0000","last_change":"2024-09-21T07:46:06.210647+0000","last_active":"2024-09-21T07:46:09.199670+0000","last_peered":"2024-09-21T07:46:09.199670+0000","last_clean":"2024-09-21T07:46:09.199670+0000","last_became_active":"2024-09-21T07:46:06.209590+0000","last_became_peered":"2024-09-21T07:46:06.209590+0000","last_unstale":"2024-09-21T07:46:09.199670+0000","last_undegraded":"2024-09-21T07:46:09.199670+0000","last_fullsized":"2024-09-21T07:46:09.199670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:40.259455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208537+0000","last_change":"2024-09-21T07:46:07.210270+0000","last_active":"2024-09-21T07:46:08.208537+0000","last_peered":"2024-09-21T07:46:08.208537+0000","last_clean":"2024-09-21T07:46:08.208537+0000","last_became_active":"2024-09-21T07:46:07.202021+0000","last_became_peered":"2024-09-21T07:46:07.202021+0000","last_unstale":"2024-09-21T07:46:08.208537+0000","last_undegraded":"2024-09-21T07:46:08.208537+0000","last_fullsized":"2024-09-21T07:46:08.208537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.223921+0000","last_change":"2024-09-21T07:46:06.212266+0000","last_active":"2024-09-21T07:46:07.223921+0000","last_peered":"2024-09-21T07:46:07.223921+0000","last_clean":"2024-09-21T07:46:07.223921+0000","last_became_active":"2024-09-21T07:46:06.210122+0000","last_became_peered":"2024-09-21T07:46:06.210122+0000","last_unstale":"2024-09-21T07:46:07.223921+0000","last_undegraded":"2024-09-21T07:46:07.223921+0000","last_fullsized":"2024-09-21T07:46:07.223921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:13.095998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399665+0000","last_change":"2024-09-21T07:46:07.216761+0000","last_active":"2024-09-21T07:46:08.399665+0000","last_peered":"2024-09-21T07:46:08.399665+0000","last_clean":"2024-09-21T07:46:08.399665+0000","last_became_active":"2024-09-21T07:46:07.215637+0000","last_became_peered":"2024-09-21T07:46:07.215637+0000","last_unstale":"2024-09-21T07:46:08.399665+0000","last_undegraded":"2024-09-21T07:46:08.399665+0000","last_fullsized":"2024-09-21T07:46:08.399665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:48.803562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205818+0000","last_change":"2024-09-21T07:46:06.206864+0000","last_active":"2024-09-21T07:46:08.205818+0000","last_peered":"2024-09-21T07:46:08.205818+0000","last_clean":"2024-09-21T07:46:08.205818+0000","last_became_active":"2024-09-21T07:46:06.206545+0000","last_became_peered":"2024-09-21T07:46:06.206545+0000","last_unstale":"2024-09-21T07:46:08.205818+0000","last_undegraded":"2024-09-21T07:46:08.205818+0000","last_fullsized":"2024-09-21T07:46:08.205818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:00.329415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199512+0000","last_change":"2024-09-21T07:46:07.236117+0000","last_active":"2024-09-21T07:46:09.199512+0000","last_peered":"2024-09-21T07:46:09.199512+0000","last_clean":"2024-09-21T07:46:09.199512+0000","last_became_active":"2024-09-21T07:46:07.233496+0000","last_became_peered":"2024-09-21T07:46:07.233496+0000","last_unstale":"2024-09-21T07:46:09.199512+0000","last_undegraded":"2024-09-21T07:46:09.199512+0000","last_fullsized":"2024-09-21T07:46:09.199512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.238063+0000","last_change":"2024-09-21T07:46:06.212633+0000","last_active":"2024-09-21T07:46:07.238063+0000","last_peered":"2024-09-21T07:46:07.238063+0000","last_clean":"2024-09-21T07:46:07.238063+0000","last_became_active":"2024-09-21T07:46:06.212240+0000","last_became_peered":"2024-09-21T07:46:06.212240+0000","last_unstale":"2024-09-21T07:46:07.238063+0000","last_undegraded":"2024-09-21T07:46:07.238063+0000","last_fullsized":"2024-09-21T07:46:07.238063+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:36.198191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521167+0000","last_change":"2024-09-21T07:46:07.216368+0000","last_active":"2024-09-21T07:46:08.521167+0000","last_peered":"2024-09-21T07:46:08.521167+0000","last_clean":"2024-09-21T07:46:08.521167+0000","last_became_active":"2024-09-21T07:46:07.216061+0000","last_became_peered":"2024-09-21T07:46:07.216061+0000","last_unstale":"2024-09-21T07:46:08.521167+0000","last_undegraded":"2024-09-21T07:46:08.521167+0000","last_fullsized":"2024-09-21T07:46:08.521167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399781+0000","last_change":"2024-09-21T07:46:06.204421+0000","last_active":"2024-09-21T07:46:08.399781+0000","last_peered":"2024-09-21T07:46:08.399781+0000","last_clean":"2024-09-21T07:46:08.399781+0000","last_became_active":"2024-09-21T07:46:06.204109+0000","last_became_peered":"2024-09-21T07:46:06.204109+0000","last_unstale":"2024-09-21T07:46:08.399781+0000","last_undegraded":"2024-09-21T07:46:08.399781+0000","last_fullsized":"2024-09-21T07:46:08.399781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:10.667453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399577+0000","last_change":"2024-09-21T07:46:07.237847+0000","last_active":"2024-09-21T07:46:08.399577+0000","last_peered":"2024-09-21T07:46:08.399577+0000","last_clean":"2024-09-21T07:46:08.399577+0000","last_became_active":"2024-09-21T07:46:07.236995+0000","last_became_peered":"2024-09-21T07:46:07.236995+0000","last_unstale":"2024-09-21T07:46:08.399577+0000","last_undegraded":"2024-09-21T07:46:08.399577+0000","last_fullsized":"2024-09-21T07:46:08.399577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:19.395558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199277+0000","last_change":"2024-09-21T07:46:06.218044+0000","last_active":"2024-09-21T07:46:09.199277+0000","last_peered":"2024-09-21T07:46:09.199277+0000","last_clean":"2024-09-21T07:46:09.199277+0000","last_became_active":"2024-09-21T07:46:06.217446+0000","last_became_peered":"2024-09-21T07:46:06.217446+0000","last_unstale":"2024-09-21T07:46:09.199277+0000","last_undegraded":"2024-09-21T07:46:09.199277+0000","last_fullsized":"2024-09-21T07:46:09.199277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:47.666243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499878+0000","last_change":"2024-09-21T07:46:07.210885+0000","last_active":"2024-09-21T07:46:09.499878+0000","last_peered":"2024-09-21T07:46:09.499878+0000","last_clean":"2024-09-21T07:46:09.499878+0000","last_became_active":"2024-09-21T07:46:07.210591+0000","last_became_peered":"2024-09-21T07:46:07.210591+0000","last_unstale":"2024-09-21T07:46:09.499878+0000","last_undegraded":"2024-09-21T07:46:09.499878+0000","last_fullsized":"2024-09-21T07:46:09.499878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.666658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207016+0000","last_change":"2024-09-21T07:46:06.202414+0000","last_active":"2024-09-21T07:46:08.207016+0000","last_peered":"2024-09-21T07:46:08.207016+0000","last_clean":"2024-09-21T07:46:08.207016+0000","last_became_active":"2024-09-21T07:46:06.201528+0000","last_became_peered":"2024-09-21T07:46:06.201528+0000","last_unstale":"2024-09-21T07:46:08.207016+0000","last_undegraded":"2024-09-21T07:46:08.207016+0000","last_fullsized":"2024-09-21T07:46:08.207016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399393+0000","last_change":"2024-09-21T07:46:07.226232+0000","last_active":"2024-09-21T07:46:08.399393+0000","last_peered":"2024-09-21T07:46:08.399393+0000","last_clean":"2024-09-21T07:46:08.399393+0000","last_became_active":"2024-09-21T07:46:07.224513+0000","last_became_peered":"2024-09-21T07:46:07.224513+0000","last_unstale":"2024-09-21T07:46:08.399393+0000","last_undegraded":"2024-09-21T07:46:08.399393+0000","last_fullsized":"2024-09-21T07:46:08.399393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:17.528901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199798+0000","last_change":"2024-09-21T07:46:06.219146+0000","last_active":"2024-09-21T07:46:09.199798+0000","last_peered":"2024-09-21T07:46:09.199798+0000","last_clean":"2024-09-21T07:46:09.199798+0000","last_became_active":"2024-09-21T07:46:06.218899+0000","last_became_peered":"2024-09-21T07:46:06.218899+0000","last_unstale":"2024-09-21T07:46:09.199798+0000","last_undegraded":"2024-09-21T07:46:09.199798+0000","last_fullsized":"2024-09-21T07:46:09.199798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:08.004063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206636+0000","last_change":"2024-09-21T07:46:07.235918+0000","last_active":"2024-09-21T07:46:08.206636+0000","last_peered":"2024-09-21T07:46:08.206636+0000","last_clean":"2024-09-21T07:46:08.206636+0000","last_became_active":"2024-09-21T07:46:07.235596+0000","last_became_peered":"2024-09-21T07:46:07.235596+0000","last_unstale":"2024-09-21T07:46:08.206636+0000","last_undegraded":"2024-09-21T07:46:08.206636+0000","last_fullsized":"2024-09-21T07:46:08.206636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208692+0000","last_change":"2024-09-21T07:46:06.208887+0000","last_active":"2024-09-21T07:46:08.208692+0000","last_peered":"2024-09-21T07:46:08.208692+0000","last_clean":"2024-09-21T07:46:08.208692+0000","last_became_active":"2024-09-21T07:46:06.208508+0000","last_became_peered":"2024-09-21T07:46:06.208508+0000","last_unstale":"2024-09-21T07:46:08.208692+0000","last_undegraded":"2024-09-21T07:46:08.208692+0000","last_fullsized":"2024-09-21T07:46:08.208692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199820+0000","last_change":"2024-09-21T07:46:07.225748+0000","last_active":"2024-09-21T07:46:09.199820+0000","last_peered":"2024-09-21T07:46:09.199820+0000","last_clean":"2024-09-21T07:46:09.199820+0000","last_became_active":"2024-09-21T07:46:07.225406+0000","last_became_peered":"2024-09-21T07:46:07.225406+0000","last_unstale":"2024-09-21T07:46:09.199820+0000","last_undegraded":"2024-09-21T07:46:09.199820+0000","last_fullsized":"2024-09-21T07:46:09.199820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.699234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208511+0000","last_change":"2024-09-21T07:46:06.209460+0000","last_active":"2024-09-21T07:46:08.208511+0000","last_peered":"2024-09-21T07:46:08.208511+0000","last_clean":"2024-09-21T07:46:08.208511+0000","last_became_active":"2024-09-21T07:46:06.207853+0000","last_became_peered":"2024-09-21T07:46:06.207853+0000","last_unstale":"2024-09-21T07:46:08.208511+0000","last_undegraded":"2024-09-21T07:46:08.208511+0000","last_fullsized":"2024-09-21T07:46:08.208511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199786+0000","last_change":"2024-09-21T07:46:07.228685+0000","last_active":"2024-09-21T07:46:09.199786+0000","last_peered":"2024-09-21T07:46:09.199786+0000","last_clean":"2024-09-21T07:46:09.199786+0000","last_became_active":"2024-09-21T07:46:07.226780+0000","last_became_peered":"2024-09-21T07:46:07.226780+0000","last_unstale":"2024-09-21T07:46:09.199786+0000","last_undegraded":"2024-09-21T07:46:09.199786+0000","last_fullsized":"2024-09-21T07:46:09.199786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:32.406624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521589+0000","last_change":"2024-09-21T07:46:06.216893+0000","last_active":"2024-09-21T07:46:08.521589+0000","last_peered":"2024-09-21T07:46:08.521589+0000","last_clean":"2024-09-21T07:46:08.521589+0000","last_became_active":"2024-09-21T07:46:06.216657+0000","last_became_peered":"2024-09-21T07:46:06.216657+0000","last_unstale":"2024-09-21T07:46:08.521589+0000","last_undegraded":"2024-09-21T07:46:08.521589+0000","last_fullsized":"2024-09-21T07:46:08.521589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399601+0000","last_change":"2024-09-21T07:46:07.210978+0000","last_active":"2024-09-21T07:46:08.399601+0000","last_peered":"2024-09-21T07:46:08.399601+0000","last_clean":"2024-09-21T07:46:08.399601+0000","last_became_active":"2024-09-21T07:46:07.210665+0000","last_became_peered":"2024-09-21T07:46:07.210665+0000","last_unstale":"2024-09-21T07:46:08.399601+0000","last_undegraded":"2024-09-21T07:46:08.399601+0000","last_fullsized":"2024-09-21T07:46:08.399601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:57.801044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499736+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:09.499736+0000","last_peered":"2024-09-21T07:46:09.499736+0000","last_clean":"2024-09-21T07:46:09.499736+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:09.499736+0000","last_undegraded":"2024-09-21T07:46:09.499736+0000","last_fullsized":"2024-09-21T07:46:09.499736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399734+0000","last_change":"2024-09-21T07:46:06.204733+0000","last_active":"2024-09-21T07:46:08.399734+0000","last_peered":"2024-09-21T07:46:08.399734+0000","last_clean":"2024-09-21T07:46:08.399734+0000","last_became_active":"2024-09-21T07:46:06.204416+0000","last_became_peered":"2024-09-21T07:46:06.204416+0000","last_unstale":"2024-09-21T07:46:08.399734+0000","last_undegraded":"2024-09-21T07:46:08.399734+0000","last_fullsized":"2024-09-21T07:46:08.399734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:18.773164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207407+0000","last_change":"2024-09-21T07:46:07.222509+0000","last_active":"2024-09-21T07:46:08.207407+0000","last_peered":"2024-09-21T07:46:08.207407+0000","last_clean":"2024-09-21T07:46:08.207407+0000","last_became_active":"2024-09-21T07:46:07.221105+0000","last_became_peered":"2024-09-21T07:46:07.221105+0000","last_unstale":"2024-09-21T07:46:08.207407+0000","last_undegraded":"2024-09-21T07:46:08.207407+0000","last_fullsized":"2024-09-21T07:46:08.207407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399939+0000","last_change":"2024-09-21T07:46:06.212101+0000","last_active":"2024-09-21T07:46:08.399939+0000","last_peered":"2024-09-21T07:46:08.399939+0000","last_clean":"2024-09-21T07:46:08.399939+0000","last_became_active":"2024-09-21T07:46:06.211043+0000","last_became_peered":"2024-09-21T07:46:06.211043+0000","last_unstale":"2024-09-21T07:46:08.399939+0000","last_undegraded":"2024-09-21T07:46:08.399939+0000","last_fullsized":"2024-09-21T07:46:08.399939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:18.001419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521036+0000","last_change":"2024-09-21T07:46:07.218927+0000","last_active":"2024-09-21T07:46:08.521036+0000","last_peered":"2024-09-21T07:46:08.521036+0000","last_clean":"2024-09-21T07:46:08.521036+0000","last_became_active":"2024-09-21T07:46:07.218594+0000","last_became_peered":"2024-09-21T07:46:07.218594+0000","last_unstale":"2024-09-21T07:46:08.521036+0000","last_undegraded":"2024-09-21T07:46:08.521036+0000","last_fullsized":"2024-09-21T07:46:08.521036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.201232+0000","last_change":"2024-09-21T07:46:06.208141+0000","last_active":"2024-09-21T07:46:07.201232+0000","last_peered":"2024-09-21T07:46:07.201232+0000","last_clean":"2024-09-21T07:46:07.201232+0000","last_became_active":"2024-09-21T07:46:06.207799+0000","last_became_peered":"2024-09-21T07:46:06.207799+0000","last_unstale":"2024-09-21T07:46:07.201232+0000","last_undegraded":"2024-09-21T07:46:07.201232+0000","last_fullsized":"2024-09-21T07:46:07.201232+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:02.343379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208515+0000","last_change":"2024-09-21T07:46:07.208889+0000","last_active":"2024-09-21T07:46:08.208515+0000","last_peered":"2024-09-21T07:46:08.208515+0000","last_clean":"2024-09-21T07:46:08.208515+0000","last_became_active":"2024-09-21T07:46:07.208365+0000","last_became_peered":"2024-09-21T07:46:07.208365+0000","last_unstale":"2024-09-21T07:46:08.208515+0000","last_undegraded":"2024-09-21T07:46:08.208515+0000","last_fullsized":"2024-09-21T07:46:08.208515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399888+0000","last_change":"2024-09-21T07:46:06.212471+0000","last_active":"2024-09-21T07:46:08.399888+0000","last_peered":"2024-09-21T07:46:08.399888+0000","last_clean":"2024-09-21T07:46:08.399888+0000","last_became_active":"2024-09-21T07:46:06.212157+0000","last_became_peered":"2024-09-21T07:46:06.212157+0000","last_unstale":"2024-09-21T07:46:08.399888+0000","last_undegraded":"2024-09-21T07:46:08.399888+0000","last_fullsized":"2024-09-21T07:46:08.399888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199360+0000","last_change":"2024-09-21T07:46:07.234186+0000","last_active":"2024-09-21T07:46:09.199360+0000","last_peered":"2024-09-21T07:46:09.199360+0000","last_clean":"2024-09-21T07:46:09.199360+0000","last_became_active":"2024-09-21T07:46:07.231069+0000","last_became_peered":"2024-09-21T07:46:07.231069+0000","last_unstale":"2024-09-21T07:46:09.199360+0000","last_undegraded":"2024-09-21T07:46:09.199360+0000","last_fullsized":"2024-09-21T07:46:09.199360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209588+0000","last_change":"2024-09-21T07:46:06.213705+0000","last_active":"2024-09-21T07:46:08.209588+0000","last_peered":"2024-09-21T07:46:08.209588+0000","last_clean":"2024-09-21T07:46:08.209588+0000","last_became_active":"2024-09-21T07:46:06.213351+0000","last_became_peered":"2024-09-21T07:46:06.213351+0000","last_unstale":"2024-09-21T07:46:08.209588+0000","last_undegraded":"2024-09-21T07:46:08.209588+0000","last_fullsized":"2024-09-21T07:46:08.209588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205777+0000","last_change":"2024-09-21T07:46:07.216921+0000","last_active":"2024-09-21T07:46:08.205777+0000","last_peered":"2024-09-21T07:46:08.205777+0000","last_clean":"2024-09-21T07:46:08.205777+0000","last_became_active":"2024-09-21T07:46:07.216430+0000","last_became_peered":"2024-09-21T07:46:07.216430+0000","last_unstale":"2024-09-21T07:46:08.205777+0000","last_undegraded":"2024-09-21T07:46:08.205777+0000","last_fullsized":"2024-09-21T07:46:08.205777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:41.114984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399677+0000","last_change":"2024-09-21T07:46:06.208651+0000","last_active":"2024-09-21T07:46:08.399677+0000","last_peered":"2024-09-21T07:46:08.399677+0000","last_clean":"2024-09-21T07:46:08.399677+0000","last_became_active":"2024-09-21T07:46:06.206073+0000","last_became_peered":"2024-09-21T07:46:06.206073+0000","last_unstale":"2024-09-21T07:46:08.399677+0000","last_undegraded":"2024-09-21T07:46:08.399677+0000","last_fullsized":"2024-09-21T07:46:08.399677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:01.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521474+0000","last_change":"2024-09-21T07:46:07.225723+0000","last_active":"2024-09-21T07:46:08.521474+0000","last_peered":"2024-09-21T07:46:08.521474+0000","last_clean":"2024-09-21T07:46:08.521474+0000","last_became_active":"2024-09-21T07:46:07.225424+0000","last_became_peered":"2024-09-21T07:46:07.225424+0000","last_unstale":"2024-09-21T07:46:08.521474+0000","last_undegraded":"2024-09-21T07:46:08.521474+0000","last_fullsized":"2024-09-21T07:46:08.521474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.196020+0000","last_change":"2024-09-21T07:46:06.203823+0000","last_active":"2024-09-21T07:46:09.196020+0000","last_peered":"2024-09-21T07:46:09.196020+0000","last_clean":"2024-09-21T07:46:09.196020+0000","last_became_active":"2024-09-21T07:46:06.203500+0000","last_became_peered":"2024-09-21T07:46:06.203500+0000","last_unstale":"2024-09-21T07:46:09.196020+0000","last_undegraded":"2024-09-21T07:46:09.196020+0000","last_fullsized":"2024-09-21T07:46:09.196020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208233+0000","last_change":"2024-09-21T07:46:07.222640+0000","last_active":"2024-09-21T07:46:08.208233+0000","last_peered":"2024-09-21T07:46:08.208233+0000","last_clean":"2024-09-21T07:46:08.208233+0000","last_became_active":"2024-09-21T07:46:07.220945+0000","last_became_peered":"2024-09-21T07:46:07.220945+0000","last_unstale":"2024-09-21T07:46:08.208233+0000","last_undegraded":"2024-09-21T07:46:08.208233+0000","last_fullsized":"2024-09-21T07:46:08.208233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:38:12.923471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400048+0000","last_change":"2024-09-21T07:46:06.215628+0000","last_active":"2024-09-21T07:46:08.400048+0000","last_peered":"2024-09-21T07:46:08.400048+0000","last_clean":"2024-09-21T07:46:08.400048+0000","last_became_active":"2024-09-21T07:46:06.215341+0000","last_became_peered":"2024-09-21T07:46:06.215341+0000","last_unstale":"2024-09-21T07:46:08.400048+0000","last_undegraded":"2024-09-21T07:46:08.400048+0000","last_fullsized":"2024-09-21T07:46:08.400048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:47.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208617+0000","last_change":"2024-09-21T07:46:07.220815+0000","last_active":"2024-09-21T07:46:08.208617+0000","last_peered":"2024-09-21T07:46:08.208617+0000","last_clean":"2024-09-21T07:46:08.208617+0000","last_became_active":"2024-09-21T07:46:07.220441+0000","last_became_peered":"2024-09-21T07:46:07.220441+0000","last_unstale":"2024-09-21T07:46:08.208617+0000","last_undegraded":"2024-09-21T07:46:08.208617+0000","last_fullsized":"2024-09-21T07:46:08.208617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.198896+0000","last_change":"2024-09-21T07:46:06.212367+0000","last_active":"2024-09-21T07:46:09.198896+0000","last_peered":"2024-09-21T07:46:09.198896+0000","last_clean":"2024-09-21T07:46:09.198896+0000","last_became_active":"2024-09-21T07:46:06.211815+0000","last_became_peered":"2024-09-21T07:46:06.211815+0000","last_unstale":"2024-09-21T07:46:09.198896+0000","last_undegraded":"2024-09-21T07:46:09.198896+0000","last_fullsized":"2024-09-21T07:46:09.198896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:22.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399778+0000","last_change":"2024-09-21T07:46:07.224826+0000","last_active":"2024-09-21T07:46:08.399778+0000","last_peered":"2024-09-21T07:46:08.399778+0000","last_clean":"2024-09-21T07:46:08.399778+0000","last_became_active":"2024-09-21T07:46:07.221235+0000","last_became_peered":"2024-09-21T07:46:07.221235+0000","last_unstale":"2024-09-21T07:46:08.399778+0000","last_undegraded":"2024-09-21T07:46:08.399778+0000","last_fullsized":"2024-09-21T07:46:08.399778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:12.490015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207301+0000","last_change":"2024-09-21T07:46:06.206432+0000","last_active":"2024-09-21T07:46:08.207301+0000","last_peered":"2024-09-21T07:46:08.207301+0000","last_clean":"2024-09-21T07:46:08.207301+0000","last_became_active":"2024-09-21T07:46:06.203791+0000","last_became_peered":"2024-09-21T07:46:06.203791+0000","last_unstale":"2024-09-21T07:46:08.207301+0000","last_undegraded":"2024-09-21T07:46:08.207301+0000","last_fullsized":"2024-09-21T07:46:08.207301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207491+0000","last_change":"2024-09-21T07:46:07.214468+0000","last_active":"2024-09-21T07:46:08.207491+0000","last_peered":"2024-09-21T07:46:08.207491+0000","last_clean":"2024-09-21T07:46:08.207491+0000","last_became_active":"2024-09-21T07:46:07.213517+0000","last_became_peered":"2024-09-21T07:46:07.213517+0000","last_unstale":"2024-09-21T07:46:08.207491+0000","last_undegraded":"2024-09-21T07:46:08.207491+0000","last_fullsized":"2024-09-21T07:46:08.207491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521331+0000","last_change":"2024-09-21T07:46:06.211420+0000","last_active":"2024-09-21T07:46:08.521331+0000","last_peered":"2024-09-21T07:46:08.521331+0000","last_clean":"2024-09-21T07:46:08.521331+0000","last_became_active":"2024-09-21T07:46:06.208177+0000","last_became_peered":"2024-09-21T07:46:06.208177+0000","last_unstale":"2024-09-21T07:46:08.521331+0000","last_undegraded":"2024-09-21T07:46:08.521331+0000","last_fullsized":"2024-09-21T07:46:08.521331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-09-21T07:46:07.216520+0000","last_change":"2024-09-21T07:46:07.213281+0000","last_active":"2024-09-21T07:46:07.216520+0000","last_peered":"2024-09-21T07:46:07.216520+0000","last_clean":"2024-09-21T07:46:07.216520+0000","last_became_active":"2024-09-21T07:46:07.212947+0000","last_became_peered":"2024-09-21T07:46:07.212947+0000","last_unstale":"2024-09-21T07:46:07.216520+0000","last_undegraded":"2024-09-21T07:46:07.216520+0000","last_fullsized":"2024-09-21T07:46:07.216520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:17.897897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399658+0000","last_change":"2024-09-21T07:46:06.206452+0000","last_active":"2024-09-21T07:46:08.399658+0000","last_peered":"2024-09-21T07:46:08.399658+0000","last_clean":"2024-09-21T07:46:08.399658+0000","last_became_active":"2024-09-21T07:46:06.206136+0000","last_became_peered":"2024-09-21T07:46:06.206136+0000","last_unstale":"2024-09-21T07:46:08.399658+0000","last_undegraded":"2024-09-21T07:46:08.399658+0000","last_fullsized":"2024-09-21T07:46:08.399658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:20.047491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400029+0000","last_change":"2024-09-21T07:46:07.211503+0000","last_active":"2024-09-21T07:46:08.400029+0000","last_peered":"2024-09-21T07:46:08.400029+0000","last_clean":"2024-09-21T07:46:08.400029+0000","last_became_active":"2024-09-21T07:46:07.211183+0000","last_became_peered":"2024-09-21T07:46:07.211183+0000","last_unstale":"2024-09-21T07:46:08.400029+0000","last_undegraded":"2024-09-21T07:46:08.400029+0000","last_fullsized":"2024-09-21T07:46:08.400029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:52.327032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399537+0000","last_change":"2024-09-21T07:46:07.228129+0000","last_active":"2024-09-21T07:46:08.399537+0000","last_peered":"2024-09-21T07:46:08.399537+0000","last_clean":"2024-09-21T07:46:08.399537+0000","last_became_active":"2024-09-21T07:46:07.227801+0000","last_became_peered":"2024-09-21T07:46:07.227801+0000","last_unstale":"2024-09-21T07:46:08.399537+0000","last_undegraded":"2024-09-21T07:46:08.399537+0000","last_fullsized":"2024-09-21T07:46:08.399537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.699234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199646+0000","last_change":"2024-09-21T07:46:06.217706+0000","last_active":"2024-09-21T07:46:09.199646+0000","last_peered":"2024-09-21T07:46:09.199646+0000","last_clean":"2024-09-21T07:46:09.199646+0000","last_became_active":"2024-09-21T07:46:06.215293+0000","last_became_peered":"2024-09-21T07:46:06.215293+0000","last_unstale":"2024-09-21T07:46:09.199646+0000","last_undegraded":"2024-09-21T07:46:09.199646+0000","last_fullsized":"2024-09-21T07:46:09.199646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:24.359404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500181+0000","last_change":"2024-09-21T07:46:07.232479+0000","last_active":"2024-09-21T07:46:09.500181+0000","last_peered":"2024-09-21T07:46:09.500181+0000","last_clean":"2024-09-21T07:46:09.500181+0000","last_became_active":"2024-09-21T07:46:07.232068+0000","last_became_peered":"2024-09-21T07:46:07.232068+0000","last_unstale":"2024-09-21T07:46:09.500181+0000","last_undegraded":"2024-09-21T07:46:09.500181+0000","last_fullsized":"2024-09-21T07:46:09.500181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:41.114984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199360+0000","last_change":"2024-09-21T07:46:06.210521+0000","last_active":"2024-09-21T07:46:09.199360+0000","last_peered":"2024-09-21T07:46:09.199360+0000","last_clean":"2024-09-21T07:46:09.199360+0000","last_became_active":"2024-09-21T07:46:06.210142+0000","last_became_peered":"2024-09-21T07:46:06.210142+0000","last_unstale":"2024-09-21T07:46:09.199360+0000","last_undegraded":"2024-09-21T07:46:09.199360+0000","last_fullsized":"2024-09-21T07:46:09.199360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:14:51.153791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208316+0000","last_change":"2024-09-21T07:46:07.210700+0000","last_active":"2024-09-21T07:46:08.208316+0000","last_peered":"2024-09-21T07:46:08.208316+0000","last_clean":"2024-09-21T07:46:08.208316+0000","last_became_active":"2024-09-21T07:46:07.203963+0000","last_became_peered":"2024-09-21T07:46:07.203963+0000","last_unstale":"2024-09-21T07:46:08.208316+0000","last_undegraded":"2024-09-21T07:46:08.208316+0000","last_fullsized":"2024-09-21T07:46:08.208316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209121+0000","last_change":"2024-09-21T07:46:06.209967+0000","last_active":"2024-09-21T07:46:08.209121+0000","last_peered":"2024-09-21T07:46:08.209121+0000","last_clean":"2024-09-21T07:46:08.209121+0000","last_became_active":"2024-09-21T07:46:06.207030+0000","last_became_peered":"2024-09-21T07:46:06.207030+0000","last_unstale":"2024-09-21T07:46:08.209121+0000","last_undegraded":"2024-09-21T07:46:08.209121+0000","last_fullsized":"2024-09-21T07:46:08.209121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:27.264294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209044+0000","last_change":"2024-09-21T07:46:07.219277+0000","last_active":"2024-09-21T07:46:08.209044+0000","last_peered":"2024-09-21T07:46:08.209044+0000","last_clean":"2024-09-21T07:46:08.209044+0000","last_became_active":"2024-09-21T07:46:07.218947+0000","last_became_peered":"2024-09-21T07:46:07.218947+0000","last_unstale":"2024-09-21T07:46:08.209044+0000","last_undegraded":"2024-09-21T07:46:08.209044+0000","last_fullsized":"2024-09-21T07:46:08.209044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208464+0000","last_change":"2024-09-21T07:46:06.209178+0000","last_active":"2024-09-21T07:46:08.208464+0000","last_peered":"2024-09-21T07:46:08.208464+0000","last_clean":"2024-09-21T07:46:08.208464+0000","last_became_active":"2024-09-21T07:46:06.201083+0000","last_became_peered":"2024-09-21T07:46:06.201083+0000","last_unstale":"2024-09-21T07:46:08.208464+0000","last_undegraded":"2024-09-21T07:46:08.208464+0000","last_fullsized":"2024-09-21T07:46:08.208464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476744,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476744,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476741,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476741,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509444,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509445,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509445,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":225280,"data_stored":73659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:46:12.112 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:46:12.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2024-09-21T07:46:12.112 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-09-21T07:46:12.172 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-09-21T07:46:12.170+0000 7fd6bee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff827e00640 1 -- 172.21.15.88:0/23741049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff808003a00 msgr2=0x7ff808003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff827e00640 1 --2- 172.21.15.88:0/23741049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff808003a00 0x7ff808003e70 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7ff814009b20 tx=0x7ff808005830 comp rx=0 tx=0).stop 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff825e00640 1 -- 172.21.15.88:0/23741049 reap_dead start 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff827e00640 1 -- 172.21.15.88:0/23741049 shutdown_connections 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff827e00640 1 -- 172.21.15.88:0/23741049 >> 172.21.15.88:0/23741049 conn(0x7ff82006d3d0 msgr2=0x7ff8200b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff827e00640 1 -- 172.21.15.88:0/23741049 shutdown_connections 2024-09-21T07:46:12.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.371+0000 7ff827e00640 1 -- 172.21.15.88:0/23741049 wait complete. 2024-09-21T07:46:12.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.373+0000 7ff827e00640 1 Processor -- start 2024-09-21T07:46:12.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.374+0000 7ff827e00640 1 -- start start 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff827e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a00 0x7ff82015f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff827e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff820157e00 0x7ff8201564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff827e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 0x7ff8200b2450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff827e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff8201571d0 con 0x7ff8200b2040 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff827e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff820157050 con 0x7ff820157e00 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff827e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff820157350 con 0x7ff808003a00 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff825e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 0x7ff8200b2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff825e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 0x7ff8200b2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49908/0 (socket says 172.21.15.88:49908) 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff825e00640 1 -- 172.21.15.88:0/2626990451 learned_addr learned my addr 172.21.15.88:0/2626990451 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:12.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff824a00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff820157e00 0x7ff8201564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.375+0000 7ff825400640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a00 0x7ff82015f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff825e00640 1 -- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a00 msgr2=0x7ff82015f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff825e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a00 0x7ff82015f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff825e00640 1 -- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff820157e00 msgr2=0x7ff8201564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff825e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff820157e00 0x7ff8201564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff825e00640 1 -- 172.21.15.88:0/2626990451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8200afdf0 con 0x7ff8200b2040 2024-09-21T07:46:12.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff825e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 0x7ff8200b2450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff8200afc90 tx=0x7ff818006660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff818011900 con 0x7ff8200b2040 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff818011aa0 con 0x7ff8200b2040 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8180103c0 con 0x7ff8200b2040 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8200b2990 con 0x7ff8200b2040 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.376+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff820157850 con 0x7ff8200b2040 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.377+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8200709c0 con 0x7ff8200b2040 2024-09-21T07:46:12.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.379+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff818010560 con 0x7ff8200b2040 2024-09-21T07:46:12.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.380+0000 7ff81d600640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff800078bc0 0x7ff80007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.380+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff81809b2a0 con 0x7ff8200b2040 2024-09-21T07:46:12.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.380+0000 7ff825400640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff800078bc0 0x7ff80007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.381+0000 7ff825400640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff800078bc0 0x7ff80007b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff814009b20 tx=0x7ff81402d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:12.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.384+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff8200b2990 con 0x7ff8200b2040 2024-09-21T07:46:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.571+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7ff8200afa70 con 0x7ff8200b2040 2024-09-21T07:46:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.572+0000 7ff81d600640 1 -- 172.21.15.88:0/2626990451 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v31) ==== 86+0+160 (secure 0 0 0) 0x7ff820070f00 con 0x7ff8200b2040 2024-09-21T07:46:12.574 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:12.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff800078bc0 msgr2=0x7ff80007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff800078bc0 0x7ff80007b0d0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff814009b20 tx=0x7ff81402d040 comp rx=0 tx=0).stop 2024-09-21T07:46:12.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 msgr2=0x7ff8200b2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 0x7ff8200b2450 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff8200afc90 tx=0x7ff818006660 comp rx=0 tx=0).stop 2024-09-21T07:46:12.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 shutdown_connections 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff800078bc0 0x7ff80007b0d0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8200b2040 0x7ff8200b2450 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff820157e00 0x7ff8201564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 --2- 172.21.15.88:0/2626990451 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff808003a00 0x7ff82015f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 >> 172.21.15.88:0/2626990451 conn(0x7ff82006d3d0 msgr2=0x7ff82015b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 shutdown_connections 2024-09-21T07:46:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.575+0000 7ff827e00640 1 -- 172.21.15.88:0/2626990451 wait complete. 2024-09-21T07:46:12.601 INFO:teuthology.orchestra.run.smithi088.stdout:[{"version":31,"timestamp":"2024-09-21T07:45:54.115662+0000","name":"","changes":[{"name":"osd.2/osd_mclock_max_capacity_iops_ssd","new_value":"3413.387841"}]}] 2024-09-21T07:46:12.601 INFO:tasks.ceph_manager:config epoch is 31 2024-09-21T07:46:12.602 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T07:46:12.602 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-09-21T07:46:12.602 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-09-21T07:46:12.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 -- 172.21.15.88:0/3282638611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ae2d0 msgr2=0x7f88c80c7610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 --2- 172.21.15.88:0/3282638611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ae2d0 0x7f88c80c7610 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f88b0005fd0 tx=0x7f88b002a540 comp rx=0 tx=0).stop 2024-09-21T07:46:12.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 -- 172.21.15.88:0/3282638611 shutdown_connections 2024-09-21T07:46:12.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 --2- 172.21.15.88:0/3282638611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ae2d0 0x7f88c80c7610 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 -- 172.21.15.88:0/3282638611 >> 172.21.15.88:0/3282638611 conn(0x7f88c80b2ff0 msgr2=0x7f88c80b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:12.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 -- 172.21.15.88:0/3282638611 shutdown_connections 2024-09-21T07:46:12.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.822+0000 7f88ce800640 1 -- 172.21.15.88:0/3282638611 wait complete. 2024-09-21T07:46:12.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.824+0000 7f88ce800640 1 Processor -- start 2024-09-21T07:46:12.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.824+0000 7f88ce800640 1 -- start start 2024-09-21T07:46:12.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88ce800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88c80ae2d0 0x7f88c80b5980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88ce800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88c80ae6e0 0x7f88c80cd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88ce800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 0x7f88c814bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88ce800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f88c80b2230 con 0x7f88c80ce7c0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88ce800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f88c80b20b0 con 0x7f88c80ae2d0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88ce800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f88c80c7b50 con 0x7f88c80ae6e0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 0x7f88c814bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 0x7f88c814bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49920/0 (socket says 172.21.15.88:49920) 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 -- 172.21.15.88:0/1580627070 learned_addr learned my addr 172.21.15.88:0/1580627070 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88bea00640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88c80ae6e0 0x7f88c80cd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7400640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88c80ae2d0 0x7f88c80b5980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 -- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88c80ae6e0 msgr2=0x7f88c80cd230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88c80ae6e0 0x7f88c80cd230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 -- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88c80ae2d0 msgr2=0x7f88c80b5980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88c80ae2d0 0x7f88c80b5980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.826+0000 7f88c7e00640 1 -- 172.21.15.88:0/1580627070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88c80cd770 con 0x7f88c80ce7c0 2024-09-21T07:46:12.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.827+0000 7f88c7e00640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 0x7f88c814bd10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c760 tx=0x7f88c000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.827+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c0017070 con 0x7f88c80ce7c0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.827+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88c000ce10 con 0x7f88c80ce7c0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.827+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88c815fc00 con 0x7f88c80ce7c0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.827+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c00052b0 con 0x7f88c80ce7c0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.827+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88c80b4ea0 con 0x7f88c80ce7c0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.828+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88c8070920 con 0x7f88c80ce7c0 2024-09-21T07:46:12.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.829+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f88c001a070 con 0x7f88c80ce7c0 2024-09-21T07:46:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.830+0000 7f88bfe00640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f88a4078bc0 0x7f88a407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.830+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f88c009a2e0 con 0x7f88c80ce7c0 2024-09-21T07:46:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.831+0000 7f88c7400640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f88a4078bc0 0x7f88a407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.831+0000 7f88c7400640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f88a4078bc0 0x7f88a407b0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88b8006fd0 tx=0x7f88b8009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:12.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:12.835+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f88c8070920 con 0x7f88c80ce7c0 2024-09-21T07:46:13.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.068+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f88c815fe80 con 0x7f88c80ce7c0 2024-09-21T07:46:13.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.071+0000 7f88bfe00640 1 -- 172.21.15.88:0/1580627070 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) ==== 74+0+194722 (secure 0 0 0) 0x7f88c00620d0 con 0x7f88c80ce7c0 2024-09-21T07:46:13.076 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:13.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f88a4078bc0 msgr2=0x7f88a407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f88a4078bc0 0x7f88a407b0d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f88b8006fd0 tx=0x7f88b8009060 comp rx=0 tx=0).stop 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 msgr2=0x7f88c814bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 0x7f88c814bd10 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f88c000c760 tx=0x7f88c000cc20 comp rx=0 tx=0).stop 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 shutdown_connections 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f88a4078bc0 0x7f88a407b0d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f88c80ae6e0 0x7f88c80cd230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88c80ce7c0 0x7f88c814bd10 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 --2- 172.21.15.88:0/1580627070 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88c80ae2d0 0x7f88c80b5980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 >> 172.21.15.88:0/1580627070 conn(0x7f88c80b2ff0 msgr2=0x7f88c8070210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 shutdown_connections 2024-09-21T07:46:13.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.075+0000 7f88ce800640 1 -- 172.21.15.88:0/1580627070 wait complete. 2024-09-21T07:46:13.098 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":5,"flags":0,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3294411971},{"type":"v1","addr":"172.21.15.175:6801","nonce":3294411971}]},"active_addr":"172.21.15.175:6801/3294411971","active_change":"2024-09-21T07:45:43.841149+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":4116,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":3741301681}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":1554442378}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":2139739944}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.175:0","nonce":1671168534}]}]} 2024-09-21T07:46:13.103 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-09-21T07:46:13.103 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-09-21T07:46:13.103 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:13.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 -- 172.21.15.88:0/1354037170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa60815d760 msgr2=0x7fa6081535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1354037170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa60815d760 0x7fa6081535c0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8005fd0 tx=0x7fa5f802a540 comp rx=0 tx=0).stop 2024-09-21T07:46:13.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 -- 172.21.15.88:0/1354037170 shutdown_connections 2024-09-21T07:46:13.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1354037170 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6080b3550 0x7fa6080b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1354037170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa60815d760 0x7fa6081535c0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 -- 172.21.15.88:0/1354037170 >> 172.21.15.88:0/1354037170 conn(0x7fa60806d730 msgr2=0x7fa6080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 -- 172.21.15.88:0/1354037170 shutdown_connections 2024-09-21T07:46:13.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.321+0000 7fa60fa00640 1 -- 172.21.15.88:0/1354037170 wait complete. 2024-09-21T07:46:13.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.323+0000 7fa60fa00640 1 Processor -- start 2024-09-21T07:46:13.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.323+0000 7fa60fa00640 1 -- start start 2024-09-21T07:46:13.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60fa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 0x7fa60809b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080b3550 0x7fa60814aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815d760 0x7fa60809de40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60fa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa60806f240 con 0x7fa6080ad0e0 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6080b4e30 con 0x7fa6080b3550 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6080b3ea0 con 0x7fa60815d760 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 0x7fa60809b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 0x7fa60809b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49930/0 (socket says 172.21.15.88:49930) 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60d000640 1 -- 172.21.15.88:0/1793331419 learned_addr learned my addr 172.21.15.88:0/1793331419 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60da00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815d760 0x7fa60809de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa607e00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080b3550 0x7fa60814aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.325+0000 7fa60d000640 1 -- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815d760 msgr2=0x7fa60809de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60d000640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815d760 0x7fa60809de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60d000640 1 -- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080b3550 msgr2=0x7fa60814aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60d000640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080b3550 0x7fa60814aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60d000640 1 -- 172.21.15.88:0/1793331419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa608073680 con 0x7fa6080ad0e0 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa607e00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080b3550 0x7fa60814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60d000640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 0x7fa60809b320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000c00 tx=0x7fa5f8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f803e030 con 0x7fa6080ad0e0 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5f8003e00 con 0x7fa6080ad0e0 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa608073960 con 0x7fa6080ad0e0 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5f8035040 con 0x7fa6080ad0e0 2024-09-21T07:46:13.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.326+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa608073f20 con 0x7fa6080ad0e0 2024-09-21T07:46:13.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.327+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6080b42f0 con 0x7fa6080ad0e0 2024-09-21T07:46:13.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.329+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa5f8041030 con 0x7fa6080ad0e0 2024-09-21T07:46:13.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.330+0000 7fa604c00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa5e8078b90 0x7fa5e807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.330+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa5f80bb3f0 con 0x7fa6080ad0e0 2024-09-21T07:46:13.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.331+0000 7fa607e00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa5e8078b90 0x7fa5e807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.331+0000 7fa607e00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa5e8078b90 0x7fa5e807b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa60809d4f0 tx=0x7fa5f4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:13.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.335+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa5f8083950 con 0x7fa6080ad0e0 2024-09-21T07:46:13.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.520+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa608156f60 con 0x7fa6080ad0e0 2024-09-21T07:46:13.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.521+0000 7fa604c00640 1 -- 172.21.15.88:0/1793331419 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fa5f8083250 con 0x7fa6080ad0e0 2024-09-21T07:46:13.524 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:13.524 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa5e8078b90 msgr2=0x7fa5e807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa5e8078b90 0x7fa5e807b0a0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa60809d4f0 tx=0x7fa5f4009040 comp rx=0 tx=0).stop 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 msgr2=0x7fa60809b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 0x7fa60809b320 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fa5f8000c00 tx=0x7fa5f8003910 comp rx=0 tx=0).stop 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 shutdown_connections 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa5e8078b90 0x7fa5e807b0a0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa60815d760 0x7fa60809de40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6080b3550 0x7fa60814aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.523+0000 7fa60fa00640 1 --2- 172.21.15.88:0/1793331419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6080ad0e0 0x7fa60809b320 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.524+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 >> 172.21.15.88:0/1793331419 conn(0x7fa60806d730 msgr2=0x7fa6080b0c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.524+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 shutdown_connections 2024-09-21T07:46:13.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.524+0000 7fa60fa00640 1 -- 172.21.15.88:0/1793331419 wait complete. 2024-09-21T07:46:13.547 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-09-21T07:46:13.547 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 -- 172.21.15.88:0/2231250217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92015fae0 msgr2=0x7fb9200b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 --2- 172.21.15.88:0/2231250217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92015fae0 0x7fb9200b3750 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb920072810 tx=0x7fb91802aa30 comp rx=0 tx=0).stop 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 -- 172.21.15.88:0/2231250217 shutdown_connections 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 --2- 172.21.15.88:0/2231250217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb92015fae0 0x7fb9200b3750 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 --2- 172.21.15.88:0/2231250217 >> v2:172.21.15.175:3300/0 conn(0x7fb9200aebc0 0x7fb920152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 -- 172.21.15.88:0/2231250217 >> 172.21.15.88:0/2231250217 conn(0x7fb92006d730 msgr2=0x7fb9200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 -- 172.21.15.88:0/2231250217 shutdown_connections 2024-09-21T07:46:13.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.765+0000 7fb924e00640 1 -- 172.21.15.88:0/2231250217 wait complete. 2024-09-21T07:46:13.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.768+0000 7fb924e00640 1 Processor -- start 2024-09-21T07:46:13.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.768+0000 7fb924e00640 1 -- start start 2024-09-21T07:46:13.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb924e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9200aebc0 0x7fb9201f2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb924e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9200b2750 0x7fb9200b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb924e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 0x7fb9200b1200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb924e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9200b40d0 con 0x7fb9201f2f50 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb924e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb9200b3f50 con 0x7fb9200b2750 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb924e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb9200b4250 con 0x7fb9200aebc0 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb91e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 0x7fb9200b1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb91e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 0x7fb9200b1200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49940/0 (socket says 172.21.15.88:49940) 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.769+0000 7fb91e800640 1 -- 172.21.15.88:0/3822398432 learned_addr learned my addr 172.21.15.88:0/3822398432 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91de00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9200aebc0 0x7fb9201f2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91d400640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9200b2750 0x7fb9200b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91e800640 1 -- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9200aebc0 msgr2=0x7fb9201f2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91e800640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9200aebc0 0x7fb9201f2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91e800640 1 -- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9200b2750 msgr2=0x7fb9200b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91e800640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9200b2750 0x7fb9200b0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91e800640 1 -- 172.21.15.88:0/3822398432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb92014d750 con 0x7fb9201f2f50 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91de00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9200aebc0 0x7fb9201f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91d400640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9200b2750 0x7fb9200b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb91e800640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 0x7fb9200b1200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb918005fd0 tx=0x7fb918003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918033050 con 0x7fb9201f2f50 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.770+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb918004540 con 0x7fb9201f2f50 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.771+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb918038450 con 0x7fb9201f2f50 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.771+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb92014da30 con 0x7fb9201f2f50 2024-09-21T07:46:13.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.771+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9201591b0 con 0x7fb9201f2f50 2024-09-21T07:46:13.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.772+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9200b4770 con 0x7fb9201f2f50 2024-09-21T07:46:13.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.773+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb918036070 con 0x7fb9201f2f50 2024-09-21T07:46:13.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.774+0000 7fb916000640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb900078bc0 0x7fb90007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:13.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.774+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb9180bc090 con 0x7fb9201f2f50 2024-09-21T07:46:13.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.775+0000 7fb91de00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb900078bc0 0x7fb90007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:13.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.775+0000 7fb91de00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb900078bc0 0x7fb90007b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb910009b80 tx=0x7fb910005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:13.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.779+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb9180844c0 con 0x7fb9201f2f50 2024-09-21T07:46:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.968+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb92015fca0 con 0x7fb9201f2f50 2024-09-21T07:46:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.969+0000 7fb916000640 1 -- 172.21.15.88:0/3822398432 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fb918083dc0 con 0x7fb9201f2f50 2024-09-21T07:46:13.972 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:13.972 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:13.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb900078bc0 msgr2=0x7fb90007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb900078bc0 0x7fb90007b0d0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fb910009b80 tx=0x7fb910005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:13.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 msgr2=0x7fb9200b1200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:13.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 0x7fb9200b1200 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb918005fd0 tx=0x7fb918003f90 comp rx=0 tx=0).stop 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 shutdown_connections 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb900078bc0 0x7fb90007b0d0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9201f2f50 0x7fb9200b1200 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb9200b2750 0x7fb9200b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 --2- 172.21.15.88:0/3822398432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb9200aebc0 0x7fb9201f2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 >> 172.21.15.88:0/3822398432 conn(0x7fb92006d730 msgr2=0x7fb92015cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 shutdown_connections 2024-09-21T07:46:13.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:13.972+0000 7fb924e00640 1 -- 172.21.15.88:0/3822398432 wait complete. 2024-09-21T07:46:13.996 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-09-21T07:46:13.996 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-09-21T07:46:13.997 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-09-21T07:46:13.997 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-09-21T07:46:13.997 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-09-21T07:46:13.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-09-21T07:46:13.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-09-21T07:46:13.999 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-09-21T07:46:14.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.385+0000 7fb27e200640 1 -- 172.21.15.88:0/3133322675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270054720 msgr2=0x7fb270054b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.385+0000 7fb27e200640 1 --2- 172.21.15.88:0/3133322675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270054720 0x7fb270054b30 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb27806b360 tx=0x7fb2780a6670 comp rx=0 tx=0).stop 2024-09-21T07:46:14.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.385+0000 7fb27e200640 1 -- 172.21.15.88:0/3133322675 shutdown_connections 2024-09-21T07:46:14.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.385+0000 7fb27e200640 1 --2- 172.21.15.88:0/3133322675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270054720 0x7fb270054b30 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.385+0000 7fb27e200640 1 --2- 172.21.15.88:0/3133322675 >> v2:172.21.15.175:3300/0 conn(0x7fb270052360 0x7fb2700e7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.385+0000 7fb27e200640 1 -- 172.21.15.88:0/3133322675 >> 172.21.15.88:0/3133322675 conn(0x7fb27001d9a0 msgr2=0x7fb270039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.386+0000 7fb27e200640 1 -- 172.21.15.88:0/3133322675 shutdown_connections 2024-09-21T07:46:14.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.386+0000 7fb27e200640 1 -- 172.21.15.88:0/3133322675 wait complete. 2024-09-21T07:46:14.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.391+0000 7fb27e200640 1 Processor -- start 2024-09-21T07:46:14.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.392+0000 7fb27e200640 1 -- start start 2024-09-21T07:46:14.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.392+0000 7ff048a00640 1 -- 172.21.15.88:0/691534591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c056da0 msgr2=0x7ff03c057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.392+0000 7ff048a00640 1 --2- 172.21.15.88:0/691534591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c056da0 0x7ff03c057c80 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7ff04406ad00 tx=0x7ff0440a65e0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.393+0000 7ff048a00640 1 -- 172.21.15.88:0/691534591 shutdown_connections 2024-09-21T07:46:14.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.393+0000 7ff048a00640 1 --2- 172.21.15.88:0/691534591 >> v2:172.21.15.88:3300/0 conn(0x7ff03c054360 0x7ff03c03c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.393+0000 7ff048a00640 1 --2- 172.21.15.88:0/691534591 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c056da0 0x7ff03c057c80 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.393+0000 7ff048a00640 1 -- 172.21.15.88:0/691534591 >> 172.21.15.88:0/691534591 conn(0x7ff03c03a5f0 msgr2=0x7ff03c03aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.395+0000 7fb27e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb270051630 0x7fb2700d8700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.395+0000 7fb27e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270052360 0x7fb2700322e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.395+0000 7fb27e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb270054720 0x7fb270032820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.395+0000 7fb27e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb270052770 con 0x7fb270054720 2024-09-21T07:46:14.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.395+0000 7fb27e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb270033ec0 con 0x7fb270051630 2024-09-21T07:46:14.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.395+0000 7fb27e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2700528f0 con 0x7fb270052360 2024-09-21T07:46:14.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.394+0000 7ff048a00640 1 -- 172.21.15.88:0/691534591 shutdown_connections 2024-09-21T07:46:14.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.396+0000 7fb277400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb270051630 0x7fb2700d8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.396+0000 7fb277400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb270051630 0x7fb2700d8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:60690/0 (socket says 172.21.15.88:60690) 2024-09-21T07:46:14.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.396+0000 7fb277400640 1 -- 172.21.15.88:0/2399947017 learned_addr learned my addr 172.21.15.88:0/2399947017 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.396+0000 7fb277e00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb270054720 0x7fb270032820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb276a00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270052360 0x7fb2700322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb276a00640 1 -- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb270051630 msgr2=0x7fb2700d8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb276a00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb270051630 0x7fb2700d8700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb276a00640 1 -- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb270054720 msgr2=0x7fb270032820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb276a00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb270054720 0x7fb270032820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.394+0000 7ff048a00640 1 -- 172.21.15.88:0/691534591 wait complete. 2024-09-21T07:46:14.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb276a00640 1 -- 172.21.15.88:0/2399947017 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb270050930 con 0x7fb270052360 2024-09-21T07:46:14.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.397+0000 7fb277e00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb270054720 0x7fb270032820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:14.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7fb276a00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270052360 0x7fb2700322e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c3f0 tx=0x7fb26c00c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7ff048a00640 1 Processor -- start 2024-09-21T07:46:14.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7febe7a00640 1 -- 172.21.15.88:0/4161958041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00d32d0 msgr2=0x7febe00b41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7febe7a00640 1 --2- 172.21.15.88:0/4161958041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00d32d0 0x7febe00b41a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7febe008d5e0 tx=0x7febd802a920 comp rx=0 tx=0).stop 2024-09-21T07:46:14.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7ff048a00640 1 -- start start 2024-09-21T07:46:14.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7febe7a00640 1 -- 172.21.15.88:0/4161958041 shutdown_connections 2024-09-21T07:46:14.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7febe7a00640 1 --2- 172.21.15.88:0/4161958041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00d32d0 0x7febe00b41a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.398+0000 7febe7a00640 1 --2- 172.21.15.88:0/4161958041 >> v2:172.21.15.175:3300/0 conn(0x7febe00d2500 0x7febe00b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.399+0000 7febe7a00640 1 -- 172.21.15.88:0/4161958041 >> 172.21.15.88:0/4161958041 conn(0x7febe008bfe0 msgr2=0x7febe00d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.399+0000 7febe7a00640 1 -- 172.21.15.88:0/4161958041 shutdown_connections 2024-09-21T07:46:14.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.399+0000 7febe7a00640 1 -- 172.21.15.88:0/4161958041 wait complete. 2024-09-21T07:46:14.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.400+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011790 con 0x7fb270052360 2024-09-21T07:46:14.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.400+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb26c017070 con 0x7fb270052360 2024-09-21T07:46:14.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.400+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb26c011cc0 con 0x7fb270052360 2024-09-21T07:46:14.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.400+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27004de20 con 0x7fb270052360 2024-09-21T07:46:14.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.400+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27004e3e0 con 0x7fb270052360 2024-09-21T07:46:14.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 -- 172.21.15.88:0/4234098375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94054360 msgr2=0x7f3f94057c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 --2- 172.21.15.88:0/4234098375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94054360 0x7f3f94057c80 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f3f94011f00 tx=0x7f3f8802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 -- 172.21.15.88:0/4234098375 shutdown_connections 2024-09-21T07:46:14.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 --2- 172.21.15.88:0/4234098375 >> v2:172.21.15.175:3300/0 conn(0x7f3f940e2f40 0x7f3f9403c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 --2- 172.21.15.88:0/4234098375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94054360 0x7f3f94057c80 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 -- 172.21.15.88:0/4234098375 >> 172.21.15.88:0/4234098375 conn(0x7f3f9403a5f0 msgr2=0x7f3f9403aa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7f3fa1400640 1 -- 172.21.15.88:0/4234098375 shutdown_connections 2024-09-21T07:46:14.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.401+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb254001030 con 0x7fb270052360 2024-09-21T07:46:14.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff048a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff03c054360 0x7ff03c038e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff048a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c056da0 0x7ff03c0515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff048a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c0e2f40 0x7ff03c04d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff048a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c007030 con 0x7ff03c056da0 2024-09-21T07:46:14.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff048a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff03c006eb0 con 0x7ff03c0e2f40 2024-09-21T07:46:14.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff048a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff03c0071b0 con 0x7ff03c054360 2024-09-21T07:46:14.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7f17be800640 1 -- 172.21.15.88:0/3832863278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00574b0 msgr2=0x7f17b00e6d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7f17be800640 1 --2- 172.21.15.88:0/3832863278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00574b0 0x7f17b00e6d90 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009830 tx=0x7f17ac02a290 comp rx=0 tx=0).stop 2024-09-21T07:46:14.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.402+0000 7f3fa1400640 1 -- 172.21.15.88:0/4234098375 wait complete. 2024-09-21T07:46:14.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff042a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c056da0 0x7ff03c0515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff042a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c056da0 0x7ff03c0515c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49994/0 (socket says 172.21.15.88:49994) 2024-09-21T07:46:14.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7ff042a00640 1 -- 172.21.15.88:0/1997921899 learned_addr learned my addr 172.21.15.88:0/1997921899 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7f17be800640 1 -- 172.21.15.88:0/3832863278 shutdown_connections 2024-09-21T07:46:14.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7f17be800640 1 --2- 172.21.15.88:0/3832863278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b0051280 0x7f17b0051690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7f17be800640 1 --2- 172.21.15.88:0/3832863278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00574b0 0x7f17b00e6d90 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.403+0000 7f17be800640 1 -- 172.21.15.88:0/3832863278 >> 172.21.15.88:0/3832863278 conn(0x7f17b0056990 msgr2=0x7f17b0056da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff043400640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff03c054360 0x7ff03c038e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7febe7a00640 1 Processor -- start 2024-09-21T07:46:14.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff042a00640 1 -- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff03c054360 msgr2=0x7ff03c038e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7febe7a00640 1 -- start start 2024-09-21T07:46:14.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7f17be800640 1 -- 172.21.15.88:0/3832863278 shutdown_connections 2024-09-21T07:46:14.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7f17be800640 1 -- 172.21.15.88:0/3832863278 wait complete. 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff042a00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff03c054360 0x7ff03c038e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff043e00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c0e2f40 0x7ff03c04d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff042a00640 1 -- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c0e2f40 msgr2=0x7ff03c04d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff042a00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c0e2f40 0x7ff03c04d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff042a00640 1 -- 172.21.15.88:0/1997921899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff03c039740 con 0x7ff03c056da0 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.404+0000 7ff043e00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff03c0e2f40 0x7ff03c04d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.405+0000 7f3fa1400640 1 Processor -- start 2024-09-21T07:46:14.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.406+0000 7f3fa1400640 1 -- start start 2024-09-21T07:46:14.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.406+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb26c01c460 con 0x7fb270052360 2024-09-21T07:46:14.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.406+0000 7ff042a00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c056da0 0x7ff03c0515c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c370 tx=0x7ff03800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.407+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038006890 con 0x7ff03c056da0 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.407+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff038006b20 con 0x7ff03c056da0 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.407+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038006890 con 0x7ff03c056da0 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.408+0000 7febe7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00b9d60 0x7febe00cbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.408+0000 7fb26b400640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb258078b90 0x7fb25807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.408+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb26c09ac60 con 0x7fb270052360 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7fb26b400640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7fb258080e60 0x7fb2580832b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 --> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb26c01be50 con 0x7fb258080e60 2024-09-21T07:46:14.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fb26c09b050 con 0x7fb270052360 2024-09-21T07:46:14.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.408+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff03c04c790 con 0x7ff03c056da0 2024-09-21T07:46:14.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.408+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff03c04cd50 con 0x7ff03c056da0 2024-09-21T07:46:14.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f17be800640 1 Processor -- start 2024-09-21T07:46:14.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f17be800640 1 -- start start 2024-09-21T07:46:14.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3fa1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f94054360 0x7f3f940d8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3fa1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94056da0 0x7f3f940508e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3fa1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f940e2f40 0x7f3f94050e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3fa1400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94007110 con 0x7f3f94056da0 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3fa1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f94006f90 con 0x7f3f940e2f40 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3fa1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f94007290 con 0x7f3f94054360 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7febe7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe00d2500 0x7febe00c9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.410+0000 7fb277e00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7fb258080e60 0x7fb2580832b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3f9be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f94054360 0x7f3f940d8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3f9be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f94054360 0x7f3f940d8000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:54930/0 (socket says 172.21.15.88:54930) 2024-09-21T07:46:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7f3f9be00640 1 -- 172.21.15.88:0/1102221296 learned_addr learned my addr 172.21.15.88:0/1102221296 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.410+0000 7f3f9b400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94056da0 0x7f3f940508e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.410+0000 7f3fa0a00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f940e2f40 0x7f3f94050e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.410+0000 7fb277400640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb258078b90 0x7fb25807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7febe7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe00d32d0 0x7febe00ca300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7febe7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7febe00b4e10 con 0x7febe00b9d60 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7febe7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7febe00b4c90 con 0x7febe00d2500 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7febe7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7febe00b4f90 con 0x7febe00d32d0 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.409+0000 7febe5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00b9d60 0x7febe00cbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.410+0000 7febe5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00b9d60 0x7febe00cbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50014/0 (socket says 172.21.15.88:50014) 2024-09-21T07:46:14.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.410+0000 7febe5000640 1 -- 172.21.15.88:0/2572544710 learned_addr learned my addr 172.21.15.88:0/2572544710 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7fb277400640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb258078b90 0x7fb25807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb264009b70 tx=0x7fb264005f30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7f3f9be00640 1 -- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f940e2f40 msgr2=0x7f3f94050e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7f3f9be00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f940e2f40 0x7f3f94050e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7f3f9be00640 1 -- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94056da0 msgr2=0x7f3f940508e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7f3f9be00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94056da0 0x7f3f940508e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7f3f9be00640 1 -- 172.21.15.88:0/1102221296 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f940d8540 con 0x7f3f94054360 2024-09-21T07:46:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7f3f9b400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f94056da0 0x7f3f940508e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7febdfe00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe00d2500 0x7febe00c9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7febe5a00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe00d32d0 0x7febe00ca300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7f3f9be00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f94054360 0x7f3f940d8000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c06bab0 tx=0x7f3f9c06a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7febe5000640 1 -- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe00d32d0 msgr2=0x7febe00ca300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.411+0000 7febe5000640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7febe00d32d0 0x7febe00ca300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7febe5000640 1 -- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe00d2500 msgr2=0x7febe00c9dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7febe5000640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7febe00d2500 0x7febe00c9dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7febe5000640 1 -- 172.21.15.88:0/2572544710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febe01f94c0 con 0x7febe00b9d60 2024-09-21T07:46:14.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7fb277e00640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7fb258080e60 0x7fb2580832b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c069430 con 0x7f3f94054360 2024-09-21T07:46:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f9c0696c0 con 0x7f3f94054360 2024-09-21T07:46:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7febe5000640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00b9d60 0x7febe00cbd00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7febd400e7c0 tx=0x7febd400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f9c093400 con 0x7f3f94054360 2024-09-21T07:46:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f94052c20 con 0x7f3f94054360 2024-09-21T07:46:14.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.413+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f9404fe00 con 0x7f3f94054360 2024-09-21T07:46:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.412+0000 7f17be800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17b0051280 0x7f17b017ab30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.413+0000 7f17be800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b00574b0 0x7f17b017d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17be800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00e72d0 0x7f17b017dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17be800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0051f60 con 0x7f17b0051280 2024-09-21T07:46:14.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f3f94000bc0 con 0x7f3f94054360 2024-09-21T07:46:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17be800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f17b0051de0 con 0x7f17b00e72d0 2024-09-21T07:46:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd400ccc0 con 0x7febe00b9d60 2024-09-21T07:46:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7febd4010430 con 0x7febe00b9d60 2024-09-21T07:46:14.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febd4010860 con 0x7febe00b9d60 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17be800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f17b00520e0 con 0x7f17b00574b0 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17bd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17b0051280 0x7f17b017ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17bd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17b0051280 0x7f17b017ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50034/0 (socket says 172.21.15.88:50034) 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17bd400640 1 -- 172.21.15.88:0/1746900550 learned_addr learned my addr 172.21.15.88:0/1746900550 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17bca00640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b00574b0 0x7f17b017d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7f17bde00640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00e72d0 0x7f17b017dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7f17bd400640 1 -- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b00574b0 msgr2=0x7f17b017d6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7f17bd400640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17b00574b0 0x7f17b017d6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7f17bd400640 1 -- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00e72d0 msgr2=0x7f17b017dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7f17bd400640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17b00e72d0 0x7f17b017dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7f17bd400640 1 -- 172.21.15.88:0/1746900550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17b017b070 con 0x7f17b0051280 2024-09-21T07:46:14.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febe01f97a0 con 0x7febe00b9d60 2024-09-21T07:46:14.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.415+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febe01f6c50 con 0x7febe00b9d60 2024-09-21T07:46:14.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== osd.0 v2:172.21.15.88:6808/2605666815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb26c01be50 con 0x7fb258080e60 2024-09-21T07:46:14.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff03c0076d0 con 0x7ff03c056da0 2024-09-21T07:46:14.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.414+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ff038019020 con 0x7ff03c056da0 2024-09-21T07:46:14.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.416+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febe008caf0 con 0x7febe00b9d60 2024-09-21T07:46:14.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.416+0000 7f17bd400640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17b0051280 0x7f17b017ab30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009890 tx=0x7f17ac003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.417+0000 7fd5aea00640 1 -- 172.21.15.88:0/1580922362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c003ce0 msgr2=0x7fd58c0040f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.417+0000 7fd5aea00640 1 --2- 172.21.15.88:0/1580922362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd58c003ce0 0x7fd58c0040f0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd5a0009a90 tx=0x7fd58c005830 comp rx=0 tx=0).stop 2024-09-21T07:46:14.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7fd5aca00640 1 -- 172.21.15.88:0/1580922362 reap_dead start 2024-09-21T07:46:14.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.416+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f3f9c0935a0 con 0x7f3f94054360 2024-09-21T07:46:14.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7fd5aea00640 1 -- 172.21.15.88:0/1580922362 shutdown_connections 2024-09-21T07:46:14.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7fd5aea00640 1 -- 172.21.15.88:0/1580922362 >> 172.21.15.88:0/1580922362 conn(0x7fd5a806d3d0 msgr2=0x7fd5a80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac037040 con 0x7f17b0051280 2024-09-21T07:46:14.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f17ac040070 con 0x7f17b0051280 2024-09-21T07:46:14.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17ac032440 con 0x7f17b0051280 2024-09-21T07:46:14.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7f3f8fe00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3f7c078b90 0x7f3f7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7fd5aea00640 1 -- 172.21.15.88:0/1580922362 shutdown_connections 2024-09-21T07:46:14.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.419+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f3f9c116300 con 0x7f3f94054360 2024-09-21T07:46:14.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.419+0000 7f3f9b400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3f7c078b90 0x7f3f7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.419+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7febd4010a00 con 0x7febe00b9d60 2024-09-21T07:46:14.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.419+0000 7fd5aea00640 1 -- 172.21.15.88:0/1580922362 wait complete. 2024-09-21T07:46:14.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.417+0000 7ff037400640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff024078d90 0x7ff02407b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.417+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ff03809a010 con 0x7ff03c056da0 2024-09-21T07:46:14.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7ff037400640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff024080f80 0x7ff0240833d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 --> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff024083aa0 con 0x7ff024080f80 2024-09-21T07:46:14.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7ff03809c5b0 con 0x7ff03c056da0 2024-09-21T07:46:14.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.420+0000 7f3f8fe00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7f3f7c080e00 0x7f3f7c083250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17b004a760 con 0x7f17b0051280 2024-09-21T07:46:14.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.418+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17b004c440 con 0x7f17b0051280 2024-09-21T07:46:14.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.419+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1794001030 con 0x7f17b0051280 2024-09-21T07:46:14.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.420+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 --> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f3f9c070e50 con 0x7f3f7c080e00 2024-09-21T07:46:14.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.420+0000 7ff043400640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff024078d90 0x7ff02407b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.420+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f3f9c11d050 con 0x7f3f94054360 2024-09-21T07:46:14.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.420+0000 7f3fa0a00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7f3f7c080e00 0x7f3f7c083250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.420+0000 7ff043e00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff024080f80 0x7ff0240833d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 -- 172.21.15.88:0/4257742466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd000 msgr2=0x7f042c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 --2- 172.21.15.88:0/4257742466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd000 0x7f042c0cf570 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f042c075570 tx=0x7f041c02c200 comp rx=0 tx=0).stop 2024-09-21T07:46:14.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f3f9b400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3f7c078b90 0x7f3f7c07b0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f88033000 tx=0x7f3f88004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7ff043400640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff024078d90 0x7ff02407b2a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff03c006880 tx=0x7ff04406cc60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7febdcc00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7febc0078c90 0x7febc007b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7febd409c220 con 0x7febe00b9d60 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7febdcc00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7febc0080f00 0x7febc0083350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 --> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febd401bec0 con 0x7febc0080f00 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7febd409c610 con 0x7febe00b9d60 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7febdfe00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7febc0078c90 0x7febc007b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f1be2400640 1 -- 172.21.15.88:0/3539323329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0b3d60 msgr2=0x7f1bdc0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 -- 172.21.15.88:0/4257742466 shutdown_connections 2024-09-21T07:46:14.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 --2- 172.21.15.88:0/4257742466 >> v2:172.21.15.88:3300/0 conn(0x7f042c0cd410 0x7f042c0cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 --2- 172.21.15.88:0/4257742466 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd000 0x7f042c0cf570 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 -- 172.21.15.88:0/4257742466 >> 172.21.15.88:0/4257742466 conn(0x7f042c06d3d0 msgr2=0x7f042c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 -- 172.21.15.88:0/4257742466 shutdown_connections 2024-09-21T07:46:14.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.421+0000 7f0432800640 1 -- 172.21.15.88:0/4257742466 wait complete. 2024-09-21T07:46:14.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f1be2400640 1 --2- 172.21.15.88:0/3539323329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0b3d60 0x7f1bdc0af510 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1bc4013f30 tx=0x7f1bc4032aa0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f3fa0a00640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7f3f7c080e00 0x7f3f7c083250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f1be2400640 1 -- 172.21.15.88:0/3539323329 shutdown_connections 2024-09-21T07:46:14.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f1be2400640 1 --2- 172.21.15.88:0/3539323329 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0b3d60 0x7f1bdc0af510 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f1be2400640 1 --2- 172.21.15.88:0/3539323329 >> v2:172.21.15.88:3300/0 conn(0x7f1bdc0711c0 0x7f1bdc0701e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f1be2400640 1 -- 172.21.15.88:0/3539323329 >> 172.21.15.88:0/3539323329 conn(0x7f1bdc06d3d0 msgr2=0x7f1bdc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.422+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f17ac045460 con 0x7f17b0051280 2024-09-21T07:46:14.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.423+0000 7ff043e00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff024080f80 0x7ff0240833d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.423+0000 7fd5aea00640 1 Processor -- start 2024-09-21T07:46:14.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.424+0000 7fd5aea00640 1 -- start start 2024-09-21T07:46:14.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.423+0000 7f1be2400640 1 -- 172.21.15.88:0/3539323329 shutdown_connections 2024-09-21T07:46:14.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.423+0000 7f1be2400640 1 -- 172.21.15.88:0/3539323329 wait complete. 2024-09-21T07:46:14.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.424+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== osd.3 v2:172.21.15.88:6824/1227287750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f3f9c070e50 con 0x7f3f7c080e00 2024-09-21T07:46:14.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.423+0000 7febe5a00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7febc0080f00 0x7febc0083350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.424+0000 7febdfe00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7febc0078c90 0x7febc007b1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7febe006e5e0 tx=0x7febcc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f0432800640 1 Processor -- start 2024-09-21T07:46:14.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.424+0000 7f17b5600640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1798078be0 0x7f179807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.424+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f17ac0bab30 con 0x7f17b0051280 2024-09-21T07:46:14.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f17b5600640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f1798080eb0 0x7f1798083300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 --> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f17ac02eec0 con 0x7f1798080eb0 2024-09-21T07:46:14.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f17ac0bf2a0 con 0x7f17b0051280 2024-09-21T07:46:14.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== osd.1 v2:172.21.15.88:6800/3750368380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff024083aa0 con 0x7ff024080f80 2024-09-21T07:46:14.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f17bde00640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f1798080eb0 0x7f1798083300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f17bca00640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1798078be0 0x7f179807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.425+0000 7f0432800640 1 -- start start 2024-09-21T07:46:14.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7febe5a00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7febc0080f00 0x7febc0083350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5aea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5a80cfd80 0x7fd5a81f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5aea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a81f4a10 0x7fd5a81f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5aea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5a81f5360 0x7fd5a8157490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5aea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a8070470 con 0x7fd5a81f5360 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5aea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5a80702f0 con 0x7fd5a81f4a10 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5aea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5a80705f0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.426+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5a80cfd80 0x7fd5a81f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7fd5a7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5a80cfd80 0x7fd5a81f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:54972/0 (socket says 172.21.15.88:54972) 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7fd5a7e00640 1 -- 172.21.15.88:0/2510962245 learned_addr learned my addr 172.21.15.88:0/2510962245 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7fd5aca00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5a81f5360 0x7fd5a8157490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7fd5a7400640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a81f4a10 0x7fd5a81f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f0432800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f042c0b3d60 0x7f042c159790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f0432800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042c0cd000 0x7f042c0c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f0432800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd410 0x7f042c0c6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f0432800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0b4380 con 0x7f042c0b3d60 2024-09-21T07:46:14.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== osd.4 v2:172.21.15.175:6810/1473903548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febd401bec0 con 0x7febc0080f00 2024-09-21T07:46:14.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f0432800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f042c0711c0 con 0x7f042c0cd410 2024-09-21T07:46:14.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f0432800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f042c071340 con 0x7f042c0cd000 2024-09-21T07:46:14.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7fd5a7e00640 1 -- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a81f4a10 msgr2=0x7fd5a81f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7fd5a7e00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5a81f4a10 0x7fd5a81f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7fd5a7e00640 1 -- 172.21.15.88:0/2510962245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5a81f5360 msgr2=0x7fd5a8157490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f17bde00640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f1798080eb0 0x7f1798083300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.427+0000 7f17bca00640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1798078be0 0x7f179807b0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17b00542e0 tx=0x7f17a4007eb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== osd.2 v2:172.21.15.88:6809/4047473449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f17ac02eec0 con 0x7f1798080eb0 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7f042b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f042c0b3d60 0x7f042c159790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7f042b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f042c0b3d60 0x7f042c159790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50068/0 (socket says 172.21.15.88:50068) 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7f042b400640 1 -- 172.21.15.88:0/1098063101 learned_addr learned my addr 172.21.15.88:0/1098063101 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.428+0000 7f042aa00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042c0cd000 0x7f042c0c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f042be00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd410 0x7f042c0c6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f042b400640 1 -- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042c0cd000 msgr2=0x7f042c0c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f042b400640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042c0cd000 0x7f042c0c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f042b400640 1 -- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd410 msgr2=0x7f042c0c6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f042b400640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f042c0cd410 0x7f042c0c6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7f042b400640 1 -- 172.21.15.88:0/1098063101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f042c15a1c0 con 0x7f042c0b3d60 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7fd5a7e00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5a81f5360 0x7fd5a8157490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7fd5a7e00640 1 -- 172.21.15.88:0/2510962245 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5a8155af0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.429+0000 7fd5aca00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5a81f5360 0x7fd5a8157490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:14.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7fd5a7e00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5a80cfd80 0x7fd5a81f42f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c760 tx=0x7fd59400cc20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7f042b400640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f042c0b3d60 0x7f042c159790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f042400bd60 tx=0x7f0424009ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7f1be2400640 1 Processor -- start 2024-09-21T07:46:14.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594016020 con 0x7fd5a80cfd80 2024-09-21T07:46:14.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5940053e0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd594006af0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5a8155dd0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0424019070 con 0x7f042c0b3d60 2024-09-21T07:46:14.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f042400cd30 con 0x7f042c0b3d60 2024-09-21T07:46:14.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04240146a0 con 0x7f042c0b3d60 2024-09-21T07:46:14.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.430+0000 7f1be2400640 1 -- start start 2024-09-21T07:46:14.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.431+0000 7f0432800640 1 -- 172.21.15.88:0/1098063101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f042c159cd0 con 0x7f042c0b3d60 2024-09-21T07:46:14.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.431+0000 7f0432800640 1 -- 172.21.15.88:0/1098063101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f042c15b430 con 0x7f042c0b3d60 2024-09-21T07:46:14.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.432+0000 7f0432800640 1 -- 172.21.15.88:0/1098063101 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f042c15bdd0 con 0x7f042c0b3d60 2024-09-21T07:46:14.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.432+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5a80c4bb0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.433+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd5a80b3d60 con 0x7fd5a80cfd80 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1be2400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bdc0711c0 0x7f1bdc1f1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1be2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bdc0b3d60 0x7f1bdc0c51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1be2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0cd000 0x7f1bdc15d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1bd3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bdc0b3d60 0x7f1bdc0c51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1be2400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0b42f0 con 0x7f1bdc0711c0 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1bd3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bdc0b3d60 0x7f1bdc0c51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:60822/0 (socket says 172.21.15.88:60822) 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.434+0000 7f1bdb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bdc0711c0 0x7f1bdc1f1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.435+0000 7f1bdbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0cd000 0x7f1bdc15d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.435+0000 7f1bd3e00640 1 -- 172.21.15.88:0/2731078299 learned_addr learned my addr 172.21.15.88:0/2731078299 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.435+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1bdc0b4170 con 0x7f1bdc0b3d60 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.435+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1bdc0cd410 con 0x7f1bdc0cd000 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.436+0000 7f1bdbe00640 1 -- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bdc0b3d60 msgr2=0x7f1bdc0c51a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.436+0000 7f1bdbe00640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bdc0b3d60 0x7f1bdc0c51a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.436+0000 7f1bdbe00640 1 -- 172.21.15.88:0/2731078299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bdc0711c0 msgr2=0x7f1bdc1f1fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.436+0000 7f1bd3e00640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1bdc0b3d60 0x7f1bdc0c51a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.436+0000 7f1bdbe00640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1bdc0711c0 0x7f1bdc1f1fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.437+0000 7f1bdbe00640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1bdc0c7540 con 0x7f1bdc0cd000 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.437+0000 7f1bdbe00640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0cd000 0x7f1bdc15d460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400ca70 tx=0x7f1bd400cf30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.436+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd594006c90 con 0x7fd5a80cfd80 2024-09-21T07:46:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7fd59fe00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd588078b90 0x7fd58807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4016020 con 0x7f1bdc0cd000 2024-09-21T07:46:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1bd400ac00 con 0x7f1bdc0cd000 2024-09-21T07:46:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4012420 con 0x7f1bdc0cd000 2024-09-21T07:46:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd59409b040 con 0x7fd5a80cfd80 2024-09-21T07:46:14.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1bdc0c7820 con 0x7f1bdc0cd000 2024-09-21T07:46:14.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f1bdc0c57e0 con 0x7f1bdc0cd000 2024-09-21T07:46:14.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.439+0000 7fd5a7400640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd588078b90 0x7fd58807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7fd59fe00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7fd588080de0 0x7fd588083230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 --> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd594010e50 con 0x7fd588080de0 2024-09-21T07:46:14.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7fd59409b4d0 con 0x7fd5a80cfd80 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7fd5aca00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7fd588080de0 0x7fd588083230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1bdc0c7290 con 0x7f1bdc0cd000 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.438+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f042401c070 con 0x7f042c0b3d60 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7f0423400640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f040c078b90 0x7f040c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f042409b2d0 con 0x7f042c0b3d60 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7f0423400640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f040c080de0 0x7f040c083230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 --> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0424012e50 con 0x7f040c080de0 2024-09-21T07:46:14.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.440+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f042409b680 con 0x7f042c0b3d60 2024-09-21T07:46:14.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.441+0000 7fd5a7400640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd588078b90 0x7fd58807b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8073080 tx=0x7fd598000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.441+0000 7f042aa00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f040c078b90 0x7f040c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.441+0000 7fd5aca00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7fd588080de0 0x7fd588083230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.441+0000 7f042be00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f040c080de0 0x7f040c083230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.442+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 --> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb254002dd0 con 0x7fb258080e60 2024-09-21T07:46:14.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.442+0000 7f042aa00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f040c078b90 0x7f040c07b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f041c036000 tx=0x7f041c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.442+0000 7f042be00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f040c080de0 0x7f040c083230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.443+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== osd.5 v2:172.21.15.175:6821/4267764075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd594010e50 con 0x7fd588080de0 2024-09-21T07:46:14.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.443+0000 7fb26b400640 1 -- 172.21.15.88:0/2399947017 <== osd.0 v2:172.21.15.88:6808/2605666815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fb254002dd0 con 0x7fb258080e60 2024-09-21T07:46:14.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.443+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1bd4019020 con 0x7f1bdc0cd000 2024-09-21T07:46:14.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.444+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== osd.6 v2:172.21.15.175:6818/443779952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0424012e50 con 0x7f040c080de0 2024-09-21T07:46:14.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.444+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7fb258080e60 msgr2=0x7fb2580832b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.444+0000 7fb27e200640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7fb258080e60 0x7fb2580832b0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.445+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb258078b90 msgr2=0x7fb25807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.445+0000 7fb27e200640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb258078b90 0x7fb25807b0a0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fb264009b70 tx=0x7fb264005f30 comp rx=0 tx=0).stop 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.445+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270052360 msgr2=0x7fb2700322e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.445+0000 7fb27e200640 1 --2- 172.21.15.88:0/2399947017 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb270052360 0x7fb2700322e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb26c00c3f0 tx=0x7fb26c00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.446+0000 7f1bd3400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bb8078be0 0x7f1bb807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.446+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1bd409aec0 con 0x7f1bdc0cd000 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.447+0000 7f1bd3400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f1bb8080eb0 0x7f1bb8083300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.447+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1bd4015e70 con 0x7f1bb8080eb0 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.447+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) ==== 24+0+0 (secure 0 0 0) 0x7f1bd409b270 con 0x7f1bdc0cd000 2024-09-21T07:46:14.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.447+0000 7f1bdb400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bb8078be0 0x7f1bb807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.448+0000 7fb277e00640 1 -- 172.21.15.88:0/2399947017 reap_dead start 2024-09-21T07:46:14.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.448+0000 7f1bdb400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bb8078be0 0x7f1bb807b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0098e0 tx=0x7f1bcc009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.448+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 shutdown_connections 2024-09-21T07:46:14.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.448+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 >> 172.21.15.88:0/2399947017 conn(0x7fb27001d9a0 msgr2=0x7fb270052ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.448+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 shutdown_connections 2024-09-21T07:46:14.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.448+0000 7fb27e200640 1 -- 172.21.15.88:0/2399947017 wait complete. 2024-09-21T07:46:14.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.449+0000 7f1bd3e00640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f1bb8080eb0 0x7f1bb8083300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.450+0000 7f1bd3e00640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f1bb8080eb0 0x7f1bb8083300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.451+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 --> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff03c0394f0 con 0x7ff024080f80 2024-09-21T07:46:14.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.451+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== osd.7 v2:172.21.15.175:6802/1199047837 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1bd4015e70 con 0x7f1bb8080eb0 2024-09-21T07:46:14.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.452+0000 7ff037400640 1 -- 172.21.15.88:0/1997921899 <== osd.1 v2:172.21.15.88:6800/3750368380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7ff03c0394f0 con 0x7ff024080f80 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.452+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff024080f80 msgr2=0x7ff0240833d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.452+0000 7ff048a00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff024080f80 0x7ff0240833d0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff024078d90 msgr2=0x7ff02407b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff048a00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff024078d90 0x7ff02407b2a0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff03c006880 tx=0x7ff04406cc60 comp rx=0 tx=0).stop 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c056da0 msgr2=0x7ff03c0515c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff048a00640 1 --2- 172.21.15.88:0/1997921899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff03c056da0 0x7ff03c0515c0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7ff03800c370 tx=0x7ff03800c830 comp rx=0 tx=0).stop 2024-09-21T07:46:14.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 --> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f3f94007830 con 0x7f3f7c080e00 2024-09-21T07:46:14.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff043e00640 1 -- 172.21.15.88:0/1997921899 reap_dead start 2024-09-21T07:46:14.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 shutdown_connections 2024-09-21T07:46:14.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.454+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 >> 172.21.15.88:0/1997921899 conn(0x7ff03c03a5f0 msgr2=0x7ff03c0e56a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.455+0000 7f3f8fe00640 1 -- 172.21.15.88:0/1102221296 <== osd.3 v2:172.21.15.88:6824/1227287750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f3f94007830 con 0x7f3f7c080e00 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.455+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7f3f7c080e00 msgr2=0x7f3f7c083250 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 shutdown_connections 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7ff048a00640 1 -- 172.21.15.88:0/1997921899 wait complete. 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.455+0000 7f3fa1400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7f3f7c080e00 0x7f3f7c083250 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3f7c078b90 msgr2=0x7f3f7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3f7c078b90 0x7f3f7c07b0a0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f3f88033000 tx=0x7f3f88004730 comp rx=0 tx=0).stop 2024-09-21T07:46:14.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f94054360 msgr2=0x7f3f940d8000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 --2- 172.21.15.88:0/1102221296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f94054360 0x7f3f940d8000 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3f9c06bab0 tx=0x7f3f9c06a030 comp rx=0 tx=0).stop 2024-09-21T07:46:14.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.455+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 --> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1794002dd0 con 0x7f1798080eb0 2024-09-21T07:46:14.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa0a00640 1 -- 172.21.15.88:0/1102221296 reap_dead start 2024-09-21T07:46:14.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 shutdown_connections 2024-09-21T07:46:14.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 >> 172.21.15.88:0/1102221296 conn(0x7f3f9403a5f0 msgr2=0x7f3f9403c8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f17b5600640 1 -- 172.21.15.88:0/1746900550 <== osd.2 v2:172.21.15.88:6809/4047473449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1794002dd0 con 0x7f1798080eb0 2024-09-21T07:46:14.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.456+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 shutdown_connections 2024-09-21T07:46:14.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.457+0000 7f3fa1400640 1 -- 172.21.15.88:0/1102221296 wait complete. 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.457+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f1798080eb0 msgr2=0x7f1798083300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.457+0000 7f17be800640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f1798080eb0 0x7f1798083300 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.458+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1798078be0 msgr2=0x7f179807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1798078be0 0x7f179807b0f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f17b00542e0 tx=0x7f17a4007eb0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17b0051280 msgr2=0x7f17b017ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 --2- 172.21.15.88:0/1746900550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f17b0051280 0x7f17b017ab30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f17ac009890 tx=0x7f17ac003b90 comp rx=0 tx=0).stop 2024-09-21T07:46:14.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 --> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febe00b41a0 con 0x7febc0080f00 2024-09-21T07:46:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17bde00640 1 -- 172.21.15.88:0/1746900550 reap_dead start 2024-09-21T07:46:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 shutdown_connections 2024-09-21T07:46:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 >> 172.21.15.88:0/1746900550 conn(0x7f17b0056990 msgr2=0x7f17b00397c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 shutdown_connections 2024-09-21T07:46:14.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7f17be800640 1 -- 172.21.15.88:0/1746900550 wait complete. 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.459+0000 7febdcc00640 1 -- 172.21.15.88:0/2572544710 <== osd.4 v2:172.21.15.175:6810/1473903548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7febe00b41a0 con 0x7febc0080f00 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.460+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7febc0080f00 msgr2=0x7febc0083350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.460+0000 7febe7a00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7febc0080f00 0x7febc0083350 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7febc0078c90 msgr2=0x7febc007b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe7a00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7febc0078c90 0x7febc007b1a0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7febe006e5e0 tx=0x7febcc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00b9d60 msgr2=0x7febe00cbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe7a00640 1 --2- 172.21.15.88:0/2572544710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7febe00b9d60 0x7febe00cbd00 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7febd400e7c0 tx=0x7febd400ec80 comp rx=0 tx=0).stop 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe5a00640 1 -- 172.21.15.88:0/2572544710 reap_dead start 2024-09-21T07:46:14.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 shutdown_connections 2024-09-21T07:46:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.461+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 >> 172.21.15.88:0/2572544710 conn(0x7febe008bfe0 msgr2=0x7febe00b5460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.462+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 shutdown_connections 2024-09-21T07:46:14.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.462+0000 7febe7a00640 1 -- 172.21.15.88:0/2572544710 wait complete. 2024-09-21T07:46:14.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.474+0000 7f0432800640 1 -- 172.21.15.88:0/1098063101 --> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f042c0af510 con 0x7f040c080de0 2024-09-21T07:46:14.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.475+0000 7f0423400640 1 -- 172.21.15.88:0/1098063101 <== osd.6 v2:172.21.15.175:6818/443779952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f042c0af510 con 0x7f040c080de0 2024-09-21T07:46:14.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 --> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fd5a80743f0 con 0x7fd588080de0 2024-09-21T07:46:14.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f040c080de0 msgr2=0x7f040c083230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7f0420c00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f040c080de0 0x7f040c083230 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f040c078b90 msgr2=0x7f040c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7f0420c00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f040c078b90 0x7f040c07b0a0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f041c036000 tx=0x7f041c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:14.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f042c0b3d60 msgr2=0x7f042c159790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7f0420c00640 1 --2- 172.21.15.88:0/1098063101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f042c0b3d60 0x7f042c159790 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f042400bd60 tx=0x7f0424009ef0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.476+0000 7fd59fe00640 1 -- 172.21.15.88:0/2510962245 <== osd.5 v2:172.21.15.175:6821/4267764075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fd5a80743f0 con 0x7fd588080de0 2024-09-21T07:46:14.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.477+0000 7f042be00640 1 -- 172.21.15.88:0/1098063101 reap_dead start 2024-09-21T07:46:14.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.477+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 shutdown_connections 2024-09-21T07:46:14.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.477+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 >> 172.21.15.88:0/1098063101 conn(0x7f042c06d3d0 msgr2=0x7f042c0b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.477+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 shutdown_connections 2024-09-21T07:46:14.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7f0420c00640 1 -- 172.21.15.88:0/1098063101 wait complete. 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7fd588080de0 msgr2=0x7fd588083230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7fd588080de0 0x7fd588083230 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd588078b90 msgr2=0x7fd58807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd588078b90 0x7fd58807b0a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8073080 tx=0x7fd598000ed0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5a80cfd80 msgr2=0x7fd5a81f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 --2- 172.21.15.88:0/2510962245 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5a80cfd80 0x7fd5a81f42f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fd59400c760 tx=0x7fd59400cc20 comp rx=0 tx=0).stop 2024-09-21T07:46:14.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aca00640 1 -- 172.21.15.88:0/2510962245 reap_dead start 2024-09-21T07:46:14.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 shutdown_connections 2024-09-21T07:46:14.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.478+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 >> 172.21.15.88:0/2510962245 conn(0x7fd5a806d3d0 msgr2=0x7fd5a8158770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.479+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 shutdown_connections 2024-09-21T07:46:14.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.479+0000 7fd5aea00640 1 -- 172.21.15.88:0/2510962245 wait complete. 2024-09-21T07:46:14.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.482+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 --> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1bdc0af510 con 0x7f1bb8080eb0 2024-09-21T07:46:14.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.483+0000 7f1bd3400640 1 -- 172.21.15.88:0/2731078299 <== osd.7 v2:172.21.15.175:6802/1199047837 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f1bdc0af510 con 0x7f1bb8080eb0 2024-09-21T07:46:14.486 INFO:teuthology.orchestra.run.smithi088.stdout:68719476742 2024-09-21T07:46:14.486 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T07:46:14.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.484+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f1bb8080eb0 msgr2=0x7f1bb8083300 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.484+0000 7f1be2400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f1bb8080eb0 0x7f1bb8083300 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bb8078be0 msgr2=0x7f1bb807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1bb8078be0 0x7f1bb807b0f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0098e0 tx=0x7f1bcc009060 comp rx=0 tx=0).stop 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0cd000 msgr2=0x7f1bdc15d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 --2- 172.21.15.88:0/2731078299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1bdc0cd000 0x7f1bdc15d460 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400ca70 tx=0x7f1bd400cf30 comp rx=0 tx=0).stop 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1bdbe00640 1 -- 172.21.15.88:0/2731078299 reap_dead start 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 shutdown_connections 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 >> 172.21.15.88:0/2731078299 conn(0x7f1bdc06d3d0 msgr2=0x7f1bdc06fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 shutdown_connections 2024-09-21T07:46:14.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.485+0000 7f1be2400640 1 -- 172.21.15.88:0/2731078299 wait complete. 2024-09-21T07:46:14.497 INFO:teuthology.orchestra.run.smithi088.stdout:68719476745 2024-09-21T07:46:14.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T07:46:14.500 INFO:teuthology.orchestra.run.smithi088.stdout:68719476742 2024-09-21T07:46:14.501 INFO:teuthology.orchestra.run.smithi088.stdout:68719476745 2024-09-21T07:46:14.501 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T07:46:14.502 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T07:46:14.508 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T07:46:14.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T07:46:14.515 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T07:46:14.516 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T07:46:14.516 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T07:46:14.516 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T07:46:14.523 INFO:teuthology.orchestra.run.smithi088.stdout:60129542151 2024-09-21T07:46:14.523 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T07:46:14.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.873+0000 7fca07200640 1 -- 172.21.15.88:0/2650316934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00054190 msgr2=0x7fca00007550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.873+0000 7fca07200640 1 --2- 172.21.15.88:0/2650316934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00054190 0x7fca00007550 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4009830 tx=0x7fc9f402a3f0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.873+0000 7fca07200640 1 -- 172.21.15.88:0/2650316934 shutdown_connections 2024-09-21T07:46:14.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.874+0000 7fca07200640 1 --2- 172.21.15.88:0/2650316934 >> v2:172.21.15.175:3300/0 conn(0x7fca00053860 0x7fca0003c450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.874+0000 7fca07200640 1 --2- 172.21.15.88:0/2650316934 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00054190 0x7fca00007550 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.874+0000 7fca07200640 1 -- 172.21.15.88:0/2650316934 >> 172.21.15.88:0/2650316934 conn(0x7fca0003a680 msgr2=0x7fca0003aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.875+0000 7fca07200640 1 -- 172.21.15.88:0/2650316934 shutdown_connections 2024-09-21T07:46:14.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.875+0000 7fca07200640 1 -- 172.21.15.88:0/2650316934 wait complete. 2024-09-21T07:46:14.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa156a00640 1 -- 172.21.15.88:0/2969364759 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150030c80 con 0x7fa15806ef10 2024-09-21T07:46:14.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa15e000640 1 -- 172.21.15.88:0/2969364759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 msgr2=0x7fa15806f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa15e000640 1 --2- 172.21.15.88:0/2969364759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa15806f320 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa150005fd0 tx=0x7fa15002a930 comp rx=0 tx=0).stop 2024-09-21T07:46:14.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa15e000640 1 -- 172.21.15.88:0/2969364759 shutdown_connections 2024-09-21T07:46:14.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa15e000640 1 --2- 172.21.15.88:0/2969364759 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa15806f320 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa15e000640 1 --2- 172.21.15.88:0/2969364759 >> v2:172.21.15.175:3300/0 conn(0x7fa1580b41a0 0x7fa1580b45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.876+0000 7fa15e000640 1 -- 172.21.15.88:0/2969364759 >> 172.21.15.88:0/2969364759 conn(0x7fa15806d5b0 msgr2=0x7fa1580b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.877+0000 7fa15e000640 1 -- 172.21.15.88:0/2969364759 shutdown_connections 2024-09-21T07:46:14.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.879+0000 7fca07200640 1 Processor -- start 2024-09-21T07:46:14.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.879+0000 7fca07200640 1 -- start start 2024-09-21T07:46:14.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.880+0000 7ffa90a00640 1 -- 172.21.15.88:0/840985163 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c036070 con 0x7ffa940b7f40 2024-09-21T07:46:14.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.877+0000 7fa15e000640 1 -- 172.21.15.88:0/2969364759 wait complete. 2024-09-21T07:46:14.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.881+0000 7ffa93200640 1 -- 172.21.15.88:0/840985163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b7f40 msgr2=0x7ffa940b5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.881+0000 7ffa93200640 1 --2- 172.21.15.88:0/840985163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b7f40 0x7ffa940b5f70 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c005fd0 tx=0x7ffa8c02a7e0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.881+0000 7fa15e000640 1 Processor -- start 2024-09-21T07:46:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7fa15e000640 1 -- start start 2024-09-21T07:46:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7ffa93200640 1 -- 172.21.15.88:0/840985163 shutdown_connections 2024-09-21T07:46:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7ffa93200640 1 --2- 172.21.15.88:0/840985163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa940b7f40 0x7ffa940b5f70 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7ffa93200640 1 --2- 172.21.15.88:0/840985163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa9415e770 0x7ffa940b7a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7ffa93200640 1 -- 172.21.15.88:0/840985163 >> 172.21.15.88:0/840985163 conn(0x7ffa940d44f0 msgr2=0x7ffa940d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7ffa93200640 1 -- 172.21.15.88:0/840985163 shutdown_connections 2024-09-21T07:46:14.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.882+0000 7ffa93200640 1 -- 172.21.15.88:0/840985163 wait complete. 2024-09-21T07:46:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.883+0000 7fca07200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00053860 0x7fca000e3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.883+0000 7fca07200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 0x7fca001824d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.884+0000 7fca07200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca000e3200 0x7fca0004e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.884+0000 7fca07200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca0003cb10 con 0x7fca00054190 2024-09-21T07:46:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.884+0000 7fca07200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca0003c990 con 0x7fca000e3200 2024-09-21T07:46:14.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.884+0000 7fca07200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca0003cc90 con 0x7fca00053860 2024-09-21T07:46:14.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.885+0000 7fca05400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 0x7fca001824d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.885+0000 7fca05400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 0x7fca001824d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50102/0 (socket says 172.21.15.88:50102) 2024-09-21T07:46:14.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.885+0000 7fca05400640 1 -- 172.21.15.88:0/3842234878 learned_addr learned my addr 172.21.15.88:0/3842234878 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.885+0000 7fa15e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa1580995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fa15e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1580b41a0 0x7fa1581541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.885+0000 7f1048600640 1 -- 172.21.15.88:0/1348469576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400711c0 msgr2=0x7f10400701e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.885+0000 7f1048600640 1 --2- 172.21.15.88:0/1348469576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400711c0 0x7f10400701e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10400cdd60 tx=0x7f103402aa30 comp rx=0 tx=0).stop 2024-09-21T07:46:14.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7f1048600640 1 -- 172.21.15.88:0/1348469576 shutdown_connections 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7f1048600640 1 --2- 172.21.15.88:0/1348469576 >> v2:172.21.15.175:3300/0 conn(0x7f10400b3d60 0x7f10400af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7f1048600640 1 --2- 172.21.15.88:0/1348469576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400711c0 0x7f10400701e0 secure :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f10400cdd60 tx=0x7f103402aa30 comp rx=0 tx=0).stop 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7f1048600640 1 -- 172.21.15.88:0/1348469576 >> 172.21.15.88:0/1348469576 conn(0x7f104006d3d0 msgr2=0x7f10400b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fa157400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa1580995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fa157400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa1580995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:55054/0 (socket says 172.21.15.88:55054) 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fca05400640 1 -- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00053860 msgr2=0x7fca000e3840 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fca05400640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00053860 0x7fca000e3840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fca05400640 1 -- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca000e3200 msgr2=0x7fca0004e6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:46:14.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fca05400640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca000e3200 0x7fca0004e6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fca05400640 1 -- 172.21.15.88:0/3842234878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca00009ae0 con 0x7fca00054190 2024-09-21T07:46:14.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fa157400640 1 -- 172.21.15.88:0/1991923253 learned_addr learned my addr 172.21.15.88:0/1991923253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7f1048600640 1 -- 172.21.15.88:0/1348469576 shutdown_connections 2024-09-21T07:46:14.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.886+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1580b4e60 0x7fa158154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa15806f9e0 con 0x7fa1580b41a0 2024-09-21T07:46:14.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa15806f860 con 0x7fa1580b4e60 2024-09-21T07:46:14.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7ffa93200640 1 Processor -- start 2024-09-21T07:46:14.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa15806fb60 con 0x7fa15806ef10 2024-09-21T07:46:14.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7f1048600640 1 -- 172.21.15.88:0/1348469576 wait complete. 2024-09-21T07:46:14.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7ffa93200640 1 -- start start 2024-09-21T07:46:14.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa157e00640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1580b4e60 0x7fa158154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa157400640 1 -- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1580b4e60 msgr2=0x7fa158154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa14ea00640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1580b41a0 0x7fa1581541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa157400640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1580b4e60 0x7fa158154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.887+0000 7fa157400640 1 -- 172.21.15.88:0/1991923253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1580b41a0 msgr2=0x7fa1581541c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.888+0000 7fa157400640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1580b41a0 0x7fa1581541c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.888+0000 7fa157400640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1581624a0 con 0x7fa15806ef10 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.888+0000 7fa14ea00640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1580b41a0 0x7fa1581541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.888+0000 7fca05400640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 0x7fca001824d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800c3d0 tx=0x7fc9f800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.888+0000 7fa157400640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa1580995e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c550 tx=0x7fa14000ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.888+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8011790 con 0x7fca00054190 2024-09-21T07:46:14.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f8017070 con 0x7fca00054190 2024-09-21T07:46:14.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8011cc0 con 0x7fca00054190 2024-09-21T07:46:14.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fe6c3200640 1 -- 172.21.15.88:0/3187177133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 msgr2=0x7fe6bc005de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fe6c3200640 1 --2- 172.21.15.88:0/3187177133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 0x7fe6bc005de0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4009820 tx=0x7fe6b402a2b0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140017070 con 0x7fa15806ef10 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1400035f0 con 0x7fa15806ef10 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa140003a40 con 0x7fa15806ef10 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa158162780 con 0x7fa15806ef10 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1581579c0 con 0x7fa15806ef10 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.890+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca00009d80 con 0x7fca00054190 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.890+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca0000a340 con 0x7fca00054190 2024-09-21T07:46:14.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.890+0000 7ffa93200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 0x7ffa940700c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fe6c3200640 1 -- 172.21.15.88:0/3187177133 shutdown_connections 2024-09-21T07:46:14.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fe6c3200640 1 --2- 172.21.15.88:0/3187177133 >> v2:172.21.15.175:3300/0 conn(0x7fe6bc039d70 0x7fe6bc054e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fe6c3200640 1 --2- 172.21.15.88:0/3187177133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 0x7fe6bc005de0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.889+0000 7fe6c3200640 1 -- 172.21.15.88:0/3187177133 >> 172.21.15.88:0/3187177133 conn(0x7fe6bc01d9a0 msgr2=0x7fe6bc039660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.890+0000 7ffa93200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa9415e770 0x7ffa94201ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7fe6c3200640 1 -- 172.21.15.88:0/3187177133 shutdown_connections 2024-09-21T07:46:14.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7f1048600640 1 Processor -- start 2024-09-21T07:46:14.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca000e3de0 con 0x7fca00054190 2024-09-21T07:46:14.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7f1048600640 1 -- start start 2024-09-21T07:46:14.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7fe6c3200640 1 -- 172.21.15.88:0/3187177133 wait complete. 2024-09-21T07:46:14.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7ffa91e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 0x7ffa940700c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7ffa93200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa94202000 0x7ffa940cd9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7ffa93200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa940d3a30 con 0x7ffa9415e770 2024-09-21T07:46:14.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7ffa93200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa940b6920 con 0x7ffa94202000 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.891+0000 7ffa93200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa940d3bb0 con 0x7ffa940b7f40 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.892+0000 7ffa91e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 0x7ffa940700c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:55066/0 (socket says 172.21.15.88:55066) 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.892+0000 7ffa91e00640 1 -- 172.21.15.88:0/1108586043 learned_addr learned my addr 172.21.15.88:0/1108586043 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.892+0000 7ffa92800640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa94202000 0x7ffa940cd9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7ffa92800640 1 -- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 msgr2=0x7ffa940700c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7ffa92800640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 0x7ffa940700c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7ffa92800640 1 -- 172.21.15.88:0/1108586043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa9415e770 msgr2=0x7ffa94201ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7ffa92800640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa9415e770 0x7ffa94201ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7ffa92800640 1 -- 172.21.15.88:0/1108586043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa94070910 con 0x7ffa94202000 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.893+0000 7ffa91e00640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 0x7ffa940700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:14.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.894+0000 7ffa92800640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa94202000 0x7ffa940cd9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009000 tx=0x7ffa8c032c60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.894+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa14001a070 con 0x7fa15806ef10 2024-09-21T07:46:14.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.894+0000 7f1048600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 0x7f10401f10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.894+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa15806f320 con 0x7fa15806ef10 2024-09-21T07:46:14.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7f1048600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400cd000 0x7f104015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c041020 con 0x7ffa94202000 2024-09-21T07:46:14.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa8c032e50 con 0x7ffa94202000 2024-09-21T07:46:14.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7f1048600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 0x7f104015d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7f1045c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 0x7f10401f10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7f1045c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 0x7f10401f10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50128/0 (socket says 172.21.15.88:50128) 2024-09-21T07:46:14.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa8c036040 con 0x7ffa94202000 2024-09-21T07:46:14.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.895+0000 7fe6c3200640 1 Processor -- start 2024-09-21T07:46:14.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1048600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10400b42f0 con 0x7f10400b3d60 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1048600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10400b4170 con 0x7f10400cd000 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1046600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 0x7f104015d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1046600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 0x7f104015d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:55078/0 (socket says 172.21.15.88:55078) 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1046600640 1 -- 172.21.15.88:0/379812300 learned_addr learned my addr 172.21.15.88:0/379812300 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1045200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400cd000 0x7f104015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7fe6c3200640 1 -- start start 2024-09-21T07:46:14.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10400cd410 con 0x7f1040158460 2024-09-21T07:46:14.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7fa14fe00640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa134078b90 0x7fa13407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fa14009a350 con 0x7fa15806ef10 2024-09-21T07:46:14.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.897+0000 7fa14ea00640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa134078b90 0x7fa13407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.897+0000 7ffa93200640 1 -- 172.21.15.88:0/1108586043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa94070bf0 con 0x7ffa94202000 2024-09-21T07:46:14.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.897+0000 7ffa93200640 1 -- 172.21.15.88:0/1108586043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa94073390 con 0x7ffa94202000 2024-09-21T07:46:14.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.896+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9f801c460 con 0x7fca00054190 2024-09-21T07:46:14.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.897+0000 7fc9fe000640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e8078be0 0x7fc9e807b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc9f809aa30 con 0x7fca00054190 2024-09-21T07:46:14.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.897+0000 7f1045c00640 1 -- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 msgr2=0x7f104015d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7fa14ea00640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa134078b90 0x7fa13407b0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1580ad8b0 tx=0x7fa148005d10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7f1045c00640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 0x7f104015d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7f1045c00640 1 -- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400cd000 msgr2=0x7f104015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7f1045c00640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400cd000 0x7f104015d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7ffa93200640 1 -- 172.21.15.88:0/1108586043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa94070680 con 0x7ffa94202000 2024-09-21T07:46:14.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fca05e00640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e8078be0 0x7fc9e807b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.898+0000 7f1045c00640 1 -- 172.21.15.88:0/379812300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10400c6d20 con 0x7f10400b3d60 2024-09-21T07:46:14.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7f1046600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 0x7f104015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:14.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7f1045200640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400cd000 0x7f104015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:14.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fe6c3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6bc006320 0x7fe6bc103aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fe6c3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 0x7fe6bc04a960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fe6c3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 0x7fe6bc04cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fe6c3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc055370 con 0x7fe6bc006320 2024-09-21T07:46:14.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7f1045c00640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 0x7f10401f10e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10400cf360 tx=0x7f1034003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fe6c3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6bc0554f0 con 0x7fe6bc039d70 2024-09-21T07:46:14.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.899+0000 7fe6c3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6bc0341f0 con 0x7fe6bc0067a0 2024-09-21T07:46:14.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 0x7fe6bc04cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 0x7fe6bc04cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:60894/0 (socket says 172.21.15.88:60894) 2024-09-21T07:46:14.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 -- 172.21.15.88:0/2542341495 learned_addr learned my addr 172.21.15.88:0/2542341495 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c1400640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 0x7fe6bc04a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 -- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 msgr2=0x7fe6bc04a960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 0x7fe6bc04a960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 -- 172.21.15.88:0/2542341495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6bc006320 msgr2=0x7fe6bc103aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6bc006320 0x7fe6bc103aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fca05e00640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e8078be0 0x7fc9e807b0f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4034000 tx=0x7fc9f4004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.900+0000 7fe6c2800640 1 -- 172.21.15.88:0/2542341495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6bc1042f0 con 0x7fe6bc039d70 2024-09-21T07:46:14.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.901+0000 7fe6c2800640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 0x7fe6bc04cdd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4051230 tx=0x7fe6c4075480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.902+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1400626f0 con 0x7fa15806ef10 2024-09-21T07:46:14.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.902+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa8c044020 con 0x7ffa94202000 2024-09-21T07:46:14.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.904+0000 7ffa8a000640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa70078c10 0x7ffa7007b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.904+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7ffa8c0bbec0 con 0x7ffa94202000 2024-09-21T07:46:14.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.904+0000 7ffa91e00640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa70078c10 0x7ffa7007b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4087070 con 0x7fe6bc039d70 2024-09-21T07:46:14.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c40737d0 con 0x7fe6bc039d70 2024-09-21T07:46:14.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4073c00 con 0x7fe6bc039d70 2024-09-21T07:46:14.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7ffa91e00640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa70078c10 0x7ffa7007b120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0016c0 tx=0x7ffa7c001d10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034046070 con 0x7f10400b3d60 2024-09-21T07:46:14.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.906+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa8c04b070 con 0x7ffa94202000 2024-09-21T07:46:14.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.906+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10340042d0 con 0x7f10400b3d60 2024-09-21T07:46:14.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.906+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10401f2350 con 0x7f10400b3d60 2024-09-21T07:46:14.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.906+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10401f1620 con 0x7f10400b3d60 2024-09-21T07:46:14.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6bc19dc20 con 0x7fe6bc039d70 2024-09-21T07:46:14.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.905+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6bc19e1e0 con 0x7fe6bc039d70 2024-09-21T07:46:14.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.906+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc9f8062f50 con 0x7fca00054190 2024-09-21T07:46:14.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.908+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10400cbd10 con 0x7f10400b3d60 2024-09-21T07:46:14.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.908+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe69c005290 con 0x7fe6bc039d70 2024-09-21T07:46:14.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.908+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1034033050 con 0x7f10400b3d60 2024-09-21T07:46:14.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.909+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe6c408a070 con 0x7fe6bc039d70 2024-09-21T07:46:14.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.911+0000 7fe6ba000640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6a4078c60 0x7fe6a407b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.912+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe6c4105450 con 0x7fe6bc039d70 2024-09-21T07:46:14.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.912+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1034036070 con 0x7f10400b3d60 2024-09-21T07:46:14.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.913+0000 7fe6c1e00640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6a4078c60 0x7fe6a407b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.914+0000 7fe6c1e00640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6a4078c60 0x7fe6a407b170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4034000 tx=0x7fe6b40046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.914+0000 7f103d600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1020078c60 0x7f102007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.914+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f10340bbef0 con 0x7f10400b3d60 2024-09-21T07:46:14.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.916+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6c40cd2b0 con 0x7fe6bc039d70 2024-09-21T07:46:14.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.920+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10400cbd10 con 0x7f10400b3d60 2024-09-21T07:46:14.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.921+0000 7f1045200640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1020078c60 0x7f102007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.925+0000 7f1045200640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1020078c60 0x7f102007b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f102c00a870 tx=0x7f102c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.950+0000 7fc1d5a00640 1 -- 172.21.15.88:0/3584831904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d0162ab0 msgr2=0x7fc1d0072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.950+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/3584831904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d0162ab0 0x7fc1d0072e60 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc005fd0 tx=0x7fc1bc02a540 comp rx=0 tx=0).stop 2024-09-21T07:46:14.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.950+0000 7fc1d5a00640 1 -- 172.21.15.88:0/3584831904 shutdown_connections 2024-09-21T07:46:14.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.950+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/3584831904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d0162ab0 0x7fc1d0072e60 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.950+0000 7fc1d5a00640 1 -- 172.21.15.88:0/3584831904 >> 172.21.15.88:0/3584831904 conn(0x7fc1d00b1c60 msgr2=0x7fc1d00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.951+0000 7fc1d5a00640 1 -- 172.21.15.88:0/3584831904 shutdown_connections 2024-09-21T07:46:14.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.951+0000 7fc1d5a00640 1 -- 172.21.15.88:0/3584831904 wait complete. 2024-09-21T07:46:14.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.954+0000 7fc1d5a00640 1 Processor -- start 2024-09-21T07:46:14.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.955+0000 7fc1d5a00640 1 -- start start 2024-09-21T07:46:14.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.956+0000 7f9f55000640 1 -- 172.21.15.88:0/2365342775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48053c00 msgr2=0x7f9f48036340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.956+0000 7f9f55000640 1 --2- 172.21.15.88:0/2365342775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48053c00 0x7f9f48036340 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c0098e0 tx=0x7f9f3c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.956+0000 7f9f55000640 1 -- 172.21.15.88:0/2365342775 shutdown_connections 2024-09-21T07:46:14.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.957+0000 7f9f55000640 1 --2- 172.21.15.88:0/2365342775 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48053c00 0x7f9f48036340 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.957+0000 7f9f55000640 1 -- 172.21.15.88:0/2365342775 >> 172.21.15.88:0/2365342775 conn(0x7f9f48051420 msgr2=0x7f9f48051860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:14.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.957+0000 7f9f55000640 1 -- 172.21.15.88:0/2365342775 shutdown_connections 2024-09-21T07:46:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.958+0000 7fc1d5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 0x7fc1d0152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.957+0000 7f9f55000640 1 -- 172.21.15.88:0/2365342775 wait complete. 2024-09-21T07:46:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.958+0000 7fc1d5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d006f210 0x7fc1d009d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.959+0000 7fc1d5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1d0160d60 0x7fc1d0161170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.959+0000 7fc1d5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0154350 con 0x7fc1d006ee00 2024-09-21T07:46:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.959+0000 7fc1d5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1d0160d60 0x7fc1d0161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.959+0000 7fc1cfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 0x7fc1d0152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.960+0000 7fc1cfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 0x7fc1d0152f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50174/0 (socket says 172.21.15.88:50174) 2024-09-21T07:46:14.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.960+0000 7fc1cfe00640 1 -- 172.21.15.88:0/1079914974 learned_addr learned my addr 172.21.15.88:0/1079914974 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.960+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1d0073eb0 con 0x7fc1d006f210 2024-09-21T07:46:14.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.960+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc1d01544d0 con 0x7fc1d0160d60 2024-09-21T07:46:14.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.961+0000 7fc1cfe00640 1 -- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1d0160d60 msgr2=0x7fc1d0161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.961+0000 7fc1cfe00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1d0160d60 0x7fc1d0161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.961+0000 7fc1cfe00640 1 -- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d006f210 msgr2=0x7fc1d009d7d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T07:46:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.962+0000 7f9f55000640 1 Processor -- start 2024-09-21T07:46:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.962+0000 7fc1cfe00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d006f210 0x7fc1d009d7d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.962+0000 7fc1cfe00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1d01fd090 con 0x7fc1d006ee00 2024-09-21T07:46:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.962+0000 7fc1d5000640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1d0160d60 0x7fc1d0161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.962+0000 7f9f55000640 1 -- start start 2024-09-21T07:46:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.962+0000 7fc1cfe00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 0x7fc1d0152f60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000b420 tx=0x7fc1c000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.963+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011830 con 0x7fc1d006ee00 2024-09-21T07:46:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.963+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc1c0016070 con 0x7fc1d006ee00 2024-09-21T07:46:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.963+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1d01616b0 con 0x7fc1d006ee00 2024-09-21T07:46:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.963+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1c0011d60 con 0x7fc1d006ee00 2024-09-21T07:46:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.964+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1d01fd5e0 con 0x7fc1d006ee00 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7f9f55000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f48052360 0x7f9f480e1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1d01623a0 con 0x7fc1d006ee00 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7f9f55000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48052770 0x7f9f480d84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7f9f55000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 0x7f9f480d89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7f9f55000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9f48006770 con 0x7f9f48052770 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7f9f55000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f480065f0 con 0x7f9f48052360 2024-09-21T07:46:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.967+0000 7f9f55000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9f4800b560 con 0x7f9f48053c00 2024-09-21T07:46:14.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 0x7f9f480d89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 0x7f9f480d89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:55112/0 (socket says 172.21.15.88:55112) 2024-09-21T07:46:14.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 -- 172.21.15.88:0/2778651515 learned_addr learned my addr 172.21.15.88:0/2778651515 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:14.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4f400640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f48052360 0x7f9f480e1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 -- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f48052360 msgr2=0x7f9f480e1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f48052360 0x7f9f480e1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 -- 172.21.15.88:0/2778651515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48052770 msgr2=0x7f9f480d84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:14.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48052770 0x7f9f480d84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:14.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.968+0000 7f9f4fe00640 1 -- 172.21.15.88:0/2778651515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f48046e80 con 0x7f9f48053c00 2024-09-21T07:46:14.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.970+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc1c001b460 con 0x7fc1d006ee00 2024-09-21T07:46:14.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.971+0000 7fc1c7e00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc1b0078b90 0x7fc1b007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.971+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fc1c009a880 con 0x7fc1d006ee00 2024-09-21T07:46:14.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.970+0000 7f9f4fe00640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 0x7f9f480d89e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f50051230 tx=0x7f9f5006a030 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.972+0000 7fc1cf400640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc1b0078b90 0x7fc1b007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.972+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50063a00 con 0x7f9f48053c00 2024-09-21T07:46:14.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.972+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f50063c90 con 0x7f9f48053c00 2024-09-21T07:46:14.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.972+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f50069630 con 0x7f9f48053c00 2024-09-21T07:46:14.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.974+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f480ed6b0 con 0x7f9f48053c00 2024-09-21T07:46:14.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.974+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f480edc70 con 0x7f9f48053c00 2024-09-21T07:46:14.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.976+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f48036340 con 0x7f9f48053c00 2024-09-21T07:46:14.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.976+0000 7fc1cf400640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc1b0078b90 0x7fc1b007b0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc033000 tx=0x7fc1bc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:14.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.978+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9f500697d0 con 0x7f9f48053c00 2024-09-21T07:46:14.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.980+0000 7f9f43400640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9f30078b90 0x7f9f3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:14.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.980+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f9f500da340 con 0x7f9f48053c00 2024-09-21T07:46:14.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.980+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc1c0020020 con 0x7fc1d006ee00 2024-09-21T07:46:14.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.981+0000 7f9f4f400640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9f30078b90 0x7f9f3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:14.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.983+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f500de7d0 con 0x7f9f48053c00 2024-09-21T07:46:14.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:14.983+0000 7f9f4f400640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9f30078b90 0x7f9f3007b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c034000 tx=0x7f9f3c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:15.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.012+0000 7f104d600640 1 -- 172.21.15.88:0/88668689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480b3220 msgr2=0x7f10480b3630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.012+0000 7f104d600640 1 --2- 172.21.15.88:0/88668689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480b3220 0x7f10480b3630 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f1040005fd0 tx=0x7f104002a8c0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.012+0000 7f104d600640 1 -- 172.21.15.88:0/88668689 shutdown_connections 2024-09-21T07:46:15.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.012+0000 7f104d600640 1 --2- 172.21.15.88:0/88668689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480b3220 0x7f10480b3630 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.012+0000 7f104d600640 1 -- 172.21.15.88:0/88668689 >> 172.21.15.88:0/88668689 conn(0x7f104808dd70 msgr2=0x7f104809d9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.013+0000 7f104d600640 1 -- 172.21.15.88:0/88668689 shutdown_connections 2024-09-21T07:46:15.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.013+0000 7f104d600640 1 -- 172.21.15.88:0/88668689 wait complete. 2024-09-21T07:46:15.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.017+0000 7f104d600640 1 Processor -- start 2024-09-21T07:46:15.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.018+0000 7f104d600640 1 -- start start 2024-09-21T07:46:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f104d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 0x7f10481fa880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f104d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10480b3220 0x7f1048158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f1047e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 0x7f10481fa880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f104d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480d0480 0x7f10480b1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f104d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1048162610 con 0x7f10480d0480 2024-09-21T07:46:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f104d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1048162490 con 0x7f104808c100 2024-09-21T07:46:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f104d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10480d3c40 con 0x7f10480b3220 2024-09-21T07:46:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f1047e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 0x7f10481fa880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:60918/0 (socket says 172.21.15.88:60918) 2024-09-21T07:46:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.021+0000 7f1047e00640 1 -- 172.21.15.88:0/3789026543 learned_addr learned my addr 172.21.15.88:0/3789026543 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:15.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.022+0000 7f1047e00640 1 -- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10480b3220 msgr2=0x7f1048158410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:46:15.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.022+0000 7f1047e00640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10480b3220 0x7f1048158410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.022+0000 7f1047e00640 1 -- 172.21.15.88:0/3789026543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480d0480 msgr2=0x7f10480b1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.022+0000 7f1047e00640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480d0480 0x7f10480b1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.023+0000 7f1047e00640 1 -- 172.21.15.88:0/3789026543 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104815e140 con 0x7f104808c100 2024-09-21T07:46:15.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.023+0000 7f1047e00640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 0x7f10481fa880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f103800c430 tx=0x7f103800c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:15.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.023+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038011790 con 0x7f104808c100 2024-09-21T07:46:15.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.023+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1038017070 con 0x7f104808c100 2024-09-21T07:46:15.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.024+0000 7f104d600640 1 -- 172.21.15.88:0/3789026543 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104815e420 con 0x7f104808c100 2024-09-21T07:46:15.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.024+0000 7f104d600640 1 -- 172.21.15.88:0/3789026543 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104815e9e0 con 0x7f104808c100 2024-09-21T07:46:15.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.024+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038011cc0 con 0x7f104808c100 2024-09-21T07:46:15.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.029+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f103801c460 con 0x7f104808c100 2024-09-21T07:46:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.029+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c005290 con 0x7f104808c100 2024-09-21T07:46:15.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.031+0000 7f103fe00640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1028078bc0 0x7f102807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:15.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.031+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f103809abc0 con 0x7f104808c100 2024-09-21T07:46:15.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.033+0000 7f1047400640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1028078bc0 0x7f102807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:15.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.034+0000 7f1047400640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1028078bc0 0x7f102807b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f104806e900 tx=0x7f10340079b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:15.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.036+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1038062f60 con 0x7f104808c100 2024-09-21T07:46:15.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.330+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fa158163370 con 0x7fa15806ef10 2024-09-21T07:46:15.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.331+0000 7fa14fe00640 1 -- 172.21.15.88:0/1991923253 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fa140001210 con 0x7fa15806ef10 2024-09-21T07:46:15.333 INFO:teuthology.orchestra.run.smithi088.stdout:68719476744 2024-09-21T07:46:15.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.335+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa134078b90 msgr2=0x7fa13407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa134078b90 0x7fa13407b0a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fa1580ad8b0 tx=0x7fa148005d10 comp rx=0 tx=0).stop 2024-09-21T07:46:15.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 msgr2=0x7fa1580995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa1580995e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa14000c550 tx=0x7fa14000ca10 comp rx=0 tx=0).stop 2024-09-21T07:46:15.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 shutdown_connections 2024-09-21T07:46:15.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa134078b90 0x7fa13407b0a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1580b4e60 0x7fa158154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1580b41a0 0x7fa1581541c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 --2- 172.21.15.88:0/1991923253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa15806ef10 0x7fa1580995e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.336+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 >> 172.21.15.88:0/1991923253 conn(0x7fa15806d5b0 msgr2=0x7fa1580b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.337+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 shutdown_connections 2024-09-21T07:46:15.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.337+0000 7fa15e000640 1 -- 172.21.15.88:0/1991923253 wait complete. 2024-09-21T07:46:15.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.338+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fca0003d1b0 con 0x7fca00054190 2024-09-21T07:46:15.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.339+0000 7fc9fe000640 1 -- 172.21.15.88:0/3842234878 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9f8062850 con 0x7fca00054190 2024-09-21T07:46:15.341 INFO:teuthology.orchestra.run.smithi088.stdout:68719476741 2024-09-21T07:46:15.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.343+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e8078be0 msgr2=0x7fc9e807b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.343+0000 7fca07200640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e8078be0 0x7fc9e807b0f0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4034000 tx=0x7fc9f4004730 comp rx=0 tx=0).stop 2024-09-21T07:46:15.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.343+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 msgr2=0x7fca001824d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.344+0000 7fca07200640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 0x7fca001824d0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fc9f800c3d0 tx=0x7fc9f800c890 comp rx=0 tx=0).stop 2024-09-21T07:46:15.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.344+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 shutdown_connections 2024-09-21T07:46:15.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.344+0000 7fca07200640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e8078be0 0x7fc9e807b0f0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.344+0000 7fca07200640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca000e3200 0x7fca0004e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.345+0000 7fca07200640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca00054190 0x7fca001824d0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.345+0000 7fca07200640 1 --2- 172.21.15.88:0/3842234878 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca00053860 0x7fca000e3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.345+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 >> 172.21.15.88:0/3842234878 conn(0x7fca0003a680 msgr2=0x7fca00050ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.345+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 shutdown_connections 2024-09-21T07:46:15.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.345+0000 7fca07200640 1 -- 172.21.15.88:0/3842234878 wait complete. 2024-09-21T07:46:15.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.348+0000 7ffa93200640 1 -- 172.21.15.88:0/1108586043 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7ffa940d07a0 con 0x7ffa94202000 2024-09-21T07:46:15.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.349+0000 7ffa8a000640 1 -- 172.21.15.88:0/1108586043 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7ffa940d07a0 con 0x7ffa94202000 2024-09-21T07:46:15.354 INFO:teuthology.orchestra.run.smithi088.stdout:68719476741 2024-09-21T07:46:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 -- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa70078c10 msgr2=0x7ffa7007b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa70078c10 0x7ffa7007b120 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7ffa7c0016c0 tx=0x7ffa7c001d10 comp rx=0 tx=0).stop 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 -- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa94202000 msgr2=0x7ffa940cd9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa94202000 0x7ffa940cd9a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c009000 tx=0x7ffa8c032c60 comp rx=0 tx=0).stop 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 -- 172.21.15.88:0/1108586043 shutdown_connections 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa70078c10 0x7ffa7007b120 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa94202000 0x7ffa940cd9a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa9415e770 0x7ffa94201ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 --2- 172.21.15.88:0/1108586043 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa940b7f40 0x7ffa940700c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.353+0000 7ffa83400640 1 -- 172.21.15.88:0/1108586043 >> 172.21.15.88:0/1108586043 conn(0x7ffa940d44f0 msgr2=0x7ffa940d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.354+0000 7ffa83400640 1 -- 172.21.15.88:0/1108586043 shutdown_connections 2024-09-21T07:46:15.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.354+0000 7ffa83400640 1 -- 172.21.15.88:0/1108586043 wait complete. 2024-09-21T07:46:15.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.368+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7fe69c005580 con 0x7fe6bc039d70 2024-09-21T07:46:15.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.369+0000 7fe6ba000640 1 -- 172.21.15.88:0/2542341495 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe6c40d1ce0 con 0x7fe6bc039d70 2024-09-21T07:46:15.371 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T07:46:15.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.369+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f10400af510 con 0x7f10400b3d60 2024-09-21T07:46:15.374 INFO:teuthology.orchestra.run.smithi088.stdout:68719476744 2024-09-21T07:46:15.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.370+0000 7f103d600640 1 -- 172.21.15.88:0/379812300 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f103404f020 con 0x7f10400b3d60 2024-09-21T07:46:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6a4078c60 msgr2=0x7fe6a407b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6a4078c60 0x7fe6a407b170 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4034000 tx=0x7fe6b40046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 msgr2=0x7fe6bc04cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 0x7fe6bc04cdd0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4051230 tx=0x7fe6c4075480 comp rx=0 tx=0).stop 2024-09-21T07:46:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1020078c60 msgr2=0x7f102007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7f1048600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1020078c60 0x7f102007b170 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f102c00a870 tx=0x7f102c008040 comp rx=0 tx=0).stop 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 shutdown_connections 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6a4078c60 0x7fe6a407b170 secure :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fe6b4034000 tx=0x7fe6b40046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6bc0067a0 0x7fe6bc04a960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6bc039d70 0x7fe6bc04cdd0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 --2- 172.21.15.88:0/2542341495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6bc006320 0x7fe6bc103aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 >> 172.21.15.88:0/2542341495 conn(0x7fe6bc01d9a0 msgr2=0x7fe6bc0fe800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 msgr2=0x7f10401f10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 shutdown_connections 2024-09-21T07:46:15.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7fe6c3200640 1 -- 172.21.15.88:0/2542341495 wait complete. 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.374+0000 7f1048600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 0x7f10401f10e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f10400cf360 tx=0x7f1034003f10 comp rx=0 tx=0).stop 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.375+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 shutdown_connections 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.375+0000 7f1048600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1020078c60 0x7f102007b170 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.375+0000 7f1048600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1040158460 0x7f104015d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.375+0000 7f1048600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10400cd000 0x7f104015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.375+0000 7f1048600640 1 --2- 172.21.15.88:0/379812300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10400b3d60 0x7f10401f10e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.375+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 >> 172.21.15.88:0/379812300 conn(0x7f104006d3d0 msgr2=0x7f10400aeb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.376+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 shutdown_connections 2024-09-21T07:46:15.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.376+0000 7f1048600640 1 -- 172.21.15.88:0/379812300 wait complete. 2024-09-21T07:46:15.380 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476744 for osd.3 2024-09-21T07:46:15.391 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.0 2024-09-21T07:46:15.396 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476741 for osd.1 2024-09-21T07:46:15.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.403+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f9f480075e0 con 0x7f9f48053c00 2024-09-21T07:46:15.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.403+0000 7f9f43400640 1 -- 172.21.15.88:0/2778651515 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9f500e2bb0 con 0x7f9f48053c00 2024-09-21T07:46:15.406 INFO:teuthology.orchestra.run.smithi088.stdout:64424509444 2024-09-21T07:46:15.408 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.4 2024-09-21T07:46:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9f30078b90 msgr2=0x7f9f3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9f30078b90 0x7f9f3007b0a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9f3c034000 tx=0x7f9f3c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 msgr2=0x7f9f480d89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 0x7f9f480d89e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9f50051230 tx=0x7f9f5006a030 comp rx=0 tx=0).stop 2024-09-21T07:46:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 shutdown_connections 2024-09-21T07:46:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9f30078b90 0x7f9f3007b0a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f48053c00 0x7f9f480d89e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f48052770 0x7f9f480d84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 --2- 172.21.15.88:0/2778651515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f48052360 0x7f9f480e1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 >> 172.21.15.88:0/2778651515 conn(0x7f9f48051420 msgr2=0x7f9f480dcc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 shutdown_connections 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7f9f55000640 1 -- 172.21.15.88:0/2778651515 wait complete. 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.407+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fc1d0070d00 con 0x7fc1d006ee00 2024-09-21T07:46:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.408+0000 7fc1c7e00640 1 -- 172.21.15.88:0/1079914974 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc1d01616b0 con 0x7fc1d006ee00 2024-09-21T07:46:15.411 INFO:teuthology.orchestra.run.smithi088.stdout:64424509445 2024-09-21T07:46:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc1b0078b90 msgr2=0x7fc1b007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc1b0078b90 0x7fc1b007b0a0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fc1bc033000 tx=0x7fc1bc004730 comp rx=0 tx=0).stop 2024-09-21T07:46:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 msgr2=0x7fc1d0152f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 0x7fc1d0152f60 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fc1c000b420 tx=0x7fc1c000b8e0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 shutdown_connections 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc1b0078b90 0x7fc1b007b0a0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc1d0160d60 0x7fc1d0161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1d006f210 0x7fc1d009d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 --2- 172.21.15.88:0/1079914974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1d006ee00 0x7fc1d0152f60 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 >> 172.21.15.88:0/1079914974 conn(0x7fc1d00b1c60 msgr2=0x7fc1d01539c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 shutdown_connections 2024-09-21T07:46:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.410+0000 7fc1d5a00640 1 -- 172.21.15.88:0/1079914974 wait complete. 2024-09-21T07:46:15.425 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476744 for osd.2 2024-09-21T07:46:15.431 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509444 for osd.6 2024-09-21T07:46:15.439 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509445 for osd.5 2024-09-21T07:46:15.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.440+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f101c005580 con 0x7f104808c100 2024-09-21T07:46:15.443 INFO:teuthology.orchestra.run.smithi088.stdout:60129542150 2024-09-21T07:46:15.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.440+0000 7f103fe00640 1 -- 172.21.15.88:0/3789026543 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1038062860 con 0x7f104808c100 2024-09-21T07:46:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1028078bc0 msgr2=0x7f102807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1028078bc0 0x7f102807b0d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f104806e900 tx=0x7f10340079b0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 msgr2=0x7f10481fa880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 0x7f10481fa880 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f103800c430 tx=0x7f103800c8f0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 shutdown_connections 2024-09-21T07:46:15.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1028078bc0 0x7f102807b0d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10480b3220 0x7f1048158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10480d0480 0x7f10480b1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 --2- 172.21.15.88:0/3789026543 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f104808c100 0x7f10481fa880 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 >> 172.21.15.88:0/3789026543 conn(0x7f104808dd70 msgr2=0x7f104815c430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 shutdown_connections 2024-09-21T07:46:15.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:15.443+0000 7f103d600640 1 -- 172.21.15.88:0/3789026543 wait complete. 2024-09-21T07:46:15.467 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542150 for osd.7 2024-09-21T07:46:16.381 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-09-21T07:46:16.392 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-09-21T07:46:16.397 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-09-21T07:46:16.409 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-09-21T07:46:16.425 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-09-21T07:46:16.432 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-09-21T07:46:16.440 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-09-21T07:46:16.469 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-09-21T07:46:16.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.732+0000 7fe669c00640 1 -- 172.21.15.88:0/3644320156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 msgr2=0x7fe66415a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.732+0000 7fe669c00640 1 --2- 172.21.15.88:0/3644320156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 0x7fe66415a6d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fe650009830 tx=0x7fe65002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.733+0000 7fe669c00640 1 -- 172.21.15.88:0/3644320156 shutdown_connections 2024-09-21T07:46:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.733+0000 7fe669c00640 1 --2- 172.21.15.88:0/3644320156 >> v2:172.21.15.88:3300/0 conn(0x7fe664073e40 0x7fe6640ad7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.733+0000 7fe669c00640 1 --2- 172.21.15.88:0/3644320156 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 0x7fe66415a6d0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.733+0000 7fe669c00640 1 -- 172.21.15.88:0/3644320156 >> 172.21.15.88:0/3644320156 conn(0x7fe6640b27f0 msgr2=0x7fe66406cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.734+0000 7fe669c00640 1 -- 172.21.15.88:0/3644320156 shutdown_connections 2024-09-21T07:46:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.734+0000 7fe669c00640 1 -- 172.21.15.88:0/3644320156 wait complete. 2024-09-21T07:46:16.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.739+0000 7fe669c00640 1 Processor -- start 2024-09-21T07:46:16.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.739+0000 7fe669c00640 1 -- start start 2024-09-21T07:46:16.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.742+0000 7fe669c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 0x7fe66415bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.742+0000 7fe669c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 0x7fe664155db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.743+0000 7fe669c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe664157a00 0x7fe6640b43b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.743+0000 7fe663400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 0x7fe664155db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.743+0000 7fe663400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 0x7fe664155db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48584/0 (socket says 172.21.15.88:48584) 2024-09-21T07:46:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.744+0000 7fe663400640 1 -- 172.21.15.88:0/691820278 learned_addr learned my addr 172.21.15.88:0/691820278 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.744+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6640704b0 con 0x7fe6640ad390 2024-09-21T07:46:16.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.744+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe664070330 con 0x7fe664157a00 2024-09-21T07:46:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.744+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe664070630 con 0x7fe664071030 2024-09-21T07:46:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.744+0000 7fe669200640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe664157a00 0x7fe6640b43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.744+0000 7fe663e00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 0x7fe66415bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.745+0000 7fe669200640 1 -- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 msgr2=0x7fe66415bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.745+0000 7fe669200640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 0x7fe66415bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.745+0000 7fe669200640 1 -- 172.21.15.88:0/691820278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 msgr2=0x7fe664155db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.745+0000 7fe669200640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 0x7fe664155db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.745+0000 7fe669200640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe664157050 con 0x7fe664157a00 2024-09-21T07:46:16.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.746+0000 7fe663400640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 0x7fe664155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:16.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.746+0000 7fe669200640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe664157a00 0x7fe6640b43b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00ebb0 tx=0x7fe65c00c6a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.747+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c003b70 con 0x7fe664157a00 2024-09-21T07:46:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.747+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6641572f0 con 0x7fe664157a00 2024-09-21T07:46:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.748+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe65c003d10 con 0x7fe664157a00 2024-09-21T07:46:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.748+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe65c003b70 con 0x7fe664157a00 2024-09-21T07:46:16.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.748+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66409d610 con 0x7fe664157a00 2024-09-21T07:46:16.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.752+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe66415a6d0 con 0x7fe664157a00 2024-09-21T07:46:16.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.753+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe65c004000 con 0x7fe664157a00 2024-09-21T07:46:16.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.754+0000 7fe65be00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe644078dc0 0x7fe64407b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.755+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe65c09b460 con 0x7fe664157a00 2024-09-21T07:46:16.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.755+0000 7fe663e00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe644078dc0 0x7fe64407b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.757+0000 7fe663e00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe644078dc0 0x7fe64407b2d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe650034000 tx=0x7fe650004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.761+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe65c063240 con 0x7fe664157a00 2024-09-21T07:46:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.796+0000 7f78d6200640 1 -- 172.21.15.88:0/466258974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 msgr2=0x7f78d0072e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.797+0000 7f78d6200640 1 --2- 172.21.15.88:0/466258974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d0072e60 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f78d0072630 tx=0x7f78bc02a4f0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.797+0000 7f78d6200640 1 -- 172.21.15.88:0/466258974 shutdown_connections 2024-09-21T07:46:16.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.797+0000 7f78d6200640 1 --2- 172.21.15.88:0/466258974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d0072e60 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.797+0000 7f78d6200640 1 --2- 172.21.15.88:0/466258974 >> v2:172.21.15.175:3300/0 conn(0x7f78d006ee00 0x7f78d0162640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.797+0000 7f78d6200640 1 -- 172.21.15.88:0/466258974 >> 172.21.15.88:0/466258974 conn(0x7f78d00b1c60 msgr2=0x7f78d00b2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.798+0000 7f78d6200640 1 -- 172.21.15.88:0/466258974 shutdown_connections 2024-09-21T07:46:16.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.799+0000 7f78d6200640 1 -- 172.21.15.88:0/466258974 wait complete. 2024-09-21T07:46:16.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.803+0000 7f78d6200640 1 Processor -- start 2024-09-21T07:46:16.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.803+0000 7f78d6200640 1 -- start start 2024-09-21T07:46:16.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.804+0000 7f0a0aa00640 1 -- 172.21.15.88:0/2654668469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc009b90 msgr2=0x7f09fc0df170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.804+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/2654668469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc009b90 0x7f09fc0df170 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f09f8009820 tx=0x7f09f802a520 comp rx=0 tx=0).stop 2024-09-21T07:46:16.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.804+0000 7f0a0aa00640 1 -- 172.21.15.88:0/2654668469 shutdown_connections 2024-09-21T07:46:16.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.804+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/2654668469 >> v2:172.21.15.88:3300/0 conn(0x7f09fc036b30 0x7f09fc006650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.804+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/2654668469 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc009b90 0x7f09fc0df170 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.805+0000 7f0a0aa00640 1 -- 172.21.15.88:0/2654668469 >> 172.21.15.88:0/2654668469 conn(0x7f09fc051410 msgr2=0x7f09fc051850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.805+0000 7f0a0aa00640 1 -- 172.21.15.88:0/2654668469 shutdown_connections 2024-09-21T07:46:16.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.805+0000 7f0a0aa00640 1 -- 172.21.15.88:0/2654668469 wait complete. 2024-09-21T07:46:16.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.806+0000 7f7d10800640 1 -- 172.21.15.88:0/4249240049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080af3f0 msgr2=0x7f7d0815eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78d006ee00 0x7f78d0150d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.806+0000 7f7d10800640 1 --2- 172.21.15.88:0/4249240049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080af3f0 0x7f7d0815eca0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7d08071de0 tx=0x7f7cfc02a5a0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d006f210 0x7f78d009d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d009da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f7d10800640 1 -- 172.21.15.88:0/4249240049 shutdown_connections 2024-09-21T07:46:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f7d10800640 1 --2- 172.21.15.88:0/4249240049 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d0815f1e0 0x7f7d080d00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f7d10800640 1 --2- 172.21.15.88:0/4249240049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d080af3f0 0x7f7d0815eca0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f7d10800640 1 --2- 172.21.15.88:0/4249240049 >> v2:172.21.15.175:3300/0 conn(0x7f7d0815add0 0x7f7d0815e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f7d10800640 1 -- 172.21.15.88:0/4249240049 >> 172.21.15.88:0/4249240049 conn(0x7f7d080742a0 msgr2=0x7f7d080746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0073ac0 con 0x7f78d0155a80 2024-09-21T07:46:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78d0073940 con 0x7f78d006f210 2024-09-21T07:46:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78d0073c40 con 0x7f78d006ee00 2024-09-21T07:46:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.807+0000 7f78d4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78d006ee00 0x7f78d0150d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f7d10800640 1 -- 172.21.15.88:0/4249240049 shutdown_connections 2024-09-21T07:46:16.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78cfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d006f210 0x7f78d009d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78d5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d009da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78cfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d006f210 0x7f78d009d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:43580/0 (socket says 172.21.15.88:43580) 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78cfe00640 1 -- 172.21.15.88:0/972222619 learned_addr learned my addr 172.21.15.88:0/972222619 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78d5800640 1 -- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78d006ee00 msgr2=0x7f78d0150d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78d5800640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78d006ee00 0x7f78d0150d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78d5800640 1 -- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d006f210 msgr2=0x7f78d009d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78d5800640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d006f210 0x7f78d009d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f78d5800640 1 -- 172.21.15.88:0/972222619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d009c4a0 con 0x7f78d0155a80 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.809+0000 7f78d5800640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d009da80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78d0075310 tx=0x7f78bc003980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.808+0000 7f7d10800640 1 -- 172.21.15.88:0/4249240049 wait complete. 2024-09-21T07:46:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.810+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc03e030 con 0x7f78d0155a80 2024-09-21T07:46:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.810+0000 7f0a0aa00640 1 Processor -- start 2024-09-21T07:46:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.811+0000 7f0a0aa00640 1 -- start start 2024-09-21T07:46:16.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.811+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d009c780 con 0x7f78d0155a80 2024-09-21T07:46:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.812+0000 7f7d10800640 1 Processor -- start 2024-09-21T07:46:16.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.812+0000 7f7d10800640 1 -- start start 2024-09-21T07:46:16.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.811+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d009cd40 con 0x7f78d0155a80 2024-09-21T07:46:16.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.812+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78bc0047a0 con 0x7f78d0155a80 2024-09-21T07:46:16.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.812+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78bc03e030 con 0x7f78d0155a80 2024-09-21T07:46:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.814+0000 7f0a0aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 0x7f09fc02e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.814+0000 7f0a0aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc009b90 0x7f09fc174ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.814+0000 7f0a03e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 0x7f09fc02e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.814+0000 7f0a0aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc0deaa0 0x7f09fc175020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.814+0000 7f0a03e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 0x7f09fc02e0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48632/0 (socket says 172.21.15.88:48632) 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f0a0aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc006b90 con 0x7f09fc005c50 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f0a0aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09fc00be30 con 0x7f09fc009b90 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f1b0e200640 1 -- 172.21.15.88:0/3051432455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 msgr2=0x7f1b080b6730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3051432455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080b6730 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f1b080d2c20 tx=0x7f1b0002aa50 comp rx=0 tx=0).stop 2024-09-21T07:46:16.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f0a03e00640 1 -- 172.21.15.88:0/3930660232 learned_addr learned my addr 172.21.15.88:0/3930660232 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f1b0e200640 1 -- 172.21.15.88:0/3051432455 shutdown_connections 2024-09-21T07:46:16.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09fc006d10 con 0x7f09fc0deaa0 2024-09-21T07:46:16.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f0a03400640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc009b90 0x7f09fc174ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.816+0000 7f0a08a00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc0deaa0 0x7f09fc175020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3051432455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080b6730 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3051432455 >> v2:172.21.15.175:3300/0 conn(0x7f1b080b78c0 0x7f1b080d0330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.815+0000 7f1b0e200640 1 -- 172.21.15.88:0/3051432455 >> 172.21.15.88:0/3051432455 conn(0x7f1b0808bfe0 msgr2=0x7f1b080d4660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.816+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78a8005290 con 0x7f78d0155a80 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.816+0000 7f7d10800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d080af3f0 0x7f7d081544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f7d10800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d0815add0 0x7f7d081e60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f7d10800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d0815f1e0 0x7f7d080ae100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f1b0e200640 1 -- 172.21.15.88:0/3051432455 shutdown_connections 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f0a03e00640 1 -- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc0deaa0 msgr2=0x7f09fc175020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f0a03e00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc0deaa0 0x7f09fc175020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f0a03e00640 1 -- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc009b90 msgr2=0x7f09fc174ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f0a03e00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc009b90 0x7f09fc174ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f0a03e00640 1 -- 172.21.15.88:0/3930660232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09fc020bf0 con 0x7f09fc005c50 2024-09-21T07:46:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d0e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d0815f1e0 0x7f7d080ae100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d0e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d0815f1e0 0x7f7d080ae100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:43604/0 (socket says 172.21.15.88:43604) 2024-09-21T07:46:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d0e800640 1 -- 172.21.15.88:0/4240429693 learned_addr learned my addr 172.21.15.88:0/4240429693 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f0a03e00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 0x7f09fc02e0b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000c00 tx=0x7f09f8033f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d080d05f0 con 0x7f7d0815add0 2024-09-21T07:46:16.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d08074fd0 con 0x7f7d0815f1e0 2024-09-21T07:46:16.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d0de00640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d080af3f0 0x7f7d081544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d080d0770 con 0x7f7d080af3f0 2024-09-21T07:46:16.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.818+0000 7f7d0d400640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d0815add0 0x7f7d081e60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8004020 con 0x7f09fc005c50 2024-09-21T07:46:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09fc020ed0 con 0x7f09fc005c50 2024-09-21T07:46:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09fc02fe90 con 0x7f09fc005c50 2024-09-21T07:46:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.817+0000 7f1b0e200640 1 -- 172.21.15.88:0/3051432455 wait complete. 2024-09-21T07:46:16.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f09f8042070 con 0x7f09fc005c50 2024-09-21T07:46:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f8004020 con 0x7f09fc005c50 2024-09-21T07:46:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.821+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09e0005290 con 0x7f09fc005c50 2024-09-21T07:46:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.821+0000 7f1b0e200640 1 Processor -- start 2024-09-21T07:46:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.821+0000 7f1b0e200640 1 -- start start 2024-09-21T07:46:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f7d0de00640 1 -- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d0815f1e0 msgr2=0x7f7d080ae100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f7d0de00640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d0815f1e0 0x7f7d080ae100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.819+0000 7f7d0de00640 1 -- 172.21.15.88:0/4240429693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d0815add0 msgr2=0x7f7d081e60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.823+0000 7f7d0de00640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d0815add0 0x7f7d081e60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.823+0000 7f7d0de00640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d081fd530 con 0x7f7d080af3f0 2024-09-21T07:46:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.823+0000 7f7d0d400640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d0815add0 0x7f7d081e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:16.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.823+0000 7f7d0de00640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d080af3f0 0x7f7d081544c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e9d0 tx=0x7f7cf400ee90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.821+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f78bc041030 con 0x7f78d0155a80 2024-09-21T07:46:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.823+0000 7f78ccc00640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f78b0078dc0 0x7f78b007b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.823+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f78bc0baf10 con 0x7f78d0155a80 2024-09-21T07:46:16.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.824+0000 7f78d4e00640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f78b0078dc0 0x7f78b007b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.824+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4015030 con 0x7f7d080af3f0 2024-09-21T07:46:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.824+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7cf4008c30 con 0x7f7d080af3f0 2024-09-21T07:46:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.824+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cf4008e80 con 0x7f7d080af3f0 2024-09-21T07:46:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.824+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d081fda30 con 0x7f7d080af3f0 2024-09-21T07:46:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.824+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d081fdff0 con 0x7f7d080af3f0 2024-09-21T07:46:16.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.825+0000 7f78d4e00640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f78b0078dc0 0x7f78b007b2d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78c8005fd0 tx=0x7f78c8005bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.826+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080ce050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.826+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b080b74b0 0x7f1b0815d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.826+0000 7f1b0e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b080b78c0 0x7f1b0815dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.826+0000 7f1b0e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080b54e0 con 0x7f1b080b6320 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.827+0000 7f1b0e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b080b5360 con 0x7f1b080b78c0 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.827+0000 7f1b0e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b08169330 con 0x7f1b080b74b0 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.827+0000 7f1b07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080ce050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.827+0000 7f1b07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080ce050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48648/0 (socket says 172.21.15.88:48648) 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.827+0000 7f1b07400640 1 -- 172.21.15.88:0/3732716823 learned_addr learned my addr 172.21.15.88:0/3732716823 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.827+0000 7f1b06a00640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b080b74b0 0x7f1b0815d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.826+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d080d0cd0 con 0x7f7d080af3f0 2024-09-21T07:46:16.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.828+0000 7f1b07400640 1 -- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b080b74b0 msgr2=0x7f1b0815d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.828+0000 7f1b07400640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b080b74b0 0x7f1b0815d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.828+0000 7f1b07400640 1 -- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b080b78c0 msgr2=0x7f1b0815dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.828+0000 7f1b07400640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b080b78c0 0x7f1b0815dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.828+0000 7f1b07400640 1 -- 172.21.15.88:0/3732716823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b08167920 con 0x7f1b080b6320 2024-09-21T07:46:16.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.829+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f7cf400f040 con 0x7f7d080af3f0 2024-09-21T07:46:16.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.829+0000 7f1b07400640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080ce050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c3c0 tx=0x7f1af800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.829+0000 7f1b06a00640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b080b74b0 0x7f1b0815d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:16.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.831+0000 7f7d02000640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7ce8078b90 0x7f7ce807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.831+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f7cf409a750 con 0x7f7d080af3f0 2024-09-21T07:46:16.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.832+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78bc083340 con 0x7f78d0155a80 2024-09-21T07:46:16.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.832+0000 7f7d0d400640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7ce8078b90 0x7f7ce807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.833+0000 7f7d0d400640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7ce8078b90 0x7f7ce807b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc033000 tx=0x7f7cfc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.834+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8017070 con 0x7f1b080b6320 2024-09-21T07:46:16.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.834+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1af80053c0 con 0x7f1b080b6320 2024-09-21T07:46:16.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.834+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1af8003850 con 0x7f1b080b6320 2024-09-21T07:46:16.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.835+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b080b2900 con 0x7f1b080b6320 2024-09-21T07:46:16.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.835+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b08167ec0 con 0x7f1b080b6320 2024-09-21T07:46:16.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.836+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7cf4062530 con 0x7f7d080af3f0 2024-09-21T07:46:16.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.835+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f09f80041e0 con 0x7f09fc005c50 2024-09-21T07:46:16.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.837+0000 7f09f7e00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f09e4078d90 0x7f09e407b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.837+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f09f80bfa80 con 0x7f09fc005c50 2024-09-21T07:46:16.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.837+0000 7f0a03400640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f09e4078d90 0x7f09e407b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.838+0000 7f0a03400640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f09e4078d90 0x7f09e407b2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000b60 tx=0x7f0a04070040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.839+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b0808caf0 con 0x7f1b080b6320 2024-09-21T07:46:16.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.844+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1af801a070 con 0x7f1b080b6320 2024-09-21T07:46:16.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.845+0000 7f1aff400640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ae8078c10 0x7f1ae807b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.846+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1af809a350 con 0x7f1b080b6320 2024-09-21T07:46:16.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.846+0000 7f1b06a00640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ae8078c10 0x7f1ae807b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.847+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b0808caf0 con 0x7f1b080b6320 2024-09-21T07:46:16.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.847+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09f8087f30 con 0x7f09fc005c50 2024-09-21T07:46:16.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.854+0000 7f1b06a00640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ae8078c10 0x7f1ae807b120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b08069680 tx=0x7f1af4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.873+0000 7f113b000640 1 -- 172.21.15.88:0/1740450909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415cfe0 msgr2=0x7f113415d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.873+0000 7f113b000640 1 --2- 172.21.15.88:0/1740450909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415cfe0 0x7f113415d3f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f111c005fd0 tx=0x7f111c02a850 comp rx=0 tx=0).stop 2024-09-21T07:46:16.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.873+0000 7f113b000640 1 -- 172.21.15.88:0/1740450909 shutdown_connections 2024-09-21T07:46:16.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.873+0000 7f113b000640 1 --2- 172.21.15.88:0/1740450909 >> v2:172.21.15.88:3300/0 conn(0x7f113415d930 0x7f11341663b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.873+0000 7f113b000640 1 --2- 172.21.15.88:0/1740450909 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415cfe0 0x7f113415d3f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.873+0000 7f113b000640 1 -- 172.21.15.88:0/1740450909 >> 172.21.15.88:0/1740450909 conn(0x7f11340742a0 msgr2=0x7f11340746b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.874+0000 7f113b000640 1 -- 172.21.15.88:0/1740450909 shutdown_connections 2024-09-21T07:46:16.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.874+0000 7f113b000640 1 -- 172.21.15.88:0/1740450909 wait complete. 2024-09-21T07:46:16.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.878+0000 7f113b000640 1 Processor -- start 2024-09-21T07:46:16.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.879+0000 7f113b000640 1 -- start start 2024-09-21T07:46:16.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.882+0000 7f113b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 0x7f1134154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.882+0000 7f5c9f200640 1 -- 172.21.15.88:0/509968905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c98054190 msgr2=0x7f5c98007550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.882+0000 7f5c9f200640 1 --2- 172.21.15.88:0/509968905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c98054190 0x7f5c98007550 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f5c98057090 tx=0x7f5c8802a4f0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.882+0000 7f1133e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 0x7f1134154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.882+0000 7f1133e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 0x7f1134154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48678/0 (socket says 172.21.15.88:48678) 2024-09-21T07:46:16.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.882+0000 7f1133e00640 1 -- 172.21.15.88:0/1954334820 learned_addr learned my addr 172.21.15.88:0/1954334820 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113415cfe0 0x7f113414fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415d930 0x7f11340b4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 -- 172.21.15.88:0/509968905 shutdown_connections 2024-09-21T07:46:16.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 --2- 172.21.15.88:0/509968905 >> v2:172.21.15.175:3300/0 conn(0x7f5c98053690 0x7f5c98053aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 --2- 172.21.15.88:0/509968905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c980e3200 0x7f5c98053150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 --2- 172.21.15.88:0/509968905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c98054190 0x7f5c98007550 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 -- 172.21.15.88:0/509968905 >> 172.21.15.88:0/509968905 conn(0x7f5c9803a680 msgr2=0x7f5c9803aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1134166c80 con 0x7f11340cefc0 2024-09-21T07:46:16.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 -- 172.21.15.88:0/509968905 shutdown_connections 2024-09-21T07:46:16.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.883+0000 7f5c9f200640 1 -- 172.21.15.88:0/509968905 wait complete. 2024-09-21T07:46:16.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1134166b00 con 0x7f113415cfe0 2024-09-21T07:46:16.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f1139000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415d930 0x7f11340b4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 -- 172.21.15.88:0/3902178507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600b9d60 msgr2=0x7f5a600d2bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 --2- 172.21.15.88:0/3902178507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600b9d60 0x7f5a600d2bc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c005fd0 tx=0x7f5a4c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 -- 172.21.15.88:0/3902178507 shutdown_connections 2024-09-21T07:46:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 --2- 172.21.15.88:0/3902178507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600b9d60 0x7f5a600d2bc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 --2- 172.21.15.88:0/3902178507 >> v2:172.21.15.175:3300/0 conn(0x7f5a600d2500 0x7f5a600b3a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 -- 172.21.15.88:0/3902178507 >> 172.21.15.88:0/3902178507 conn(0x7f5a6008bfe0 msgr2=0x7f5a600d4690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:16.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f112b400640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113415cfe0 0x7f113414fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 -- 172.21.15.88:0/3902178507 shutdown_connections 2024-09-21T07:46:16.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f5a65800640 1 -- 172.21.15.88:0/3902178507 wait complete. 2024-09-21T07:46:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.884+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1134166e00 con 0x7f113415d930 2024-09-21T07:46:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.885+0000 7f1139000640 1 -- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113415cfe0 msgr2=0x7f113414fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.885+0000 7f1139000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113415cfe0 0x7f113414fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.885+0000 7f1139000640 1 -- 172.21.15.88:0/1954334820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 msgr2=0x7f1134154940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.885+0000 7f1139000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 0x7f1134154940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.886+0000 7f1139000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f113414ffa0 con 0x7f113415d930 2024-09-21T07:46:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.886+0000 7f1133e00640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 0x7f1134154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.886+0000 7f1139000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415d930 0x7f11340b4220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f11341602e0 tx=0x7f112c00ead0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.886+0000 7f5a65800640 1 Processor -- start 2024-09-21T07:46:16.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.887+0000 7f5a65800640 1 -- start start 2024-09-21T07:46:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.887+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c004220 con 0x7f113415d930 2024-09-21T07:46:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.887+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f112c005020 con 0x7f113415d930 2024-09-21T07:46:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.888+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f112c005470 con 0x7f113415d930 2024-09-21T07:46:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.888+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11341fd4c0 con 0x7f113415d930 2024-09-21T07:46:16.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.888+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7f11341fda80 con 0x7f113415d930 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.888+0000 7f5a65800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 0x7f5a6015d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.888+0000 7f5a65800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a600d2500 0x7f5a601f5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.888+0000 7f5a65800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600d3100 0x7f5a60159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a65800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600cfdb0 con 0x7f5a600b9d60 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a65800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a600cfc30 con 0x7f5a600d2500 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a65800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a600cff30 con 0x7f5a600d3100 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a5e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a600d2500 0x7f5a601f5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a5ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 0x7f5a6015d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a5ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 0x7f5a6015d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48682/0 (socket says 172.21.15.88:48682) 2024-09-21T07:46:16.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a5ea00640 1 -- 172.21.15.88:0/3720752030 learned_addr learned my addr 172.21.15.88:0/3720752030 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.889+0000 7f5a5f400640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600d3100 0x7f5a60159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5ea00640 1 -- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600d3100 msgr2=0x7f5a60159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5ea00640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600d3100 0x7f5a60159d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5ea00640 1 -- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a600d2500 msgr2=0x7f5a601f5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5ea00640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a600d2500 0x7f5a601f5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5ea00640 1 -- 172.21.15.88:0/3720752030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a601f9320 con 0x7f5a600b9d60 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5e000640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a600d2500 0x7f5a601f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f5a5ea00640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 0x7f5a6015d910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000d740 tx=0x7f5a5000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.891+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a50017070 con 0x7f5a600b9d60 2024-09-21T07:46:16.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.891+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6015a910 con 0x7f5a600b9d60 2024-09-21T07:46:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.891+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6015aed0 con 0x7f5a600b9d60 2024-09-21T07:46:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.891+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5a50009d20 con 0x7f5a600b9d60 2024-09-21T07:46:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.891+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a500106c0 con 0x7f5a600b9d60 2024-09-21T07:46:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.892+0000 7f5c9f200640 1 Processor -- start 2024-09-21T07:46:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.890+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11341fd230 con 0x7f113415d930 2024-09-21T07:46:16.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.893+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f112c00c7e0 con 0x7f113415d930 2024-09-21T07:46:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.893+0000 7f5c9f200640 1 -- start start 2024-09-21T07:46:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.895+0000 7f1130c00640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1110078c60 0x7f111007b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.895+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f112c09b250 con 0x7f113415d930 2024-09-21T07:46:16.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.895+0000 7f1133e00640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1110078c60 0x7f111007b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.895+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5a5001a070 con 0x7f5a600b9d60 2024-09-21T07:46:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.896+0000 7f5c9f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c98053690 0x7f5c98182720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5c9f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c98054190 0x7f5c9804ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f1133e00640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1110078c60 0x7f111007b170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11240046b0 tx=0x7f1124007f20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5c9de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c98053690 0x7f5c98182720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5c9de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c98053690 0x7f5c98182720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:58180/0 (socket says 172.21.15.88:58180) 2024-09-21T07:46:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5c9de00640 1 -- 172.21.15.88:0/4233818545 learned_addr learned my addr 172.21.15.88:0/4233818545 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:16.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.896+0000 7f5a56a00640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5a40078b90 0x7f5a4007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.896+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a38005290 con 0x7f5a600b9d60 2024-09-21T07:46:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5a5009aec0 con 0x7f5a600b9d60 2024-09-21T07:46:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c980e3200 0x7f5c9803d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980e71d0 con 0x7f5c980e3200 2024-09-21T07:46:16.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.897+0000 7f5a5e000640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5a40078b90 0x7f5a4007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.898+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5c980e7050 con 0x7f5c98054190 2024-09-21T07:46:16.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.898+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c980e7350 con 0x7f5c98053690 2024-09-21T07:46:16.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.898+0000 7f5c9d400640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c98054190 0x7f5c9804ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.898+0000 7f5c9e800640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c980e3200 0x7f5c9803d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.898+0000 7f5a5e000640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5a40078b90 0x7f5a4007b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c032000 tx=0x7f5a4c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f5c9d400640 1 -- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c98053690 msgr2=0x7f5c98182720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f5c9d400640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c98053690 0x7f5c98182720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f5c9d400640 1 -- 172.21.15.88:0/4233818545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c980e3200 msgr2=0x7f5c9803d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:16.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f5c9d400640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c980e3200 0x7f5c9803d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:16.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f112c063670 con 0x7f113415d930 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f5c9d400640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c980e63e0 con 0x7f5c98054190 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.899+0000 7f5c9e800640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c980e3200 0x7f5c9803d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.903+0000 7f5c9d400640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c98054190 0x7f5c9804ef20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d680 tx=0x7f5c9000db40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.903+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90017070 con 0x7f5c98054190 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.903+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c90005310 con 0x7f5c98054190 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.903+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c90009820 con 0x7f5c98054190 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.904+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c980e66c0 con 0x7f5c98054190 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.904+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c98181f20 con 0x7f5c98054190 2024-09-21T07:46:16.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.906+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c98007550 con 0x7f5c98054190 2024-09-21T07:46:16.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.910+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5a50062c10 con 0x7f5a600b9d60 2024-09-21T07:46:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.914+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5c900099c0 con 0x7f5c98054190 2024-09-21T07:46:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.915+0000 7f5c96000640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5c7c078b90 0x7f5c7c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:16.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.915+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5c9009b190 con 0x7f5c98054190 2024-09-21T07:46:16.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.917+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c90062f60 con 0x7f5c98054190 2024-09-21T07:46:16.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.917+0000 7f5c9de00640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5c7c078b90 0x7f5c7c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:16.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:16.921+0000 7f5c9de00640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5c7c078b90 0x7f5c7c07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c88000c00 tx=0x7f5c88004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.199+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe66415c510 con 0x7fe664157a00 2024-09-21T07:46:17.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.200+0000 7fe65be00640 1 -- 172.21.15.88:0/691820278 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe65c067c70 con 0x7fe664157a00 2024-09-21T07:46:17.203 INFO:teuthology.orchestra.run.smithi088.stdout:68719476746 2024-09-21T07:46:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.204+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe644078dc0 msgr2=0x7fe64407b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.204+0000 7fe669c00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe644078dc0 0x7fe64407b2d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe650034000 tx=0x7fe650004730 comp rx=0 tx=0).stop 2024-09-21T07:46:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.205+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe664157a00 msgr2=0x7fe6640b43b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.205+0000 7fe669c00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe664157a00 0x7fe6640b43b0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe65c00ebb0 tx=0x7fe65c00c6a0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.205+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 shutdown_connections 2024-09-21T07:46:17.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.205+0000 7fe669c00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe644078dc0 0x7fe64407b2d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.205+0000 7fe669c00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe664157a00 0x7fe6640b43b0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.205+0000 7fe669c00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6640ad390 0x7fe664155db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.206+0000 7fe669c00640 1 --2- 172.21.15.88:0/691820278 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe664071030 0x7fe66415bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.206+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 >> 172.21.15.88:0/691820278 conn(0x7fe6640b27f0 msgr2=0x7fe6640b0bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.206+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 shutdown_connections 2024-09-21T07:46:17.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.206+0000 7fe669c00640 1 -- 172.21.15.88:0/691820278 wait complete. 2024-09-21T07:46:17.249 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476746 for osd.3 2024-09-21T07:46:17.249 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.267+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f78a8005580 con 0x7f78d0155a80 2024-09-21T07:46:17.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.267+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f7d080cdcd0 con 0x7f7d080af3f0 2024-09-21T07:46:17.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.268+0000 7f78ccc00640 1 -- 172.21.15.88:0/972222619 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f78bc082c40 con 0x7f78d0155a80 2024-09-21T07:46:17.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.268+0000 7f7d02000640 1 -- 172.21.15.88:0/4240429693 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f7cf4066f60 con 0x7f7d080af3f0 2024-09-21T07:46:17.270 INFO:teuthology.orchestra.run.smithi088.stdout:68719476743 2024-09-21T07:46:17.271 INFO:teuthology.orchestra.run.smithi088.stdout:68719476746 2024-09-21T07:46:17.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f78b0078dc0 msgr2=0x7f78b007b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7ce8078b90 msgr2=0x7f7ce807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f7d10800640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7ce8078b90 0x7f7ce807b0a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc033000 tx=0x7f7cfc004730 comp rx=0 tx=0).stop 2024-09-21T07:46:17.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d080af3f0 msgr2=0x7f7d081544c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f7d10800640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d080af3f0 0x7f7d081544c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7cf400e9d0 tx=0x7f7cf400ee90 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f78b0078dc0 0x7f78b007b2d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f78c8005fd0 tx=0x7f78c8005bc0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 msgr2=0x7f78d009da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d009da80 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f78d0075310 tx=0x7f78bc003980 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 shutdown_connections 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f78b0078dc0 0x7f78b007b2d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78d0155a80 0x7f78d009da80 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78d006f210 0x7f78d009d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 --2- 172.21.15.88:0/972222619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78d006ee00 0x7f78d0150d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 >> 172.21.15.88:0/972222619 conn(0x7f78d00b1c60 msgr2=0x7f78d00d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 shutdown_connections 2024-09-21T07:46:17.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.272+0000 7f78d6200640 1 -- 172.21.15.88:0/972222619 wait complete. 2024-09-21T07:46:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.273+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 shutdown_connections 2024-09-21T07:46:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.273+0000 7f7d10800640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7ce8078b90 0x7f7ce807b0a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.273+0000 7f7d10800640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d0815f1e0 0x7f7d080ae100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.273+0000 7f7d10800640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d0815add0 0x7f7d081e60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.273+0000 7f7d10800640 1 --2- 172.21.15.88:0/4240429693 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d080af3f0 0x7f7d081544c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.273+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 >> 172.21.15.88:0/4240429693 conn(0x7f7d080742a0 msgr2=0x7f7d080b4260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.274+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 shutdown_connections 2024-09-21T07:46:17.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.274+0000 7f7d10800640 1 -- 172.21.15.88:0/4240429693 wait complete. 2024-09-21T07:46:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.286+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f09e0005850 con 0x7f09fc005c50 2024-09-21T07:46:17.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.287+0000 7f09f7e00640 1 -- 172.21.15.88:0/3930660232 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f09f8087830 con 0x7f09fc005c50 2024-09-21T07:46:17.290 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.290+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f09e4078d90 msgr2=0x7f09e407b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.290+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f09e4078d90 0x7f09e407b2a0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a04000b60 tx=0x7f0a04070040 comp rx=0 tx=0).stop 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.290+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 msgr2=0x7f09fc02e0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.290+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 0x7f09fc02e0b0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f09f8000c00 tx=0x7f09f8033f10 comp rx=0 tx=0).stop 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 shutdown_connections 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f09e4078d90 0x7f09e407b2a0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f09fc0deaa0 0x7f09fc175020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09fc009b90 0x7f09fc174ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 --2- 172.21.15.88:0/3930660232 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09fc005c50 0x7f09fc02e0b0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 >> 172.21.15.88:0/3930660232 conn(0x7f09fc051410 msgr2=0x7f09fc053d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 shutdown_connections 2024-09-21T07:46:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.291+0000 7f0a0aa00640 1 -- 172.21.15.88:0/3930660232 wait complete. 2024-09-21T07:46:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.294+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f1b080b6730 con 0x7f1b080b6320 2024-09-21T07:46:17.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.295+0000 7f1aff400640 1 -- 172.21.15.88:0/3732716823 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f1af8021020 con 0x7f1b080b6320 2024-09-21T07:46:17.297 INFO:teuthology.orchestra.run.smithi088.stdout:68719476743 2024-09-21T07:46:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.298+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ae8078c10 msgr2=0x7f1ae807b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.298+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ae8078c10 0x7f1ae807b120 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b08069680 tx=0x7f1af4008040 comp rx=0 tx=0).stop 2024-09-21T07:46:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.298+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 msgr2=0x7f1b080ce050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.298+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080ce050 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f1af800c3c0 tx=0x7f1af800c880 comp rx=0 tx=0).stop 2024-09-21T07:46:17.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 shutdown_connections 2024-09-21T07:46:17.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ae8078c10 0x7f1ae807b120 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b080b74b0 0x7f1b0815d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b080b78c0 0x7f1b0815dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 --2- 172.21.15.88:0/3732716823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b080b6320 0x7f1b080ce050 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 >> 172.21.15.88:0/3732716823 conn(0x7f1b0808bfe0 msgr2=0x7f1b0806e8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 shutdown_connections 2024-09-21T07:46:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.299+0000 7f1b0e200640 1 -- 172.21.15.88:0/3732716823 wait complete. 2024-09-21T07:46:17.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.307+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f113415d3f0 con 0x7f113415d930 2024-09-21T07:46:17.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.308+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f5c980e6140 con 0x7f5c98054190 2024-09-21T07:46:17.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.308+0000 7f1130c00640 1 -- 172.21.15.88:0/1954334820 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f112c062f70 con 0x7f113415d930 2024-09-21T07:46:17.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.309+0000 7f5c96000640 1 -- 172.21.15.88:0/4233818545 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5c90067990 con 0x7f5c98054190 2024-09-21T07:46:17.311 INFO:teuthology.orchestra.run.smithi088.stdout:64424509446 2024-09-21T07:46:17.311 INFO:teuthology.orchestra.run.smithi088.stdout:60129542151 2024-09-21T07:46:17.312 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476743 for osd.0 2024-09-21T07:46:17.312 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.311+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1110078c60 msgr2=0x7f111007b170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.311+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1110078c60 0x7f111007b170 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f11240046b0 tx=0x7f1124007f20 comp rx=0 tx=0).stop 2024-09-21T07:46:17.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.311+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415d930 msgr2=0x7f11340b4220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.311+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415d930 0x7f11340b4220 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f11341602e0 tx=0x7f112c00ead0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 shutdown_connections 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1110078c60 0x7f111007b170 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f113415d930 0x7f11340b4220 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f113415cfe0 0x7f113414fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 --2- 172.21.15.88:0/1954334820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11340cefc0 0x7f1134154940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 >> 172.21.15.88:0/1954334820 conn(0x7f11340742a0 msgr2=0x7f1134150290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 shutdown_connections 2024-09-21T07:46:17.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f113b000640 1 -- 172.21.15.88:0/1954334820 wait complete. 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5c7c078b90 msgr2=0x7f5c7c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5c7c078b90 0x7f5c7c07b0a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f5c88000c00 tx=0x7f5c88004730 comp rx=0 tx=0).stop 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c98054190 msgr2=0x7f5c9804ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c98054190 0x7f5c9804ef20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5c9000d680 tx=0x7f5c9000db40 comp rx=0 tx=0).stop 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 shutdown_connections 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5c7c078b90 0x7f5c7c07b0a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c980e3200 0x7f5c9803d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c98054190 0x7f5c9804ef20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 --2- 172.21.15.88:0/4233818545 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c98053690 0x7f5c98182720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.312+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 >> 172.21.15.88:0/4233818545 conn(0x7f5c9803a680 msgr2=0x7f5c9804fbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.313+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 shutdown_connections 2024-09-21T07:46:17.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.313+0000 7f5c9f200640 1 -- 172.21.15.88:0/4233818545 wait complete. 2024-09-21T07:46:17.320 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476745 got 68719476746 for osd.2 2024-09-21T07:46:17.320 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.333 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2024-09-21T07:46:17.333 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 68719476742 got 68719476743 for osd.1 2024-09-21T07:46:17.334 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.333+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f5a38005580 con 0x7f5a600b9d60 2024-09-21T07:46:17.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.334+0000 7f5a56a00640 1 -- 172.21.15.88:0/3720752030 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f5a50067640 con 0x7f5a600b9d60 2024-09-21T07:46:17.336 INFO:teuthology.orchestra.run.smithi088.stdout:64424509447 2024-09-21T07:46:17.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5a40078b90 msgr2=0x7f5a4007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5a40078b90 0x7f5a4007b0a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f5a4c032000 tx=0x7f5a4c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 msgr2=0x7f5a6015d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 0x7f5a6015d910 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f5a5000d740 tx=0x7f5a5000dc00 comp rx=0 tx=0).stop 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 shutdown_connections 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5a40078b90 0x7f5a4007b0a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a600d3100 0x7f5a60159d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a600d2500 0x7f5a601f5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 --2- 172.21.15.88:0/3720752030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a600b9d60 0x7f5a6015d910 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 >> 172.21.15.88:0/3720752030 conn(0x7f5a6008bfe0 msgr2=0x7f5a600b54e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.336+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 shutdown_connections 2024-09-21T07:46:17.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.337+0000 7f5a65800640 1 -- 172.21.15.88:0/3720752030 wait complete. 2024-09-21T07:46:17.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509445 got 64424509446 for osd.6 2024-09-21T07:46:17.353 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.353 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542151 got 60129542151 for osd.7 2024-09-21T07:46:17.353 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.360 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.5 2024-09-21T07:46:17.360 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:17.360 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:46:17.360 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 -- 172.21.15.88:0/1326847177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44140aebc0 msgr2=0x7f4414152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 --2- 172.21.15.88:0/1326847177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44140aebc0 0x7f4414152920 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4404005fd0 tx=0x7f440402a540 comp rx=0 tx=0).stop 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 -- 172.21.15.88:0/1326847177 shutdown_connections 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 --2- 172.21.15.88:0/1326847177 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44140b2750 0x7f441415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 --2- 172.21.15.88:0/1326847177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44140aebc0 0x7f4414152920 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 -- 172.21.15.88:0/1326847177 >> 172.21.15.88:0/1326847177 conn(0x7f441406d730 msgr2=0x7f44140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 -- 172.21.15.88:0/1326847177 shutdown_connections 2024-09-21T07:46:17.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.618+0000 7f441a800640 1 -- 172.21.15.88:0/1326847177 wait complete. 2024-09-21T07:46:17.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.620+0000 7f441a800640 1 Processor -- start 2024-09-21T07:46:17.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.621+0000 7f441a800640 1 -- start start 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f441a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44140aebc0 0x7f44141f98d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f441a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44140b2750 0x7f441414b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f441a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 0x7f44140cacc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f441a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f441415c4a0 con 0x7f441415f910 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f441a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44140b4a20 con 0x7f44140aebc0 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f441a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f441415c620 con 0x7f44140b2750 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f4413e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 0x7f44140cacc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f4413e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 0x7f44140cacc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48722/0 (socket says 172.21.15.88:48722) 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.622+0000 7f4413e00640 1 -- 172.21.15.88:0/4223922007 learned_addr learned my addr 172.21.15.88:0/4223922007 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413400640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44140aebc0 0x7f44141f98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4412a00640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44140b2750 0x7f441414b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413e00640 1 -- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44140b2750 msgr2=0x7f441414b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413e00640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44140b2750 0x7f441414b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413e00640 1 -- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44140aebc0 msgr2=0x7f44141f98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413e00640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44140aebc0 0x7f44141f98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413e00640 1 -- 172.21.15.88:0/4223922007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44140cba40 con 0x7f441415f910 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f4413e00640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 0x7f44140cacc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c730 tx=0x7f440c00cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c003b70 con 0x7f441415f910 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f440c003d10 con 0x7f441415f910 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f440c0052d0 con 0x7f441415f910 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.623+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44140cb760 con 0x7f441415f910 2024-09-21T07:46:17.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.624+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44141f90d0 con 0x7f441415f910 2024-09-21T07:46:17.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.625+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f441415cb40 con 0x7f441415f910 2024-09-21T07:46:17.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.626+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f440c005470 con 0x7f441415f910 2024-09-21T07:46:17.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.627+0000 7f440b400640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f43f4078b90 0x7f43f407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:17.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.627+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f440c09a980 con 0x7f441415f910 2024-09-21T07:46:17.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.628+0000 7f4413400640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f43f4078b90 0x7f43f407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:17.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.628+0000 7f4413400640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f43f4078b90 0x7f43f407b0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4404033000 tx=0x7f4404004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:17.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.632+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f441415fd20 con 0x7f441415f910 2024-09-21T07:46:17.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.817+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4414069820 con 0x7f43f4078b90 2024-09-21T07:46:17.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.823+0000 7f440b400640 1 -- 172.21.15.88:0/4223922007 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251821 (secure 0 0 0) 0x7f4414069820 con 0x7f43f4078b90 2024-09-21T07:46:17.827 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:17.829 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:46:17.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.826+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f43f4078b90 msgr2=0x7f43f407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.826+0000 7f441a800640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f43f4078b90 0x7f43f407b0a0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f4404033000 tx=0x7f4404004730 comp rx=0 tx=0).stop 2024-09-21T07:46:17.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 msgr2=0x7f44140cacc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:17.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 0x7f44140cacc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f440c00c730 tx=0x7f440c00cb30 comp rx=0 tx=0).stop 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 shutdown_connections 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f43f4078b90 0x7f43f407b0a0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44140b2750 0x7f441414b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f441415f910 0x7f44140cacc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 --2- 172.21.15.88:0/4223922007 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44140aebc0 0x7f44141f98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 >> 172.21.15.88:0/4223922007 conn(0x7f441406d730 msgr2=0x7f44140b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 shutdown_connections 2024-09-21T07:46:17.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:17.827+0000 7f441a800640 1 -- 172.21.15.88:0/4223922007 wait complete. 2024-09-21T07:46:17.853 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":30,"stamp":"2024-09-21T07:46:15.856589+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219256,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755464,"statfs":{"total":773094113280,"available":772869595136,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.669544"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199749+0000","last_change":"2024-09-21T07:46:07.234192+0000","last_active":"2024-09-21T07:46:09.199749+0000","last_peered":"2024-09-21T07:46:09.199749+0000","last_clean":"2024-09-21T07:46:09.199749+0000","last_became_active":"2024-09-21T07:46:07.231852+0000","last_became_peered":"2024-09-21T07:46:07.231852+0000","last_unstale":"2024-09-21T07:46:09.199749+0000","last_undegraded":"2024-09-21T07:46:09.199749+0000","last_fullsized":"2024-09-21T07:46:09.199749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208860+0000","last_change":"2024-09-21T07:46:07.219201+0000","last_active":"2024-09-21T07:46:08.208860+0000","last_peered":"2024-09-21T07:46:08.208860+0000","last_clean":"2024-09-21T07:46:08.208860+0000","last_became_active":"2024-09-21T07:46:07.218520+0000","last_became_peered":"2024-09-21T07:46:07.218520+0000","last_unstale":"2024-09-21T07:46:08.208860+0000","last_undegraded":"2024-09-21T07:46:08.208860+0000","last_fullsized":"2024-09-21T07:46:08.208860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:38.468021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208382+0000","last_change":"2024-09-21T07:46:07.225678+0000","last_active":"2024-09-21T07:46:08.208382+0000","last_peered":"2024-09-21T07:46:08.208382+0000","last_clean":"2024-09-21T07:46:08.208382+0000","last_became_active":"2024-09-21T07:46:07.225167+0000","last_became_peered":"2024-09-21T07:46:07.225167+0000","last_unstale":"2024-09-21T07:46:08.208382+0000","last_undegraded":"2024-09-21T07:46:08.208382+0000","last_fullsized":"2024-09-21T07:46:08.208382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205764+0000","last_change":"2024-09-21T07:46:07.229463+0000","last_active":"2024-09-21T07:46:08.205764+0000","last_peered":"2024-09-21T07:46:08.205764+0000","last_clean":"2024-09-21T07:46:08.205764+0000","last_became_active":"2024-09-21T07:46:07.229040+0000","last_became_peered":"2024-09-21T07:46:07.229040+0000","last_unstale":"2024-09-21T07:46:08.205764+0000","last_undegraded":"2024-09-21T07:46:08.205764+0000","last_fullsized":"2024-09-21T07:46:08.205764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:25.366313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199458+0000","last_change":"2024-09-21T07:46:07.234579+0000","last_active":"2024-09-21T07:46:09.199458+0000","last_peered":"2024-09-21T07:46:09.199458+0000","last_clean":"2024-09-21T07:46:09.199458+0000","last_became_active":"2024-09-21T07:46:07.233770+0000","last_became_peered":"2024-09-21T07:46:07.233770+0000","last_unstale":"2024-09-21T07:46:09.199458+0000","last_undegraded":"2024-09-21T07:46:09.199458+0000","last_fullsized":"2024-09-21T07:46:09.199458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205760+0000","last_change":"2024-09-21T07:46:07.222135+0000","last_active":"2024-09-21T07:46:08.205760+0000","last_peered":"2024-09-21T07:46:08.205760+0000","last_clean":"2024-09-21T07:46:08.205760+0000","last_became_active":"2024-09-21T07:46:07.221847+0000","last_became_peered":"2024-09-21T07:46:07.221847+0000","last_unstale":"2024-09-21T07:46:08.205760+0000","last_undegraded":"2024-09-21T07:46:08.205760+0000","last_fullsized":"2024-09-21T07:46:08.205760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:09.010971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199222+0000","last_change":"2024-09-21T07:46:07.231015+0000","last_active":"2024-09-21T07:46:09.199222+0000","last_peered":"2024-09-21T07:46:09.199222+0000","last_clean":"2024-09-21T07:46:09.199222+0000","last_became_active":"2024-09-21T07:46:07.230719+0000","last_became_peered":"2024-09-21T07:46:07.230719+0000","last_unstale":"2024-09-21T07:46:09.199222+0000","last_undegraded":"2024-09-21T07:46:09.199222+0000","last_fullsized":"2024-09-21T07:46:09.199222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208796+0000","last_change":"2024-09-21T07:46:07.217547+0000","last_active":"2024-09-21T07:46:08.208796+0000","last_peered":"2024-09-21T07:46:08.208796+0000","last_clean":"2024-09-21T07:46:08.208796+0000","last_became_active":"2024-09-21T07:46:07.215538+0000","last_became_peered":"2024-09-21T07:46:07.215538+0000","last_unstale":"2024-09-21T07:46:08.208796+0000","last_undegraded":"2024-09-21T07:46:08.208796+0000","last_fullsized":"2024-09-21T07:46:08.208796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:00.117867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500477+0000","last_change":"2024-09-21T07:46:07.201245+0000","last_active":"2024-09-21T07:46:09.500477+0000","last_peered":"2024-09-21T07:46:09.500477+0000","last_clean":"2024-09-21T07:46:09.500477+0000","last_became_active":"2024-09-21T07:46:07.200940+0000","last_became_peered":"2024-09-21T07:46:07.200940+0000","last_unstale":"2024-09-21T07:46:09.500477+0000","last_undegraded":"2024-09-21T07:46:09.500477+0000","last_fullsized":"2024-09-21T07:46:09.500477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:19.780073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199520+0000","last_change":"2024-09-21T07:46:07.226919+0000","last_active":"2024-09-21T07:46:09.199520+0000","last_peered":"2024-09-21T07:46:09.199520+0000","last_clean":"2024-09-21T07:46:09.199520+0000","last_became_active":"2024-09-21T07:46:07.226619+0000","last_became_peered":"2024-09-21T07:46:07.226619+0000","last_unstale":"2024-09-21T07:46:09.199520+0000","last_undegraded":"2024-09-21T07:46:09.199520+0000","last_fullsized":"2024-09-21T07:46:09.199520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:47.470275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199269+0000","last_change":"2024-09-21T07:46:07.225286+0000","last_active":"2024-09-21T07:46:09.199269+0000","last_peered":"2024-09-21T07:46:09.199269+0000","last_clean":"2024-09-21T07:46:09.199269+0000","last_became_active":"2024-09-21T07:46:07.223018+0000","last_became_peered":"2024-09-21T07:46:07.223018+0000","last_unstale":"2024-09-21T07:46:09.199269+0000","last_undegraded":"2024-09-21T07:46:09.199269+0000","last_fullsized":"2024-09-21T07:46:09.199269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199545+0000","last_change":"2024-09-21T07:46:07.229988+0000","last_active":"2024-09-21T07:46:09.199545+0000","last_peered":"2024-09-21T07:46:09.199545+0000","last_clean":"2024-09-21T07:46:09.199545+0000","last_became_active":"2024-09-21T07:46:07.229716+0000","last_became_peered":"2024-09-21T07:46:07.229716+0000","last_unstale":"2024-09-21T07:46:09.199545+0000","last_undegraded":"2024-09-21T07:46:09.199545+0000","last_fullsized":"2024-09-21T07:46:09.199545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521098+0000","last_change":"2024-09-21T07:46:07.238701+0000","last_active":"2024-09-21T07:46:08.521098+0000","last_peered":"2024-09-21T07:46:08.521098+0000","last_clean":"2024-09-21T07:46:08.521098+0000","last_became_active":"2024-09-21T07:46:07.238406+0000","last_became_peered":"2024-09-21T07:46:07.238406+0000","last_unstale":"2024-09-21T07:46:08.521098+0000","last_undegraded":"2024-09-21T07:46:08.521098+0000","last_fullsized":"2024-09-21T07:46:08.521098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521245+0000","last_change":"2024-09-21T07:46:07.211337+0000","last_active":"2024-09-21T07:46:08.521245+0000","last_peered":"2024-09-21T07:46:08.521245+0000","last_clean":"2024-09-21T07:46:08.521245+0000","last_became_active":"2024-09-21T07:46:07.210983+0000","last_became_peered":"2024-09-21T07:46:07.210983+0000","last_unstale":"2024-09-21T07:46:08.521245+0000","last_undegraded":"2024-09-21T07:46:08.521245+0000","last_fullsized":"2024-09-21T07:46:08.521245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:43.997921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209926+0000","last_change":"2024-09-21T07:46:07.225928+0000","last_active":"2024-09-21T07:46:08.209926+0000","last_peered":"2024-09-21T07:46:08.209926+0000","last_clean":"2024-09-21T07:46:08.209926+0000","last_became_active":"2024-09-21T07:46:07.225685+0000","last_became_peered":"2024-09-21T07:46:07.225685+0000","last_unstale":"2024-09-21T07:46:08.209926+0000","last_undegraded":"2024-09-21T07:46:08.209926+0000","last_fullsized":"2024-09-21T07:46:08.209926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399827+0000","last_change":"2024-09-21T07:46:07.211743+0000","last_active":"2024-09-21T07:46:08.399827+0000","last_peered":"2024-09-21T07:46:08.399827+0000","last_clean":"2024-09-21T07:46:08.399827+0000","last_became_active":"2024-09-21T07:46:07.209957+0000","last_became_peered":"2024-09-21T07:46:07.209957+0000","last_unstale":"2024-09-21T07:46:08.399827+0000","last_undegraded":"2024-09-21T07:46:08.399827+0000","last_fullsized":"2024-09-21T07:46:08.399827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:18.660926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400124+0000","last_change":"2024-09-21T07:46:07.228920+0000","last_active":"2024-09-21T07:46:08.400124+0000","last_peered":"2024-09-21T07:46:08.400124+0000","last_clean":"2024-09-21T07:46:08.400124+0000","last_became_active":"2024-09-21T07:46:07.227861+0000","last_became_peered":"2024-09-21T07:46:07.227861+0000","last_unstale":"2024-09-21T07:46:08.400124+0000","last_undegraded":"2024-09-21T07:46:08.400124+0000","last_fullsized":"2024-09-21T07:46:08.400124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:46:34.920768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032613+0000","last_change":"2024-09-21T07:46:07.221477+0000","last_active":"2024-09-21T07:46:09.032613+0000","last_peered":"2024-09-21T07:46:09.032613+0000","last_clean":"2024-09-21T07:46:09.032613+0000","last_became_active":"2024-09-21T07:46:07.217967+0000","last_became_peered":"2024-09-21T07:46:07.217967+0000","last_unstale":"2024-09-21T07:46:09.032613+0000","last_undegraded":"2024-09-21T07:46:09.032613+0000","last_fullsized":"2024-09-21T07:46:09.032613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:34.633793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399789+0000","last_change":"2024-09-21T07:46:07.221262+0000","last_active":"2024-09-21T07:46:08.399789+0000","last_peered":"2024-09-21T07:46:08.399789+0000","last_clean":"2024-09-21T07:46:08.399789+0000","last_became_active":"2024-09-21T07:46:07.221018+0000","last_became_peered":"2024-09-21T07:46:07.221018+0000","last_unstale":"2024-09-21T07:46:08.399789+0000","last_undegraded":"2024-09-21T07:46:08.399789+0000","last_fullsized":"2024-09-21T07:46:08.399789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:46.384301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199556+0000","last_change":"2024-09-21T07:46:07.228997+0000","last_active":"2024-09-21T07:46:09.199556+0000","last_peered":"2024-09-21T07:46:09.199556+0000","last_clean":"2024-09-21T07:46:09.199556+0000","last_became_active":"2024-09-21T07:46:07.227078+0000","last_became_peered":"2024-09-21T07:46:07.227078+0000","last_unstale":"2024-09-21T07:46:09.199556+0000","last_undegraded":"2024-09-21T07:46:09.199556+0000","last_fullsized":"2024-09-21T07:46:09.199556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207714+0000","last_change":"2024-09-21T07:46:07.220220+0000","last_active":"2024-09-21T07:46:08.207714+0000","last_peered":"2024-09-21T07:46:08.207714+0000","last_clean":"2024-09-21T07:46:08.207714+0000","last_became_active":"2024-09-21T07:46:07.219890+0000","last_became_peered":"2024-09-21T07:46:07.219890+0000","last_unstale":"2024-09-21T07:46:08.207714+0000","last_undegraded":"2024-09-21T07:46:08.207714+0000","last_fullsized":"2024-09-21T07:46:08.207714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499999+0000","last_change":"2024-09-21T07:46:07.227381+0000","last_active":"2024-09-21T07:46:09.499999+0000","last_peered":"2024-09-21T07:46:09.499999+0000","last_clean":"2024-09-21T07:46:09.499999+0000","last_became_active":"2024-09-21T07:46:07.225810+0000","last_became_peered":"2024-09-21T07:46:07.225810+0000","last_unstale":"2024-09-21T07:46:09.499999+0000","last_undegraded":"2024-09-21T07:46:09.499999+0000","last_fullsized":"2024-09-21T07:46:09.499999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:23.838546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.210100+0000","last_change":"2024-09-21T07:46:07.215158+0000","last_active":"2024-09-21T07:46:08.210100+0000","last_peered":"2024-09-21T07:46:08.210100+0000","last_clean":"2024-09-21T07:46:08.210100+0000","last_became_active":"2024-09-21T07:46:07.214842+0000","last_became_peered":"2024-09-21T07:46:07.214842+0000","last_unstale":"2024-09-21T07:46:08.210100+0000","last_undegraded":"2024-09-21T07:46:08.210100+0000","last_fullsized":"2024-09-21T07:46:08.210100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399861+0000","last_change":"2024-09-21T07:46:07.209932+0000","last_active":"2024-09-21T07:46:08.399861+0000","last_peered":"2024-09-21T07:46:08.399861+0000","last_clean":"2024-09-21T07:46:08.399861+0000","last_became_active":"2024-09-21T07:46:07.208985+0000","last_became_peered":"2024-09-21T07:46:07.208985+0000","last_unstale":"2024-09-21T07:46:08.399861+0000","last_undegraded":"2024-09-21T07:46:08.399861+0000","last_fullsized":"2024-09-21T07:46:08.399861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:32.406624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208288+0000","last_change":"2024-09-21T07:46:07.218857+0000","last_active":"2024-09-21T07:46:08.208288+0000","last_peered":"2024-09-21T07:46:08.208288+0000","last_clean":"2024-09-21T07:46:08.208288+0000","last_became_active":"2024-09-21T07:46:07.218490+0000","last_became_peered":"2024-09-21T07:46:07.218490+0000","last_unstale":"2024-09-21T07:46:08.208288+0000","last_undegraded":"2024-09-21T07:46:08.208288+0000","last_fullsized":"2024-09-21T07:46:08.208288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206165+0000","last_change":"2024-09-21T07:46:07.218802+0000","last_active":"2024-09-21T07:46:08.206165+0000","last_peered":"2024-09-21T07:46:08.206165+0000","last_clean":"2024-09-21T07:46:08.206165+0000","last_became_active":"2024-09-21T07:46:07.216480+0000","last_became_peered":"2024-09-21T07:46:07.216480+0000","last_unstale":"2024-09-21T07:46:08.206165+0000","last_undegraded":"2024-09-21T07:46:08.206165+0000","last_fullsized":"2024-09-21T07:46:08.206165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521582+0000","last_change":"2024-09-21T07:46:07.213848+0000","last_active":"2024-09-21T07:46:08.521582+0000","last_peered":"2024-09-21T07:46:08.521582+0000","last_clean":"2024-09-21T07:46:08.521582+0000","last_became_active":"2024-09-21T07:46:07.212350+0000","last_became_peered":"2024-09-21T07:46:07.212350+0000","last_unstale":"2024-09-21T07:46:08.521582+0000","last_undegraded":"2024-09-21T07:46:08.521582+0000","last_fullsized":"2024-09-21T07:46:08.521582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521460+0000","last_change":"2024-09-21T07:46:07.231247+0000","last_active":"2024-09-21T07:46:08.521460+0000","last_peered":"2024-09-21T07:46:08.521460+0000","last_clean":"2024-09-21T07:46:08.521460+0000","last_became_active":"2024-09-21T07:46:07.230913+0000","last_became_peered":"2024-09-21T07:46:07.230913+0000","last_unstale":"2024-09-21T07:46:08.521460+0000","last_undegraded":"2024-09-21T07:46:08.521460+0000","last_fullsized":"2024-09-21T07:46:08.521460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208726+0000","last_change":"2024-09-21T07:46:07.212029+0000","last_active":"2024-09-21T07:46:08.208726+0000","last_peered":"2024-09-21T07:46:08.208726+0000","last_clean":"2024-09-21T07:46:08.208726+0000","last_became_active":"2024-09-21T07:46:07.211701+0000","last_became_peered":"2024-09-21T07:46:07.211701+0000","last_unstale":"2024-09-21T07:46:08.208726+0000","last_undegraded":"2024-09-21T07:46:08.208726+0000","last_fullsized":"2024-09-21T07:46:08.208726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399383+0000","last_change":"2024-09-21T07:46:07.209431+0000","last_active":"2024-09-21T07:46:08.399383+0000","last_peered":"2024-09-21T07:46:08.399383+0000","last_clean":"2024-09-21T07:46:08.399383+0000","last_became_active":"2024-09-21T07:46:07.208558+0000","last_became_peered":"2024-09-21T07:46:07.208558+0000","last_unstale":"2024-09-21T07:46:08.399383+0000","last_undegraded":"2024-09-21T07:46:08.399383+0000","last_fullsized":"2024-09-21T07:46:08.399383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032488+0000","last_change":"2024-09-21T07:46:07.213929+0000","last_active":"2024-09-21T07:46:09.032488+0000","last_peered":"2024-09-21T07:46:09.032488+0000","last_clean":"2024-09-21T07:46:09.032488+0000","last_became_active":"2024-09-21T07:46:07.213456+0000","last_became_peered":"2024-09-21T07:46:07.213456+0000","last_unstale":"2024-09-21T07:46:09.032488+0000","last_undegraded":"2024-09-21T07:46:09.032488+0000","last_fullsized":"2024-09-21T07:46:09.032488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:08.925890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205701+0000","last_change":"2024-09-21T07:46:07.218008+0000","last_active":"2024-09-21T07:46:08.205701+0000","last_peered":"2024-09-21T07:46:08.205701+0000","last_clean":"2024-09-21T07:46:08.205701+0000","last_became_active":"2024-09-21T07:46:07.217294+0000","last_became_peered":"2024-09-21T07:46:07.217294+0000","last_unstale":"2024-09-21T07:46:08.205701+0000","last_undegraded":"2024-09-21T07:46:08.205701+0000","last_fullsized":"2024-09-21T07:46:08.205701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:48.673151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500036+0000","last_change":"2024-09-21T07:46:07.203504+0000","last_active":"2024-09-21T07:46:09.500036+0000","last_peered":"2024-09-21T07:46:09.500036+0000","last_clean":"2024-09-21T07:46:09.500036+0000","last_became_active":"2024-09-21T07:46:07.203135+0000","last_became_peered":"2024-09-21T07:46:07.203135+0000","last_unstale":"2024-09-21T07:46:09.500036+0000","last_undegraded":"2024-09-21T07:46:09.500036+0000","last_fullsized":"2024-09-21T07:46:09.500036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:11.674361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208870+0000","last_change":"2024-09-21T07:46:06.210978+0000","last_active":"2024-09-21T07:46:08.208870+0000","last_peered":"2024-09-21T07:46:08.208870+0000","last_clean":"2024-09-21T07:46:08.208870+0000","last_became_active":"2024-09-21T07:46:06.210615+0000","last_became_peered":"2024-09-21T07:46:06.210615+0000","last_unstale":"2024-09-21T07:46:08.208870+0000","last_undegraded":"2024-09-21T07:46:08.208870+0000","last_fullsized":"2024-09-21T07:46:08.208870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:12.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207933+0000","last_change":"2024-09-21T07:46:07.236139+0000","last_active":"2024-09-21T07:46:08.207933+0000","last_peered":"2024-09-21T07:46:08.207933+0000","last_clean":"2024-09-21T07:46:08.207933+0000","last_became_active":"2024-09-21T07:46:07.235357+0000","last_became_peered":"2024-09-21T07:46:07.235357+0000","last_unstale":"2024-09-21T07:46:08.207933+0000","last_undegraded":"2024-09-21T07:46:08.207933+0000","last_fullsized":"2024-09-21T07:46:08.207933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399662+0000","last_change":"2024-09-21T07:46:06.218149+0000","last_active":"2024-09-21T07:46:08.399662+0000","last_peered":"2024-09-21T07:46:08.399662+0000","last_clean":"2024-09-21T07:46:08.399662+0000","last_became_active":"2024-09-21T07:46:06.217846+0000","last_became_peered":"2024-09-21T07:46:06.217846+0000","last_unstale":"2024-09-21T07:46:08.399662+0000","last_undegraded":"2024-09-21T07:46:08.399662+0000","last_fullsized":"2024-09-21T07:46:08.399662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.520888+0000","last_change":"2024-09-21T07:46:06.213787+0000","last_active":"2024-09-21T07:46:08.520888+0000","last_peered":"2024-09-21T07:46:08.520888+0000","last_clean":"2024-09-21T07:46:08.520888+0000","last_became_active":"2024-09-21T07:46:06.213454+0000","last_became_peered":"2024-09-21T07:46:06.213454+0000","last_unstale":"2024-09-21T07:46:08.520888+0000","last_undegraded":"2024-09-21T07:46:08.520888+0000","last_fullsized":"2024-09-21T07:46:08.520888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208744+0000","last_change":"2024-09-21T07:46:07.217173+0000","last_active":"2024-09-21T07:46:08.208744+0000","last_peered":"2024-09-21T07:46:08.208744+0000","last_clean":"2024-09-21T07:46:08.208744+0000","last_became_active":"2024-09-21T07:46:07.216857+0000","last_became_peered":"2024-09-21T07:46:07.216857+0000","last_unstale":"2024-09-21T07:46:08.208744+0000","last_undegraded":"2024-09-21T07:46:08.208744+0000","last_fullsized":"2024-09-21T07:46:08.208744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399914+0000","last_change":"2024-09-21T07:46:06.210041+0000","last_active":"2024-09-21T07:46:08.399914+0000","last_peered":"2024-09-21T07:46:08.399914+0000","last_clean":"2024-09-21T07:46:08.399914+0000","last_became_active":"2024-09-21T07:46:06.209744+0000","last_became_peered":"2024-09-21T07:46:06.209744+0000","last_unstale":"2024-09-21T07:46:08.399914+0000","last_undegraded":"2024-09-21T07:46:08.399914+0000","last_fullsized":"2024-09-21T07:46:08.399914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207314+0000","last_change":"2024-09-21T07:46:07.217122+0000","last_active":"2024-09-21T07:46:08.207314+0000","last_peered":"2024-09-21T07:46:08.207314+0000","last_clean":"2024-09-21T07:46:08.207314+0000","last_became_active":"2024-09-21T07:46:07.216831+0000","last_became_peered":"2024-09-21T07:46:07.216831+0000","last_unstale":"2024-09-21T07:46:08.207314+0000","last_undegraded":"2024-09-21T07:46:08.207314+0000","last_fullsized":"2024-09-21T07:46:08.207314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500434+0000","last_change":"2024-09-21T07:46:06.202767+0000","last_active":"2024-09-21T07:46:09.500434+0000","last_peered":"2024-09-21T07:46:09.500434+0000","last_clean":"2024-09-21T07:46:09.500434+0000","last_became_active":"2024-09-21T07:46:06.200291+0000","last_became_peered":"2024-09-21T07:46:06.200291+0000","last_unstale":"2024-09-21T07:46:09.500434+0000","last_undegraded":"2024-09-21T07:46:09.500434+0000","last_fullsized":"2024-09-21T07:46:09.500434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206404+0000","last_change":"2024-09-21T07:46:07.216776+0000","last_active":"2024-09-21T07:46:08.206404+0000","last_peered":"2024-09-21T07:46:08.206404+0000","last_clean":"2024-09-21T07:46:08.206404+0000","last_became_active":"2024-09-21T07:46:07.214952+0000","last_became_peered":"2024-09-21T07:46:07.214952+0000","last_unstale":"2024-09-21T07:46:08.206404+0000","last_undegraded":"2024-09-21T07:46:08.206404+0000","last_fullsized":"2024-09-21T07:46:08.206404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199670+0000","last_change":"2024-09-21T07:46:06.210647+0000","last_active":"2024-09-21T07:46:09.199670+0000","last_peered":"2024-09-21T07:46:09.199670+0000","last_clean":"2024-09-21T07:46:09.199670+0000","last_became_active":"2024-09-21T07:46:06.209590+0000","last_became_peered":"2024-09-21T07:46:06.209590+0000","last_unstale":"2024-09-21T07:46:09.199670+0000","last_undegraded":"2024-09-21T07:46:09.199670+0000","last_fullsized":"2024-09-21T07:46:09.199670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:40.259455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208537+0000","last_change":"2024-09-21T07:46:07.210270+0000","last_active":"2024-09-21T07:46:08.208537+0000","last_peered":"2024-09-21T07:46:08.208537+0000","last_clean":"2024-09-21T07:46:08.208537+0000","last_became_active":"2024-09-21T07:46:07.202021+0000","last_became_peered":"2024-09-21T07:46:07.202021+0000","last_unstale":"2024-09-21T07:46:08.208537+0000","last_undegraded":"2024-09-21T07:46:08.208537+0000","last_fullsized":"2024-09-21T07:46:08.208537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032615+0000","last_change":"2024-09-21T07:46:06.212266+0000","last_active":"2024-09-21T07:46:09.032615+0000","last_peered":"2024-09-21T07:46:09.032615+0000","last_clean":"2024-09-21T07:46:09.032615+0000","last_became_active":"2024-09-21T07:46:06.210122+0000","last_became_peered":"2024-09-21T07:46:06.210122+0000","last_unstale":"2024-09-21T07:46:09.032615+0000","last_undegraded":"2024-09-21T07:46:09.032615+0000","last_fullsized":"2024-09-21T07:46:09.032615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:13.095998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399665+0000","last_change":"2024-09-21T07:46:07.216761+0000","last_active":"2024-09-21T07:46:08.399665+0000","last_peered":"2024-09-21T07:46:08.399665+0000","last_clean":"2024-09-21T07:46:08.399665+0000","last_became_active":"2024-09-21T07:46:07.215637+0000","last_became_peered":"2024-09-21T07:46:07.215637+0000","last_unstale":"2024-09-21T07:46:08.399665+0000","last_undegraded":"2024-09-21T07:46:08.399665+0000","last_fullsized":"2024-09-21T07:46:08.399665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:48.803562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205818+0000","last_change":"2024-09-21T07:46:06.206864+0000","last_active":"2024-09-21T07:46:08.205818+0000","last_peered":"2024-09-21T07:46:08.205818+0000","last_clean":"2024-09-21T07:46:08.205818+0000","last_became_active":"2024-09-21T07:46:06.206545+0000","last_became_peered":"2024-09-21T07:46:06.206545+0000","last_unstale":"2024-09-21T07:46:08.205818+0000","last_undegraded":"2024-09-21T07:46:08.205818+0000","last_fullsized":"2024-09-21T07:46:08.205818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:00.329415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199512+0000","last_change":"2024-09-21T07:46:07.236117+0000","last_active":"2024-09-21T07:46:09.199512+0000","last_peered":"2024-09-21T07:46:09.199512+0000","last_clean":"2024-09-21T07:46:09.199512+0000","last_became_active":"2024-09-21T07:46:07.233496+0000","last_became_peered":"2024-09-21T07:46:07.233496+0000","last_unstale":"2024-09-21T07:46:09.199512+0000","last_undegraded":"2024-09-21T07:46:09.199512+0000","last_fullsized":"2024-09-21T07:46:09.199512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032504+0000","last_change":"2024-09-21T07:46:06.212633+0000","last_active":"2024-09-21T07:46:09.032504+0000","last_peered":"2024-09-21T07:46:09.032504+0000","last_clean":"2024-09-21T07:46:09.032504+0000","last_became_active":"2024-09-21T07:46:06.212240+0000","last_became_peered":"2024-09-21T07:46:06.212240+0000","last_unstale":"2024-09-21T07:46:09.032504+0000","last_undegraded":"2024-09-21T07:46:09.032504+0000","last_fullsized":"2024-09-21T07:46:09.032504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:36.198191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521167+0000","last_change":"2024-09-21T07:46:07.216368+0000","last_active":"2024-09-21T07:46:08.521167+0000","last_peered":"2024-09-21T07:46:08.521167+0000","last_clean":"2024-09-21T07:46:08.521167+0000","last_became_active":"2024-09-21T07:46:07.216061+0000","last_became_peered":"2024-09-21T07:46:07.216061+0000","last_unstale":"2024-09-21T07:46:08.521167+0000","last_undegraded":"2024-09-21T07:46:08.521167+0000","last_fullsized":"2024-09-21T07:46:08.521167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399781+0000","last_change":"2024-09-21T07:46:06.204421+0000","last_active":"2024-09-21T07:46:08.399781+0000","last_peered":"2024-09-21T07:46:08.399781+0000","last_clean":"2024-09-21T07:46:08.399781+0000","last_became_active":"2024-09-21T07:46:06.204109+0000","last_became_peered":"2024-09-21T07:46:06.204109+0000","last_unstale":"2024-09-21T07:46:08.399781+0000","last_undegraded":"2024-09-21T07:46:08.399781+0000","last_fullsized":"2024-09-21T07:46:08.399781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:10.667453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399577+0000","last_change":"2024-09-21T07:46:07.237847+0000","last_active":"2024-09-21T07:46:08.399577+0000","last_peered":"2024-09-21T07:46:08.399577+0000","last_clean":"2024-09-21T07:46:08.399577+0000","last_became_active":"2024-09-21T07:46:07.236995+0000","last_became_peered":"2024-09-21T07:46:07.236995+0000","last_unstale":"2024-09-21T07:46:08.399577+0000","last_undegraded":"2024-09-21T07:46:08.399577+0000","last_fullsized":"2024-09-21T07:46:08.399577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:19.395558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199277+0000","last_change":"2024-09-21T07:46:06.218044+0000","last_active":"2024-09-21T07:46:09.199277+0000","last_peered":"2024-09-21T07:46:09.199277+0000","last_clean":"2024-09-21T07:46:09.199277+0000","last_became_active":"2024-09-21T07:46:06.217446+0000","last_became_peered":"2024-09-21T07:46:06.217446+0000","last_unstale":"2024-09-21T07:46:09.199277+0000","last_undegraded":"2024-09-21T07:46:09.199277+0000","last_fullsized":"2024-09-21T07:46:09.199277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:47.666243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499878+0000","last_change":"2024-09-21T07:46:07.210885+0000","last_active":"2024-09-21T07:46:09.499878+0000","last_peered":"2024-09-21T07:46:09.499878+0000","last_clean":"2024-09-21T07:46:09.499878+0000","last_became_active":"2024-09-21T07:46:07.210591+0000","last_became_peered":"2024-09-21T07:46:07.210591+0000","last_unstale":"2024-09-21T07:46:09.499878+0000","last_undegraded":"2024-09-21T07:46:09.499878+0000","last_fullsized":"2024-09-21T07:46:09.499878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.666658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207016+0000","last_change":"2024-09-21T07:46:06.202414+0000","last_active":"2024-09-21T07:46:08.207016+0000","last_peered":"2024-09-21T07:46:08.207016+0000","last_clean":"2024-09-21T07:46:08.207016+0000","last_became_active":"2024-09-21T07:46:06.201528+0000","last_became_peered":"2024-09-21T07:46:06.201528+0000","last_unstale":"2024-09-21T07:46:08.207016+0000","last_undegraded":"2024-09-21T07:46:08.207016+0000","last_fullsized":"2024-09-21T07:46:08.207016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399393+0000","last_change":"2024-09-21T07:46:07.226232+0000","last_active":"2024-09-21T07:46:08.399393+0000","last_peered":"2024-09-21T07:46:08.399393+0000","last_clean":"2024-09-21T07:46:08.399393+0000","last_became_active":"2024-09-21T07:46:07.224513+0000","last_became_peered":"2024-09-21T07:46:07.224513+0000","last_unstale":"2024-09-21T07:46:08.399393+0000","last_undegraded":"2024-09-21T07:46:08.399393+0000","last_fullsized":"2024-09-21T07:46:08.399393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:17.528901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199798+0000","last_change":"2024-09-21T07:46:06.219146+0000","last_active":"2024-09-21T07:46:09.199798+0000","last_peered":"2024-09-21T07:46:09.199798+0000","last_clean":"2024-09-21T07:46:09.199798+0000","last_became_active":"2024-09-21T07:46:06.218899+0000","last_became_peered":"2024-09-21T07:46:06.218899+0000","last_unstale":"2024-09-21T07:46:09.199798+0000","last_undegraded":"2024-09-21T07:46:09.199798+0000","last_fullsized":"2024-09-21T07:46:09.199798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:08.004063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206636+0000","last_change":"2024-09-21T07:46:07.235918+0000","last_active":"2024-09-21T07:46:08.206636+0000","last_peered":"2024-09-21T07:46:08.206636+0000","last_clean":"2024-09-21T07:46:08.206636+0000","last_became_active":"2024-09-21T07:46:07.235596+0000","last_became_peered":"2024-09-21T07:46:07.235596+0000","last_unstale":"2024-09-21T07:46:08.206636+0000","last_undegraded":"2024-09-21T07:46:08.206636+0000","last_fullsized":"2024-09-21T07:46:08.206636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208692+0000","last_change":"2024-09-21T07:46:06.208887+0000","last_active":"2024-09-21T07:46:08.208692+0000","last_peered":"2024-09-21T07:46:08.208692+0000","last_clean":"2024-09-21T07:46:08.208692+0000","last_became_active":"2024-09-21T07:46:06.208508+0000","last_became_peered":"2024-09-21T07:46:06.208508+0000","last_unstale":"2024-09-21T07:46:08.208692+0000","last_undegraded":"2024-09-21T07:46:08.208692+0000","last_fullsized":"2024-09-21T07:46:08.208692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199820+0000","last_change":"2024-09-21T07:46:07.225748+0000","last_active":"2024-09-21T07:46:09.199820+0000","last_peered":"2024-09-21T07:46:09.199820+0000","last_clean":"2024-09-21T07:46:09.199820+0000","last_became_active":"2024-09-21T07:46:07.225406+0000","last_became_peered":"2024-09-21T07:46:07.225406+0000","last_unstale":"2024-09-21T07:46:09.199820+0000","last_undegraded":"2024-09-21T07:46:09.199820+0000","last_fullsized":"2024-09-21T07:46:09.199820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.699234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208511+0000","last_change":"2024-09-21T07:46:06.209460+0000","last_active":"2024-09-21T07:46:08.208511+0000","last_peered":"2024-09-21T07:46:08.208511+0000","last_clean":"2024-09-21T07:46:08.208511+0000","last_became_active":"2024-09-21T07:46:06.207853+0000","last_became_peered":"2024-09-21T07:46:06.207853+0000","last_unstale":"2024-09-21T07:46:08.208511+0000","last_undegraded":"2024-09-21T07:46:08.208511+0000","last_fullsized":"2024-09-21T07:46:08.208511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199786+0000","last_change":"2024-09-21T07:46:07.228685+0000","last_active":"2024-09-21T07:46:09.199786+0000","last_peered":"2024-09-21T07:46:09.199786+0000","last_clean":"2024-09-21T07:46:09.199786+0000","last_became_active":"2024-09-21T07:46:07.226780+0000","last_became_peered":"2024-09-21T07:46:07.226780+0000","last_unstale":"2024-09-21T07:46:09.199786+0000","last_undegraded":"2024-09-21T07:46:09.199786+0000","last_fullsized":"2024-09-21T07:46:09.199786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:32.406624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521589+0000","last_change":"2024-09-21T07:46:06.216893+0000","last_active":"2024-09-21T07:46:08.521589+0000","last_peered":"2024-09-21T07:46:08.521589+0000","last_clean":"2024-09-21T07:46:08.521589+0000","last_became_active":"2024-09-21T07:46:06.216657+0000","last_became_peered":"2024-09-21T07:46:06.216657+0000","last_unstale":"2024-09-21T07:46:08.521589+0000","last_undegraded":"2024-09-21T07:46:08.521589+0000","last_fullsized":"2024-09-21T07:46:08.521589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399601+0000","last_change":"2024-09-21T07:46:07.210978+0000","last_active":"2024-09-21T07:46:08.399601+0000","last_peered":"2024-09-21T07:46:08.399601+0000","last_clean":"2024-09-21T07:46:08.399601+0000","last_became_active":"2024-09-21T07:46:07.210665+0000","last_became_peered":"2024-09-21T07:46:07.210665+0000","last_unstale":"2024-09-21T07:46:08.399601+0000","last_undegraded":"2024-09-21T07:46:08.399601+0000","last_fullsized":"2024-09-21T07:46:08.399601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:57.801044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499736+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:09.499736+0000","last_peered":"2024-09-21T07:46:09.499736+0000","last_clean":"2024-09-21T07:46:09.499736+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:09.499736+0000","last_undegraded":"2024-09-21T07:46:09.499736+0000","last_fullsized":"2024-09-21T07:46:09.499736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399734+0000","last_change":"2024-09-21T07:46:06.204733+0000","last_active":"2024-09-21T07:46:08.399734+0000","last_peered":"2024-09-21T07:46:08.399734+0000","last_clean":"2024-09-21T07:46:08.399734+0000","last_became_active":"2024-09-21T07:46:06.204416+0000","last_became_peered":"2024-09-21T07:46:06.204416+0000","last_unstale":"2024-09-21T07:46:08.399734+0000","last_undegraded":"2024-09-21T07:46:08.399734+0000","last_fullsized":"2024-09-21T07:46:08.399734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:18.773164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207407+0000","last_change":"2024-09-21T07:46:07.222509+0000","last_active":"2024-09-21T07:46:08.207407+0000","last_peered":"2024-09-21T07:46:08.207407+0000","last_clean":"2024-09-21T07:46:08.207407+0000","last_became_active":"2024-09-21T07:46:07.221105+0000","last_became_peered":"2024-09-21T07:46:07.221105+0000","last_unstale":"2024-09-21T07:46:08.207407+0000","last_undegraded":"2024-09-21T07:46:08.207407+0000","last_fullsized":"2024-09-21T07:46:08.207407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399939+0000","last_change":"2024-09-21T07:46:06.212101+0000","last_active":"2024-09-21T07:46:08.399939+0000","last_peered":"2024-09-21T07:46:08.399939+0000","last_clean":"2024-09-21T07:46:08.399939+0000","last_became_active":"2024-09-21T07:46:06.211043+0000","last_became_peered":"2024-09-21T07:46:06.211043+0000","last_unstale":"2024-09-21T07:46:08.399939+0000","last_undegraded":"2024-09-21T07:46:08.399939+0000","last_fullsized":"2024-09-21T07:46:08.399939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:18.001419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521036+0000","last_change":"2024-09-21T07:46:07.218927+0000","last_active":"2024-09-21T07:46:08.521036+0000","last_peered":"2024-09-21T07:46:08.521036+0000","last_clean":"2024-09-21T07:46:08.521036+0000","last_became_active":"2024-09-21T07:46:07.218594+0000","last_became_peered":"2024-09-21T07:46:07.218594+0000","last_unstale":"2024-09-21T07:46:08.521036+0000","last_undegraded":"2024-09-21T07:46:08.521036+0000","last_fullsized":"2024-09-21T07:46:08.521036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032272+0000","last_change":"2024-09-21T07:46:06.208141+0000","last_active":"2024-09-21T07:46:09.032272+0000","last_peered":"2024-09-21T07:46:09.032272+0000","last_clean":"2024-09-21T07:46:09.032272+0000","last_became_active":"2024-09-21T07:46:06.207799+0000","last_became_peered":"2024-09-21T07:46:06.207799+0000","last_unstale":"2024-09-21T07:46:09.032272+0000","last_undegraded":"2024-09-21T07:46:09.032272+0000","last_fullsized":"2024-09-21T07:46:09.032272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:02.343379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208515+0000","last_change":"2024-09-21T07:46:07.208889+0000","last_active":"2024-09-21T07:46:08.208515+0000","last_peered":"2024-09-21T07:46:08.208515+0000","last_clean":"2024-09-21T07:46:08.208515+0000","last_became_active":"2024-09-21T07:46:07.208365+0000","last_became_peered":"2024-09-21T07:46:07.208365+0000","last_unstale":"2024-09-21T07:46:08.208515+0000","last_undegraded":"2024-09-21T07:46:08.208515+0000","last_fullsized":"2024-09-21T07:46:08.208515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399888+0000","last_change":"2024-09-21T07:46:06.212471+0000","last_active":"2024-09-21T07:46:08.399888+0000","last_peered":"2024-09-21T07:46:08.399888+0000","last_clean":"2024-09-21T07:46:08.399888+0000","last_became_active":"2024-09-21T07:46:06.212157+0000","last_became_peered":"2024-09-21T07:46:06.212157+0000","last_unstale":"2024-09-21T07:46:08.399888+0000","last_undegraded":"2024-09-21T07:46:08.399888+0000","last_fullsized":"2024-09-21T07:46:08.399888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199360+0000","last_change":"2024-09-21T07:46:07.234186+0000","last_active":"2024-09-21T07:46:09.199360+0000","last_peered":"2024-09-21T07:46:09.199360+0000","last_clean":"2024-09-21T07:46:09.199360+0000","last_became_active":"2024-09-21T07:46:07.231069+0000","last_became_peered":"2024-09-21T07:46:07.231069+0000","last_unstale":"2024-09-21T07:46:09.199360+0000","last_undegraded":"2024-09-21T07:46:09.199360+0000","last_fullsized":"2024-09-21T07:46:09.199360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209588+0000","last_change":"2024-09-21T07:46:06.213705+0000","last_active":"2024-09-21T07:46:08.209588+0000","last_peered":"2024-09-21T07:46:08.209588+0000","last_clean":"2024-09-21T07:46:08.209588+0000","last_became_active":"2024-09-21T07:46:06.213351+0000","last_became_peered":"2024-09-21T07:46:06.213351+0000","last_unstale":"2024-09-21T07:46:08.209588+0000","last_undegraded":"2024-09-21T07:46:08.209588+0000","last_fullsized":"2024-09-21T07:46:08.209588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205777+0000","last_change":"2024-09-21T07:46:07.216921+0000","last_active":"2024-09-21T07:46:08.205777+0000","last_peered":"2024-09-21T07:46:08.205777+0000","last_clean":"2024-09-21T07:46:08.205777+0000","last_became_active":"2024-09-21T07:46:07.216430+0000","last_became_peered":"2024-09-21T07:46:07.216430+0000","last_unstale":"2024-09-21T07:46:08.205777+0000","last_undegraded":"2024-09-21T07:46:08.205777+0000","last_fullsized":"2024-09-21T07:46:08.205777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:41.114984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399677+0000","last_change":"2024-09-21T07:46:06.208651+0000","last_active":"2024-09-21T07:46:08.399677+0000","last_peered":"2024-09-21T07:46:08.399677+0000","last_clean":"2024-09-21T07:46:08.399677+0000","last_became_active":"2024-09-21T07:46:06.206073+0000","last_became_peered":"2024-09-21T07:46:06.206073+0000","last_unstale":"2024-09-21T07:46:08.399677+0000","last_undegraded":"2024-09-21T07:46:08.399677+0000","last_fullsized":"2024-09-21T07:46:08.399677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:01.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521474+0000","last_change":"2024-09-21T07:46:07.225723+0000","last_active":"2024-09-21T07:46:08.521474+0000","last_peered":"2024-09-21T07:46:08.521474+0000","last_clean":"2024-09-21T07:46:08.521474+0000","last_became_active":"2024-09-21T07:46:07.225424+0000","last_became_peered":"2024-09-21T07:46:07.225424+0000","last_unstale":"2024-09-21T07:46:08.521474+0000","last_undegraded":"2024-09-21T07:46:08.521474+0000","last_fullsized":"2024-09-21T07:46:08.521474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.196020+0000","last_change":"2024-09-21T07:46:06.203823+0000","last_active":"2024-09-21T07:46:09.196020+0000","last_peered":"2024-09-21T07:46:09.196020+0000","last_clean":"2024-09-21T07:46:09.196020+0000","last_became_active":"2024-09-21T07:46:06.203500+0000","last_became_peered":"2024-09-21T07:46:06.203500+0000","last_unstale":"2024-09-21T07:46:09.196020+0000","last_undegraded":"2024-09-21T07:46:09.196020+0000","last_fullsized":"2024-09-21T07:46:09.196020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208233+0000","last_change":"2024-09-21T07:46:07.222640+0000","last_active":"2024-09-21T07:46:08.208233+0000","last_peered":"2024-09-21T07:46:08.208233+0000","last_clean":"2024-09-21T07:46:08.208233+0000","last_became_active":"2024-09-21T07:46:07.220945+0000","last_became_peered":"2024-09-21T07:46:07.220945+0000","last_unstale":"2024-09-21T07:46:08.208233+0000","last_undegraded":"2024-09-21T07:46:08.208233+0000","last_fullsized":"2024-09-21T07:46:08.208233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:38:12.923471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400048+0000","last_change":"2024-09-21T07:46:06.215628+0000","last_active":"2024-09-21T07:46:08.400048+0000","last_peered":"2024-09-21T07:46:08.400048+0000","last_clean":"2024-09-21T07:46:08.400048+0000","last_became_active":"2024-09-21T07:46:06.215341+0000","last_became_peered":"2024-09-21T07:46:06.215341+0000","last_unstale":"2024-09-21T07:46:08.400048+0000","last_undegraded":"2024-09-21T07:46:08.400048+0000","last_fullsized":"2024-09-21T07:46:08.400048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:47.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208617+0000","last_change":"2024-09-21T07:46:07.220815+0000","last_active":"2024-09-21T07:46:08.208617+0000","last_peered":"2024-09-21T07:46:08.208617+0000","last_clean":"2024-09-21T07:46:08.208617+0000","last_became_active":"2024-09-21T07:46:07.220441+0000","last_became_peered":"2024-09-21T07:46:07.220441+0000","last_unstale":"2024-09-21T07:46:08.208617+0000","last_undegraded":"2024-09-21T07:46:08.208617+0000","last_fullsized":"2024-09-21T07:46:08.208617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.198896+0000","last_change":"2024-09-21T07:46:06.212367+0000","last_active":"2024-09-21T07:46:09.198896+0000","last_peered":"2024-09-21T07:46:09.198896+0000","last_clean":"2024-09-21T07:46:09.198896+0000","last_became_active":"2024-09-21T07:46:06.211815+0000","last_became_peered":"2024-09-21T07:46:06.211815+0000","last_unstale":"2024-09-21T07:46:09.198896+0000","last_undegraded":"2024-09-21T07:46:09.198896+0000","last_fullsized":"2024-09-21T07:46:09.198896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:22.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399778+0000","last_change":"2024-09-21T07:46:07.224826+0000","last_active":"2024-09-21T07:46:08.399778+0000","last_peered":"2024-09-21T07:46:08.399778+0000","last_clean":"2024-09-21T07:46:08.399778+0000","last_became_active":"2024-09-21T07:46:07.221235+0000","last_became_peered":"2024-09-21T07:46:07.221235+0000","last_unstale":"2024-09-21T07:46:08.399778+0000","last_undegraded":"2024-09-21T07:46:08.399778+0000","last_fullsized":"2024-09-21T07:46:08.399778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:12.490015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207301+0000","last_change":"2024-09-21T07:46:06.206432+0000","last_active":"2024-09-21T07:46:08.207301+0000","last_peered":"2024-09-21T07:46:08.207301+0000","last_clean":"2024-09-21T07:46:08.207301+0000","last_became_active":"2024-09-21T07:46:06.203791+0000","last_became_peered":"2024-09-21T07:46:06.203791+0000","last_unstale":"2024-09-21T07:46:08.207301+0000","last_undegraded":"2024-09-21T07:46:08.207301+0000","last_fullsized":"2024-09-21T07:46:08.207301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207491+0000","last_change":"2024-09-21T07:46:07.214468+0000","last_active":"2024-09-21T07:46:08.207491+0000","last_peered":"2024-09-21T07:46:08.207491+0000","last_clean":"2024-09-21T07:46:08.207491+0000","last_became_active":"2024-09-21T07:46:07.213517+0000","last_became_peered":"2024-09-21T07:46:07.213517+0000","last_unstale":"2024-09-21T07:46:08.207491+0000","last_undegraded":"2024-09-21T07:46:08.207491+0000","last_fullsized":"2024-09-21T07:46:08.207491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521331+0000","last_change":"2024-09-21T07:46:06.211420+0000","last_active":"2024-09-21T07:46:08.521331+0000","last_peered":"2024-09-21T07:46:08.521331+0000","last_clean":"2024-09-21T07:46:08.521331+0000","last_became_active":"2024-09-21T07:46:06.208177+0000","last_became_peered":"2024-09-21T07:46:06.208177+0000","last_unstale":"2024-09-21T07:46:08.521331+0000","last_undegraded":"2024-09-21T07:46:08.521331+0000","last_fullsized":"2024-09-21T07:46:08.521331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032387+0000","last_change":"2024-09-21T07:46:07.213281+0000","last_active":"2024-09-21T07:46:09.032387+0000","last_peered":"2024-09-21T07:46:09.032387+0000","last_clean":"2024-09-21T07:46:09.032387+0000","last_became_active":"2024-09-21T07:46:07.212947+0000","last_became_peered":"2024-09-21T07:46:07.212947+0000","last_unstale":"2024-09-21T07:46:09.032387+0000","last_undegraded":"2024-09-21T07:46:09.032387+0000","last_fullsized":"2024-09-21T07:46:09.032387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:24.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399658+0000","last_change":"2024-09-21T07:46:06.206452+0000","last_active":"2024-09-21T07:46:08.399658+0000","last_peered":"2024-09-21T07:46:08.399658+0000","last_clean":"2024-09-21T07:46:08.399658+0000","last_became_active":"2024-09-21T07:46:06.206136+0000","last_became_peered":"2024-09-21T07:46:06.206136+0000","last_unstale":"2024-09-21T07:46:08.399658+0000","last_undegraded":"2024-09-21T07:46:08.399658+0000","last_fullsized":"2024-09-21T07:46:08.399658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:20.047491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400029+0000","last_change":"2024-09-21T07:46:07.211503+0000","last_active":"2024-09-21T07:46:08.400029+0000","last_peered":"2024-09-21T07:46:08.400029+0000","last_clean":"2024-09-21T07:46:08.400029+0000","last_became_active":"2024-09-21T07:46:07.211183+0000","last_became_peered":"2024-09-21T07:46:07.211183+0000","last_unstale":"2024-09-21T07:46:08.400029+0000","last_undegraded":"2024-09-21T07:46:08.400029+0000","last_fullsized":"2024-09-21T07:46:08.400029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:52.327032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399537+0000","last_change":"2024-09-21T07:46:07.228129+0000","last_active":"2024-09-21T07:46:08.399537+0000","last_peered":"2024-09-21T07:46:08.399537+0000","last_clean":"2024-09-21T07:46:08.399537+0000","last_became_active":"2024-09-21T07:46:07.227801+0000","last_became_peered":"2024-09-21T07:46:07.227801+0000","last_unstale":"2024-09-21T07:46:08.399537+0000","last_undegraded":"2024-09-21T07:46:08.399537+0000","last_fullsized":"2024-09-21T07:46:08.399537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.699234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199646+0000","last_change":"2024-09-21T07:46:06.217706+0000","last_active":"2024-09-21T07:46:09.199646+0000","last_peered":"2024-09-21T07:46:09.199646+0000","last_clean":"2024-09-21T07:46:09.199646+0000","last_became_active":"2024-09-21T07:46:06.215293+0000","last_became_peered":"2024-09-21T07:46:06.215293+0000","last_unstale":"2024-09-21T07:46:09.199646+0000","last_undegraded":"2024-09-21T07:46:09.199646+0000","last_fullsized":"2024-09-21T07:46:09.199646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:24.359404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500181+0000","last_change":"2024-09-21T07:46:07.232479+0000","last_active":"2024-09-21T07:46:09.500181+0000","last_peered":"2024-09-21T07:46:09.500181+0000","last_clean":"2024-09-21T07:46:09.500181+0000","last_became_active":"2024-09-21T07:46:07.232068+0000","last_became_peered":"2024-09-21T07:46:07.232068+0000","last_unstale":"2024-09-21T07:46:09.500181+0000","last_undegraded":"2024-09-21T07:46:09.500181+0000","last_fullsized":"2024-09-21T07:46:09.500181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:41.114984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199360+0000","last_change":"2024-09-21T07:46:06.210521+0000","last_active":"2024-09-21T07:46:09.199360+0000","last_peered":"2024-09-21T07:46:09.199360+0000","last_clean":"2024-09-21T07:46:09.199360+0000","last_became_active":"2024-09-21T07:46:06.210142+0000","last_became_peered":"2024-09-21T07:46:06.210142+0000","last_unstale":"2024-09-21T07:46:09.199360+0000","last_undegraded":"2024-09-21T07:46:09.199360+0000","last_fullsized":"2024-09-21T07:46:09.199360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:14:51.153791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208316+0000","last_change":"2024-09-21T07:46:07.210700+0000","last_active":"2024-09-21T07:46:08.208316+0000","last_peered":"2024-09-21T07:46:08.208316+0000","last_clean":"2024-09-21T07:46:08.208316+0000","last_became_active":"2024-09-21T07:46:07.203963+0000","last_became_peered":"2024-09-21T07:46:07.203963+0000","last_unstale":"2024-09-21T07:46:08.208316+0000","last_undegraded":"2024-09-21T07:46:08.208316+0000","last_fullsized":"2024-09-21T07:46:08.208316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209121+0000","last_change":"2024-09-21T07:46:06.209967+0000","last_active":"2024-09-21T07:46:08.209121+0000","last_peered":"2024-09-21T07:46:08.209121+0000","last_clean":"2024-09-21T07:46:08.209121+0000","last_became_active":"2024-09-21T07:46:06.207030+0000","last_became_peered":"2024-09-21T07:46:06.207030+0000","last_unstale":"2024-09-21T07:46:08.209121+0000","last_undegraded":"2024-09-21T07:46:08.209121+0000","last_fullsized":"2024-09-21T07:46:08.209121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:27.264294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209044+0000","last_change":"2024-09-21T07:46:07.219277+0000","last_active":"2024-09-21T07:46:08.209044+0000","last_peered":"2024-09-21T07:46:08.209044+0000","last_clean":"2024-09-21T07:46:08.209044+0000","last_became_active":"2024-09-21T07:46:07.218947+0000","last_became_peered":"2024-09-21T07:46:07.218947+0000","last_unstale":"2024-09-21T07:46:08.209044+0000","last_undegraded":"2024-09-21T07:46:08.209044+0000","last_fullsized":"2024-09-21T07:46:08.209044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208464+0000","last_change":"2024-09-21T07:46:06.209178+0000","last_active":"2024-09-21T07:46:08.208464+0000","last_peered":"2024-09-21T07:46:08.208464+0000","last_clean":"2024-09-21T07:46:08.208464+0000","last_became_active":"2024-09-21T07:46:06.201083+0000","last_became_peered":"2024-09-21T07:46:06.201083+0000","last_unstale":"2024-09-21T07:46:08.208464+0000","last_undegraded":"2024-09-21T07:46:08.208464+0000","last_fullsized":"2024-09-21T07:46:08.208464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:46:17.857 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 -- 172.21.15.88:0/438123867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a00ad0e0 msgr2=0x7fe8a00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 --2- 172.21.15.88:0/438123867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a00ad4f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fe888009830 tx=0x7fe88802a480 comp rx=0 tx=0).stop 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 -- 172.21.15.88:0/438123867 shutdown_connections 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 --2- 172.21.15.88:0/438123867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a00ad4f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 --2- 172.21.15.88:0/438123867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a015d760 0x7fe8a01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 -- 172.21.15.88:0/438123867 >> 172.21.15.88:0/438123867 conn(0x7fe8a006d730 msgr2=0x7fe8a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 -- 172.21.15.88:0/438123867 shutdown_connections 2024-09-21T07:46:18.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.111+0000 7fe8a6400640 1 -- 172.21.15.88:0/438123867 wait complete. 2024-09-21T07:46:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.113+0000 7fe8a6400640 1 Processor -- start 2024-09-21T07:46:18.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.114+0000 7fe8a6400640 1 -- start start 2024-09-21T07:46:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe8a6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a01567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe8a6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 0x7fe8a015baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe8a6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a0160310 0x7fe8a015bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe8a6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a006f220 con 0x7fe8a015d760 2024-09-21T07:46:18.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe8a6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a006f0a0 con 0x7fe8a00ad0e0 2024-09-21T07:46:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe8a6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe8a00b4c20 con 0x7fe8a0160310 2024-09-21T07:46:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe897e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 0x7fe8a015baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe897e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 0x7fe8a015baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48742/0 (socket says 172.21.15.88:48742) 2024-09-21T07:46:18.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.115+0000 7fe897e00640 1 -- 172.21.15.88:0/755461243 learned_addr learned my addr 172.21.15.88:0/755461243 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe89f400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a01567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe89fe00640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a0160310 0x7fe8a015bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897e00640 1 -- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a0160310 msgr2=0x7fe8a015bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897e00640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a0160310 0x7fe8a015bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897e00640 1 -- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8a00ad0e0 msgr2=0x7fe8a01567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897e00640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a01567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897e00640 1 -- 172.21.15.88:0/755461243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a01f5680 con 0x7fe8a015d760 2024-09-21T07:46:18.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe89f400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a01567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe89fe00640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a0160310 0x7fe8a015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897e00640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 0x7fe8a015baa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe888009950 tx=0x7fe888033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888005c90 con 0x7fe8a015d760 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe888031e50 con 0x7fe8a015d760 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe888033310 con 0x7fe8a015d760 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a01f5940 con 0x7fe8a015d760 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.116+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a01f5f00 con 0x7fe8a015d760 2024-09-21T07:46:18.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.118+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a0154370 con 0x7fe8a015d760 2024-09-21T07:46:18.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.119+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe8880334b0 con 0x7fe8a015d760 2024-09-21T07:46:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.120+0000 7fe897400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe87c078b90 0x7fe87c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.120+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe8880bc1d0 con 0x7fe8a015d760 2024-09-21T07:46:18.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.120+0000 7fe89f400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe87c078b90 0x7fe87c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.121+0000 7fe89f400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe87c078b90 0x7fe87c07b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe890009aa0 tx=0x7fe890005db0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.124+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe88803f030 con 0x7fe8a015d760 2024-09-21T07:46:18.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.307+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe8a0160fa0 con 0x7fe87c078b90 2024-09-21T07:46:18.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.312+0000 7fe897400640 1 -- 172.21.15.88:0/755461243 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251822 (secure 0 0 0) 0x7fe8a0160fa0 con 0x7fe87c078b90 2024-09-21T07:46:18.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:18.318 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:46:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe87c078b90 msgr2=0x7fe87c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe87c078b90 0x7fe87c07b0a0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe890009aa0 tx=0x7fe890005db0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 msgr2=0x7fe8a015baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 0x7fe8a015baa0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fe888009950 tx=0x7fe888033c90 comp rx=0 tx=0).stop 2024-09-21T07:46:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 shutdown_connections 2024-09-21T07:46:18.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe87c078b90 0x7fe87c07b0a0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe8a0160310 0x7fe8a015bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a015d760 0x7fe8a015baa0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 --2- 172.21.15.88:0/755461243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe8a00ad0e0 0x7fe8a01567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 >> 172.21.15.88:0/755461243 conn(0x7fe8a006d730 msgr2=0x7fe8a00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 shutdown_connections 2024-09-21T07:46:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.316+0000 7fe8a6400640 1 -- 172.21.15.88:0/755461243 wait complete. 2024-09-21T07:46:18.343 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2024-09-21T07:46:17.857176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219256,"kb_used_data":3016,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755464,"statfs":{"total":773094113280,"available":772869595136,"internally_reserved":0,"allocated":3088384,"data_stored":1824236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.002781"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199749+0000","last_change":"2024-09-21T07:46:07.234192+0000","last_active":"2024-09-21T07:46:09.199749+0000","last_peered":"2024-09-21T07:46:09.199749+0000","last_clean":"2024-09-21T07:46:09.199749+0000","last_became_active":"2024-09-21T07:46:07.231852+0000","last_became_peered":"2024-09-21T07:46:07.231852+0000","last_unstale":"2024-09-21T07:46:09.199749+0000","last_undegraded":"2024-09-21T07:46:09.199749+0000","last_fullsized":"2024-09-21T07:46:09.199749+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208860+0000","last_change":"2024-09-21T07:46:07.219201+0000","last_active":"2024-09-21T07:46:08.208860+0000","last_peered":"2024-09-21T07:46:08.208860+0000","last_clean":"2024-09-21T07:46:08.208860+0000","last_became_active":"2024-09-21T07:46:07.218520+0000","last_became_peered":"2024-09-21T07:46:07.218520+0000","last_unstale":"2024-09-21T07:46:08.208860+0000","last_undegraded":"2024-09-21T07:46:08.208860+0000","last_fullsized":"2024-09-21T07:46:08.208860+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:38.468021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208382+0000","last_change":"2024-09-21T07:46:07.225678+0000","last_active":"2024-09-21T07:46:08.208382+0000","last_peered":"2024-09-21T07:46:08.208382+0000","last_clean":"2024-09-21T07:46:08.208382+0000","last_became_active":"2024-09-21T07:46:07.225167+0000","last_became_peered":"2024-09-21T07:46:07.225167+0000","last_unstale":"2024-09-21T07:46:08.208382+0000","last_undegraded":"2024-09-21T07:46:08.208382+0000","last_fullsized":"2024-09-21T07:46:08.208382+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205764+0000","last_change":"2024-09-21T07:46:07.229463+0000","last_active":"2024-09-21T07:46:08.205764+0000","last_peered":"2024-09-21T07:46:08.205764+0000","last_clean":"2024-09-21T07:46:08.205764+0000","last_became_active":"2024-09-21T07:46:07.229040+0000","last_became_peered":"2024-09-21T07:46:07.229040+0000","last_unstale":"2024-09-21T07:46:08.205764+0000","last_undegraded":"2024-09-21T07:46:08.205764+0000","last_fullsized":"2024-09-21T07:46:08.205764+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:25.366313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199458+0000","last_change":"2024-09-21T07:46:07.234579+0000","last_active":"2024-09-21T07:46:09.199458+0000","last_peered":"2024-09-21T07:46:09.199458+0000","last_clean":"2024-09-21T07:46:09.199458+0000","last_became_active":"2024-09-21T07:46:07.233770+0000","last_became_peered":"2024-09-21T07:46:07.233770+0000","last_unstale":"2024-09-21T07:46:09.199458+0000","last_undegraded":"2024-09-21T07:46:09.199458+0000","last_fullsized":"2024-09-21T07:46:09.199458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205760+0000","last_change":"2024-09-21T07:46:07.222135+0000","last_active":"2024-09-21T07:46:08.205760+0000","last_peered":"2024-09-21T07:46:08.205760+0000","last_clean":"2024-09-21T07:46:08.205760+0000","last_became_active":"2024-09-21T07:46:07.221847+0000","last_became_peered":"2024-09-21T07:46:07.221847+0000","last_unstale":"2024-09-21T07:46:08.205760+0000","last_undegraded":"2024-09-21T07:46:08.205760+0000","last_fullsized":"2024-09-21T07:46:08.205760+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:09.010971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199222+0000","last_change":"2024-09-21T07:46:07.231015+0000","last_active":"2024-09-21T07:46:09.199222+0000","last_peered":"2024-09-21T07:46:09.199222+0000","last_clean":"2024-09-21T07:46:09.199222+0000","last_became_active":"2024-09-21T07:46:07.230719+0000","last_became_peered":"2024-09-21T07:46:07.230719+0000","last_unstale":"2024-09-21T07:46:09.199222+0000","last_undegraded":"2024-09-21T07:46:09.199222+0000","last_fullsized":"2024-09-21T07:46:09.199222+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208796+0000","last_change":"2024-09-21T07:46:07.217547+0000","last_active":"2024-09-21T07:46:08.208796+0000","last_peered":"2024-09-21T07:46:08.208796+0000","last_clean":"2024-09-21T07:46:08.208796+0000","last_became_active":"2024-09-21T07:46:07.215538+0000","last_became_peered":"2024-09-21T07:46:07.215538+0000","last_unstale":"2024-09-21T07:46:08.208796+0000","last_undegraded":"2024-09-21T07:46:08.208796+0000","last_fullsized":"2024-09-21T07:46:08.208796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:00.117867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500477+0000","last_change":"2024-09-21T07:46:07.201245+0000","last_active":"2024-09-21T07:46:09.500477+0000","last_peered":"2024-09-21T07:46:09.500477+0000","last_clean":"2024-09-21T07:46:09.500477+0000","last_became_active":"2024-09-21T07:46:07.200940+0000","last_became_peered":"2024-09-21T07:46:07.200940+0000","last_unstale":"2024-09-21T07:46:09.500477+0000","last_undegraded":"2024-09-21T07:46:09.500477+0000","last_fullsized":"2024-09-21T07:46:09.500477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:19.780073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199520+0000","last_change":"2024-09-21T07:46:07.226919+0000","last_active":"2024-09-21T07:46:09.199520+0000","last_peered":"2024-09-21T07:46:09.199520+0000","last_clean":"2024-09-21T07:46:09.199520+0000","last_became_active":"2024-09-21T07:46:07.226619+0000","last_became_peered":"2024-09-21T07:46:07.226619+0000","last_unstale":"2024-09-21T07:46:09.199520+0000","last_undegraded":"2024-09-21T07:46:09.199520+0000","last_fullsized":"2024-09-21T07:46:09.199520+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:42:47.470275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199269+0000","last_change":"2024-09-21T07:46:07.225286+0000","last_active":"2024-09-21T07:46:09.199269+0000","last_peered":"2024-09-21T07:46:09.199269+0000","last_clean":"2024-09-21T07:46:09.199269+0000","last_became_active":"2024-09-21T07:46:07.223018+0000","last_became_peered":"2024-09-21T07:46:07.223018+0000","last_unstale":"2024-09-21T07:46:09.199269+0000","last_undegraded":"2024-09-21T07:46:09.199269+0000","last_fullsized":"2024-09-21T07:46:09.199269+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199545+0000","last_change":"2024-09-21T07:46:07.229988+0000","last_active":"2024-09-21T07:46:09.199545+0000","last_peered":"2024-09-21T07:46:09.199545+0000","last_clean":"2024-09-21T07:46:09.199545+0000","last_became_active":"2024-09-21T07:46:07.229716+0000","last_became_peered":"2024-09-21T07:46:07.229716+0000","last_unstale":"2024-09-21T07:46:09.199545+0000","last_undegraded":"2024-09-21T07:46:09.199545+0000","last_fullsized":"2024-09-21T07:46:09.199545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521098+0000","last_change":"2024-09-21T07:46:07.238701+0000","last_active":"2024-09-21T07:46:08.521098+0000","last_peered":"2024-09-21T07:46:08.521098+0000","last_clean":"2024-09-21T07:46:08.521098+0000","last_became_active":"2024-09-21T07:46:07.238406+0000","last_became_peered":"2024-09-21T07:46:07.238406+0000","last_unstale":"2024-09-21T07:46:08.521098+0000","last_undegraded":"2024-09-21T07:46:08.521098+0000","last_fullsized":"2024-09-21T07:46:08.521098+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521245+0000","last_change":"2024-09-21T07:46:07.211337+0000","last_active":"2024-09-21T07:46:08.521245+0000","last_peered":"2024-09-21T07:46:08.521245+0000","last_clean":"2024-09-21T07:46:08.521245+0000","last_became_active":"2024-09-21T07:46:07.210983+0000","last_became_peered":"2024-09-21T07:46:07.210983+0000","last_unstale":"2024-09-21T07:46:08.521245+0000","last_undegraded":"2024-09-21T07:46:08.521245+0000","last_fullsized":"2024-09-21T07:46:08.521245+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:43.997921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209926+0000","last_change":"2024-09-21T07:46:07.225928+0000","last_active":"2024-09-21T07:46:08.209926+0000","last_peered":"2024-09-21T07:46:08.209926+0000","last_clean":"2024-09-21T07:46:08.209926+0000","last_became_active":"2024-09-21T07:46:07.225685+0000","last_became_peered":"2024-09-21T07:46:07.225685+0000","last_unstale":"2024-09-21T07:46:08.209926+0000","last_undegraded":"2024-09-21T07:46:08.209926+0000","last_fullsized":"2024-09-21T07:46:08.209926+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399827+0000","last_change":"2024-09-21T07:46:07.211743+0000","last_active":"2024-09-21T07:46:08.399827+0000","last_peered":"2024-09-21T07:46:08.399827+0000","last_clean":"2024-09-21T07:46:08.399827+0000","last_became_active":"2024-09-21T07:46:07.209957+0000","last_became_peered":"2024-09-21T07:46:07.209957+0000","last_unstale":"2024-09-21T07:46:08.399827+0000","last_undegraded":"2024-09-21T07:46:08.399827+0000","last_fullsized":"2024-09-21T07:46:08.399827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:18.660926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400124+0000","last_change":"2024-09-21T07:46:07.228920+0000","last_active":"2024-09-21T07:46:08.400124+0000","last_peered":"2024-09-21T07:46:08.400124+0000","last_clean":"2024-09-21T07:46:08.400124+0000","last_became_active":"2024-09-21T07:46:07.227861+0000","last_became_peered":"2024-09-21T07:46:07.227861+0000","last_unstale":"2024-09-21T07:46:08.400124+0000","last_undegraded":"2024-09-21T07:46:08.400124+0000","last_fullsized":"2024-09-21T07:46:08.400124+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:46:34.920768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032613+0000","last_change":"2024-09-21T07:46:07.221477+0000","last_active":"2024-09-21T07:46:09.032613+0000","last_peered":"2024-09-21T07:46:09.032613+0000","last_clean":"2024-09-21T07:46:09.032613+0000","last_became_active":"2024-09-21T07:46:07.217967+0000","last_became_peered":"2024-09-21T07:46:07.217967+0000","last_unstale":"2024-09-21T07:46:09.032613+0000","last_undegraded":"2024-09-21T07:46:09.032613+0000","last_fullsized":"2024-09-21T07:46:09.032613+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:34.633793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399789+0000","last_change":"2024-09-21T07:46:07.221262+0000","last_active":"2024-09-21T07:46:08.399789+0000","last_peered":"2024-09-21T07:46:08.399789+0000","last_clean":"2024-09-21T07:46:08.399789+0000","last_became_active":"2024-09-21T07:46:07.221018+0000","last_became_peered":"2024-09-21T07:46:07.221018+0000","last_unstale":"2024-09-21T07:46:08.399789+0000","last_undegraded":"2024-09-21T07:46:08.399789+0000","last_fullsized":"2024-09-21T07:46:08.399789+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:46.384301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199556+0000","last_change":"2024-09-21T07:46:07.228997+0000","last_active":"2024-09-21T07:46:09.199556+0000","last_peered":"2024-09-21T07:46:09.199556+0000","last_clean":"2024-09-21T07:46:09.199556+0000","last_became_active":"2024-09-21T07:46:07.227078+0000","last_became_peered":"2024-09-21T07:46:07.227078+0000","last_unstale":"2024-09-21T07:46:09.199556+0000","last_undegraded":"2024-09-21T07:46:09.199556+0000","last_fullsized":"2024-09-21T07:46:09.199556+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207714+0000","last_change":"2024-09-21T07:46:07.220220+0000","last_active":"2024-09-21T07:46:08.207714+0000","last_peered":"2024-09-21T07:46:08.207714+0000","last_clean":"2024-09-21T07:46:08.207714+0000","last_became_active":"2024-09-21T07:46:07.219890+0000","last_became_peered":"2024-09-21T07:46:07.219890+0000","last_unstale":"2024-09-21T07:46:08.207714+0000","last_undegraded":"2024-09-21T07:46:08.207714+0000","last_fullsized":"2024-09-21T07:46:08.207714+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499999+0000","last_change":"2024-09-21T07:46:07.227381+0000","last_active":"2024-09-21T07:46:09.499999+0000","last_peered":"2024-09-21T07:46:09.499999+0000","last_clean":"2024-09-21T07:46:09.499999+0000","last_became_active":"2024-09-21T07:46:07.225810+0000","last_became_peered":"2024-09-21T07:46:07.225810+0000","last_unstale":"2024-09-21T07:46:09.499999+0000","last_undegraded":"2024-09-21T07:46:09.499999+0000","last_fullsized":"2024-09-21T07:46:09.499999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:23.838546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.210100+0000","last_change":"2024-09-21T07:46:07.215158+0000","last_active":"2024-09-21T07:46:08.210100+0000","last_peered":"2024-09-21T07:46:08.210100+0000","last_clean":"2024-09-21T07:46:08.210100+0000","last_became_active":"2024-09-21T07:46:07.214842+0000","last_became_peered":"2024-09-21T07:46:07.214842+0000","last_unstale":"2024-09-21T07:46:08.210100+0000","last_undegraded":"2024-09-21T07:46:08.210100+0000","last_fullsized":"2024-09-21T07:46:08.210100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:50.833198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399861+0000","last_change":"2024-09-21T07:46:07.209932+0000","last_active":"2024-09-21T07:46:08.399861+0000","last_peered":"2024-09-21T07:46:08.399861+0000","last_clean":"2024-09-21T07:46:08.399861+0000","last_became_active":"2024-09-21T07:46:07.208985+0000","last_became_peered":"2024-09-21T07:46:07.208985+0000","last_unstale":"2024-09-21T07:46:08.399861+0000","last_undegraded":"2024-09-21T07:46:08.399861+0000","last_fullsized":"2024-09-21T07:46:08.399861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:32.406624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208288+0000","last_change":"2024-09-21T07:46:07.218857+0000","last_active":"2024-09-21T07:46:08.208288+0000","last_peered":"2024-09-21T07:46:08.208288+0000","last_clean":"2024-09-21T07:46:08.208288+0000","last_became_active":"2024-09-21T07:46:07.218490+0000","last_became_peered":"2024-09-21T07:46:07.218490+0000","last_unstale":"2024-09-21T07:46:08.208288+0000","last_undegraded":"2024-09-21T07:46:08.208288+0000","last_fullsized":"2024-09-21T07:46:08.208288+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:42.378122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206165+0000","last_change":"2024-09-21T07:46:07.218802+0000","last_active":"2024-09-21T07:46:08.206165+0000","last_peered":"2024-09-21T07:46:08.206165+0000","last_clean":"2024-09-21T07:46:08.206165+0000","last_became_active":"2024-09-21T07:46:07.216480+0000","last_became_peered":"2024-09-21T07:46:07.216480+0000","last_unstale":"2024-09-21T07:46:08.206165+0000","last_undegraded":"2024-09-21T07:46:08.206165+0000","last_fullsized":"2024-09-21T07:46:08.206165+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521582+0000","last_change":"2024-09-21T07:46:07.213848+0000","last_active":"2024-09-21T07:46:08.521582+0000","last_peered":"2024-09-21T07:46:08.521582+0000","last_clean":"2024-09-21T07:46:08.521582+0000","last_became_active":"2024-09-21T07:46:07.212350+0000","last_became_peered":"2024-09-21T07:46:07.212350+0000","last_unstale":"2024-09-21T07:46:08.521582+0000","last_undegraded":"2024-09-21T07:46:08.521582+0000","last_fullsized":"2024-09-21T07:46:08.521582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521460+0000","last_change":"2024-09-21T07:46:07.231247+0000","last_active":"2024-09-21T07:46:08.521460+0000","last_peered":"2024-09-21T07:46:08.521460+0000","last_clean":"2024-09-21T07:46:08.521460+0000","last_became_active":"2024-09-21T07:46:07.230913+0000","last_became_peered":"2024-09-21T07:46:07.230913+0000","last_unstale":"2024-09-21T07:46:08.521460+0000","last_undegraded":"2024-09-21T07:46:08.521460+0000","last_fullsized":"2024-09-21T07:46:08.521460+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208726+0000","last_change":"2024-09-21T07:46:07.212029+0000","last_active":"2024-09-21T07:46:08.208726+0000","last_peered":"2024-09-21T07:46:08.208726+0000","last_clean":"2024-09-21T07:46:08.208726+0000","last_became_active":"2024-09-21T07:46:07.211701+0000","last_became_peered":"2024-09-21T07:46:07.211701+0000","last_unstale":"2024-09-21T07:46:08.208726+0000","last_undegraded":"2024-09-21T07:46:08.208726+0000","last_fullsized":"2024-09-21T07:46:08.208726+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:04.684334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399383+0000","last_change":"2024-09-21T07:46:07.209431+0000","last_active":"2024-09-21T07:46:08.399383+0000","last_peered":"2024-09-21T07:46:08.399383+0000","last_clean":"2024-09-21T07:46:08.399383+0000","last_became_active":"2024-09-21T07:46:07.208558+0000","last_became_peered":"2024-09-21T07:46:07.208558+0000","last_unstale":"2024-09-21T07:46:08.399383+0000","last_undegraded":"2024-09-21T07:46:08.399383+0000","last_fullsized":"2024-09-21T07:46:08.399383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:38.943851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032488+0000","last_change":"2024-09-21T07:46:07.213929+0000","last_active":"2024-09-21T07:46:09.032488+0000","last_peered":"2024-09-21T07:46:09.032488+0000","last_clean":"2024-09-21T07:46:09.032488+0000","last_became_active":"2024-09-21T07:46:07.213456+0000","last_became_peered":"2024-09-21T07:46:07.213456+0000","last_unstale":"2024-09-21T07:46:09.032488+0000","last_undegraded":"2024-09-21T07:46:09.032488+0000","last_fullsized":"2024-09-21T07:46:09.032488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:08.925890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205701+0000","last_change":"2024-09-21T07:46:07.218008+0000","last_active":"2024-09-21T07:46:08.205701+0000","last_peered":"2024-09-21T07:46:08.205701+0000","last_clean":"2024-09-21T07:46:08.205701+0000","last_became_active":"2024-09-21T07:46:07.217294+0000","last_became_peered":"2024-09-21T07:46:07.217294+0000","last_unstale":"2024-09-21T07:46:08.205701+0000","last_undegraded":"2024-09-21T07:46:08.205701+0000","last_fullsized":"2024-09-21T07:46:08.205701+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:48.673151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500036+0000","last_change":"2024-09-21T07:46:07.203504+0000","last_active":"2024-09-21T07:46:09.500036+0000","last_peered":"2024-09-21T07:46:09.500036+0000","last_clean":"2024-09-21T07:46:09.500036+0000","last_became_active":"2024-09-21T07:46:07.203135+0000","last_became_peered":"2024-09-21T07:46:07.203135+0000","last_unstale":"2024-09-21T07:46:09.500036+0000","last_undegraded":"2024-09-21T07:46:09.500036+0000","last_fullsized":"2024-09-21T07:46:09.500036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:11.674361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1b","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208870+0000","last_change":"2024-09-21T07:46:06.210978+0000","last_active":"2024-09-21T07:46:08.208870+0000","last_peered":"2024-09-21T07:46:08.208870+0000","last_clean":"2024-09-21T07:46:08.208870+0000","last_became_active":"2024-09-21T07:46:06.210615+0000","last_became_peered":"2024-09-21T07:46:06.210615+0000","last_unstale":"2024-09-21T07:46:08.208870+0000","last_undegraded":"2024-09-21T07:46:08.208870+0000","last_fullsized":"2024-09-21T07:46:08.208870+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:30:12.178363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207933+0000","last_change":"2024-09-21T07:46:07.236139+0000","last_active":"2024-09-21T07:46:08.207933+0000","last_peered":"2024-09-21T07:46:08.207933+0000","last_clean":"2024-09-21T07:46:08.207933+0000","last_became_active":"2024-09-21T07:46:07.235357+0000","last_became_peered":"2024-09-21T07:46:07.235357+0000","last_unstale":"2024-09-21T07:46:08.207933+0000","last_undegraded":"2024-09-21T07:46:08.207933+0000","last_fullsized":"2024-09-21T07:46:08.207933+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399662+0000","last_change":"2024-09-21T07:46:06.218149+0000","last_active":"2024-09-21T07:46:08.399662+0000","last_peered":"2024-09-21T07:46:08.399662+0000","last_clean":"2024-09-21T07:46:08.399662+0000","last_became_active":"2024-09-21T07:46:06.217846+0000","last_became_peered":"2024-09-21T07:46:06.217846+0000","last_unstale":"2024-09-21T07:46:08.399662+0000","last_undegraded":"2024-09-21T07:46:08.399662+0000","last_fullsized":"2024-09-21T07:46:08.399662+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.520888+0000","last_change":"2024-09-21T07:46:06.213787+0000","last_active":"2024-09-21T07:46:08.520888+0000","last_peered":"2024-09-21T07:46:08.520888+0000","last_clean":"2024-09-21T07:46:08.520888+0000","last_became_active":"2024-09-21T07:46:06.213454+0000","last_became_peered":"2024-09-21T07:46:06.213454+0000","last_unstale":"2024-09-21T07:46:08.520888+0000","last_undegraded":"2024-09-21T07:46:08.520888+0000","last_fullsized":"2024-09-21T07:46:08.520888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208744+0000","last_change":"2024-09-21T07:46:07.217173+0000","last_active":"2024-09-21T07:46:08.208744+0000","last_peered":"2024-09-21T07:46:08.208744+0000","last_clean":"2024-09-21T07:46:08.208744+0000","last_became_active":"2024-09-21T07:46:07.216857+0000","last_became_peered":"2024-09-21T07:46:07.216857+0000","last_unstale":"2024-09-21T07:46:08.208744+0000","last_undegraded":"2024-09-21T07:46:08.208744+0000","last_fullsized":"2024-09-21T07:46:08.208744+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399914+0000","last_change":"2024-09-21T07:46:06.210041+0000","last_active":"2024-09-21T07:46:08.399914+0000","last_peered":"2024-09-21T07:46:08.399914+0000","last_clean":"2024-09-21T07:46:08.399914+0000","last_became_active":"2024-09-21T07:46:06.209744+0000","last_became_peered":"2024-09-21T07:46:06.209744+0000","last_unstale":"2024-09-21T07:46:08.399914+0000","last_undegraded":"2024-09-21T07:46:08.399914+0000","last_fullsized":"2024-09-21T07:46:08.399914+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207314+0000","last_change":"2024-09-21T07:46:07.217122+0000","last_active":"2024-09-21T07:46:08.207314+0000","last_peered":"2024-09-21T07:46:08.207314+0000","last_clean":"2024-09-21T07:46:08.207314+0000","last_became_active":"2024-09-21T07:46:07.216831+0000","last_became_peered":"2024-09-21T07:46:07.216831+0000","last_unstale":"2024-09-21T07:46:08.207314+0000","last_undegraded":"2024-09-21T07:46:08.207314+0000","last_fullsized":"2024-09-21T07:46:08.207314+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:54.268520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500434+0000","last_change":"2024-09-21T07:46:06.202767+0000","last_active":"2024-09-21T07:46:09.500434+0000","last_peered":"2024-09-21T07:46:09.500434+0000","last_clean":"2024-09-21T07:46:09.500434+0000","last_became_active":"2024-09-21T07:46:06.200291+0000","last_became_peered":"2024-09-21T07:46:06.200291+0000","last_unstale":"2024-09-21T07:46:09.500434+0000","last_undegraded":"2024-09-21T07:46:09.500434+0000","last_fullsized":"2024-09-21T07:46:09.500434+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:47.513196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206404+0000","last_change":"2024-09-21T07:46:07.216776+0000","last_active":"2024-09-21T07:46:08.206404+0000","last_peered":"2024-09-21T07:46:08.206404+0000","last_clean":"2024-09-21T07:46:08.206404+0000","last_became_active":"2024-09-21T07:46:07.214952+0000","last_became_peered":"2024-09-21T07:46:07.214952+0000","last_unstale":"2024-09-21T07:46:08.206404+0000","last_undegraded":"2024-09-21T07:46:08.206404+0000","last_fullsized":"2024-09-21T07:46:08.206404+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:21.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199670+0000","last_change":"2024-09-21T07:46:06.210647+0000","last_active":"2024-09-21T07:46:09.199670+0000","last_peered":"2024-09-21T07:46:09.199670+0000","last_clean":"2024-09-21T07:46:09.199670+0000","last_became_active":"2024-09-21T07:46:06.209590+0000","last_became_peered":"2024-09-21T07:46:06.209590+0000","last_unstale":"2024-09-21T07:46:09.199670+0000","last_undegraded":"2024-09-21T07:46:09.199670+0000","last_fullsized":"2024-09-21T07:46:09.199670+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:48:40.259455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208537+0000","last_change":"2024-09-21T07:46:07.210270+0000","last_active":"2024-09-21T07:46:08.208537+0000","last_peered":"2024-09-21T07:46:08.208537+0000","last_clean":"2024-09-21T07:46:08.208537+0000","last_became_active":"2024-09-21T07:46:07.202021+0000","last_became_peered":"2024-09-21T07:46:07.202021+0000","last_unstale":"2024-09-21T07:46:08.208537+0000","last_undegraded":"2024-09-21T07:46:08.208537+0000","last_fullsized":"2024-09-21T07:46:08.208537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.15","version":"22'3","reported_seq":18,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032615+0000","last_change":"2024-09-21T07:46:06.212266+0000","last_active":"2024-09-21T07:46:09.032615+0000","last_peered":"2024-09-21T07:46:09.032615+0000","last_clean":"2024-09-21T07:46:09.032615+0000","last_became_active":"2024-09-21T07:46:06.210122+0000","last_became_peered":"2024-09-21T07:46:06.210122+0000","last_unstale":"2024-09-21T07:46:09.032615+0000","last_undegraded":"2024-09-21T07:46:09.032615+0000","last_fullsized":"2024-09-21T07:46:09.032615+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:13.095998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399665+0000","last_change":"2024-09-21T07:46:07.216761+0000","last_active":"2024-09-21T07:46:08.399665+0000","last_peered":"2024-09-21T07:46:08.399665+0000","last_clean":"2024-09-21T07:46:08.399665+0000","last_became_active":"2024-09-21T07:46:07.215637+0000","last_became_peered":"2024-09-21T07:46:07.215637+0000","last_unstale":"2024-09-21T07:46:08.399665+0000","last_undegraded":"2024-09-21T07:46:08.399665+0000","last_fullsized":"2024-09-21T07:46:08.399665+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:48.803562+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.14","version":"22'6","reported_seq":21,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205818+0000","last_change":"2024-09-21T07:46:06.206864+0000","last_active":"2024-09-21T07:46:08.205818+0000","last_peered":"2024-09-21T07:46:08.205818+0000","last_clean":"2024-09-21T07:46:08.205818+0000","last_became_active":"2024-09-21T07:46:06.206545+0000","last_became_peered":"2024-09-21T07:46:06.206545+0000","last_unstale":"2024-09-21T07:46:08.205818+0000","last_undegraded":"2024-09-21T07:46:08.205818+0000","last_fullsized":"2024-09-21T07:46:08.205818+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:00.329415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199512+0000","last_change":"2024-09-21T07:46:07.236117+0000","last_active":"2024-09-21T07:46:09.199512+0000","last_peered":"2024-09-21T07:46:09.199512+0000","last_clean":"2024-09-21T07:46:09.199512+0000","last_became_active":"2024-09-21T07:46:07.233496+0000","last_became_peered":"2024-09-21T07:46:07.233496+0000","last_unstale":"2024-09-21T07:46:09.199512+0000","last_undegraded":"2024-09-21T07:46:09.199512+0000","last_fullsized":"2024-09-21T07:46:09.199512+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:23.747478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.13","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032504+0000","last_change":"2024-09-21T07:46:06.212633+0000","last_active":"2024-09-21T07:46:09.032504+0000","last_peered":"2024-09-21T07:46:09.032504+0000","last_clean":"2024-09-21T07:46:09.032504+0000","last_became_active":"2024-09-21T07:46:06.212240+0000","last_became_peered":"2024-09-21T07:46:06.212240+0000","last_unstale":"2024-09-21T07:46:09.032504+0000","last_undegraded":"2024-09-21T07:46:09.032504+0000","last_fullsized":"2024-09-21T07:46:09.032504+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:36.198191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521167+0000","last_change":"2024-09-21T07:46:07.216368+0000","last_active":"2024-09-21T07:46:08.521167+0000","last_peered":"2024-09-21T07:46:08.521167+0000","last_clean":"2024-09-21T07:46:08.521167+0000","last_became_active":"2024-09-21T07:46:07.216061+0000","last_became_peered":"2024-09-21T07:46:07.216061+0000","last_unstale":"2024-09-21T07:46:08.521167+0000","last_undegraded":"2024-09-21T07:46:08.521167+0000","last_fullsized":"2024-09-21T07:46:08.521167+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399781+0000","last_change":"2024-09-21T07:46:06.204421+0000","last_active":"2024-09-21T07:46:08.399781+0000","last_peered":"2024-09-21T07:46:08.399781+0000","last_clean":"2024-09-21T07:46:08.399781+0000","last_became_active":"2024-09-21T07:46:06.204109+0000","last_became_peered":"2024-09-21T07:46:06.204109+0000","last_unstale":"2024-09-21T07:46:08.399781+0000","last_undegraded":"2024-09-21T07:46:08.399781+0000","last_fullsized":"2024-09-21T07:46:08.399781+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:10.667453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399577+0000","last_change":"2024-09-21T07:46:07.237847+0000","last_active":"2024-09-21T07:46:08.399577+0000","last_peered":"2024-09-21T07:46:08.399577+0000","last_clean":"2024-09-21T07:46:08.399577+0000","last_became_active":"2024-09-21T07:46:07.236995+0000","last_became_peered":"2024-09-21T07:46:07.236995+0000","last_unstale":"2024-09-21T07:46:08.399577+0000","last_undegraded":"2024-09-21T07:46:08.399577+0000","last_fullsized":"2024-09-21T07:46:08.399577+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:53:19.395558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199277+0000","last_change":"2024-09-21T07:46:06.218044+0000","last_active":"2024-09-21T07:46:09.199277+0000","last_peered":"2024-09-21T07:46:09.199277+0000","last_clean":"2024-09-21T07:46:09.199277+0000","last_became_active":"2024-09-21T07:46:06.217446+0000","last_became_peered":"2024-09-21T07:46:06.217446+0000","last_unstale":"2024-09-21T07:46:09.199277+0000","last_undegraded":"2024-09-21T07:46:09.199277+0000","last_fullsized":"2024-09-21T07:46:09.199277+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:47.666243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499878+0000","last_change":"2024-09-21T07:46:07.210885+0000","last_active":"2024-09-21T07:46:09.499878+0000","last_peered":"2024-09-21T07:46:09.499878+0000","last_clean":"2024-09-21T07:46:09.499878+0000","last_became_active":"2024-09-21T07:46:07.210591+0000","last_became_peered":"2024-09-21T07:46:07.210591+0000","last_unstale":"2024-09-21T07:46:09.499878+0000","last_undegraded":"2024-09-21T07:46:09.499878+0000","last_fullsized":"2024-09-21T07:46:09.499878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:33.666658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.10","version":"22'8","reported_seq":23,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207016+0000","last_change":"2024-09-21T07:46:06.202414+0000","last_active":"2024-09-21T07:46:08.207016+0000","last_peered":"2024-09-21T07:46:08.207016+0000","last_clean":"2024-09-21T07:46:08.207016+0000","last_became_active":"2024-09-21T07:46:06.201528+0000","last_became_peered":"2024-09-21T07:46:06.201528+0000","last_unstale":"2024-09-21T07:46:08.207016+0000","last_undegraded":"2024-09-21T07:46:08.207016+0000","last_fullsized":"2024-09-21T07:46:08.207016+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399393+0000","last_change":"2024-09-21T07:46:07.226232+0000","last_active":"2024-09-21T07:46:08.399393+0000","last_peered":"2024-09-21T07:46:08.399393+0000","last_clean":"2024-09-21T07:46:08.399393+0000","last_became_active":"2024-09-21T07:46:07.224513+0000","last_became_peered":"2024-09-21T07:46:07.224513+0000","last_unstale":"2024-09-21T07:46:08.399393+0000","last_undegraded":"2024-09-21T07:46:08.399393+0000","last_fullsized":"2024-09-21T07:46:08.399393+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:17.528901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199798+0000","last_change":"2024-09-21T07:46:06.219146+0000","last_active":"2024-09-21T07:46:09.199798+0000","last_peered":"2024-09-21T07:46:09.199798+0000","last_clean":"2024-09-21T07:46:09.199798+0000","last_became_active":"2024-09-21T07:46:06.218899+0000","last_became_peered":"2024-09-21T07:46:06.218899+0000","last_unstale":"2024-09-21T07:46:09.199798+0000","last_undegraded":"2024-09-21T07:46:09.199798+0000","last_fullsized":"2024-09-21T07:46:09.199798+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:08.004063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.206636+0000","last_change":"2024-09-21T07:46:07.235918+0000","last_active":"2024-09-21T07:46:08.206636+0000","last_peered":"2024-09-21T07:46:08.206636+0000","last_clean":"2024-09-21T07:46:08.206636+0000","last_became_active":"2024-09-21T07:46:07.235596+0000","last_became_peered":"2024-09-21T07:46:07.235596+0000","last_unstale":"2024-09-21T07:46:08.206636+0000","last_undegraded":"2024-09-21T07:46:08.206636+0000","last_fullsized":"2024-09-21T07:46:08.206636+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:23.403670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.e","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208692+0000","last_change":"2024-09-21T07:46:06.208887+0000","last_active":"2024-09-21T07:46:08.208692+0000","last_peered":"2024-09-21T07:46:08.208692+0000","last_clean":"2024-09-21T07:46:08.208692+0000","last_became_active":"2024-09-21T07:46:06.208508+0000","last_became_peered":"2024-09-21T07:46:06.208508+0000","last_unstale":"2024-09-21T07:46:08.208692+0000","last_undegraded":"2024-09-21T07:46:08.208692+0000","last_fullsized":"2024-09-21T07:46:08.208692+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:32.659749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199820+0000","last_change":"2024-09-21T07:46:07.225748+0000","last_active":"2024-09-21T07:46:09.199820+0000","last_peered":"2024-09-21T07:46:09.199820+0000","last_clean":"2024-09-21T07:46:09.199820+0000","last_became_active":"2024-09-21T07:46:07.225406+0000","last_became_peered":"2024-09-21T07:46:07.225406+0000","last_unstale":"2024-09-21T07:46:09.199820+0000","last_undegraded":"2024-09-21T07:46:09.199820+0000","last_fullsized":"2024-09-21T07:46:09.199820+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.699234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208511+0000","last_change":"2024-09-21T07:46:06.209460+0000","last_active":"2024-09-21T07:46:08.208511+0000","last_peered":"2024-09-21T07:46:08.208511+0000","last_clean":"2024-09-21T07:46:08.208511+0000","last_became_active":"2024-09-21T07:46:06.207853+0000","last_became_peered":"2024-09-21T07:46:06.207853+0000","last_unstale":"2024-09-21T07:46:08.208511+0000","last_undegraded":"2024-09-21T07:46:08.208511+0000","last_fullsized":"2024-09-21T07:46:08.208511+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199786+0000","last_change":"2024-09-21T07:46:07.228685+0000","last_active":"2024-09-21T07:46:09.199786+0000","last_peered":"2024-09-21T07:46:09.199786+0000","last_clean":"2024-09-21T07:46:09.199786+0000","last_became_active":"2024-09-21T07:46:07.226780+0000","last_became_peered":"2024-09-21T07:46:07.226780+0000","last_unstale":"2024-09-21T07:46:09.199786+0000","last_undegraded":"2024-09-21T07:46:09.199786+0000","last_fullsized":"2024-09-21T07:46:09.199786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:32.406624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521589+0000","last_change":"2024-09-21T07:46:06.216893+0000","last_active":"2024-09-21T07:46:08.521589+0000","last_peered":"2024-09-21T07:46:08.521589+0000","last_clean":"2024-09-21T07:46:08.521589+0000","last_became_active":"2024-09-21T07:46:06.216657+0000","last_became_peered":"2024-09-21T07:46:06.216657+0000","last_unstale":"2024-09-21T07:46:08.521589+0000","last_undegraded":"2024-09-21T07:46:08.521589+0000","last_fullsized":"2024-09-21T07:46:08.521589+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:01.635309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399601+0000","last_change":"2024-09-21T07:46:07.210978+0000","last_active":"2024-09-21T07:46:08.399601+0000","last_peered":"2024-09-21T07:46:08.399601+0000","last_clean":"2024-09-21T07:46:08.399601+0000","last_became_active":"2024-09-21T07:46:07.210665+0000","last_became_peered":"2024-09-21T07:46:07.210665+0000","last_unstale":"2024-09-21T07:46:08.399601+0000","last_undegraded":"2024-09-21T07:46:08.399601+0000","last_fullsized":"2024-09-21T07:46:08.399601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:57.801044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":111,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.499736+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:09.499736+0000","last_peered":"2024-09-21T07:46:09.499736+0000","last_clean":"2024-09-21T07:46:09.499736+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:09.499736+0000","last_undegraded":"2024-09-21T07:46:09.499736+0000","last_fullsized":"2024-09-21T07:46:09.499736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399734+0000","last_change":"2024-09-21T07:46:06.204733+0000","last_active":"2024-09-21T07:46:08.399734+0000","last_peered":"2024-09-21T07:46:08.399734+0000","last_clean":"2024-09-21T07:46:08.399734+0000","last_became_active":"2024-09-21T07:46:06.204416+0000","last_became_peered":"2024-09-21T07:46:06.204416+0000","last_unstale":"2024-09-21T07:46:08.399734+0000","last_undegraded":"2024-09-21T07:46:08.399734+0000","last_fullsized":"2024-09-21T07:46:08.399734+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:18.773164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207407+0000","last_change":"2024-09-21T07:46:07.222509+0000","last_active":"2024-09-21T07:46:08.207407+0000","last_peered":"2024-09-21T07:46:08.207407+0000","last_clean":"2024-09-21T07:46:08.207407+0000","last_became_active":"2024-09-21T07:46:07.221105+0000","last_became_peered":"2024-09-21T07:46:07.221105+0000","last_unstale":"2024-09-21T07:46:08.207407+0000","last_undegraded":"2024-09-21T07:46:08.207407+0000","last_fullsized":"2024-09-21T07:46:08.207407+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399939+0000","last_change":"2024-09-21T07:46:06.212101+0000","last_active":"2024-09-21T07:46:08.399939+0000","last_peered":"2024-09-21T07:46:08.399939+0000","last_clean":"2024-09-21T07:46:08.399939+0000","last_became_active":"2024-09-21T07:46:06.211043+0000","last_became_peered":"2024-09-21T07:46:06.211043+0000","last_unstale":"2024-09-21T07:46:08.399939+0000","last_undegraded":"2024-09-21T07:46:08.399939+0000","last_fullsized":"2024-09-21T07:46:08.399939+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:18.001419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521036+0000","last_change":"2024-09-21T07:46:07.218927+0000","last_active":"2024-09-21T07:46:08.521036+0000","last_peered":"2024-09-21T07:46:08.521036+0000","last_clean":"2024-09-21T07:46:08.521036+0000","last_became_active":"2024-09-21T07:46:07.218594+0000","last_became_peered":"2024-09-21T07:46:07.218594+0000","last_unstale":"2024-09-21T07:46:08.521036+0000","last_undegraded":"2024-09-21T07:46:08.521036+0000","last_fullsized":"2024-09-21T07:46:08.521036+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:37:38.711473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032272+0000","last_change":"2024-09-21T07:46:06.208141+0000","last_active":"2024-09-21T07:46:09.032272+0000","last_peered":"2024-09-21T07:46:09.032272+0000","last_clean":"2024-09-21T07:46:09.032272+0000","last_became_active":"2024-09-21T07:46:06.207799+0000","last_became_peered":"2024-09-21T07:46:06.207799+0000","last_unstale":"2024-09-21T07:46:09.032272+0000","last_undegraded":"2024-09-21T07:46:09.032272+0000","last_fullsized":"2024-09-21T07:46:09.032272+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:02.343379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208515+0000","last_change":"2024-09-21T07:46:07.208889+0000","last_active":"2024-09-21T07:46:08.208515+0000","last_peered":"2024-09-21T07:46:08.208515+0000","last_clean":"2024-09-21T07:46:08.208515+0000","last_became_active":"2024-09-21T07:46:07.208365+0000","last_became_peered":"2024-09-21T07:46:07.208365+0000","last_unstale":"2024-09-21T07:46:08.208515+0000","last_undegraded":"2024-09-21T07:46:08.208515+0000","last_fullsized":"2024-09-21T07:46:08.208515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:45.974915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399888+0000","last_change":"2024-09-21T07:46:06.212471+0000","last_active":"2024-09-21T07:46:08.399888+0000","last_peered":"2024-09-21T07:46:08.399888+0000","last_clean":"2024-09-21T07:46:08.399888+0000","last_became_active":"2024-09-21T07:46:06.212157+0000","last_became_peered":"2024-09-21T07:46:06.212157+0000","last_unstale":"2024-09-21T07:46:08.399888+0000","last_undegraded":"2024-09-21T07:46:08.399888+0000","last_fullsized":"2024-09-21T07:46:08.399888+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199360+0000","last_change":"2024-09-21T07:46:07.234186+0000","last_active":"2024-09-21T07:46:09.199360+0000","last_peered":"2024-09-21T07:46:09.199360+0000","last_clean":"2024-09-21T07:46:09.199360+0000","last_became_active":"2024-09-21T07:46:07.231069+0000","last_became_peered":"2024-09-21T07:46:07.231069+0000","last_unstale":"2024-09-21T07:46:09.199360+0000","last_undegraded":"2024-09-21T07:46:09.199360+0000","last_fullsized":"2024-09-21T07:46:09.199360+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:05.674987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209588+0000","last_change":"2024-09-21T07:46:06.213705+0000","last_active":"2024-09-21T07:46:08.209588+0000","last_peered":"2024-09-21T07:46:08.209588+0000","last_clean":"2024-09-21T07:46:08.209588+0000","last_became_active":"2024-09-21T07:46:06.213351+0000","last_became_peered":"2024-09-21T07:46:06.213351+0000","last_unstale":"2024-09-21T07:46:08.209588+0000","last_undegraded":"2024-09-21T07:46:08.209588+0000","last_fullsized":"2024-09-21T07:46:08.209588+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.205777+0000","last_change":"2024-09-21T07:46:07.216921+0000","last_active":"2024-09-21T07:46:08.205777+0000","last_peered":"2024-09-21T07:46:08.205777+0000","last_clean":"2024-09-21T07:46:08.205777+0000","last_became_active":"2024-09-21T07:46:07.216430+0000","last_became_peered":"2024-09-21T07:46:07.216430+0000","last_unstale":"2024-09-21T07:46:08.205777+0000","last_undegraded":"2024-09-21T07:46:08.205777+0000","last_fullsized":"2024-09-21T07:46:08.205777+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:41.114984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399677+0000","last_change":"2024-09-21T07:46:06.208651+0000","last_active":"2024-09-21T07:46:08.399677+0000","last_peered":"2024-09-21T07:46:08.399677+0000","last_clean":"2024-09-21T07:46:08.399677+0000","last_became_active":"2024-09-21T07:46:06.206073+0000","last_became_peered":"2024-09-21T07:46:06.206073+0000","last_unstale":"2024-09-21T07:46:08.399677+0000","last_undegraded":"2024-09-21T07:46:08.399677+0000","last_fullsized":"2024-09-21T07:46:08.399677+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:01.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521474+0000","last_change":"2024-09-21T07:46:07.225723+0000","last_active":"2024-09-21T07:46:08.521474+0000","last_peered":"2024-09-21T07:46:08.521474+0000","last_clean":"2024-09-21T07:46:08.521474+0000","last_became_active":"2024-09-21T07:46:07.225424+0000","last_became_peered":"2024-09-21T07:46:07.225424+0000","last_unstale":"2024-09-21T07:46:08.521474+0000","last_undegraded":"2024-09-21T07:46:08.521474+0000","last_fullsized":"2024-09-21T07:46:08.521474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:03.397828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"23'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.196020+0000","last_change":"2024-09-21T07:46:06.203823+0000","last_active":"2024-09-21T07:46:09.196020+0000","last_peered":"2024-09-21T07:46:09.196020+0000","last_clean":"2024-09-21T07:46:09.196020+0000","last_became_active":"2024-09-21T07:46:06.203500+0000","last_became_peered":"2024-09-21T07:46:06.203500+0000","last_unstale":"2024-09-21T07:46:09.196020+0000","last_undegraded":"2024-09-21T07:46:09.196020+0000","last_fullsized":"2024-09-21T07:46:09.196020+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:22.831637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208233+0000","last_change":"2024-09-21T07:46:07.222640+0000","last_active":"2024-09-21T07:46:08.208233+0000","last_peered":"2024-09-21T07:46:08.208233+0000","last_clean":"2024-09-21T07:46:08.208233+0000","last_became_active":"2024-09-21T07:46:07.220945+0000","last_became_peered":"2024-09-21T07:46:07.220945+0000","last_unstale":"2024-09-21T07:46:08.208233+0000","last_undegraded":"2024-09-21T07:46:08.208233+0000","last_fullsized":"2024-09-21T07:46:08.208233+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:38:12.923471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400048+0000","last_change":"2024-09-21T07:46:06.215628+0000","last_active":"2024-09-21T07:46:08.400048+0000","last_peered":"2024-09-21T07:46:08.400048+0000","last_clean":"2024-09-21T07:46:08.400048+0000","last_became_active":"2024-09-21T07:46:06.215341+0000","last_became_peered":"2024-09-21T07:46:06.215341+0000","last_unstale":"2024-09-21T07:46:08.400048+0000","last_undegraded":"2024-09-21T07:46:08.400048+0000","last_fullsized":"2024-09-21T07:46:08.400048+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:47.831344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208617+0000","last_change":"2024-09-21T07:46:07.220815+0000","last_active":"2024-09-21T07:46:08.208617+0000","last_peered":"2024-09-21T07:46:08.208617+0000","last_clean":"2024-09-21T07:46:08.208617+0000","last_became_active":"2024-09-21T07:46:07.220441+0000","last_became_peered":"2024-09-21T07:46:07.220441+0000","last_unstale":"2024-09-21T07:46:08.208617+0000","last_undegraded":"2024-09-21T07:46:08.208617+0000","last_fullsized":"2024-09-21T07:46:08.208617+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:16.690174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.198896+0000","last_change":"2024-09-21T07:46:06.212367+0000","last_active":"2024-09-21T07:46:09.198896+0000","last_peered":"2024-09-21T07:46:09.198896+0000","last_clean":"2024-09-21T07:46:09.198896+0000","last_became_active":"2024-09-21T07:46:06.211815+0000","last_became_peered":"2024-09-21T07:46:06.211815+0000","last_unstale":"2024-09-21T07:46:09.198896+0000","last_undegraded":"2024-09-21T07:46:09.198896+0000","last_fullsized":"2024-09-21T07:46:09.198896+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:22.909532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399778+0000","last_change":"2024-09-21T07:46:07.224826+0000","last_active":"2024-09-21T07:46:08.399778+0000","last_peered":"2024-09-21T07:46:08.399778+0000","last_clean":"2024-09-21T07:46:08.399778+0000","last_became_active":"2024-09-21T07:46:07.221235+0000","last_became_peered":"2024-09-21T07:46:07.221235+0000","last_unstale":"2024-09-21T07:46:08.399778+0000","last_undegraded":"2024-09-21T07:46:08.399778+0000","last_fullsized":"2024-09-21T07:46:08.399778+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:12.490015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207301+0000","last_change":"2024-09-21T07:46:06.206432+0000","last_active":"2024-09-21T07:46:08.207301+0000","last_peered":"2024-09-21T07:46:08.207301+0000","last_clean":"2024-09-21T07:46:08.207301+0000","last_became_active":"2024-09-21T07:46:06.203791+0000","last_became_peered":"2024-09-21T07:46:06.203791+0000","last_unstale":"2024-09-21T07:46:08.207301+0000","last_undegraded":"2024-09-21T07:46:08.207301+0000","last_fullsized":"2024-09-21T07:46:08.207301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:33.511076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.207491+0000","last_change":"2024-09-21T07:46:07.214468+0000","last_active":"2024-09-21T07:46:08.207491+0000","last_peered":"2024-09-21T07:46:08.207491+0000","last_clean":"2024-09-21T07:46:08.207491+0000","last_became_active":"2024-09-21T07:46:07.213517+0000","last_became_peered":"2024-09-21T07:46:07.213517+0000","last_unstale":"2024-09-21T07:46:08.207491+0000","last_undegraded":"2024-09-21T07:46:08.207491+0000","last_fullsized":"2024-09-21T07:46:08.207491+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:45:25.873000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.521331+0000","last_change":"2024-09-21T07:46:06.211420+0000","last_active":"2024-09-21T07:46:08.521331+0000","last_peered":"2024-09-21T07:46:08.521331+0000","last_clean":"2024-09-21T07:46:08.521331+0000","last_became_active":"2024-09-21T07:46:06.208177+0000","last_became_peered":"2024-09-21T07:46:06.208177+0000","last_unstale":"2024-09-21T07:46:08.521331+0000","last_undegraded":"2024-09-21T07:46:08.521331+0000","last_fullsized":"2024-09-21T07:46:08.521331+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:41.931954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.032387+0000","last_change":"2024-09-21T07:46:07.213281+0000","last_active":"2024-09-21T07:46:09.032387+0000","last_peered":"2024-09-21T07:46:09.032387+0000","last_clean":"2024-09-21T07:46:09.032387+0000","last_became_active":"2024-09-21T07:46:07.212947+0000","last_became_peered":"2024-09-21T07:46:07.212947+0000","last_unstale":"2024-09-21T07:46:09.032387+0000","last_undegraded":"2024-09-21T07:46:09.032387+0000","last_fullsized":"2024-09-21T07:46:09.032387+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:58:24.283108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.b","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399658+0000","last_change":"2024-09-21T07:46:06.206452+0000","last_active":"2024-09-21T07:46:08.399658+0000","last_peered":"2024-09-21T07:46:08.399658+0000","last_clean":"2024-09-21T07:46:08.399658+0000","last_became_active":"2024-09-21T07:46:06.206136+0000","last_became_peered":"2024-09-21T07:46:06.206136+0000","last_unstale":"2024-09-21T07:46:08.399658+0000","last_undegraded":"2024-09-21T07:46:08.399658+0000","last_fullsized":"2024-09-21T07:46:08.399658+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:20.047491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.400029+0000","last_change":"2024-09-21T07:46:07.211503+0000","last_active":"2024-09-21T07:46:08.400029+0000","last_peered":"2024-09-21T07:46:08.400029+0000","last_clean":"2024-09-21T07:46:08.400029+0000","last_became_active":"2024-09-21T07:46:07.211183+0000","last_became_peered":"2024-09-21T07:46:07.211183+0000","last_unstale":"2024-09-21T07:46:08.400029+0000","last_undegraded":"2024-09-21T07:46:08.400029+0000","last_fullsized":"2024-09-21T07:46:08.400029+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:52.327032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.399537+0000","last_change":"2024-09-21T07:46:07.228129+0000","last_active":"2024-09-21T07:46:08.399537+0000","last_peered":"2024-09-21T07:46:08.399537+0000","last_clean":"2024-09-21T07:46:08.399537+0000","last_became_active":"2024-09-21T07:46:07.227801+0000","last_became_peered":"2024-09-21T07:46:07.227801+0000","last_unstale":"2024-09-21T07:46:08.399537+0000","last_undegraded":"2024-09-21T07:46:08.399537+0000","last_fullsized":"2024-09-21T07:46:08.399537+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:53:59.699234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1c","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199646+0000","last_change":"2024-09-21T07:46:06.217706+0000","last_active":"2024-09-21T07:46:09.199646+0000","last_peered":"2024-09-21T07:46:09.199646+0000","last_clean":"2024-09-21T07:46:09.199646+0000","last_became_active":"2024-09-21T07:46:06.215293+0000","last_became_peered":"2024-09-21T07:46:06.215293+0000","last_unstale":"2024-09-21T07:46:09.199646+0000","last_undegraded":"2024-09-21T07:46:09.199646+0000","last_fullsized":"2024-09-21T07:46:09.199646+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:24.359404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.500181+0000","last_change":"2024-09-21T07:46:07.232479+0000","last_active":"2024-09-21T07:46:09.500181+0000","last_peered":"2024-09-21T07:46:09.500181+0000","last_clean":"2024-09-21T07:46:09.500181+0000","last_became_active":"2024-09-21T07:46:07.232068+0000","last_became_peered":"2024-09-21T07:46:07.232068+0000","last_unstale":"2024-09-21T07:46:09.500181+0000","last_undegraded":"2024-09-21T07:46:09.500181+0000","last_fullsized":"2024-09-21T07:46:09.500181+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:41.114984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"22'2","reported_seq":17,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:09.199360+0000","last_change":"2024-09-21T07:46:06.210521+0000","last_active":"2024-09-21T07:46:09.199360+0000","last_peered":"2024-09-21T07:46:09.199360+0000","last_clean":"2024-09-21T07:46:09.199360+0000","last_became_active":"2024-09-21T07:46:06.210142+0000","last_became_peered":"2024-09-21T07:46:06.210142+0000","last_unstale":"2024-09-21T07:46:09.199360+0000","last_undegraded":"2024-09-21T07:46:09.199360+0000","last_fullsized":"2024-09-21T07:46:09.199360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:14:51.153791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208316+0000","last_change":"2024-09-21T07:46:07.210700+0000","last_active":"2024-09-21T07:46:08.208316+0000","last_peered":"2024-09-21T07:46:08.208316+0000","last_clean":"2024-09-21T07:46:08.208316+0000","last_became_active":"2024-09-21T07:46:07.203963+0000","last_became_peered":"2024-09-21T07:46:07.203963+0000","last_unstale":"2024-09-21T07:46:08.208316+0000","last_undegraded":"2024-09-21T07:46:08.208316+0000","last_fullsized":"2024-09-21T07:46:08.208316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:32.500930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1e","version":"22'1","reported_seq":16,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209121+0000","last_change":"2024-09-21T07:46:06.209967+0000","last_active":"2024-09-21T07:46:08.209121+0000","last_peered":"2024-09-21T07:46:08.209121+0000","last_clean":"2024-09-21T07:46:08.209121+0000","last_became_active":"2024-09-21T07:46:06.207030+0000","last_became_peered":"2024-09-21T07:46:06.207030+0000","last_unstale":"2024-09-21T07:46:08.209121+0000","last_undegraded":"2024-09-21T07:46:08.209121+0000","last_fullsized":"2024-09-21T07:46:08.209121+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:34:27.264294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.209044+0000","last_change":"2024-09-21T07:46:07.219277+0000","last_active":"2024-09-21T07:46:08.209044+0000","last_peered":"2024-09-21T07:46:08.209044+0000","last_clean":"2024-09-21T07:46:08.209044+0000","last_became_active":"2024-09-21T07:46:07.218947+0000","last_became_peered":"2024-09-21T07:46:07.218947+0000","last_unstale":"2024-09-21T07:46:08.209044+0000","last_undegraded":"2024-09-21T07:46:08.209044+0000","last_fullsized":"2024-09-21T07:46:08.209044+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:06.176676+0000","last_clean_scrub_stamp":"2024-09-21T07:46:06.176676+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:33:00.301927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1f","version":"22'4","reported_seq":19,"reported_epoch":23,"state":"active+clean","last_fresh":"2024-09-21T07:46:08.208464+0000","last_change":"2024-09-21T07:46:06.209178+0000","last_active":"2024-09-21T07:46:08.208464+0000","last_peered":"2024-09-21T07:46:08.208464+0000","last_clean":"2024-09-21T07:46:08.208464+0000","last_became_active":"2024-09-21T07:46:06.201083+0000","last_became_peered":"2024-09-21T07:46:06.201083+0000","last_unstale":"2024-09-21T07:46:08.208464+0000","last_undegraded":"2024-09-21T07:46:08.208464+0000","last_fullsized":"2024-09-21T07:46:08.208464+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:05.169767+0000","last_clean_scrub_stamp":"2024-09-21T07:46:05.169767+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:08:40.108075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476746,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":233472,"data_stored":80417,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476746,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":241664,"data_stored":81447,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476743,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27836,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344004,"statfs":{"total":96636764160,"available":96608260096,"internally_reserved":0,"allocated":827392,"data_stored":670273,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476743,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":237568,"data_stored":79891,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509446,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":229376,"data_stored":79835,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509447,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":831488,"data_stored":670295,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509447,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27272,"kb_used_data":244,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344568,"statfs":{"total":96636764160,"available":96608837632,"internally_reserved":0,"allocated":249856,"data_stored":82119,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542151,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27264,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344576,"statfs":{"total":96636764160,"available":96608845824,"internally_reserved":0,"allocated":237568,"data_stored":79959,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:46:18.347 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:46:18.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T07:46:18.348 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-09-21T07:46:18.348 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 -- 172.21.15.88:0/3810156349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0073c10 msgr2=0x7f94a006f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 --2- 172.21.15.88:0/3810156349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0073c10 0x7f94a006f0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f948c005fd0 tx=0x7f948c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 -- 172.21.15.88:0/3810156349 shutdown_connections 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 --2- 172.21.15.88:0/3810156349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a0073c10 0x7f94a006f0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 -- 172.21.15.88:0/3810156349 >> 172.21.15.88:0/3810156349 conn(0x7f94a006d730 msgr2=0x7f94a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 -- 172.21.15.88:0/3810156349 shutdown_connections 2024-09-21T07:46:18.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.599+0000 7f94a7c00640 1 -- 172.21.15.88:0/3810156349 wait complete. 2024-09-21T07:46:18.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.602+0000 7f94a7c00640 1 Processor -- start 2024-09-21T07:46:18.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.602+0000 7f94a7c00640 1 -- start start 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f94a7c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a006f5e0 0x7f94a01455b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f94a7c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 0x7f94a0158930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f94a7c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a0158e70 0x7f94a01e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f94a7c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94a00b3df0 con 0x7f94a00af630 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f94a7c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f94a00b3c70 con 0x7f94a006f5e0 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f94a7c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94a00b3f70 con 0x7f94a0158e70 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f949fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 0x7f94a0158930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f949fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 0x7f94a0158930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48748/0 (socket says 172.21.15.88:48748) 2024-09-21T07:46:18.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.603+0000 7f949fe00640 1 -- 172.21.15.88:0/833090346 learned_addr learned my addr 172.21.15.88:0/833090346 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f94a5200640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a006f5e0 0x7f94a01455b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f94a5c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a0158e70 0x7f94a01e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949fe00640 1 -- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a0158e70 msgr2=0x7f94a01e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949fe00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a0158e70 0x7f94a01e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949fe00640 1 -- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a006f5e0 msgr2=0x7f94a01455b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949fe00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a006f5e0 0x7f94a01455b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949fe00640 1 -- 172.21.15.88:0/833090346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94a01f9630 con 0x7f94a00af630 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f94a5c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a0158e70 0x7f94a01e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f94a5200640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a006f5e0 0x7f94a01455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949fe00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 0x7f94a0158930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f948c000c00 tx=0x7f948c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c03e030 con 0x7f94a00af630 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f948c003ca0 con 0x7f94a00af630 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94a01f9910 con 0x7f94a00af630 2024-09-21T07:46:18.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.604+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f948c035040 con 0x7f94a00af630 2024-09-21T07:46:18.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.605+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94a01f9ed0 con 0x7f94a00af630 2024-09-21T07:46:18.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.606+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94a00b4460 con 0x7f94a00af630 2024-09-21T07:46:18.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.607+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f948c041030 con 0x7f94a00af630 2024-09-21T07:46:18.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.608+0000 7f949cc00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9480078bc0 0x7f948007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:18.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.608+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f948c0bb360 con 0x7f94a00af630 2024-09-21T07:46:18.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.608+0000 7f94a5200640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9480078bc0 0x7f948007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:18.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.609+0000 7f94a5200640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9480078bc0 0x7f948007b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9494006fd0 tx=0x7f9494008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:18.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.612+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f948c0837a0 con 0x7f94a00af630 2024-09-21T07:46:18.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.835+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f94a00ae3b0 con 0x7f94a00af630 2024-09-21T07:46:18.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.836+0000 7f949cc00640 1 -- 172.21.15.88:0/833090346 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f948c0830a0 con 0x7f94a00af630 2024-09-21T07:46:18.838 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:18.838 INFO:teuthology.orchestra.run.smithi088.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9480078bc0 msgr2=0x7f948007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9480078bc0 0x7f948007b0d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f9494006fd0 tx=0x7f9494008040 comp rx=0 tx=0).stop 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 msgr2=0x7f94a0158930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 0x7f94a0158930 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f948c000c00 tx=0x7f948c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 shutdown_connections 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9480078bc0 0x7f948007b0d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f94a0158e70 0x7f94a01e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94a00af630 0x7f94a0158930 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 --2- 172.21.15.88:0/833090346 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94a006f5e0 0x7f94a01455b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 >> 172.21.15.88:0/833090346 conn(0x7f94a006d730 msgr2=0x7f94a01521e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:18.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 shutdown_connections 2024-09-21T07:46:18.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:18.838+0000 7f94a7c00640 1 -- 172.21.15.88:0/833090346 wait complete. 2024-09-21T07:46:18.862 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-09-21T07:46:18.862 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 -- 172.21.15.88:0/3125213167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280b9840 msgr2=0x7fee280d4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 --2- 172.21.15.88:0/3125213167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280b9840 0x7fee280d4fc0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fee281632f0 tx=0x7fee1c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 -- 172.21.15.88:0/3125213167 shutdown_connections 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 --2- 172.21.15.88:0/3125213167 >> v2:172.21.15.175:3300/0 conn(0x7fee280d2850 0x7fee280cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 --2- 172.21.15.88:0/3125213167 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee280b35c0 0x7fee280d2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 --2- 172.21.15.88:0/3125213167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280b9840 0x7fee280d4fc0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.086+0000 7fee2e800640 1 -- 172.21.15.88:0/3125213167 >> 172.21.15.88:0/3125213167 conn(0x7fee280d44a0 msgr2=0x7fee280d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.087+0000 7fee2e800640 1 -- 172.21.15.88:0/3125213167 shutdown_connections 2024-09-21T07:46:19.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.087+0000 7fee2e800640 1 -- 172.21.15.88:0/3125213167 wait complete. 2024-09-21T07:46:19.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.089+0000 7fee2e800640 1 Processor -- start 2024-09-21T07:46:19.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.089+0000 7fee2e800640 1 -- start start 2024-09-21T07:46:19.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.090+0000 7fee2e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee280b35c0 0x7fee280ca550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.090+0000 7fee2e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee280b9840 0x7fee280cc310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.090+0000 7fee2e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280d2850 0x7fee280cc850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2e800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee280d0d10 con 0x7fee280d2850 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee280d0600 con 0x7fee280b35c0 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee280d0e90 con 0x7fee280b9840 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280d2850 0x7fee280cc850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee280b35c0 0x7fee280ca550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee280b35c0 0x7fee280ca550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:43706/0 (socket says 172.21.15.88:43706) 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2d400640 1 -- 172.21.15.88:0/1529822730 learned_addr learned my addr 172.21.15.88:0/1529822730 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2ca00640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee280b9840 0x7fee280cc310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2de00640 1 -- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee280b9840 msgr2=0x7fee280cc310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2de00640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee280b9840 0x7fee280cc310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2de00640 1 -- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee280b35c0 msgr2=0x7fee280ca550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2de00640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee280b35c0 0x7fee280ca550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.091+0000 7fee2de00640 1 -- 172.21.15.88:0/1529822730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee280b5120 con 0x7fee280d2850 2024-09-21T07:46:19.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.092+0000 7fee2de00640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280d2850 0x7fee280cc850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee2000be50 tx=0x7fee2000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.092+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20019070 con 0x7fee280d2850 2024-09-21T07:46:19.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.092+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee20014440 con 0x7fee280d2850 2024-09-21T07:46:19.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.092+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee280cd0d0 con 0x7fee280d2850 2024-09-21T07:46:19.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.092+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20014890 con 0x7fee280d2850 2024-09-21T07:46:19.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.092+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee280cd690 con 0x7fee280d2850 2024-09-21T07:46:19.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.093+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee280cce40 con 0x7fee280d2850 2024-09-21T07:46:19.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.094+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fee2001c070 con 0x7fee280d2850 2024-09-21T07:46:19.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.095+0000 7fee25600640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fee08078b90 0x7fee0807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.095+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fee2009adb0 con 0x7fee280d2850 2024-09-21T07:46:19.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.096+0000 7fee2d400640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fee08078b90 0x7fee0807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.096+0000 7fee2d400640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fee08078b90 0x7fee0807b0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009660 tx=0x7fee1c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.100+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee2009f070 con 0x7fee280d2850 2024-09-21T07:46:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.321+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fee2808cc70 con 0x7fee280d2850 2024-09-21T07:46:19.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.322+0000 7fee25600640 1 -- 172.21.15.88:0/1529822730 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7fee2808cc70 con 0x7fee280d2850 2024-09-21T07:46:19.324 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:19.324 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:19.325 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fee08078b90 msgr2=0x7fee0807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fee08078b90 0x7fee0807b0a0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fee1c009660 tx=0x7fee1c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280d2850 msgr2=0x7fee280cc850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280d2850 0x7fee280cc850 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fee2000be50 tx=0x7fee2000bf40 comp rx=0 tx=0).stop 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 shutdown_connections 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fee08078b90 0x7fee0807b0a0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee280b9840 0x7fee280cc310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee280d2850 0x7fee280cc850 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 --2- 172.21.15.88:0/1529822730 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee280b35c0 0x7fee280ca550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 >> 172.21.15.88:0/1529822730 conn(0x7fee280d44a0 msgr2=0x7fee2806de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.324+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 shutdown_connections 2024-09-21T07:46:19.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.325+0000 7fee2e800640 1 -- 172.21.15.88:0/1529822730 wait complete. 2024-09-21T07:46:19.348 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 -- 172.21.15.88:0/1226992119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c0ad9c0 msgr2=0x7fec3c0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 --2- 172.21.15.88:0/1226992119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c0ad9c0 0x7fec3c0af170 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fec2c009830 tx=0x7fec2c02a530 comp rx=0 tx=0).stop 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 -- 172.21.15.88:0/1226992119 shutdown_connections 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 --2- 172.21.15.88:0/1226992119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c0ad9c0 0x7fec3c0af170 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 -- 172.21.15.88:0/1226992119 >> 172.21.15.88:0/1226992119 conn(0x7fec3c0b1e40 msgr2=0x7fec3c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 -- 172.21.15.88:0/1226992119 shutdown_connections 2024-09-21T07:46:19.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.567+0000 7fec3b200640 1 -- 172.21.15.88:0/1226992119 wait complete. 2024-09-21T07:46:19.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.569+0000 7fec3b200640 1 Processor -- start 2024-09-21T07:46:19.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.569+0000 7fec3b200640 1 -- start start 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec3c0ad9c0 0x7fec3c0cb9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec3c0b2f60 0x7fec3c09cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 0x7fec3c09d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c06fb20 con 0x7fec3c15a0f0 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec3c06f9a0 con 0x7fec3c0ad9c0 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec3c15d670 con 0x7fec3c0b2f60 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 0x7fec3c09d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 0x7fec3c09d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48772/0 (socket says 172.21.15.88:48772) 2024-09-21T07:46:19.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 -- 172.21.15.88:0/2450161998 learned_addr learned my addr 172.21.15.88:0/2450161998 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:19.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec39e00640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec3c0ad9c0 0x7fec3c0cb9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec39400640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec3c0b2f60 0x7fec3c09cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 -- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec3c0b2f60 msgr2=0x7fec3c09cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec3c0b2f60 0x7fec3c09cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 -- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec3c0ad9c0 msgr2=0x7fec3c0cb9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec3c0ad9c0 0x7fec3c0cb9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.571+0000 7fec3a800640 1 -- 172.21.15.88:0/2450161998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec3c0c9e60 con 0x7fec3c15a0f0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.572+0000 7fec3a800640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 0x7fec3c09d2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec3400ca80 tx=0x7fec3400cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.572+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34016020 con 0x7fec3c15a0f0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.572+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec3400ad20 con 0x7fec3c15a0f0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.572+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec34012680 con 0x7fec3c15a0f0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.572+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec3c0ca140 con 0x7fec3c15a0f0 2024-09-21T07:46:19.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.572+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec3c157b90 con 0x7fec3c15a0f0 2024-09-21T07:46:19.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.573+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec3c0cbfc0 con 0x7fec3c15a0f0 2024-09-21T07:46:19.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.574+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec34019020 con 0x7fec3c15a0f0 2024-09-21T07:46:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.576+0000 7fec32000640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec1c078b90 0x7fec1c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.576+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fec3409b1b0 con 0x7fec3c15a0f0 2024-09-21T07:46:19.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.576+0000 7fec39e00640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec1c078b90 0x7fec1c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:19.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.577+0000 7fec39e00640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec1c078b90 0x7fec1c07b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec2c034000 tx=0x7fec2c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:19.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.581+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec34020020 con 0x7fec3c15a0f0 2024-09-21T07:46:19.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.807+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fec3c0af170 con 0x7fec3c15a0f0 2024-09-21T07:46:19.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.808+0000 7fec32000640 1 -- 172.21.15.88:0/2450161998 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7fec34063030 con 0x7fec3c15a0f0 2024-09-21T07:46:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:19.810 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:19.810 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:19.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec1c078b90 msgr2=0x7fec1c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec1c078b90 0x7fec1c07b0a0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fec2c034000 tx=0x7fec2c004730 comp rx=0 tx=0).stop 2024-09-21T07:46:19.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 msgr2=0x7fec3c09d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:19.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 0x7fec3c09d2c0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec3400ca80 tx=0x7fec3400cf40 comp rx=0 tx=0).stop 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 shutdown_connections 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec1c078b90 0x7fec1c07b0a0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec3c0b2f60 0x7fec3c09cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec3c15a0f0 0x7fec3c09d2c0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 --2- 172.21.15.88:0/2450161998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec3c0ad9c0 0x7fec3c0cb9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 >> 172.21.15.88:0/2450161998 conn(0x7fec3c0b1e40 msgr2=0x7fec3c15a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 shutdown_connections 2024-09-21T07:46:19.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:19.810+0000 7fec3b200640 1 -- 172.21.15.88:0/2450161998 wait complete. 2024-09-21T07:46:19.836 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 -- 172.21.15.88:0/3829834000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc0af630 msgr2=0x7fbfcc073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 --2- 172.21.15.88:0/3829834000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc0af630 0x7fbfcc073500 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbfcc0b5080 tx=0x7fbfc002a560 comp rx=0 tx=0).stop 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 -- 172.21.15.88:0/3829834000 shutdown_connections 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 --2- 172.21.15.88:0/3829834000 >> v2:172.21.15.175:3300/0 conn(0x7fbfcc06f5e0 0x7fbfcc162560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 --2- 172.21.15.88:0/3829834000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc073a40 0x7fbfcc06f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 --2- 172.21.15.88:0/3829834000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc0af630 0x7fbfcc073500 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 -- 172.21.15.88:0/3829834000 >> 172.21.15.88:0/3829834000 conn(0x7fbfcc06d730 msgr2=0x7fbfcc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 -- 172.21.15.88:0/3829834000 shutdown_connections 2024-09-21T07:46:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.051+0000 7fbfd3600640 1 -- 172.21.15.88:0/3829834000 wait complete. 2024-09-21T07:46:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.054+0000 7fbfd3600640 1 Processor -- start 2024-09-21T07:46:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.054+0000 7fbfd3600640 1 -- start start 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfd3600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc06f5e0 0x7fbfcc154d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfd3600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 0x7fbfcc1f8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfd3600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0af630 0x7fbfcc1f93d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfd3600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc162e30 con 0x7fbfcc073a40 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfd3600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbfcc162cb0 con 0x7fbfcc06f5e0 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfd3600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbfcc162fb0 con 0x7fbfcc0af630 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfcbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 0x7fbfcc1f8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfcbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 0x7fbfcc1f8e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48794/0 (socket says 172.21.15.88:48794) 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.055+0000 7fbfcbe00640 1 -- 172.21.15.88:0/2832173019 learned_addr learned my addr 172.21.15.88:0/2832173019 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfd0c00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc06f5e0 0x7fbfcc154d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfd1600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0af630 0x7fbfcc1f93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfcbe00640 1 -- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0af630 msgr2=0x7fbfcc1f93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfcbe00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0af630 0x7fbfcc1f93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfcbe00640 1 -- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc06f5e0 msgr2=0x7fbfcc154d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfcbe00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc06f5e0 0x7fbfcc154d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfcbe00640 1 -- 172.21.15.88:0/2832173019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfcc1552d0 con 0x7fbfcc073a40 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfd0c00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc06f5e0 0x7fbfcc154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfcbe00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 0x7fbfcc1f8e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e750 tx=0x7fbfb800ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb800ccc0 con 0x7fbfcc073a40 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbfb800ce60 con 0x7fbfcc073a40 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfcc14fa30 con 0x7fbfcc073a40 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.056+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfb8005280 con 0x7fbfcc073a40 2024-09-21T07:46:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.057+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfcc14fff0 con 0x7fbfcc073a40 2024-09-21T07:46:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.057+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbfcc155500 con 0x7fbfcc073a40 2024-09-21T07:46:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.059+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbfb8005420 con 0x7fbfcc073a40 2024-09-21T07:46:20.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.060+0000 7fbfc8c00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbfac078bc0 0x7fbfac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.060+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fbfb809b680 con 0x7fbfcc073a40 2024-09-21T07:46:20.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.060+0000 7fbfd0c00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbfac078bc0 0x7fbfac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.061+0000 7fbfd0c00640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbfac078bc0 0x7fbfac07b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0033000 tx=0x7fbfc0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:20.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.064+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbfb800ee60 con 0x7fbfcc073a40 2024-09-21T07:46:20.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.251+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbfcc0add40 con 0x7fbfcc073a40 2024-09-21T07:46:20.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.252+0000 7fbfc8c00640 1 -- 172.21.15.88:0/2832173019 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fbfb8063a20 con 0x7fbfcc073a40 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:20.255 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:20.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbfac078bc0 msgr2=0x7fbfac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbfac078bc0 0x7fbfac07b0d0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbfc0033000 tx=0x7fbfc0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 msgr2=0x7fbfcc1f8e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 0x7fbfcc1f8e90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbfb800e750 tx=0x7fbfb800ec10 comp rx=0 tx=0).stop 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 shutdown_connections 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbfac078bc0 0x7fbfac07b0d0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbfcc0af630 0x7fbfcc1f93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfcc073a40 0x7fbfcc1f8e90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 --2- 172.21.15.88:0/2832173019 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbfcc06f5e0 0x7fbfcc154d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 >> 172.21.15.88:0/2832173019 conn(0x7fbfcc06d730 msgr2=0x7fbfcc0b12a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 shutdown_connections 2024-09-21T07:46:20.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.255+0000 7fbfd3600640 1 -- 172.21.15.88:0/2832173019 wait complete. 2024-09-21T07:46:20.281 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:20.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 -- 172.21.15.88:0/3121890794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 msgr2=0x7f61200b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 --2- 172.21.15.88:0/3121890794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200b3e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f6120072810 tx=0x7f611802a980 comp rx=0 tx=0).stop 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 -- 172.21.15.88:0/3121890794 shutdown_connections 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 --2- 172.21.15.88:0/3121890794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200b3e10 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 --2- 172.21.15.88:0/3121890794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61200ad3c0 0x7f61200ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 --2- 172.21.15.88:0/3121890794 >> v2:172.21.15.175:3300/0 conn(0x7f61200704b0 0x7f6120073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 -- 172.21.15.88:0/3121890794 >> 172.21.15.88:0/3121890794 conn(0x7f612006d730 msgr2=0x7f61200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 -- 172.21.15.88:0/3121890794 shutdown_connections 2024-09-21T07:46:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.500+0000 7f6128200640 1 -- 172.21.15.88:0/3121890794 wait complete. 2024-09-21T07:46:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.502+0000 7f6128200640 1 Processor -- start 2024-09-21T07:46:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.503+0000 7f6128200640 1 -- start start 2024-09-21T07:46:20.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6128200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61200704b0 0x7f612015a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6128200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ad3c0 0x7f612015d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6128200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200ae470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6128200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f61200712b0 con 0x7f61200add10 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6128200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6120071130 con 0x7f61200ad3c0 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6128200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61200b4350 con 0x7f61200704b0 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6126200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200ae470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6126200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200ae470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48812/0 (socket says 172.21.15.88:48812) 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6126200640 1 -- 172.21.15.88:0/2557254118 learned_addr learned my addr 172.21.15.88:0/2557254118 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6124e00640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ad3c0 0x7f612015d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.504+0000 7f6125800640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61200704b0 0x7f612015a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6126200640 1 -- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61200704b0 msgr2=0x7f612015a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6126200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61200704b0 0x7f612015a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6126200640 1 -- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ad3c0 msgr2=0x7f612015d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6126200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ad3c0 0x7f612015d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6126200640 1 -- 172.21.15.88:0/2557254118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f612015daf0 con 0x7f61200add10 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6126200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200ae470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6118033040 tx=0x7f6118003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118034040 con 0x7f61200add10 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6118004540 con 0x7f61200add10 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6118045480 con 0x7f61200add10 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6120158700 con 0x7f61200add10 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.505+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61201f6810 con 0x7f61200add10 2024-09-21T07:46:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.507+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f61200ad7d0 con 0x7f61200add10 2024-09-21T07:46:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.508+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6118037020 con 0x7f61200add10 2024-09-21T07:46:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.509+0000 7f611d600640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6100078b90 0x7f610007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.509+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f61180bc4a0 con 0x7f61200add10 2024-09-21T07:46:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.509+0000 7f6125800640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6100078b90 0x7f610007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.510+0000 7f6125800640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6100078b90 0x7f610007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6114009ab0 tx=0x7f6114005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:20.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.513+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61180842a0 con 0x7f61200add10 2024-09-21T07:46:20.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.735+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f612015e420 con 0x7f61200add10 2024-09-21T07:46:20.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.736+0000 7f611d600640 1 -- 172.21.15.88:0/2557254118 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f6118088cd0 con 0x7f61200add10 2024-09-21T07:46:20.738 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:20.739 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:20.739 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6100078b90 msgr2=0x7f610007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6100078b90 0x7f610007b0a0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f6114009ab0 tx=0x7f6114005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 msgr2=0x7f61200ae470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200ae470 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f6118033040 tx=0x7f6118003f10 comp rx=0 tx=0).stop 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 shutdown_connections 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6100078b90 0x7f610007b0a0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61200add10 0x7f61200ae470 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f61200ad3c0 0x7f612015d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 --2- 172.21.15.88:0/2557254118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f61200704b0 0x7f612015a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.738+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 >> 172.21.15.88:0/2557254118 conn(0x7f612006d730 msgr2=0x7f612014db30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:20.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.739+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 shutdown_connections 2024-09-21T07:46:20.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:20.739+0000 7f6128200640 1 -- 172.21.15.88:0/2557254118 wait complete. 2024-09-21T07:46:21.167 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:46:06.691548+0000', 'modified': '2024-09-21T07:46:09.694928+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4423}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4423': {'gid': 4423, 'name': 'b', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/3640011394', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 3640011394}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 3640011394}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4423, 'qdb_cluster': [4423]} 2024-09-21T07:46:21.167 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:46:21.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:46:21.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 -- 172.21.15.88:0/3870888878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00cfc80 msgr2=0x7f83f0156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3870888878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00cfc80 0x7f83f0156e40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009860 tx=0x7f83e002a560 comp rx=0 tx=0).stop 2024-09-21T07:46:21.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 -- 172.21.15.88:0/3870888878 shutdown_connections 2024-09-21T07:46:21.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3870888878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00cfc80 0x7f83f0156e40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 -- 172.21.15.88:0/3870888878 >> 172.21.15.88:0/3870888878 conn(0x7f83f006d3d0 msgr2=0x7f83f00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 -- 172.21.15.88:0/3870888878 shutdown_connections 2024-09-21T07:46:21.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.387+0000 7f83f5c00640 1 -- 172.21.15.88:0/3870888878 wait complete. 2024-09-21T07:46:21.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.389+0000 7f83f5c00640 1 Processor -- start 2024-09-21T07:46:21.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.389+0000 7f83f5c00640 1 -- start start 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.390+0000 7f83f5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 0x7f83f00c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.390+0000 7f83f5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00cfc80 0x7f83f00ca0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.390+0000 7f83f5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f015ce30 0x7f83f00b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.390+0000 7f83f5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00cd100 con 0x7f83f00705b0 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.390+0000 7f83f5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00ccf80 con 0x7f83f015ce30 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.390+0000 7f83f5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83f00cd280 con 0x7f83f00cfc80 2024-09-21T07:46:21.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 0x7f83f00c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 0x7f83f00c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48830/0 (socket says 172.21.15.88:48830) 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 -- 172.21.15.88:0/3763842008 learned_addr learned my addr 172.21.15.88:0/3763842008 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83ee000640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00cfc80 0x7f83f00ca0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83ef400640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f015ce30 0x7f83f00b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 -- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00cfc80 msgr2=0x7f83f00ca0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00cfc80 0x7f83f00ca0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 -- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f015ce30 msgr2=0x7f83f00b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f015ce30 0x7f83f00b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 -- 172.21.15.88:0/3763842008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83f00c7920 con 0x7f83f00705b0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.391+0000 7f83eea00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 0x7f83f00c4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009780 tx=0x7f83e0033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.392+0000 7f83e6a00640 1 -- 172.21.15.88:0/3763842008 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0031d60 con 0x7f83f00705b0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.392+0000 7f83e6a00640 1 -- 172.21.15.88:0/3763842008 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83e0005020 con 0x7f83f00705b0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.392+0000 7f83e6a00640 1 -- 172.21.15.88:0/3763842008 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e0005450 con 0x7f83f00705b0 2024-09-21T07:46:21.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.392+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83f00c56b0 con 0x7f83f00705b0 2024-09-21T07:46:21.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.392+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83f00c5c70 con 0x7f83f00705b0 2024-09-21T07:46:21.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.395+0000 7f83e6a00640 1 -- 172.21.15.88:0/3763842008 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f83e00047a0 con 0x7f83f00705b0 2024-09-21T07:46:21.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.396+0000 7f83e6a00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f83d0078b90 0x7f83d007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.396+0000 7f83e6a00640 1 -- 172.21.15.88:0/3763842008 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f83e00bbe80 con 0x7f83f00705b0 2024-09-21T07:46:21.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.397+0000 7f83fcc5d740 1 Processor -- start 2024-09-21T07:46:21.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.397+0000 7f83ee000640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f83d0078b90 0x7f83d007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.397+0000 7f83fcc5d740 1 -- start start 2024-09-21T07:46:21.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.397+0000 7f83ee000640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f83d0078b90 0x7f83d007b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83f0069b20 tx=0x7f83dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b949d270 0x5571b9871d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 0x5571b98722c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9842370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5571b97c7430 con 0x5571b94743b0 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571b97b75d0 con 0x5571b949d270 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5571b97a6a00 con 0x5571b947c590 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.398+0000 7f83ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 0x5571b98722c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 0x5571b98722c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48840/0 (socket says 172.21.15.88:48840) 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 -- 172.21.15.88:0/2900724718 learned_addr learned my addr 172.21.15.88:0/2900724718 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ef400640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9842370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83eea00640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b949d270 0x5571b9871d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 -- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 msgr2=0x5571b9842370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9842370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 -- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b949d270 msgr2=0x5571b9871d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b949d270 0x5571b9871d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 -- 172.21.15.88:0/2900724718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b976f520 con 0x5571b94743b0 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ef400640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9842370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83eea00640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b949d270 0x5571b9871d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83ee000640 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 0x5571b98722c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00f9c0 tx=0x7f83dc012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362bffd5aca330ba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.399+0000 7f83db400640 1 -- 172.21.15.88:0/2900724718 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc025070 con 0x5571b94743b0 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83db400640 1 -- 172.21.15.88:0/2900724718 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83dc01b050 con 0x5571b94743b0 2024-09-21T07:46:21.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83db400640 1 -- 172.21.15.88:0/2900724718 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc014dc0 con 0x5571b94743b0 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 -- 172.21.15.88:0/2900724718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 msgr2=0x5571b98722c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 0x5571b98722c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f83dc00f9c0 tx=0x7f83dc012000 comp rx=0 tx=0).stop 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 -- 172.21.15.88:0/2900724718 shutdown_connections 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9842370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b94743b0 0x5571b98722c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/2900724718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b949d270 0x5571b9871d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 -- 172.21.15.88:0/2900724718 >> 172.21.15.88:0/2900724718 conn(0x5571b94913c0 msgr2=0x5571b9847d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.400+0000 7f83fcc5d740 1 -- 172.21.15.88:0/2900724718 shutdown_connections 2024-09-21T07:46:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.401+0000 7f83fcc5d740 1 -- 172.21.15.88:0/2900724718 wait complete. 2024-09-21T07:46:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.402+0000 7f83fcc5d740 1 -- start start 2024-09-21T07:46:21.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.402+0000 7f83fcc5d740 1 Processor -- start 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9857400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b986cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9885350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5571b974d330 con 0x5571b949d270 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571b97aa810 con 0x5571b94743b0 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5571b9517620 con 0x5571b947c590 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.403+0000 7f83fcc5d740 10 client.0 _ll_register_callbacks cb 0x5571b9752420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9885350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9885350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48854/0 (socket says 172.21.15.88:48854) 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 -- 172.21.15.88:0/1834487143 learned_addr learned my addr 172.21.15.88:0/1834487143 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:21.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83eea00640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9857400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ee000640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b986cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 -- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 msgr2=0x5571b986cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b986cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 -- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 msgr2=0x5571b9857400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9857400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 -- 172.21.15.88:0/1834487143 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b9508850 con 0x5571b949d270 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ee000640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b986cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83ef400640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9885350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f83e8011950 tx=0x7f83e8011e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2506bfeaa8292290 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.404+0000 7f83cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83cfe00640 1 -- 172.21.15.88:0/1834487143 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e801b020 con 0x5571b949d270 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83cfe00640 1 -- 172.21.15.88:0/1834487143 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f83e800fb40 con 0x5571b949d270 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83dc01b050 con 0x5571b949d270 2024-09-21T07:46:21.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83cfe00640 1 -- 172.21.15.88:0/1834487143 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f83e8025050 con 0x5571b949d270 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 10 client.4701 fetch_fsmap learned FSMap version 8 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5571b957f6b0 con 0x5571b949d270 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83cfe00640 1 -- 172.21.15.88:0/1834487143 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83dc01b050 con 0x5571b949d270 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83cfe00640 1 -- 172.21.15.88:0/1834487143 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f83e8028020 con 0x5571b949d270 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 10 client.4701 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 10 client.4701 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 20 client.4701 populate_metadata read hostname 'smithi088' 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b945be40 0x5571b986e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 4 client.4701 mds_command: new command op to 4423 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.405+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 --> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5571b97b9520 con 0x5571b945be40 2024-09-21T07:46:21.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.406+0000 7f83eea00640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b945be40 0x5571b986e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.406+0000 7f83eea00640 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b945be40 0x5571b986e820 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0f345bb03038b57 server_cookie=da48d275dcc2a881 in_seq=0 out_seq=0 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.406+0000 7f83cfe00640 10 client.4701 ms_handle_connect on v2:172.21.15.175:6835/3640011394 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83cfe00640 1 -- 172.21.15.88:0/1834487143 <== mds.0 v2:172.21.15.175:6835/3640011394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5571b97b9520 con 0x5571b945be40 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83cfe00640 10 client.4701 handle_command_reply: tid=0 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 client.4701 shutdown 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 msgr2=0x5571b9885350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9885350 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f83e8011950 tx=0x7f83e8011e10 comp rx=0 tx=0).stop 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 shutdown_connections 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b945be40 0x5571b986e820 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b986cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9885350 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83cfe00640 0 client.4701 ms_handle_reset on v2:172.21.15.175:6835/3640011394 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1834487143 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9857400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 >> 172.21.15.88:0/1834487143 conn(0x5571b94913c0 msgr2=0x5571b9883280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 shutdown_connections 2024-09-21T07:46:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1834487143 wait complete. 2024-09-21T07:46:21.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.408+0000 7f83fcc5d740 20 client.4701 trim_cache size 0 max 16384 2024-09-21T07:46:21.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.428+0000 7f83fcc5d740 1 Processor -- start 2024-09-21T07:46:21.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.428+0000 7f83fcc5d740 1 -- start start 2024-09-21T07:46:21.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b98861d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 0x5571b98afef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b949d270 0x5571b98b0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00cc7f0 con 0x5571b947c590 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f83f00cbb50 con 0x5571b94743b0 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f83f00cc1a0 con 0x5571b949d270 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 0x5571b98afef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 0x5571b98afef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48860/0 (socket says 172.21.15.88:48860) 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.429+0000 7f83ee000640 1 -- 172.21.15.88:0/3183311712 learned_addr learned my addr 172.21.15.88:0/3183311712 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83eea00640 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b98861d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ef400640 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b949d270 0x5571b98b0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ee000640 1 -- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b949d270 msgr2=0x5571b98b0430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ee000640 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b949d270 0x5571b98b0430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ee000640 1 -- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 msgr2=0x5571b98861d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ee000640 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b98861d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ee000640 1 -- 172.21.15.88:0/3183311712 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5571b9858b60 con 0x5571b947c590 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83eea00640 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b98861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83ee000640 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 0x5571b98afef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83dc020000 tx=0x7f83dc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9237c8fed0320d8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83daa00640 1 -- 172.21.15.88:0/3183311712 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc025070 con 0x5571b947c590 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83daa00640 1 -- 172.21.15.88:0/3183311712 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83dc026040 con 0x5571b947c590 2024-09-21T07:46:21.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.430+0000 7f83daa00640 1 -- 172.21.15.88:0/3183311712 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83dc020890 con 0x5571b947c590 2024-09-21T07:46:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 -- 172.21.15.88:0/3183311712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 msgr2=0x5571b98afef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 0x5571b98afef0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f83dc020000 tx=0x7f83dc000ed0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 -- 172.21.15.88:0/3183311712 shutdown_connections 2024-09-21T07:46:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b949d270 0x5571b98b0430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b947c590 0x5571b98afef0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/3183311712 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b98861d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 -- 172.21.15.88:0/3183311712 >> 172.21.15.88:0/3183311712 conn(0x5571b94913c0 msgr2=0x5571b9857e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 -- 172.21.15.88:0/3183311712 shutdown_connections 2024-09-21T07:46:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.431+0000 7f83fcc5d740 1 -- 172.21.15.88:0/3183311712 wait complete. 2024-09-21T07:46:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.432+0000 7f83fcc5d740 1 -- start start 2024-09-21T07:46:21.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.432+0000 7f83fcc5d740 1 Processor -- start 2024-09-21T07:46:21.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9910390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9903810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9871f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5571b9859450 con 0x5571b949d270 2024-09-21T07:46:21.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5571b9885890 con 0x5571b94743b0 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x5571b9859a10 con 0x5571b947c590 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83fcc5d740 10 client.0 _ll_register_callbacks cb 0x7f83e8011530 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9871f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9871f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48866/0 (socket says 172.21.15.88:48866) 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83eea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9910390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83ef400640 1 -- 172.21.15.88:0/1081785376 learned_addr learned my addr 172.21.15.88:0/1081785376 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.434+0000 7f83ee000640 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9903810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83ef400640 1 -- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 msgr2=0x5571b9903810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83ef400640 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9903810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83ef400640 1 -- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 msgr2=0x5571b9910390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83ef400640 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9910390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83ef400640 1 -- 172.21.15.88:0/1081785376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5571b9886c70 con 0x5571b949d270 2024-09-21T07:46:21.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83ef400640 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9871f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83e8001e60 tx=0x7f83e8001b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d3dfd11ac6bbcda server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83cfe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83cfe00640 1 -- 172.21.15.88:0/1081785376 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83e802e060 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83cfe00640 1 -- 172.21.15.88:0/1081785376 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f83e80050b0 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83dc026040 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83cfe00640 1 -- 172.21.15.88:0/1081785376 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) ==== 24+0+0 (secure 0 0 0) 0x7f83e8034050 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83fcc5d740 10 client.4707 fetch_fsmap learned FSMap version 8 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.435+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) -- 0x5571b9848a60 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83cfe00640 1 -- 172.21.15.88:0/1081785376 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f83dc026040 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83cfe00640 1 -- 172.21.15.88:0/1081785376 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 8) ==== 3502+0+0 (secure 0 0 0) 0x7f83e8037020 con 0x5571b949d270 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83fcc5d740 10 client.4707 fetch_fsmap finished waiting for FSMap version 8 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83fcc5d740 10 client.4707 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83fcc5d740 20 client.4707 populate_metadata read hostname 'smithi088' 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b983cf80 0x5571b9870830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83fcc5d740 4 client.4707 mds_command: new command op to 4423 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:46:21.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 --> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] -- command(tid 0: {"prefix": "status"}) -- 0x5571b97b4fd0 con 0x5571b983cf80 2024-09-21T07:46:21.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.436+0000 7f83eea00640 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b983cf80 0x5571b9870830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83eea00640 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b983cf80 0x5571b9870830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9f15d24c271a79e server_cookie=51474817f88078e8 in_seq=0 out_seq=0 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83cfe00640 10 client.4707 ms_handle_connect on v2:172.21.15.175:6835/3640011394 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83cfe00640 1 -- 172.21.15.88:0/1081785376 <== mds.0 v2:172.21.15.175:6835/3640011394 1 ==== command_reply(tid 0: 0 ) ==== 8+0+330 (crc 0 0 0) 0x5571b97b4fd0 con 0x5571b983cf80 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83cfe00640 10 client.4707 handle_command_reply: tid=0 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 client.4707 shutdown 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 msgr2=0x5571b9871f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9871f80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f83e8001e60 tx=0x7f83e8001b60 comp rx=0 tx=0).stop 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 shutdown_connections 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:6835/3640011394,v1:172.21.15.175:6837/3640011394] conn(0x5571b983cf80 0x5571b9870830 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x5571b947c590 0x5571b9903810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5571b949d270 0x5571b9871f80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83cfe00640 0 client.4707 ms_handle_reset on v2:172.21.15.175:6835/3640011394 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.437+0000 7f83fcc5d740 1 --2- 172.21.15.88:0/1081785376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x5571b94743b0 0x5571b9910390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 >> 172.21.15.88:0/1081785376 conn(0x5571b94913c0 msgr2=0x5571b9857b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 shutdown_connections 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83fcc5d740 1 -- 172.21.15.88:0/1081785376 wait complete. 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83fcc5d740 20 client.4707 trim_cache size 0 max 16384 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:46:21.443 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4423, 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 14.252059366999999, 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 8, 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 23, 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 23, 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 20.274106466999999 2024-09-21T07:46:21.444 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f83d0078b90 msgr2=0x7f83d007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f83d0078b90 0x7f83d007b0a0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f83f0069b20 tx=0x7f83dc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 msgr2=0x7f83f00c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 0x7f83f00c4e50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f83e0009780 tx=0x7f83e0033d60 comp rx=0 tx=0).stop 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 shutdown_connections 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f83d0078b90 0x7f83d007b0a0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f83f00cfc80 0x7f83f00ca0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f83f015ce30 0x7f83f00b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 --2- 172.21.15.88:0/3763842008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83f00705b0 0x7f83f00c4e50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.438+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 >> 172.21.15.88:0/3763842008 conn(0x7f83f006d3d0 msgr2=0x7f83f006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:21.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.439+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 shutdown_connections 2024-09-21T07:46:21.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:21.439+0000 7f83f5c00640 1 -- 172.21.15.88:0/3763842008 wait complete. 2024-09-21T07:46:21.464 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T07:46:21.487 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T07:46:21.487 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2024-09-21T07:46:21.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.487 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:46:21.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.523 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T07:46:21.621 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T07:46:21.624 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:46:21.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.625 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:46:21.660 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.660 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T07:46:21.757 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T07:46:21.759 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:46:21.760 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.760 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:46:21.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.800 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-09-21T07:46:21.905 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-09-21T07:46:21.930 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:46:21.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.931 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:46:21.967 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:21.967 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-09-21T07:46:22.066 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-09-21T07:46:22.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:46:22.069 INFO:tasks.kclient:client.0 config is {'syntax': 'v1'} 2024-09-21T07:46:22.069 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:46:22.069 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:22.069 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:46:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.291+0000 7f58a6200640 1 -- 172.21.15.88:0/350966609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 msgr2=0x7f58a00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.291+0000 7f58a6200640 1 --2- 172.21.15.88:0/350966609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 0x7f58a00ad4f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f588c009860 tx=0x7f588c02a510 comp rx=0 tx=0).stop 2024-09-21T07:46:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.292+0000 7f58a6200640 1 -- 172.21.15.88:0/350966609 shutdown_connections 2024-09-21T07:46:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.292+0000 7f58a6200640 1 --2- 172.21.15.88:0/350966609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 0x7f58a00ad4f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.292+0000 7f58a6200640 1 -- 172.21.15.88:0/350966609 >> 172.21.15.88:0/350966609 conn(0x7f58a006d730 msgr2=0x7f58a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:22.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.292+0000 7f58a6200640 1 -- 172.21.15.88:0/350966609 shutdown_connections 2024-09-21T07:46:22.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.292+0000 7f58a6200640 1 -- 172.21.15.88:0/350966609 wait complete. 2024-09-21T07:46:22.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.294+0000 7f58a6200640 1 Processor -- start 2024-09-21T07:46:22.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.294+0000 7f58a6200640 1 -- start start 2024-09-21T07:46:22.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.295+0000 7f58a6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 0x7f58a014a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.295+0000 7f58a6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a00b3550 0x7f58a0147e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.295+0000 7f58a6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 0x7f58a009dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.295+0000 7f58a6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00b4da0 con 0x7f58a015d760 2024-09-21T07:46:22.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.295+0000 7f58a6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58a00b4c20 con 0x7f58a00b3550 2024-09-21T07:46:22.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.295+0000 7f58a6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58a00b4070 con 0x7f58a00ad0e0 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 0x7f58a009dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 0x7f58a009dcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48874/0 (socket says 172.21.15.88:48874) 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 -- 172.21.15.88:0/3973433301 learned_addr learned my addr 172.21.15.88:0/3973433301 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589f400640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 0x7f58a014a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589ea00640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a00b3550 0x7f58a0147e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 -- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 msgr2=0x7f58a014a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 0x7f58a014a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 -- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a00b3550 msgr2=0x7f58a0147e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a00b3550 0x7f58a0147e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 -- 172.21.15.88:0/3973433301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58a014ad10 con 0x7f58a015d760 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.296+0000 7f589fe00640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 0x7f58a009dcd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f589800c700 tx=0x7f589800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.297+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5898003b70 con 0x7f58a015d760 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.297+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5898003d10 con 0x7f58a015d760 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.297+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f589800aab0 con 0x7f58a015d760 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.297+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58a014afb0 con 0x7f58a015d760 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.297+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58a014e720 con 0x7f58a015d760 2024-09-21T07:46:22.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.298+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58a00ad4f0 con 0x7f58a015d760 2024-09-21T07:46:22.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.299+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f589800ac50 con 0x7f58a015d760 2024-09-21T07:46:22.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.300+0000 7f5897400640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5880078b90 0x7f588007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:22.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.300+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f589809a9e0 con 0x7f58a015d760 2024-09-21T07:46:22.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.300+0000 7f589f400640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5880078b90 0x7f588007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:22.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.301+0000 7f589f400640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5880078b90 0x7f588007b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890007ef0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:22.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.304+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58a014ad10 con 0x7f58a015d760 2024-09-21T07:46:22.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.530+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f58a00696d0 con 0x7f58a015d760 2024-09-21T07:46:22.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.531+0000 7f5897400640 1 -- 172.21.15.88:0/3973433301 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f58980627c0 con 0x7f58a015d760 2024-09-21T07:46:22.533 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:46:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5880078b90 msgr2=0x7f588007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5880078b90 0x7f588007b0a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5890000c00 tx=0x7f5890007ef0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 msgr2=0x7f58a009dcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 0x7f58a009dcd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f589800c700 tx=0x7f589800cbc0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 shutdown_connections 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5880078b90 0x7f588007b0a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58a015d760 0x7f58a009dcd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58a00b3550 0x7f58a0147e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 --2- 172.21.15.88:0/3973433301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58a00ad0e0 0x7f58a014a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 >> 172.21.15.88:0/3973433301 conn(0x7f58a006d730 msgr2=0x7f58a01535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 shutdown_connections 2024-09-21T07:46:22.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22.533+0000 7f58a6200640 1 -- 172.21.15.88:0/3973433301 wait complete. 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:46:22.560 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:46:22.561 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:46:22.561 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:46:22.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:22.561 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:22.597 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42712sec preferred_lft 42712sec 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:46:22.598 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:22.599 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr flush dev ceph-brx 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set ceph-brx up 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:46:22.599 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:22.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:22 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:23.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:23.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:23.013 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:46:23.069 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:46:23.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:23.072 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:46:23.128 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:46:23.128 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:46:23.128 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:46:23.128 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:46:23.128 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:46:23.128 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:23.129 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:23.129 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:46:23.129 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:46:23.129 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:46:23.129 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:23.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:23.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:23.757 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:46:23.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:23.793 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-09-21T07:46:23.868 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:23.868 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:23.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:46:23.869 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T07:46:23.869 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:23.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:23 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:24.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:24.043 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T07:46:24.043 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:24.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:24.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:24.285 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:24.285 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:24.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:46:24.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:46:24.285 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:24.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:24 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:24.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:24 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:24.451 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:24.451 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:24.487 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:46:24.487 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:24.487 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:24.561 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:46:24.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:24.561 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:46:24.776 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:46:24.776 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:46:24.776 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:46:25.506 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:46:25.506 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:46:25.507 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:46:25.507 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:25.507 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:46:25.540 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:46:25.541 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:46:25.542 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:46:25.542 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:46:25.542 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:46:25.542 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:46:25.544 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:46:25.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:25.545 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:46:25.602 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:25.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:25.706 INFO:teuthology.orchestra.run.smithi088.stdout:client4716 2024-09-21T07:46:25.706 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:25.706 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:25.742 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:25.782 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:25.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:25.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:25 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:25.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:25 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:25.882 INFO:teuthology.orchestra.run.smithi088.stdout:client4716 2024-09-21T07:46:25.882 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:26.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:26.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:26.027 INFO:teuthology.orchestra.run.smithi088.stdout:client4716 2024-09-21T07:46:26.027 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1afec4d-fdf7-414b-83b4-ffa85a98fc94.client4716/status') 2024-09-21T07:46:26.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:26.138 INFO:teuthology.orchestra.run.smithi088.stdout:instance: client.4716 (3)192.168.0.1:0/1352464771 2024-09-21T07:46:26.138 INFO:teuthology.orchestra.run.smithi088.stdout:blocklisted: false 2024-09-21T07:46:26.138 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records in 2024-09-21T07:46:26.139 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records out 2024-09-21T07:46:26.139 INFO:teuthology.orchestra.run.smithi088.stderr:69 bytes copied, 5.7047e-05 s, 1.2 MB/s 2024-09-21T07:46:26.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:26.150 INFO:tasks.kclient:client.2 config is {'syntax': 'v1'} 2024-09-21T07:46:26.150 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:46:26.150 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T07:46:26.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 -- 172.21.15.88:0/2923093155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b00b2220 msgr2=0x7f46b015c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 --2- 172.21.15.88:0/2923093155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b00b2220 0x7f46b015c7c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f46a4009830 tx=0x7f46a402a480 comp rx=0 tx=0).stop 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 -- 172.21.15.88:0/2923093155 shutdown_connections 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 --2- 172.21.15.88:0/2923093155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46b0160ca0 0x7f46b006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 --2- 172.21.15.88:0/2923093155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b00b2220 0x7f46b015c7c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 -- 172.21.15.88:0/2923093155 >> 172.21.15.88:0/2923093155 conn(0x7f46b009aa50 msgr2=0x7f46b00b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 -- 172.21.15.88:0/2923093155 shutdown_connections 2024-09-21T07:46:26.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.373+0000 7f46b6600640 1 -- 172.21.15.88:0/2923093155 wait complete. 2024-09-21T07:46:26.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.376+0000 7f46b6600640 1 Processor -- start 2024-09-21T07:46:26.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.376+0000 7f46b6600640 1 -- start start 2024-09-21T07:46:26.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b006d2e0 0x7f46b01f58d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46b00b2220 0x7f46b0149330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 0x7f46b01f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46b00d17a0 con 0x7f46b0160ca0 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46b00b1620 con 0x7f46b006d2e0 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46b00d1920 con 0x7f46b00b2220 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.377+0000 7f46b5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 0x7f46b01f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 0x7f46b01f8ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49148/0 (socket says 172.21.15.88:49148) 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 -- 172.21.15.88:0/2507079607 learned_addr learned my addr 172.21.15.88:0/2507079607 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5200640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b006d2e0 0x7f46b01f58d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46afe00640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46b00b2220 0x7f46b0149330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 -- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46b00b2220 msgr2=0x7f46b0149330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46b00b2220 0x7f46b0149330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 -- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b006d2e0 msgr2=0x7f46b01f58d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b006d2e0 0x7f46b01f58d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 -- 172.21.15.88:0/2507079607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46b0149d60 con 0x7f46b0160ca0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.378+0000 7f46b5c00640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 0x7f46b01f8ec0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c760 tx=0x7f46a800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.379+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a8003da0 con 0x7f46b0160ca0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.379+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46a800ce10 con 0x7f46b0160ca0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.379+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46b01f6400 con 0x7f46b0160ca0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.379+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46a80052d0 con 0x7f46b0160ca0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.379+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46b0159f70 con 0x7f46b0160ca0 2024-09-21T07:46:26.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.380+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46b0149f30 con 0x7f46b0160ca0 2024-09-21T07:46:26.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.381+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f46a8005470 con 0x7f46b0160ca0 2024-09-21T07:46:26.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.382+0000 7f46acc00640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4690078bc0 0x7f469007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:26.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.382+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f46a809a9d0 con 0x7f46b0160ca0 2024-09-21T07:46:26.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.383+0000 7f46b5200640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4690078bc0 0x7f469007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:26.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.384+0000 7f46b5200640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4690078bc0 0x7f469007b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f46a4034000 tx=0x7f46a4004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:26.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.391+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46b006d6f0 con 0x7f46b0160ca0 2024-09-21T07:46:26.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.620+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f46b0159d60 con 0x7f46b0160ca0 2024-09-21T07:46:26.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.620+0000 7f46acc00640 1 -- 172.21.15.88:0/2507079607 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f46a8062730 con 0x7f46b0160ca0 2024-09-21T07:46:26.621 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:46:26.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4690078bc0 msgr2=0x7f469007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4690078bc0 0x7f469007b0d0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f46a4034000 tx=0x7f46a4004730 comp rx=0 tx=0).stop 2024-09-21T07:46:26.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 msgr2=0x7f46b01f8ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:26.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 0x7f46b01f8ec0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f46a800c760 tx=0x7f46a800cc20 comp rx=0 tx=0).stop 2024-09-21T07:46:26.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 shutdown_connections 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4690078bc0 0x7f469007b0d0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46b00b2220 0x7f46b0149330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46b0160ca0 0x7f46b01f8ec0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 --2- 172.21.15.88:0/2507079607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46b006d2e0 0x7f46b01f58d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 >> 172.21.15.88:0/2507079607 conn(0x7f46b009aa50 msgr2=0x7f46b0069290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 shutdown_connections 2024-09-21T07:46:26.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26.623+0000 7f46b6600640 1 -- 172.21.15.88:0/2507079607 wait complete. 2024-09-21T07:46:26.646 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:46:26.646 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:46:26.646 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:46:26.646 INFO:tasks.cephfs.mount:self.client.name = client.2 2024-09-21T07:46:26.646 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2024-09-21T07:46:26.646 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:46:26.647 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:46:26.647 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:46:26.647 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2024-09-21T07:46:26.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:26.647 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:46:26.683 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:26.683 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:26.683 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:26.683 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:26.683 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:46:26.684 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42708sec preferred_lft 42708sec 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:46:26.685 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:46:26.686 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:26.686 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:26.686 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:46:26.779 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:46:26.782 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:26.782 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:46:26.818 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:46:26.818 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:46:26.819 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:46:26.819 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:46:26.819 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:46:26.819 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:26.819 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:26.819 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:46:26.819 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:46:26.819 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:46:26.819 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:26.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:27.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:27.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:27.017 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:46:27.053 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:46:27.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:27.054 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list-id 2024-09-21T07:46:27.129 INFO:teuthology.orchestra.run.smithi088.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2024-09-21T07:46:27.130 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:27.130 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:27.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:46:27.130 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2024-09-21T07:46:27.130 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:27.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:27.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:27.306 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 96:b5:1d:c4:6a:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:46:27.361 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.362 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::94b5:1dff:fec4:6a11/64 scope link 2024-09-21T07:46:27.362 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:27.364 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:27.419 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 96:b5:1d:c4:6a:11 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:46:27.420 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:46:27.420 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.420 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::94b5:1dff:fec4:6a11/64 scope link 2024-09-21T07:46:27.420 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:27.422 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2024-09-21T07:46:27.422 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2024-09-21T07:46:27.423 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:27.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:27.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:27.642 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:27.642 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:46:27.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.1 up 2024-09-21T07:46:27.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T07:46:27.642 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:46:27.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:27.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:27.808 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:27.808 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2024-09-21T07:46:27.843 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2024-09-21T07:46:27.844 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:27.844 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2024-09-21T07:46:27.918 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:46:27.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:27.919 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T07:46:28.112 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T07:46:28.113 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T07:46:28.113 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:46:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T07:46:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T07:46:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:46:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:28.133 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=2,mds_namespace=cephfs 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs". 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:46:28.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,mds_namespace=cephfs,key=2,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:46:28.164 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.2 does not contain SELinux labels. 2024-09-21T07:46:28.164 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:46:28.165 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:46:28.165 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:46:28.165 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:46:28.167 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:46:28.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:28.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2024-09-21T07:46:28.223 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:28.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:28.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:28.313 INFO:teuthology.orchestra.run.smithi088.stdout:client4577 2024-09-21T07:46:28.313 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:28.313 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:28.349 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:28.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:28.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:28.477 INFO:teuthology.orchestra.run.smithi088.stdout:client4577 2024-09-21T07:46:28.477 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:28.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:28.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:28.566 INFO:teuthology.orchestra.run.smithi088.stdout:client4577 2024-09-21T07:46:28.566 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.2 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1afec4d-fdf7-414b-83b4-ffa85a98fc94.client4577/status') 2024-09-21T07:46:28.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:28.665 INFO:teuthology.orchestra.run.smithi088.stdout:instance: client.4577 (3)172.21.15.88:0/1528715195 2024-09-21T07:46:28.665 INFO:teuthology.orchestra.run.smithi088.stdout:blocklisted: false 2024-09-21T07:46:28.665 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records in 2024-09-21T07:46:28.666 INFO:teuthology.orchestra.run.smithi088.stderr:0+1 records out 2024-09-21T07:46:28.666 INFO:teuthology.orchestra.run.smithi088.stderr:70 bytes copied, 5.2287e-05 s, 1.3 MB/s 2024-09-21T07:46:28.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:28.676 INFO:tasks.kclient:client.1 config is {'syntax': 'v1'} 2024-09-21T07:46:28.676 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:46:28.676 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:46:28.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:46:28.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb172400640 1 -- 172.21.15.88:0/87092346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb150004d40 msgr2=0x7fb150005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb172400640 1 --2- 172.21.15.88:0/87092346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb150004d40 0x7fb150005150 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb16c0aea60 tx=0x7fb150005830 comp rx=0 tx=0).stop 2024-09-21T07:46:28.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb16be00640 1 -- 172.21.15.88:0/87092346 reap_dead start 2024-09-21T07:46:28.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb172400640 1 -- 172.21.15.88:0/87092346 shutdown_connections 2024-09-21T07:46:28.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb172400640 1 -- 172.21.15.88:0/87092346 >> 172.21.15.88:0/87092346 conn(0x7fb16c06d730 msgr2=0x7fb16c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:28.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb172400640 1 -- 172.21.15.88:0/87092346 shutdown_connections 2024-09-21T07:46:28.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.896+0000 7fb172400640 1 -- 172.21.15.88:0/87092346 wait complete. 2024-09-21T07:46:28.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.898+0000 7fb172400640 1 Processor -- start 2024-09-21T07:46:28.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.899+0000 7fb172400640 1 -- start start 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb172400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb150004d40 0x7fb16c1f95b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb172400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c1f4eb0 0x7fb16c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb172400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 0x7fb16c09cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb172400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0b4da0 con 0x7fb16c15bd30 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb172400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb16c0b4c20 con 0x7fb16c1f4eb0 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb172400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb16c160e30 con 0x7fb150004d40 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb16be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 0x7fb16c09cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb16be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 0x7fb16c09cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49170/0 (socket says 172.21.15.88:49170) 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.900+0000 7fb16be00640 1 -- 172.21.15.88:0/2871576752 learned_addr learned my addr 172.21.15.88:0/2871576752 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16b400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb150004d40 0x7fb16c1f95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb162a00640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c1f4eb0 0x7fb16c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16be00640 1 -- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb150004d40 msgr2=0x7fb16c1f95b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16be00640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb150004d40 0x7fb16c1f95b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16be00640 1 -- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c1f4eb0 msgr2=0x7fb16c15b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:28.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16be00640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c1f4eb0 0x7fb16c15b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:28.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16be00640 1 -- 172.21.15.88:0/2871576752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb16c15c140 con 0x7fb16c15bd30 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb16be00640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 0x7fb16c09cb90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb16402b660 tx=0x7fb164004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb15c009720 con 0x7fb16c15bd30 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb164004d40 con 0x7fb16c15bd30 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.901+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb16c09a620 con 0x7fb16c15bd30 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.902+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb16400c3a0 con 0x7fb16c15bd30 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.902+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb16c09abe0 con 0x7fb16c15bd30 2024-09-21T07:46:28.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.903+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb16c0ad4f0 con 0x7fb16c15bd30 2024-09-21T07:46:28.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.904+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb16402bbd0 con 0x7fb16c15bd30 2024-09-21T07:46:28.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.905+0000 7fb163e00640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb148078bc0 0x7fb14807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:28.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.905+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb16403d020 con 0x7fb16c15bd30 2024-09-21T07:46:28.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.905+0000 7fb16b400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb148078bc0 0x7fb14807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:28.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.907+0000 7fb16b400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb148078bc0 0x7fb14807b0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009d30 tx=0x7fb15c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:28.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:28.910+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb16c15c140 con 0x7fb16c15bd30 2024-09-21T07:46:29.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.133+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fb16c1608c0 con 0x7fb16c15bd30 2024-09-21T07:46:29.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.134+0000 7fb163e00640 1 -- 172.21.15.88:0/2871576752 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7fb16c0ada30 con 0x7fb16c15bd30 2024-09-21T07:46:29.135 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb148078bc0 msgr2=0x7fb14807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb148078bc0 0x7fb14807b0d0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb15c009d30 tx=0x7fb15c005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 msgr2=0x7fb16c09cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 0x7fb16c09cb90 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb16402b660 tx=0x7fb164004a10 comp rx=0 tx=0).stop 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 shutdown_connections 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb148078bc0 0x7fb14807b0d0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb16c15bd30 0x7fb16c09cb90 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb16c1f4eb0 0x7fb16c15b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 --2- 172.21.15.88:0/2871576752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb150004d40 0x7fb16c1f95b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.136+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 >> 172.21.15.88:0/2871576752 conn(0x7fb16c06d730 msgr2=0x7fb16c15f1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:29.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.137+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 shutdown_connections 2024-09-21T07:46:29.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:29.137+0000 7fb172400640 1 -- 172.21.15.88:0/2871576752 wait complete. 2024-09-21T07:46:29.158 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:46:29.159 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:46:29.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:29.159 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:46:29.199 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:29.199 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:46:29.200 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:29.201 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:46:29.201 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42656sec preferred_lft 42656sec 2024-09-21T07:46:29.201 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:46:29.201 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:29.201 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T07:46:29.201 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:29.201 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:29.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add name ceph-brx type bridge 2024-09-21T07:46:29.201 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr flush dev ceph-brx 2024-09-21T07:46:29.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set ceph-brx up 2024-09-21T07:46:29.202 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T07:46:29.202 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:29.322 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:29.602 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:29.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:29.611 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:46:29.666 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:46:29.670 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:29.670 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:46:29.719 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:46:29.719 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:46:29.719 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:46:29.719 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:46:29.720 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:46:29.720 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:29.720 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:29.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:46:29.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:46:29.720 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:46:29.720 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:29.798 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:30.297 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:30.305 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:30.305 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:46:30.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:30.341 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-09-21T07:46:30.416 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:30.416 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:30.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:46:30.416 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2024-09-21T07:46:30.416 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:30.533 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:30.585 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:30.594 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2024-09-21T07:46:30.594 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:30.671 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:30.829 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:30.837 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:30.837 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:30.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:46:30.837 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T07:46:30.837 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:30.914 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:30.998 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:31.006 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:31.006 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:46:31.041 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:46:31.041 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:31.042 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:46:31.116 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:46:31.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:31.117 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:46:31.319 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:46:31.319 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:46:31.319 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:46:31.954 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:46:31.954 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:46:31.954 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:46:31.954 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:31.954 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:31.985 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:46:31.985 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:46:31.985 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:46:31.985 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:31.986 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:31.986 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:46:31.986 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:46:31.986 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:46:31.987 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:46:31.987 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:46:31.988 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:46:31.988 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:46:31.988 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:46:31.991 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:46:31.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:31.991 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:46:32.050 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:32.128 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:32.132 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:32.140 INFO:teuthology.orchestra.run.smithi175.stdout:client4586 2024-09-21T07:46:32.140 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:32.140 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:32.176 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:32.292 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:32.295 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:32.303 INFO:teuthology.orchestra.run.smithi175.stdout:client4586 2024-09-21T07:46:32.303 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:32.381 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:32.384 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:32.391 INFO:teuthology.orchestra.run.smithi175.stdout:client4586 2024-09-21T07:46:32.392 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1afec4d-fdf7-414b-83b4-ffa85a98fc94.client4586/status') 2024-09-21T07:46:32.468 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:32.489 INFO:teuthology.orchestra.run.smithi175.stdout:instance: client.4586 (3)192.168.0.1:0/2844181427 2024-09-21T07:46:32.490 INFO:teuthology.orchestra.run.smithi175.stdout:blocklisted: false 2024-09-21T07:46:32.490 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2024-09-21T07:46:32.490 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2024-09-21T07:46:32.490 INFO:teuthology.orchestra.run.smithi175.stderr:69 bytes copied, 5.1606e-05 s, 1.3 MB/s 2024-09-21T07:46:32.492 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:32.500 INFO:tasks.kclient:client.3 config is {'syntax': 'v1'} 2024-09-21T07:46:32.500 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T07:46:32.500 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T07:46:32.501 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:46:32.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 -- 172.21.15.88:0/326861255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 msgr2=0x7f4d380d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 --2- 172.21.15.88:0/326861255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 0x7f4d380d1990 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f4d30005fd0 tx=0x7f4d3002a840 comp rx=0 tx=0).stop 2024-09-21T07:46:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 -- 172.21.15.88:0/326861255 shutdown_connections 2024-09-21T07:46:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 --2- 172.21.15.88:0/326861255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 0x7f4d380d1990 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 -- 172.21.15.88:0/326861255 >> 172.21.15.88:0/326861255 conn(0x7f4d3808bfe0 msgr2=0x7f4d3809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 -- 172.21.15.88:0/326861255 shutdown_connections 2024-09-21T07:46:32.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.729+0000 7f4d37200640 1 -- 172.21.15.88:0/326861255 wait complete. 2024-09-21T07:46:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.731+0000 7f4d37200640 1 Processor -- start 2024-09-21T07:46:32.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.731+0000 7f4d37200640 1 -- start start 2024-09-21T07:46:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d37200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 0x7f4d38152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d37200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d380b9e00 0x7f4d380a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d37200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 0x7f4d380a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d37200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380d4700 con 0x7f4d380b3160 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d37200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4d380d4580 con 0x7f4d380d1580 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d37200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4d380d4880 con 0x7f4d380b9e00 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 0x7f4d38152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 0x7f4d38152c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49182/0 (socket says 172.21.15.88:49182) 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d35e00640 1 -- 172.21.15.88:0/1970209946 learned_addr learned my addr 172.21.15.88:0/1970209946 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d36800640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 0x7f4d380a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.733+0000 7f4d2d400640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d380b9e00 0x7f4d380a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d35e00640 1 -- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d380b9e00 msgr2=0x7f4d380a13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d35e00640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d380b9e00 0x7f4d380a13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d35e00640 1 -- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 msgr2=0x7f4d380a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d35e00640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 0x7f4d380a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d35e00640 1 -- 172.21.15.88:0/1970209946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4d3809fe20 con 0x7f4d380b3160 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d2d400640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d380b9e00 0x7f4d380a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d35e00640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 0x7f4d38152c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c430 tx=0x7f4d2800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28017070 con 0x7f4d380b3160 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4d280053e0 con 0x7f4d380b3160 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4d28003850 con 0x7f4d380b3160 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4d3815efc0 con 0x7f4d380b3160 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.734+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4d3809e2e0 con 0x7f4d380b3160 2024-09-21T07:46:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.735+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4d380d1990 con 0x7f4d380b3160 2024-09-21T07:46:32.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.737+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4d2801a070 con 0x7f4d380b3160 2024-09-21T07:46:32.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.737+0000 7f4d2ea00640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4d14078b90 0x7f4d1407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:32.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.738+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f4d2809a0a0 con 0x7f4d380b3160 2024-09-21T07:46:32.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.738+0000 7f4d2d400640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4d14078b90 0x7f4d1407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:32.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.739+0000 7f4d2d400640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4d14078b90 0x7f4d1407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4d3815f8c0 tx=0x7f4d20009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:32.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.742+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4d28061e80 con 0x7f4d380b3160 2024-09-21T07:46:32.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.962+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4d380d0550 con 0x7f4d380b3160 2024-09-21T07:46:32.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.963+0000 7f4d2ea00640 1 -- 172.21.15.88:0/1970209946 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) ==== 53+0+73 (secure 0 0 0) 0x7f4d280668b0 con 0x7f4d380b3160 2024-09-21T07:46:32.964 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4d14078b90 msgr2=0x7f4d1407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4d14078b90 0x7f4d1407b0a0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f4d3815f8c0 tx=0x7f4d20009040 comp rx=0 tx=0).stop 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 msgr2=0x7f4d38152c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 0x7f4d38152c30 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4d2800c430 tx=0x7f4d2800c8f0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 shutdown_connections 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4d14078b90 0x7f4d1407b0a0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4d380b9e00 0x7f4d380a13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4d380d1580 0x7f4d380a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 --2- 172.21.15.88:0/1970209946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4d380b3160 0x7f4d38152c30 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 >> 172.21.15.88:0/1970209946 conn(0x7f4d3808bfe0 msgr2=0x7f4d38162a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 shutdown_connections 2024-09-21T07:46:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:32.965+0000 7f4d37200640 1 -- 172.21.15.88:0/1970209946 wait complete. 2024-09-21T07:46:32.991 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T07:46:32.991 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:46:32.991 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:46:32.991 INFO:tasks.cephfs.mount:self.client.name = client.3 2024-09-21T07:46:32.991 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2024-09-21T07:46:32.991 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T07:46:32.992 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T07:46:32.992 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:46:32.992 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2024-09-21T07:46:32.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:32.992 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.029 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42652sec preferred_lft 42652sec 2024-09-21T07:46:33.030 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:46:33.031 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.032 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:46:33.127 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:46:33.130 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.130 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:46:33.166 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:46:33.166 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:46:33.166 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:46:33.166 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:46:33.166 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:46:33.167 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:33.167 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:33.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:46:33.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:46:33.167 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:46:33.167 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:33.283 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:33.353 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:33.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.361 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:46:33.397 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.397 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list-id 2024-09-21T07:46:33.473 INFO:teuthology.orchestra.run.smithi175.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2024-09-21T07:46:33.473 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:33.474 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:33.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:46:33.474 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2024-09-21T07:46:33.474 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:33.590 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:33.639 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:33.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.648 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T07:46:33.703 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:33.703 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 6a:92:2e:fc:05:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::6892:2eff:fefc:5cd/64 scope link 2024-09-21T07:46:33.704 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:33.706 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.761 INFO:teuthology.orchestra.run.smithi175.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:46:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 6a:92:2e:fc:05:cd brd ff:ff:ff:ff:ff:ff link-netnsid 0 2024-09-21T07:46:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2024-09-21T07:46:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::6892:2eff:fefc:5cd/64 scope link 2024-09-21T07:46:33.762 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:46:33.764 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2024-09-21T07:46:33.764 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:33.764 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:33.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2024-09-21T07:46:33.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2024-09-21T07:46:33.764 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2024-09-21T07:46:33.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2024-09-21T07:46:33.765 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2024-09-21T07:46:33.765 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:33.842 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:33 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:33.972 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:33.980 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:46:33.980 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:46:33.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.1 up 2024-09-21T07:46:33.980 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set dev brx.1 master ceph-brx 2024-09-21T07:46:33.980 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:46:34.060 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:34.139 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:34.148 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:34.148 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2024-09-21T07:46:34.183 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2024-09-21T07:46:34.183 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:46:34.183 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2024-09-21T07:46:34.258 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:46:34.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.259 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T07:46:34.452 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T07:46:34.453 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T07:46:34.453 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:46:34.472 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T07:46:34.472 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T07:46:34.472 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:46:34.472 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:34.472 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:34.501 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=3,mds_namespace=cephfs 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs". 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:46:34.502 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,mds_namespace=cephfs,key=3,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:46:34.503 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.3 does not contain SELinux labels. 2024-09-21T07:46:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:46:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:46:34.503 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:46:34.504 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:46:34.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:46:34.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:34.507 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2024-09-21T07:46:34.562 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:34.640 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:34.644 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:34.652 INFO:teuthology.orchestra.run.smithi175.stdout:client4734 2024-09-21T07:46:34.652 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:34.652 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:34.688 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:34.807 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:34.810 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:34.818 INFO:teuthology.orchestra.run.smithi175.stdout:client4734 2024-09-21T07:46:34.818 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:46:34.896 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:46:34.899 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:34.907 INFO:teuthology.orchestra.run.smithi175.stdout:client4734 2024-09-21T07:46:34.907 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.3 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/c1afec4d-fdf7-414b-83b4-ffa85a98fc94.client4734/status') 2024-09-21T07:46:34.984 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:34 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:46:35.005 INFO:teuthology.orchestra.run.smithi175.stdout:instance: client.4734 (3)172.21.15.175:0/2362276271 2024-09-21T07:46:35.005 INFO:teuthology.orchestra.run.smithi175.stdout:blocklisted: false 2024-09-21T07:46:35.006 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records in 2024-09-21T07:46:35.006 INFO:teuthology.orchestra.run.smithi175.stderr:0+1 records out 2024-09-21T07:46:35.006 INFO:teuthology.orchestra.run.smithi175.stderr:71 bytes copied, 6.4185e-05 s, 1.1 MB/s 2024-09-21T07:46:35.008 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:46:35 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:46:35.016 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2024-09-21T07:46:35.156 INFO:tasks.cephfs_test_runner:Starting test: test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate binary .meta file on legacy subvol root doesn't break the system 2024-09-21T07:46:35.156 INFO:tasks.cephfs_test_runner:test_binary_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:46:35.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 -- 172.21.15.88:0/2407522847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be40af170 msgr2=0x7f2be40b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 --2- 172.21.15.88:0/2407522847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be40af170 0x7f2be40b2f60 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4009830 tx=0x7f2bd402a480 comp rx=0 tx=0).stop 2024-09-21T07:46:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 -- 172.21.15.88:0/2407522847 shutdown_connections 2024-09-21T07:46:35.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 --2- 172.21.15.88:0/2407522847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415a0f0 0x7f2be415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 --2- 172.21.15.88:0/2407522847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be40af170 0x7f2be40b2f60 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 -- 172.21.15.88:0/2407522847 >> 172.21.15.88:0/2407522847 conn(0x7f2be40b1e40 msgr2=0x7f2be40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:35.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 -- 172.21.15.88:0/2407522847 shutdown_connections 2024-09-21T07:46:35.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.385+0000 7f2beb200640 1 -- 172.21.15.88:0/2407522847 wait complete. 2024-09-21T07:46:35.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.387+0000 7f2beb200640 1 Processor -- start 2024-09-21T07:46:35.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.387+0000 7f2beb200640 1 -- start start 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2beb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be40ad9c0 0x7f2be4148c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2beb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be40af170 0x7f2be41f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2beb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be415a0f0 0x7f2be4157b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2beb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2be406f290 con 0x7f2be415a0f0 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2beb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2be415a5e0 con 0x7f2be40af170 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2beb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2be406f410 con 0x7f2be40ad9c0 2024-09-21T07:46:35.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2bea800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be415a0f0 0x7f2be4157b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2be9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be40ad9c0 0x7f2be4148c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2be9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be40af170 0x7f2be41f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2be9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be40af170 0x7f2be41f6750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:42388/0 (socket says 172.21.15.88:42388) 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2be9400640 1 -- 172.21.15.88:0/13458564 learned_addr learned my addr 172.21.15.88:0/13458564 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2be9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be40ad9c0 0x7f2be4148c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:43646/0 (socket says 172.21.15.88:43646) 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.389+0000 7f2bea800640 1 -- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be40ad9c0 msgr2=0x7f2be4148c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bea800640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be40ad9c0 0x7f2be4148c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bea800640 1 -- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be40af170 msgr2=0x7f2be41f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bea800640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be40af170 0x7f2be41f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bea800640 1 -- 172.21.15.88:0/13458564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be41f7480 con 0x7f2be415a0f0 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bea800640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be415a0f0 0x7f2be4157b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e990 tx=0x7f2be000ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0004d20 con 0x7f2be415a0f0 2024-09-21T07:46:35.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2be000ca60 con 0x7f2be415a0f0 2024-09-21T07:46:35.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be409ac60 con 0x7f2be415a0f0 2024-09-21T07:46:35.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0005020 con 0x7f2be415a0f0 2024-09-21T07:46:35.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.390+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be409b220 con 0x7f2be415a0f0 2024-09-21T07:46:35.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.391+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be41f71f0 con 0x7f2be415a0f0 2024-09-21T07:46:35.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.392+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f2be00051c0 con 0x7f2be415a0f0 2024-09-21T07:46:35.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.393+0000 7f2bde000640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bc8078b90 0x7f2bc807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.393+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f2be009b540 con 0x7f2be415a0f0 2024-09-21T07:46:35.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.394+0000 7f2be9e00640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bc8078b90 0x7f2bc807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.395+0000 7f2be9e00640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bc8078b90 0x7f2bc807b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8000f60 tx=0x7f2bd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:35.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.398+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2be0016030 con 0x7f2be415a0f0 2024-09-21T07:46:35.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.582+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2be409aa50 con 0x7f2be415a0f0 2024-09-21T07:46:35.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.583+0000 7f2bde000640 1 -- 172.21.15.88:0/13458564 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7f2be0063320 con 0x7f2be415a0f0 2024-09-21T07:46:35.584 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:35.585 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bc8078b90 msgr2=0x7f2bc807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bc8078b90 0x7f2bc807b0a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f2bd8000f60 tx=0x7f2bd8009040 comp rx=0 tx=0).stop 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be415a0f0 msgr2=0x7f2be4157b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be415a0f0 0x7f2be4157b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e990 tx=0x7f2be000ed90 comp rx=0 tx=0).stop 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 shutdown_connections 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2bc8078b90 0x7f2bc807b0a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2be415a0f0 0x7f2be4157b00 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be40af170 0x7f2be41f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 --2- 172.21.15.88:0/13458564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be40ad9c0 0x7f2be4148c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 >> 172.21.15.88:0/13458564 conn(0x7f2be40b1e40 msgr2=0x7f2be40cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:35.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 shutdown_connections 2024-09-21T07:46:35.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.585+0000 7f2beb200640 1 -- 172.21.15.88:0/13458564 wait complete. 2024-09-21T07:46:35.609 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T07:46:04.825550+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T07:46:05.649137+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:46:35.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:46:35.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.833+0000 7f0efec00640 1 -- 172.21.15.88:0/3517027138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80ad3d0 msgr2=0x7f0ef80ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.833+0000 7f0efec00640 1 --2- 172.21.15.88:0/3517027138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef80ad7e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0ef0005fd0 tx=0x7f0ef002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.834+0000 7f0efec00640 1 -- 172.21.15.88:0/3517027138 shutdown_connections 2024-09-21T07:46:35.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.834+0000 7f0efec00640 1 --2- 172.21.15.88:0/3517027138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef80ad7e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.834+0000 7f0efec00640 1 --2- 172.21.15.88:0/3517027138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ef8160490 0x7f0ef815d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.834+0000 7f0efec00640 1 -- 172.21.15.88:0/3517027138 >> 172.21.15.88:0/3517027138 conn(0x7f0ef806d710 msgr2=0x7f0ef80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:35.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.834+0000 7f0efec00640 1 -- 172.21.15.88:0/3517027138 shutdown_connections 2024-09-21T07:46:35.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.834+0000 7f0efec00640 1 -- 172.21.15.88:0/3517027138 wait complete. 2024-09-21T07:46:35.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.836+0000 7f0efec00640 1 Processor -- start 2024-09-21T07:46:35.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.836+0000 7f0efec00640 1 -- start start 2024-09-21T07:46:35.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef815b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80b4f00 0x7f0ef815ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ef8160490 0x7f0ef8155090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef806f0f0 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ef806ef70 con 0x7f0ef80b4f00 2024-09-21T07:46:35.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ef806f270 con 0x7f0ef8160490 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef815b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef815b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58684/0 (socket says 172.21.15.88:58684) 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 -- 172.21.15.88:0/3243451118 learned_addr learned my addr 172.21.15.88:0/3243451118 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7400640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80b4f00 0x7f0ef815ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0efcc00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ef8160490 0x7f0ef8155090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 -- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ef8160490 msgr2=0x7f0ef8155090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ef8160490 0x7f0ef8155090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 -- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80b4f00 msgr2=0x7f0ef815ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80b4f00 0x7f0ef815ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.838+0000 7f0ef7e00640 1 -- 172.21.15.88:0/3243451118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ef81f5340 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.839+0000 7f0ef7e00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef815b090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800c460 tx=0x7f0ee800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.839+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8016020 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.839+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ee8012940 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.839+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ef81f5600 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.839+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee8012d90 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.839+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ef81f5bc0 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.840+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ef806f7c0 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.841+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0ee8019020 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.842+0000 7f0eefe00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078bc0 0x7f0ed807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.842+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0ee809a580 con 0x7f0ef80ad3d0 2024-09-21T07:46:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.842+0000 7f0ef7400640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078bc0 0x7f0ed807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:35.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.843+0000 7f0ef7400640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078bc0 0x7f0ed807b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8156c00 tx=0x7f0ee4005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:35.847+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ee8020070 con 0x7f0ef80ad3d0 2024-09-21T07:46:36.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.032+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0ef80b1d50 con 0x7f0ef80ad3d0 2024-09-21T07:46:36.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.033+0000 7f0eefe00640 1 -- 172.21.15.88:0/3243451118 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) ==== 93+0+10 (secure 0 0 0) 0x7f0ee8062360 con 0x7f0ef80ad3d0 2024-09-21T07:46:36.034 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:46:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078bc0 msgr2=0x7f0ed807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078bc0 0x7f0ed807b0d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f0ef8156c00 tx=0x7f0ee4005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 msgr2=0x7f0ef815b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef815b090 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0ee800c460 tx=0x7f0ee800c920 comp rx=0 tx=0).stop 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 shutdown_connections 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078bc0 0x7f0ed807b0d0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ef8160490 0x7f0ef8155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ef80b4f00 0x7f0ef815ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 --2- 172.21.15.88:0/3243451118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ef80ad3d0 0x7f0ef815b090 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 >> 172.21.15.88:0/3243451118 conn(0x7f0ef806d710 msgr2=0x7f0ef8073970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 shutdown_connections 2024-09-21T07:46:36.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.035+0000 7f0efec00640 1 -- 172.21.15.88:0/3243451118 wait complete. 2024-09-21T07:46:36.058 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T07:46:36.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 -- 172.21.15.88:0/1791998580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 msgr2=0x7fde540af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 --2- 172.21.15.88:0/1791998580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde540af0d0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fde540b3160 tx=0x7fde4c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 -- 172.21.15.88:0/1791998580 shutdown_connections 2024-09-21T07:46:36.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 --2- 172.21.15.88:0/1791998580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde540af0d0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 --2- 172.21.15.88:0/1791998580 >> v2:172.21.15.175:3300/0 conn(0x7fde54156ca0 0x7fde540ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 -- 172.21.15.88:0/1791998580 >> 172.21.15.88:0/1791998580 conn(0x7fde5406d620 msgr2=0x7fde540b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:36.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 -- 172.21.15.88:0/1791998580 shutdown_connections 2024-09-21T07:46:36.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.280+0000 7fde53e00640 1 -- 172.21.15.88:0/1791998580 wait complete. 2024-09-21T07:46:36.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.282+0000 7fde53e00640 1 Processor -- start 2024-09-21T07:46:36.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.282+0000 7fde53e00640 1 -- start start 2024-09-21T07:46:36.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde540b3f30 0x7fde5414be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde5409bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde54156ca0 0x7fde5409c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde540af9a0 con 0x7fde540cd000 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde540af820 con 0x7fde540b3f30 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde540afb20 con 0x7fde54156ca0 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde5409bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde5409bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58698/0 (socket says 172.21.15.88:58698) 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 -- 172.21.15.88:0/1938600887 learned_addr learned my addr 172.21.15.88:0/1938600887 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde52a00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde540b3f30 0x7fde5414be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde53400640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde54156ca0 0x7fde5409c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 -- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde54156ca0 msgr2=0x7fde5409c1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde54156ca0 0x7fde5409c1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 -- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde540b3f30 msgr2=0x7fde5414be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde540b3f30 0x7fde5414be00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde4a000640 1 -- 172.21.15.88:0/1938600887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde540708e0 con 0x7fde540cd000 2024-09-21T07:46:36.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.284+0000 7fde52a00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde540b3f30 0x7fde5414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:36.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.285+0000 7fde4a000640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde5409bc70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d740 tx=0x7fde3c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.285+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c017070 con 0x7fde540cd000 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.285+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde3c00de50 con 0x7fde540cd000 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.285+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde5409dfb0 con 0x7fde540cd000 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.285+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde3c004020 con 0x7fde540cd000 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.285+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde54153e20 con 0x7fde540cd000 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.286+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde540af0d0 con 0x7fde540cd000 2024-09-21T07:46:36.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.288+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fde3c0041c0 con 0x7fde540cd000 2024-09-21T07:46:36.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.288+0000 7fde4b400640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fde30078b90 0x7fde3007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.289+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fde3c09b160 con 0x7fde540cd000 2024-09-21T07:46:36.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.289+0000 7fde52a00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fde30078b90 0x7fde3007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.290+0000 7fde52a00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fde30078b90 0x7fde3007b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde540cf590 tx=0x7fde44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:36.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.293+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde3c020020 con 0x7fde540cd000 2024-09-21T07:46:36.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.477+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fde54156460 con 0x7fde540cd000 2024-09-21T07:46:36.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.478+0000 7fde4b400640 1 -- 172.21.15.88:0/1938600887 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) ==== 104+0+11 (secure 0 0 0) 0x7fde3c063590 con 0x7fde540cd000 2024-09-21T07:46:36.479 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 32 2024-09-21T07:46:36.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fde30078b90 msgr2=0x7fde3007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fde30078b90 0x7fde3007b0a0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fde540cf590 tx=0x7fde44009040 comp rx=0 tx=0).stop 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 msgr2=0x7fde5409bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde5409bc70 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fde3c00d740 tx=0x7fde3c00dc00 comp rx=0 tx=0).stop 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 shutdown_connections 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fde30078b90 0x7fde3007b0a0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde54156ca0 0x7fde5409c1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde540cd000 0x7fde5409bc70 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 --2- 172.21.15.88:0/1938600887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde540b3f30 0x7fde5414be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 >> 172.21.15.88:0/1938600887 conn(0x7fde5406d620 msgr2=0x7fde541554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 shutdown_connections 2024-09-21T07:46:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.480+0000 7fde53e00640 1 -- 172.21.15.88:0/1938600887 wait complete. 2024-09-21T07:46:36.506 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.729+0000 7f4606c00640 1 -- 172.21.15.88:0/121108136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600073a80 msgr2=0x7f46000cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.729+0000 7f4606c00640 1 --2- 172.21.15.88:0/121108136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600073a80 0x7f46000cd610 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f45f8005fd0 tx=0x7f45f802aa90 comp rx=0 tx=0).stop 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.729+0000 7f4606c00640 1 -- 172.21.15.88:0/121108136 shutdown_connections 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.729+0000 7f4606c00640 1 --2- 172.21.15.88:0/121108136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600073a80 0x7f46000cd610 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.729+0000 7f4606c00640 1 --2- 172.21.15.88:0/121108136 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4600145a40 0x7f4600073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.729+0000 7f4606c00640 1 -- 172.21.15.88:0/121108136 >> 172.21.15.88:0/121108136 conn(0x7f460006d1c0 msgr2=0x7f460006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:36.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.730+0000 7f4606c00640 1 -- 172.21.15.88:0/121108136 shutdown_connections 2024-09-21T07:46:36.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.730+0000 7f4606c00640 1 -- 172.21.15.88:0/121108136 wait complete. 2024-09-21T07:46:36.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.732+0000 7f4606c00640 1 Processor -- start 2024-09-21T07:46:36.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.732+0000 7f4606c00640 1 -- start start 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.733+0000 7f4606c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4600073a80 0x7f460013e020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.733+0000 7f4606c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 0x7f460015ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.733+0000 7f4606c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600145a40 0x7f46000aee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.733+0000 7f4606c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46000ce130 con 0x7f4600145550 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.733+0000 7f4606c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46000cdfb0 con 0x7f4600145a40 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.733+0000 7f4606c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46000ce2b0 con 0x7f4600073a80 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 0x7f460015ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 0x7f460015ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58718/0 (socket says 172.21.15.88:58718) 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 -- 172.21.15.88:0/2887706868 learned_addr learned my addr 172.21.15.88:0/2887706868 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f4604c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600145a40 0x7f46000aee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ffe00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4600073a80 0x7f460013e020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 -- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4600073a80 msgr2=0x7f460013e020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4600073a80 0x7f460013e020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 -- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600145a40 msgr2=0x7f46000aee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600145a40 0x7f46000aee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 -- 172.21.15.88:0/2887706868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f460013e560 con 0x7f4600145550 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.734+0000 7f45ff400640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 0x7f460015ddb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c940 tx=0x7f45ec00ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.735+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec007bb0 con 0x7f4600145550 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.735+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ec007e40 con 0x7f4600145550 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.735+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ec00f690 con 0x7f4600145550 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.735+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f460015e2f0 con 0x7f4600145550 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.735+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46000ae070 con 0x7f4600145550 2024-09-21T07:46:36.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.736+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46000ce850 con 0x7f4600145550 2024-09-21T07:46:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.737+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f45ec00f830 con 0x7f4600145550 2024-09-21T07:46:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.738+0000 7f45f7e00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45e0078b90 0x7f45e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.738+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f45ec09b3f0 con 0x7f4600145550 2024-09-21T07:46:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.739+0000 7f45ffe00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45e0078b90 0x7f45e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:36.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.739+0000 7f45ffe00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45e0078b90 0x7f45e007b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f45f0006fd0 tx=0x7f45f0009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:36.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.742+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45ec014030 con 0x7f4600145550 2024-09-21T07:46:36.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.926+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f460006fe70 con 0x7f4600145550 2024-09-21T07:46:36.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.927+0000 7f45f7e00640 1 -- 172.21.15.88:0/2887706868 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) ==== 100+0+11 (secure 0 0 0) 0x7f45ec063200 con 0x7f4600145550 2024-09-21T07:46:36.928 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 64 2024-09-21T07:46:36.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45e0078b90 msgr2=0x7f45e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45e0078b90 0x7f45e007b0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f45f0006fd0 tx=0x7f45f0009060 comp rx=0 tx=0).stop 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 msgr2=0x7f460015ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 0x7f460015ddb0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f45ec00c940 tx=0x7f45ec00ce00 comp rx=0 tx=0).stop 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 shutdown_connections 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45e0078b90 0x7f45e007b0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4600145a40 0x7f46000aee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4600145550 0x7f460015ddb0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 --2- 172.21.15.88:0/2887706868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4600073a80 0x7f460013e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 >> 172.21.15.88:0/2887706868 conn(0x7f460006d1c0 msgr2=0x7f4600142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 shutdown_connections 2024-09-21T07:46:36.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:36.929+0000 7f4606c00640 1 -- 172.21.15.88:0/2887706868 wait complete. 2024-09-21T07:46:36.955 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-21T07:46:37.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 -- 172.21.15.88:0/561125196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1dc003c30 msgr2=0x7fb1dc004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 --2- 172.21.15.88:0/561125196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1dc003c30 0x7fb1dc004080 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8009b50 tx=0x7fb1dc005830 comp rx=0 tx=0).stop 2024-09-21T07:46:37.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 -- 172.21.15.88:0/561125196 shutdown_connections 2024-09-21T07:46:37.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 --2- 172.21.15.88:0/561125196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1dc003c30 0x7fb1dc004080 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 -- 172.21.15.88:0/561125196 >> 172.21.15.88:0/561125196 conn(0x7fb1f406d730 msgr2=0x7fb1f40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:37.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 -- 172.21.15.88:0/561125196 shutdown_connections 2024-09-21T07:46:37.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.178+0000 7fb1fb800640 1 -- 172.21.15.88:0/561125196 wait complete. 2024-09-21T07:46:37.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.181+0000 7fb1fb800640 1 Processor -- start 2024-09-21T07:46:37.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.181+0000 7fb1fb800640 1 -- start start 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.182+0000 7fb1fb800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc003c30 0x7fb1f4073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.182+0000 7fb1fb800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 0x7fb1f414f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.182+0000 7fb1fb800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f409b510 0x7fb1f409b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.182+0000 7fb1fb800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f40ad4f0 con 0x7fb1f409dc70 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.182+0000 7fb1fb800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1f40b4e30 con 0x7fb1dc003c30 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.182+0000 7fb1fb800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1f40ad670 con 0x7fb1f409b510 2024-09-21T07:46:37.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 0x7fb1f414f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 0x7fb1f414f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58730/0 (socket says 172.21.15.88:58730) 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 -- 172.21.15.88:0/2108183072 learned_addr learned my addr 172.21.15.88:0/2108183072 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f8e00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc003c30 0x7fb1f4073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f9800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f409b510 0x7fb1f409b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 -- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f409b510 msgr2=0x7fb1f409b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f409b510 0x7fb1f409b920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 -- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc003c30 msgr2=0x7fb1f4073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc003c30 0x7fb1f4073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 -- 172.21.15.88:0/2108183072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1f414e9a0 con 0x7fb1f409dc70 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f9800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f409b510 0x7fb1f409b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:37.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.183+0000 7fb1f3e00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 0x7fb1f414f720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0006400 tx=0x7fb1e00039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:37.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.184+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0015070 con 0x7fb1f409dc70 2024-09-21T07:46:37.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.184+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb1e0011840 con 0x7fb1f409dc70 2024-09-21T07:46:37.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.184+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1e0011c90 con 0x7fb1f409dc70 2024-09-21T07:46:37.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.184+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1f414ec40 con 0x7fb1f409dc70 2024-09-21T07:46:37.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.184+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1f414f200 con 0x7fb1f409dc70 2024-09-21T07:46:37.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.185+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1f41535c0 con 0x7fb1f409dc70 2024-09-21T07:46:37.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.186+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb1e0018070 con 0x7fb1f409dc70 2024-09-21T07:46:37.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.187+0000 7fb1f0c00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb1d4078b90 0x7fb1d407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.187+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb1e009b900 con 0x7fb1f409dc70 2024-09-21T07:46:37.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.188+0000 7fb1f8e00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb1d4078b90 0x7fb1d407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.188+0000 7fb1f8e00640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb1d4078b90 0x7fb1d407b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8035000 tx=0x7fb1e80046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:37.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.191+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1e0063770 con 0x7fb1f409dc70 2024-09-21T07:46:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.447+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fb1f409e100 con 0x7fb1f409dc70 2024-09-21T07:46:37.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.643+0000 7fb1f0c00640 1 -- 172.21.15.88:0/2108183072 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v55) ==== 161+0+0 (secure 0 0 0) 0x7fb1e00681a0 con 0x7fb1f409dc70 2024-09-21T07:46:37.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb1d4078b90 msgr2=0x7fb1d407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb1d4078b90 0x7fb1d407b0a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb1e8035000 tx=0x7fb1e80046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 msgr2=0x7fb1f414f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 0x7fb1f414f720 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fb1e0006400 tx=0x7fb1e00039c0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 shutdown_connections 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb1d4078b90 0x7fb1d407b0a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1f409b510 0x7fb1f409b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1f409dc70 0x7fb1f414f720 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 --2- 172.21.15.88:0/2108183072 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1dc003c30 0x7fb1f4073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 >> 172.21.15.88:0/2108183072 conn(0x7fb1f406d730 msgr2=0x7fb1f40b00a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 shutdown_connections 2024-09-21T07:46:37.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.645+0000 7fb1fb800640 1 -- 172.21.15.88:0/2108183072 wait complete. 2024-09-21T07:46:37.669 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:46:37.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 -- 172.21.15.88:0/2364843185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 msgr2=0x7fdbc01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 --2- 172.21.15.88:0/2364843185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc01535c0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fdbc0072d60 tx=0x7fdbb002a560 comp rx=0 tx=0).stop 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 -- 172.21.15.88:0/2364843185 shutdown_connections 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 --2- 172.21.15.88:0/2364843185 >> v2:172.21.15.175:3300/0 conn(0x7fdbc00ad0e0 0x7fdbc00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 --2- 172.21.15.88:0/2364843185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc01535c0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 -- 172.21.15.88:0/2364843185 >> 172.21.15.88:0/2364843185 conn(0x7fdbc006d730 msgr2=0x7fdbc00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 -- 172.21.15.88:0/2364843185 shutdown_connections 2024-09-21T07:46:37.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.891+0000 7fdbc5400640 1 -- 172.21.15.88:0/2364843185 wait complete. 2024-09-21T07:46:37.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.893+0000 7fdbc5400640 1 Processor -- start 2024-09-21T07:46:37.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.894+0000 7fdbc5400640 1 -- start start 2024-09-21T07:46:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbc5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc00ad0e0 0x7fdbc01f3510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbc5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc014e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbc5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc0160310 0x7fdbc01f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbc5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc00b4c20 con 0x7fdbc015d760 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbc5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc0153c70 con 0x7fdbc00ad0e0 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbc5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc00b4da0 con 0x7fdbc0160310 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbbde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc014e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbbde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc014e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58752/0 (socket says 172.21.15.88:58752) 2024-09-21T07:46:37.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.895+0000 7fdbbde00640 1 -- 172.21.15.88:0/3169337118 learned_addr learned my addr 172.21.15.88:0/3169337118 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:37.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbe800640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc00ad0e0 0x7fdbc01f3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbde00640 1 -- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc0160310 msgr2=0x7fdbc01f7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbde00640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc0160310 0x7fdbc01f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbde00640 1 -- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc00ad0e0 msgr2=0x7fdbc01f3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbde00640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc00ad0e0 0x7fdbc01f3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbde00640 1 -- 172.21.15.88:0/3169337118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc0156730 con 0x7fdbc015d760 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbe800640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc00ad0e0 0x7fdbc01f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbbde00640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc014e850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00d6d0 tx=0x7fdbac00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac017070 con 0x7fdbc015d760 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbac004310 con 0x7fdbc015d760 2024-09-21T07:46:37.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc01569a0 con 0x7fdbc015d760 2024-09-21T07:46:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.896+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbac009a80 con 0x7fdbc015d760 2024-09-21T07:46:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.897+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc01f6b40 con 0x7fdbc015d760 2024-09-21T07:46:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.897+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc0160ff0 con 0x7fdbc015d760 2024-09-21T07:46:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.899+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdbac0035f0 con 0x7fdbc015d760 2024-09-21T07:46:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.899+0000 7fdbb6a00640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdba0078bc0 0x7fdba007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:37.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.900+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fdbac09af90 con 0x7fdbc015d760 2024-09-21T07:46:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.900+0000 7fdbbe800640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdba0078bc0 0x7fdba007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:37.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.901+0000 7fdbbe800640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdba0078bc0 0x7fdba007b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0033000 tx=0x7fdbb0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:37.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:37.904+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdbac063450 con 0x7fdbc015d760 2024-09-21T07:46:38.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.092+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fdbc009e1f0 con 0x7fdbc015d760 2024-09-21T07:46:38.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.092+0000 7fdbb6a00640 1 -- 172.21.15.88:0/3169337118 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v31) ==== 59+0+2294 (secure 0 0 0) 0x7fdbac062d50 con 0x7fdbc015d760 2024-09-21T07:46:38.093 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:46:38.093 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:46:38.093 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:46:38.094 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:46:38.095 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:46:38.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdba0078bc0 msgr2=0x7fdba007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdba0078bc0 0x7fdba007b0d0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fdbb0033000 tx=0x7fdbb0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 msgr2=0x7fdbc014e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc014e850 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fdbac00d6d0 tx=0x7fdbac00db90 comp rx=0 tx=0).stop 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 shutdown_connections 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdba0078bc0 0x7fdba007b0d0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc0160310 0x7fdbc01f7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc015d760 0x7fdbc014e850 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 --2- 172.21.15.88:0/3169337118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc00ad0e0 0x7fdbc01f3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 >> 172.21.15.88:0/3169337118 conn(0x7fdbc006d730 msgr2=0x7fdbc006f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 shutdown_connections 2024-09-21T07:46:38.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.095+0000 7fdbc5400640 1 -- 172.21.15.88:0/3169337118 wait complete. 2024-09-21T07:46:38.118 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.343+0000 7f0d87800640 1 -- 172.21.15.88:0/1806226049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d68004d40 msgr2=0x7f0d68005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.343+0000 7f0d87800640 1 --2- 172.21.15.88:0/1806226049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d68004d40 0x7f0d68005150 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0d800d1be0 tx=0x7f0d68005830 comp rx=0 tx=0).stop 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.343+0000 7f0d85800640 1 -- 172.21.15.88:0/1806226049 reap_dead start 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.344+0000 7f0d87800640 1 -- 172.21.15.88:0/1806226049 shutdown_connections 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.344+0000 7f0d87800640 1 -- 172.21.15.88:0/1806226049 >> 172.21.15.88:0/1806226049 conn(0x7f0d800d2550 msgr2=0x7f0d800d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.344+0000 7f0d87800640 1 -- 172.21.15.88:0/1806226049 shutdown_connections 2024-09-21T07:46:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.344+0000 7f0d87800640 1 -- 172.21.15.88:0/1806226049 wait complete. 2024-09-21T07:46:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.346+0000 7f0d87800640 1 Processor -- start 2024-09-21T07:46:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.346+0000 7f0d87800640 1 -- start start 2024-09-21T07:46:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.347+0000 7f0d87800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d68004d40 0x7f0d801df370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.347+0000 7f0d87800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8009cf20 0x7f0d801df8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d87800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 0x7f0d801f7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d87800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d800b7630 con 0x7f0d801f6f30 2024-09-21T07:46:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d87800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d800cee20 con 0x7f0d8009cf20 2024-09-21T07:46:38.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d87800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d800b5440 con 0x7f0d68004d40 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 0x7f0d801f7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 0x7f0d801f7340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58768/0 (socket says 172.21.15.88:58768) 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 -- 172.21.15.88:0/430562375 learned_addr learned my addr 172.21.15.88:0/430562375 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d84e00640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d68004d40 0x7f0d801df370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d7fe00640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8009cf20 0x7f0d801df8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 -- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d68004d40 msgr2=0x7f0d801df370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d68004d40 0x7f0d801df370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 -- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8009cf20 msgr2=0x7f0d801df8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8009cf20 0x7f0d801df8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.348+0000 7f0d85800640 1 -- 172.21.15.88:0/430562375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d8009f9f0 con 0x7f0d801f6f30 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.349+0000 7f0d85800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 0x7f0d801f7340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0d78036040 tx=0x7f0d78033bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.349+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78043070 con 0x7f0d801f6f30 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.349+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d78033cb0 con 0x7f0d801f6f30 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.349+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d8009c970 con 0x7f0d801f6f30 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.349+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d78038040 con 0x7f0d801f6f30 2024-09-21T07:46:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.349+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d8009fc60 con 0x7f0d801f6f30 2024-09-21T07:46:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.350+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d800b6550 con 0x7f0d801f6f30 2024-09-21T07:46:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.351+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0d78003bd0 con 0x7f0d801f6f30 2024-09-21T07:46:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.352+0000 7f0d7cc00640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d60078bc0 0x7f0d6007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:38.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.352+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f0d780bae80 con 0x7f0d801f6f30 2024-09-21T07:46:38.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.353+0000 7f0d84e00640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d60078bc0 0x7f0d6007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:38.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.354+0000 7f0d84e00640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d60078bc0 0x7f0d6007b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000fb0 tx=0x7f0d74006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:38.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.357+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d78082bd0 con 0x7f0d801f6f30 2024-09-21T07:46:38.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.543+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f0d800d04e0 con 0x7f0d801f6f30 2024-09-21T07:46:38.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.547+0000 7f0d7cc00640 1 -- 172.21.15.88:0/430562375 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f0d78087600 con 0x7f0d801f6f30 2024-09-21T07:46:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d60078bc0 msgr2=0x7f0d6007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d60078bc0 0x7f0d6007b0d0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0d74000fb0 tx=0x7f0d74006040 comp rx=0 tx=0).stop 2024-09-21T07:46:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 msgr2=0x7f0d801f7340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 0x7f0d801f7340 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0d78036040 tx=0x7f0d78033bc0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 shutdown_connections 2024-09-21T07:46:38.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d60078bc0 0x7f0d6007b0d0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d801f6f30 0x7f0d801f7340 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d8009cf20 0x7f0d801df8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 --2- 172.21.15.88:0/430562375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d68004d40 0x7f0d801df370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:38.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 >> 172.21.15.88:0/430562375 conn(0x7f0d800d2550 msgr2=0x7f0d80072200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:38.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 shutdown_connections 2024-09-21T07:46:38.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:38.550+0000 7f0d87800640 1 -- 172.21.15.88:0/430562375 wait complete. 2024-09-21T07:46:38.575 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:46:38.575 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:38.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:46:38.765 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:46:38.765 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:38.766 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:38.802 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:38.802 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:38.878 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:38.878 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:38.953 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:46:39.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 -- 172.21.15.88:0/1461367485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce415ce30 msgr2=0x7f5ce415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 --2- 172.21.15.88:0/1461367485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce415ce30 0x7f5ce415d240 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5cdc005fd0 tx=0x7f5cdc02aa70 comp rx=0 tx=0).stop 2024-09-21T07:46:39.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 -- 172.21.15.88:0/1461367485 shutdown_connections 2024-09-21T07:46:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 --2- 172.21.15.88:0/1461367485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce415ce30 0x7f5ce415d240 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 --2- 172.21.15.88:0/1461367485 >> v2:172.21.15.175:3300/0 conn(0x7f5ce40cfc80 0x7f5ce4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 -- 172.21.15.88:0/1461367485 >> 172.21.15.88:0/1461367485 conn(0x7f5ce406d3d0 msgr2=0x7f5ce40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 -- 172.21.15.88:0/1461367485 shutdown_connections 2024-09-21T07:46:39.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.213+0000 7f5ceae00640 1 -- 172.21.15.88:0/1461367485 wait complete. 2024-09-21T07:46:39.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.215+0000 7f5ceae00640 1 Processor -- start 2024-09-21T07:46:39.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.215+0000 7f5ceae00640 1 -- start start 2024-09-21T07:46:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ce40705b0 0x7f5ce41f68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 0x7f5ce409ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ceae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce415ce30 0x7f5ce40ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ceae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce40cd050 con 0x7f5ce40cfc80 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ceae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ce40cced0 con 0x7f5ce415ce30 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ceae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ce40cd1d0 con 0x7f5ce40705b0 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ce3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 0x7f5ce409ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ce3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 0x7f5ce409ceb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58776/0 (socket says 172.21.15.88:58776) 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ce3400640 1 -- 172.21.15.88:0/2227180729 learned_addr learned my addr 172.21.15.88:0/2227180729 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ce3e00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ce40705b0 0x7f5ce41f68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.217+0000 7f5ce8e00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce415ce30 0x7f5ce40ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ce3400640 1 -- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ce40705b0 msgr2=0x7f5ce41f68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ce3400640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ce40705b0 0x7f5ce41f68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ce3400640 1 -- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce415ce30 msgr2=0x7f5ce40ae340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ce3400640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce415ce30 0x7f5ce40ae340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ce3400640 1 -- 172.21.15.88:0/2227180729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ce41f6df0 con 0x7f5ce40cfc80 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ce3400640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 0x7f5ce409ceb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c370 tx=0x7f5cd000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd00052d0 con 0x7f5ce40cfc80 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cd0007020 con 0x7f5ce40cfc80 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ce40743c0 con 0x7f5ce40cfc80 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cd0007470 con 0x7f5ce40cfc80 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.218+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ce4074980 con 0x7f5ce40cfc80 2024-09-21T07:46:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.220+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ce40cd6f0 con 0x7f5ce40cfc80 2024-09-21T07:46:39.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.221+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5cd00067d0 con 0x7f5ce40cfc80 2024-09-21T07:46:39.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.222+0000 7f5cdbe00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5cc4078b90 0x7f5cc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.222+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f5cd009a180 con 0x7f5ce40cfc80 2024-09-21T07:46:39.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.222+0000 7f5ce3e00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5cc4078b90 0x7f5cc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.223+0000 7f5ce3e00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5cc4078b90 0x7f5cc407b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ce409e510 tx=0x7f5cd4005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:39.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.227+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cd0061f60 con 0x7f5ce40cfc80 2024-09-21T07:46:39.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.422+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f5ce4074150 con 0x7f5ce40cfc80 2024-09-21T07:46:39.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.422+0000 7f5cdbe00640 1 -- 172.21.15.88:0/2227180729 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f5cd0066990 con 0x7f5ce40cfc80 2024-09-21T07:46:39.423 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.424+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5cc4078b90 msgr2=0x7f5cc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.424+0000 7f5ceae00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5cc4078b90 0x7f5cc407b0a0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f5ce409e510 tx=0x7f5cd4005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.424+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 msgr2=0x7f5ce409ceb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.424+0000 7f5ceae00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 0x7f5ce409ceb0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5cd000c370 tx=0x7f5cd000c830 comp rx=0 tx=0).stop 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 shutdown_connections 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5cc4078b90 0x7f5cc407b0a0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce415ce30 0x7f5ce40ae340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce40cfc80 0x7f5ce409ceb0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 --2- 172.21.15.88:0/2227180729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ce40705b0 0x7f5ce41f68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 >> 172.21.15.88:0/2227180729 conn(0x7f5ce406d3d0 msgr2=0x7f5ce406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 shutdown_connections 2024-09-21T07:46:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.425+0000 7f5ceae00640 1 -- 172.21.15.88:0/2227180729 wait complete. 2024-09-21T07:46:39.450 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:46:39.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2024-09-21T07:46:39.510 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:46:39.511 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:39.511 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:46:39.558 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:39.558 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:39.598 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:39.599 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:39.677 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:39.677 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:39.755 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:46:39.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 -- 172.21.15.88:0/1572568293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc070cc0 msgr2=0x7f55cc156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 --2- 172.21.15.88:0/1572568293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc070cc0 0x7f55cc156e40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f55b8005fd0 tx=0x7f55b802a5a0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 -- 172.21.15.88:0/1572568293 shutdown_connections 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 --2- 172.21.15.88:0/1572568293 >> v2:172.21.15.175:3300/0 conn(0x7f55cc15ce30 0x7f55cc15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 --2- 172.21.15.88:0/1572568293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc070cc0 0x7f55cc156e40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 -- 172.21.15.88:0/1572568293 >> 172.21.15.88:0/1572568293 conn(0x7f55cc06d3d0 msgr2=0x7f55cc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 -- 172.21.15.88:0/1572568293 shutdown_connections 2024-09-21T07:46:39.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.977+0000 7f55d1800640 1 -- 172.21.15.88:0/1572568293 wait complete. 2024-09-21T07:46:39.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.979+0000 7f55d1800640 1 Processor -- start 2024-09-21T07:46:39.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.979+0000 7f55d1800640 1 -- start start 2024-09-21T07:46:39.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55d1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55cc0cfc80 0x7f55cc159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55d1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55cc15ce30 0x7f55cc1f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55d1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 0x7f55cc1f93a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55d1800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0cd050 con 0x7f55cc0b19e0 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55d1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55cc0cced0 con 0x7f55cc0cfc80 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55d1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55cc0cd1d0 con 0x7f55cc15ce30 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55cb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 0x7f55cc1f93a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55ca000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55cc15ce30 0x7f55cc1f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55caa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55cc0cfc80 0x7f55cc159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55cb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 0x7f55cc1f93a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58780/0 (socket says 172.21.15.88:58780) 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.981+0000 7f55cb400640 1 -- 172.21.15.88:0/2421253262 learned_addr learned my addr 172.21.15.88:0/2421253262 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55cb400640 1 -- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55cc15ce30 msgr2=0x7f55cc1f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55cb400640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55cc15ce30 0x7f55cc1f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55cb400640 1 -- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55cc0cfc80 msgr2=0x7f55cc159b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55cb400640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55cc0cfc80 0x7f55cc159b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:39.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55cb400640 1 -- 172.21.15.88:0/2421253262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55cc15a080 con 0x7f55cc0b19e0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55cb400640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 0x7f55cc1f93a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55c400d9d0 tx=0x7f55c400de90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c4017070 con 0x7f55cc0b19e0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f55c4003970 con 0x7f55cc0b19e0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55cc0b07a0 con 0x7f55cc0b19e0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55c400be60 con 0x7f55cc0b19e0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.982+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55cc0b0d60 con 0x7f55cc0b19e0 2024-09-21T07:46:39.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.983+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55cc0cd6f0 con 0x7f55cc0b19e0 2024-09-21T07:46:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.985+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f55c4010460 con 0x7f55cc0b19e0 2024-09-21T07:46:39.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.985+0000 7f55c2a00640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f55ac078bc0 0x7f55ac07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:39.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.986+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f55c409b080 con 0x7f55cc0b19e0 2024-09-21T07:46:39.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.986+0000 7f55caa00640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f55ac078bc0 0x7f55ac07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:39.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.987+0000 7f55caa00640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f55ac078bc0 0x7f55ac07b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55cc09e510 tx=0x7f55bc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:39.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:39.990+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55cc15a080 con 0x7f55cc0b19e0 2024-09-21T07:46:40.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.186+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f55cc0afa70 con 0x7f55cc0b19e0 2024-09-21T07:46:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.187+0000 7f55c2a00640 1 -- 172.21.15.88:0/2421253262 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7f55c4062e60 con 0x7f55cc0b19e0 2024-09-21T07:46:40.188 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f55ac078bc0 msgr2=0x7f55ac07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f55ac078bc0 0x7f55ac07b0d0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f55cc09e510 tx=0x7f55bc009040 comp rx=0 tx=0).stop 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 msgr2=0x7f55cc1f93a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 0x7f55cc1f93a0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f55c400d9d0 tx=0x7f55c400de90 comp rx=0 tx=0).stop 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 shutdown_connections 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f55ac078bc0 0x7f55ac07b0d0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55cc15ce30 0x7f55cc1f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55cc0b19e0 0x7f55cc1f93a0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 --2- 172.21.15.88:0/2421253262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55cc0cfc80 0x7f55cc159b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 >> 172.21.15.88:0/2421253262 conn(0x7f55cc06d3d0 msgr2=0x7f55cc15efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:40.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.189+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 shutdown_connections 2024-09-21T07:46:40.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.190+0000 7f55d1800640 1 -- 172.21.15.88:0/2421253262 wait complete. 2024-09-21T07:46:40.213 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:46:40.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.213 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2024-09-21T07:46:40.280 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:46:40.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:40.280 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:46:40.325 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:40.325 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:40.360 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:40.360 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:40.435 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:46:40.435 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:40.511 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:46:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.772+0000 7fb4c2600640 1 -- 172.21.15.88:0/1417111497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc15e770 msgr2=0x7fb4bc0b7a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.772+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1417111497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc15e770 0x7fb4bc0b7a00 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005fd0 tx=0x7fb4a402a540 comp rx=0 tx=0).stop 2024-09-21T07:46:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.773+0000 7fb4c2600640 1 -- 172.21.15.88:0/1417111497 shutdown_connections 2024-09-21T07:46:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.773+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1417111497 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4bc0b7f40 0x7fb4bc0b5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.773+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1417111497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc15e770 0x7fb4bc0b7a00 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.773+0000 7fb4c2600640 1 -- 172.21.15.88:0/1417111497 >> 172.21.15.88:0/1417111497 conn(0x7fb4bc0d44f0 msgr2=0x7fb4bc0d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:40.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.773+0000 7fb4c2600640 1 -- 172.21.15.88:0/1417111497 shutdown_connections 2024-09-21T07:46:40.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.773+0000 7fb4c2600640 1 -- 172.21.15.88:0/1417111497 wait complete. 2024-09-21T07:46:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.775+0000 7fb4c2600640 1 Processor -- start 2024-09-21T07:46:40.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.775+0000 7fb4c2600640 1 -- start start 2024-09-21T07:46:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c2600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4bc0b7f40 0x7fb4bc1e7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c2600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4bc15e770 0x7fb4bc0aafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c2600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 0x7fb4bc149c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c2600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0d39a0 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c2600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4bc0d3820 con 0x7fb4bc0b7f40 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c2600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4bc0d3b20 con 0x7fb4bc15e770 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 0x7fb4bc149c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4bc0b7f40 0x7fb4bc1e7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 0x7fb4bc149c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58798/0 (socket says 172.21.15.88:58798) 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 -- 172.21.15.88:0/1548158847 learned_addr learned my addr 172.21.15.88:0/1548158847 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4b3e00640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4bc15e770 0x7fb4bc0aafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 -- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4bc15e770 msgr2=0x7fb4bc0aafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4bc15e770 0x7fb4bc0aafa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 -- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4bc0b7f40 msgr2=0x7fb4bc1e7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4bc0b7f40 0x7fb4bc1e7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.777+0000 7fb4c1c00640 1 -- 172.21.15.88:0/1548158847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4bc1e9420 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.778+0000 7fb4c1c00640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 0x7fb4bc149c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400eab0 tx=0x7fb4b400ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.778+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b400cea0 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.778+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb4b4004b20 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.778+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4bc1e8380 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.778+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4b4005020 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.778+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4bc1e8940 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.779+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4bc0cf6c0 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.780+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb4b40051c0 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.781+0000 7fb4b9600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb498078bc0 0x7fb49807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:40.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.781+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fb4b409b8b0 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.782+0000 7fb4c1200640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb498078bc0 0x7fb49807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:40.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.783+0000 7fb4c1200640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb498078bc0 0x7fb49807b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac001720 tx=0x7fb4ac009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:40.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.786+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4b4063690 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.987+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb4bc1e8110 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.987+0000 7fb4b9600640 1 -- 172.21.15.88:0/1548158847 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fb4b40680c0 con 0x7fb4bc0ab4e0 2024-09-21T07:46:40.988 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb498078bc0 msgr2=0x7fb49807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb498078bc0 0x7fb49807b0d0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb4ac001720 tx=0x7fb4ac009040 comp rx=0 tx=0).stop 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 msgr2=0x7fb4bc149c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 0x7fb4bc149c40 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fb4b400eab0 tx=0x7fb4b400ef70 comp rx=0 tx=0).stop 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 shutdown_connections 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb498078bc0 0x7fb49807b0d0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4bc15e770 0x7fb4bc0aafa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4bc0ab4e0 0x7fb4bc149c40 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 --2- 172.21.15.88:0/1548158847 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4bc0b7f40 0x7fb4bc1e7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:40.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 >> 172.21.15.88:0/1548158847 conn(0x7fb4bc0d44f0 msgr2=0x7fb4bc160b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:40.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 shutdown_connections 2024-09-21T07:46:40.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:40.990+0000 7fb4c2600640 1 -- 172.21.15.88:0/1548158847 wait complete. 2024-09-21T07:46:41.013 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2024-09-21T07:46:41.013 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.013 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2024-09-21T07:46:41.072 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:46:41.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.072 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2024-09-21T07:46:41.108 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:41.108 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:41.145 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:41.145 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:41.221 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:46:41.221 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:46:41.297 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:46:41.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 -- 172.21.15.88:0/2846685376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5015d540 msgr2=0x7fed50156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 --2- 172.21.15.88:0/2846685376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5015d540 0x7fed50156e40 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fed48009be0 tx=0x7fed4802a8e0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 -- 172.21.15.88:0/2846685376 shutdown_connections 2024-09-21T07:46:41.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 --2- 172.21.15.88:0/2846685376 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed5015d540 0x7fed50156e40 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 -- 172.21.15.88:0/2846685376 >> 172.21.15.88:0/2846685376 conn(0x7fed5006d3d0 msgr2=0x7fed500b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:41.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 -- 172.21.15.88:0/2846685376 shutdown_connections 2024-09-21T07:46:41.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.519+0000 7fed55c00640 1 -- 172.21.15.88:0/2846685376 wait complete. 2024-09-21T07:46:41.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.521+0000 7fed55c00640 1 Processor -- start 2024-09-21T07:46:41.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.521+0000 7fed55c00640 1 -- start start 2024-09-21T07:46:41.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed55c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed500cfc80 0x7fed50099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed55c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5015d540 0x7fed500c53d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed55c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 0x7fed500c5910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed55c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed500cd100 con 0x7fed50070d00 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed55c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed500ccf80 con 0x7fed500cfc80 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed55c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed500cd280 con 0x7fed5015d540 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 0x7fed500c5910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 0x7fed500c5910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58800/0 (socket says 172.21.15.88:58800) 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 -- 172.21.15.88:0/484192776 learned_addr learned my addr 172.21.15.88:0/484192776 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4ea00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed500cfc80 0x7fed50099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4e000640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5015d540 0x7fed500c53d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 -- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5015d540 msgr2=0x7fed500c53d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5015d540 0x7fed500c53d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 -- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed500cfc80 msgr2=0x7fed50099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed500cfc80 0x7fed50099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4f400640 1 -- 172.21.15.88:0/484192776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed501f7b10 con 0x7fed50070d00 2024-09-21T07:46:41.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.523+0000 7fed4ea00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed500cfc80 0x7fed50099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:46:41.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.524+0000 7fed4f400640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 0x7fed500c5910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed48006010 tx=0x7fed48032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.524+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48041070 con 0x7fed50070d00 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.524+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed48004470 con 0x7fed50070d00 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.524+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed50099ec0 con 0x7fed50070d00 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.524+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed48031430 con 0x7fed50070d00 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.524+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed50072f20 con 0x7fed50070d00 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.525+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed50076850 con 0x7fed50070d00 2024-09-21T07:46:41.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.526+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fed48044070 con 0x7fed50070d00 2024-09-21T07:46:41.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.527+0000 7fed46a00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fed30078bc0 0x7fed3007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:41.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.527+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fed4802e020 con 0x7fed50070d00 2024-09-21T07:46:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.528+0000 7fed4ea00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fed30078bc0 0x7fed3007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:41.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.528+0000 7fed4ea00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fed30078bc0 0x7fed3007b0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed5009e510 tx=0x7fed40009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:41.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.532+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed50076850 con 0x7fed50070d00 2024-09-21T07:46:41.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.731+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fed500cd770 con 0x7fed50070d00 2024-09-21T07:46:41.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.732+0000 7fed46a00640 1 -- 172.21.15.88:0/484192776 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v23) ==== 80+0+0 (secure 0 0 0) 0x7fed4804b070 con 0x7fed50070d00 2024-09-21T07:46:41.733 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:46:41.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fed30078bc0 msgr2=0x7fed3007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fed30078bc0 0x7fed3007b0d0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fed5009e510 tx=0x7fed40009040 comp rx=0 tx=0).stop 2024-09-21T07:46:41.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 msgr2=0x7fed500c5910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:41.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 0x7fed500c5910 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fed48006010 tx=0x7fed48032d30 comp rx=0 tx=0).stop 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 shutdown_connections 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fed30078bc0 0x7fed3007b0d0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed5015d540 0x7fed500c53d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed50070d00 0x7fed500c5910 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 --2- 172.21.15.88:0/484192776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed500cfc80 0x7fed50099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 >> 172.21.15.88:0/484192776 conn(0x7fed5006d3d0 msgr2=0x7fed500b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 shutdown_connections 2024-09-21T07:46:41.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:41.734+0000 7fed55c00640 1 -- 172.21.15.88:0/484192776 wait complete. 2024-09-21T07:46:41.759 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2024-09-21T07:46:41.759 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.759 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2024-09-21T07:46:41.819 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:46:41.819 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:46:41.819 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2024-09-21T07:46:41.855 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:46:41.855 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 -- 172.21.15.88:0/3430270615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc00b3550 msgr2=0x7f6fc00b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3430270615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc00b3550 0x7f6fc00b3960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0072d60 tx=0x7f6fb802a820 comp rx=0 tx=0).stop 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 -- 172.21.15.88:0/3430270615 shutdown_connections 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3430270615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc00b3550 0x7f6fc00b3960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3430270615 >> v2:172.21.15.175:3300/0 conn(0x7f6fc00ad0e0 0x7f6fc00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 -- 172.21.15.88:0/3430270615 >> 172.21.15.88:0/3430270615 conn(0x7f6fc006d730 msgr2=0x7f6fc00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:42.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 -- 172.21.15.88:0/3430270615 shutdown_connections 2024-09-21T07:46:42.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.074+0000 7f6fc8000640 1 -- 172.21.15.88:0/3430270615 wait complete. 2024-09-21T07:46:42.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.076+0000 7f6fc8000640 1 Processor -- start 2024-09-21T07:46:42.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.077+0000 7f6fc8000640 1 -- start start 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc00ad0e0 0x7f6fc01f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc00b3550 0x7f6fc014bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc8000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 0x7f6fc00b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc8000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00b4da0 con 0x7f6fc015d760 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc00b4c20 con 0x7f6fc00b3550 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc00b3ea0 con 0x7f6fc00ad0e0 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 0x7f6fc00b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 0x7f6fc00b1a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58818/0 (socket says 172.21.15.88:58818) 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc6000640 1 -- 172.21.15.88:0/3007168729 learned_addr learned my addr 172.21.15.88:0/3007168729 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.078+0000 7f6fc4c00640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc00b3550 0x7f6fc014bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc5600640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc00ad0e0 0x7f6fc01f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc6000640 1 -- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc00ad0e0 msgr2=0x7f6fc01f7580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc6000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc00ad0e0 0x7f6fc01f7580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc6000640 1 -- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc00b3550 msgr2=0x7f6fc014bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc6000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc00b3550 0x7f6fc014bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc6000640 1 -- 172.21.15.88:0/3007168729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc014e1e0 con 0x7f6fc015d760 2024-09-21T07:46:42.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fc6000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 0x7f6fc00b1a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009b30 tx=0x7f6fb8032be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:42.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.079+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8037040 con 0x7f6fc015d760 2024-09-21T07:46:42.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.080+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fb8042070 con 0x7f6fc015d760 2024-09-21T07:46:42.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.080+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc014e4c0 con 0x7f6fc015d760 2024-09-21T07:46:42.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.080+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fb8004040 con 0x7f6fc015d760 2024-09-21T07:46:42.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.080+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc01f7060 con 0x7f6fc015d760 2024-09-21T07:46:42.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.081+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc01f6b70 con 0x7f6fc015d760 2024-09-21T07:46:42.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.082+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6fb80041e0 con 0x7f6fc015d760 2024-09-21T07:46:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.083+0000 7f6fbd600640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6fa0078b90 0x7f6fa007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.083+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f6fb80bb9f0 con 0x7f6fc015d760 2024-09-21T07:46:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.083+0000 7f6fc5600640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6fa0078b90 0x7f6fa007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.084+0000 7f6fc5600640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6fa0078b90 0x7f6fa007b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007d60 tx=0x7f6fb4007cf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:42.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.087+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fb8083740 con 0x7f6fc015d760 2024-09-21T07:46:42.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.308+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6fc014df70 con 0x7f6fc015d760 2024-09-21T07:46:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.309+0000 7f6fbd600640 1 -- 172.21.15.88:0/3007168729 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f6fb8088170 con 0x7f6fc015d760 2024-09-21T07:46:42.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:42.310 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:42.310 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6fa0078b90 msgr2=0x7f6fa007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6fa0078b90 0x7f6fa007b0a0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4007d60 tx=0x7f6fb4007cf0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 msgr2=0x7f6fc00b1a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 0x7f6fc00b1a30 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f6fb8009b30 tx=0x7f6fb8032be0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 shutdown_connections 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6fa0078b90 0x7f6fa007b0a0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc015d760 0x7f6fc00b1a30 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc00b3550 0x7f6fc014bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 --2- 172.21.15.88:0/3007168729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc00ad0e0 0x7f6fc01f7580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 >> 172.21.15.88:0/3007168729 conn(0x7f6fc006d730 msgr2=0x7f6fc00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.311+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 shutdown_connections 2024-09-21T07:46:42.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.312+0000 7f6fc8000640 1 -- 172.21.15.88:0/3007168729 wait complete. 2024-09-21T07:46:42.338 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 -- 172.21.15.88:0/429957913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe00ad0e0 msgr2=0x7f1fe00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/429957913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe00ad0e0 0x7f1fe00ad4f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1fe0075250 tx=0x7f1fcc02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 -- 172.21.15.88:0/429957913 shutdown_connections 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/429957913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fe0160310 0x7f1fe0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/429957913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe00ad0e0 0x7f1fe00ad4f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/429957913 >> v2:172.21.15.175:3300/0 conn(0x7f1fe015d760 0x7f1fe01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 -- 172.21.15.88:0/429957913 >> 172.21.15.88:0/429957913 conn(0x7f1fe006d730 msgr2=0x7f1fe00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:42.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.554+0000 7f1fe6e00640 1 -- 172.21.15.88:0/429957913 shutdown_connections 2024-09-21T07:46:42.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.555+0000 7f1fe6e00640 1 -- 172.21.15.88:0/429957913 wait complete. 2024-09-21T07:46:42.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.557+0000 7f1fe6e00640 1 Processor -- start 2024-09-21T07:46:42.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.557+0000 7f1fe6e00640 1 -- start start 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fe00ad0e0 0x7f1fe0151d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe015d760 0x7f1fe01f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 0x7f1fe01589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00b4da0 con 0x7f1fe0160310 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe00b4c20 con 0x7f1fe015d760 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1fe0160c60 con 0x7f1fe00ad0e0 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.558+0000 7f1fe4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 0x7f1fe01589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 0x7f1fe01589d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58842/0 (socket says 172.21.15.88:58842) 2024-09-21T07:46:42.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 -- 172.21.15.88:0/1823293371 learned_addr learned my addr 172.21.15.88:0/1823293371 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:42.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fdfe00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fe00ad0e0 0x7f1fe0151d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fdf400640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe015d760 0x7f1fe01f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 -- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fe00ad0e0 msgr2=0x7f1fe0151d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fe00ad0e0 0x7f1fe0151d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 -- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe015d760 msgr2=0x7f1fe01f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe015d760 0x7f1fe01f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 -- 172.21.15.88:0/1823293371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe0073940 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fe4e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 0x7f1fe01589d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800e990 tx=0x7f1fd800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.559+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8004d20 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.560+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fd800cb00 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.560+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8005020 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.560+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe0073c20 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.560+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01f5520 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.561+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe0157200 con 0x7f1fe0160310 2024-09-21T07:46:42.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.562+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1fd80051c0 con 0x7f1fe0160310 2024-09-21T07:46:42.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.563+0000 7f1fd7e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fc0078b90 0x7f1fc007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:42.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.563+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f1fd809b510 con 0x7f1fe0160310 2024-09-21T07:46:42.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.563+0000 7f1fdfe00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fc0078b90 0x7f1fc007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:42.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.564+0000 7f1fdfe00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fc0078b90 0x7f1fc007b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000bb50 tx=0x7f1fd000b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:42.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.567+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1fe0157200 con 0x7f1fe0160310 2024-09-21T07:46:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.790+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1fe0156f90 con 0x7f1fe0160310 2024-09-21T07:46:42.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.791+0000 7f1fd7e00640 1 -- 172.21.15.88:0/1823293371 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f1fd8016030 con 0x7f1fe0160310 2024-09-21T07:46:42.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:42.792 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:42.792 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:42.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fc0078b90 msgr2=0x7f1fc007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fc0078b90 0x7f1fc007b0a0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f1fd000bb50 tx=0x7f1fd000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 msgr2=0x7f1fe01589d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 0x7f1fe01589d0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1fd800e990 tx=0x7f1fd800ee50 comp rx=0 tx=0).stop 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 shutdown_connections 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fc0078b90 0x7f1fc007b0a0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0160310 0x7f1fe01589d0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fe015d760 0x7f1fe01f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 --2- 172.21.15.88:0/1823293371 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fe00ad0e0 0x7f1fe0151d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 >> 172.21.15.88:0/1823293371 conn(0x7f1fe006d730 msgr2=0x7f1fe00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 shutdown_connections 2024-09-21T07:46:42.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:42.793+0000 7f1fe6e00640 1 -- 172.21.15.88:0/1823293371 wait complete. 2024-09-21T07:46:42.816 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:43.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.035+0000 7fd49e000640 1 -- 172.21.15.88:0/297536263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd49815d760 msgr2=0x7fd4981535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.035+0000 7fd49e000640 1 --2- 172.21.15.88:0/297536263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd49815d760 0x7fd4981535c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fd488009830 tx=0x7fd48802a480 comp rx=0 tx=0).stop 2024-09-21T07:46:43.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.036+0000 7fd49e000640 1 -- 172.21.15.88:0/297536263 shutdown_connections 2024-09-21T07:46:43.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.036+0000 7fd49e000640 1 --2- 172.21.15.88:0/297536263 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4980ad0e0 0x7fd4980ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.036+0000 7fd49e000640 1 --2- 172.21.15.88:0/297536263 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd49815d760 0x7fd4981535c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.036+0000 7fd49e000640 1 -- 172.21.15.88:0/297536263 >> 172.21.15.88:0/297536263 conn(0x7fd49806d730 msgr2=0x7fd4980b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:43.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.036+0000 7fd49e000640 1 -- 172.21.15.88:0/297536263 shutdown_connections 2024-09-21T07:46:43.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.036+0000 7fd49e000640 1 -- 172.21.15.88:0/297536263 wait complete. 2024-09-21T07:46:43.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.038+0000 7fd49e000640 1 Processor -- start 2024-09-21T07:46:43.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.038+0000 7fd49e000640 1 -- start start 2024-09-21T07:46:43.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.039+0000 7fd49e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4980ad0e0 0x7fd4981f45d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.039+0000 7fd49e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 0x7fd498151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.039+0000 7fd49e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd498160310 0x7fd4981f70b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.039+0000 7fd49e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd49806f220 con 0x7fd49815d760 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.039+0000 7fd49e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd49806f0a0 con 0x7fd4980ad0e0 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.039+0000 7fd49e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4980b4c20 con 0x7fd498160310 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 0x7fd498151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 0x7fd498151be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58848/0 (socket says 172.21.15.88:58848) 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 -- 172.21.15.88:0/1442780877 learned_addr learned my addr 172.21.15.88:0/1442780877 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd497e00640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd498160310 0x7fd4981f70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd497400640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4980ad0e0 0x7fd4981f45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 -- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd498160310 msgr2=0x7fd4981f70b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd498160310 0x7fd4981f70b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 -- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4980ad0e0 msgr2=0x7fd4981f45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4980ad0e0 0x7fd4981f45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.040+0000 7fd48ea00640 1 -- 172.21.15.88:0/1442780877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd498156450 con 0x7fd49815d760 2024-09-21T07:46:43.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.041+0000 7fd48ea00640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 0x7fd498151be0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd48000e940 tx=0x7fd48000ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.041+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48000cd80 con 0x7fd49815d760 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.041+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd480010430 con 0x7fd49815d760 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.041+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd498156730 con 0x7fd49815d760 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.041+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd480010880 con 0x7fd49815d760 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.041+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4981f40d0 con 0x7fd49815d760 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.042+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4981561c0 con 0x7fd49815d760 2024-09-21T07:46:43.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.043+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fd480010a20 con 0x7fd49815d760 2024-09-21T07:46:43.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.045+0000 7fd48fe00640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd474078b90 0x7fd47407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.045+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fd48009b810 con 0x7fd49815d760 2024-09-21T07:46:43.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.045+0000 7fd497400640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd474078b90 0x7fd47407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.046+0000 7fd497400640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd474078b90 0x7fd47407b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd488034000 tx=0x7fd488004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:43.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.049+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd480063570 con 0x7fd49815d760 2024-09-21T07:46:43.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.234+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd498160fa0 con 0x7fd49815d760 2024-09-21T07:46:43.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.235+0000 7fd48fe00640 1 -- 172.21.15.88:0/1442780877 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fd480067fa0 con 0x7fd49815d760 2024-09-21T07:46:43.238 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:43.238 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd474078b90 msgr2=0x7fd47407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd474078b90 0x7fd47407b0a0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fd488034000 tx=0x7fd488004730 comp rx=0 tx=0).stop 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 msgr2=0x7fd498151be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 0x7fd498151be0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fd48000e940 tx=0x7fd48000ee00 comp rx=0 tx=0).stop 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 shutdown_connections 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd474078b90 0x7fd47407b0a0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd498160310 0x7fd4981f70b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd49815d760 0x7fd498151be0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 --2- 172.21.15.88:0/1442780877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4980ad0e0 0x7fd4981f45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 >> 172.21.15.88:0/1442780877 conn(0x7fd49806d730 msgr2=0x7fd4980cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:43.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 shutdown_connections 2024-09-21T07:46:43.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.238+0000 7fd49e000640 1 -- 172.21.15.88:0/1442780877 wait complete. 2024-09-21T07:46:43.266 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:46:43.266 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 -- 172.21.15.88:0/2130211158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 msgr2=0x7f46880739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 --2- 172.21.15.88:0/2130211158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 0x7f46880739d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f467000a940 tx=0x7f467002b880 comp rx=0 tx=0).stop 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 -- 172.21.15.88:0/2130211158 shutdown_connections 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 --2- 172.21.15.88:0/2130211158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 0x7f46880739d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 -- 172.21.15.88:0/2130211158 >> 172.21.15.88:0/2130211158 conn(0x7f46880b1e40 msgr2=0x7f46880b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 -- 172.21.15.88:0/2130211158 shutdown_connections 2024-09-21T07:46:43.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.483+0000 7f4687e00640 1 -- 172.21.15.88:0/2130211158 wait complete. 2024-09-21T07:46:43.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.486+0000 7f4687e00640 1 Processor -- start 2024-09-21T07:46:43.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.486+0000 7f4687e00640 1 -- start start 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.487+0000 7f4687e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 0x7f468815c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.487+0000 7f4687e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46880b3520 0x7f46881e6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.487+0000 7f4687e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 0x7f4688159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.487+0000 7f4687e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f468806f360 con 0x7f46881518d0 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.487+0000 7f4687e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4688071160 con 0x7f46880735c0 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.487+0000 7f4687e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46880b4040 con 0x7f46880b3520 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 0x7f4688159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 0x7f4688159820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58850/0 (socket says 172.21.15.88:58850) 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 -- 172.21.15.88:0/1826247802 learned_addr learned my addr 172.21.15.88:0/1826247802 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:43.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4686a00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 0x7f468815c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f467fe00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46880b3520 0x7f46881e6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 -- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46880b3520 msgr2=0x7f46881e6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46880b3520 0x7f46881e6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 -- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 msgr2=0x7f468815c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 0x7f468815c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 -- 172.21.15.88:0/1826247802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46881f5940 con 0x7f46881518d0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.488+0000 7f4687400640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 0x7f4688159820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4680001f40 tx=0x7f468000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.489+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4680016020 con 0x7f46881518d0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.489+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4680012420 con 0x7f46881518d0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.489+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46800128b0 con 0x7f46881518d0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.489+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46881f9120 con 0x7f46881518d0 2024-09-21T07:46:43.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.489+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46881f96e0 con 0x7f46881518d0 2024-09-21T07:46:43.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.490+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4688074480 con 0x7f46881518d0 2024-09-21T07:46:43.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.491+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4680019020 con 0x7f46881518d0 2024-09-21T07:46:43.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.493+0000 7f467f400640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4664078b90 0x7f466407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.493+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f468009b240 con 0x7f46881518d0 2024-09-21T07:46:43.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.493+0000 7f4686a00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4664078b90 0x7f466407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.494+0000 7f4686a00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4664078b90 0x7f466407b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4678001120 tx=0x7f4678009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:43.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.497+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4688074480 con 0x7f46881518d0 2024-09-21T07:46:43.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.717+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f46880b45d0 con 0x7f46881518d0 2024-09-21T07:46:43.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.717+0000 7f467f400640 1 -- 172.21.15.88:0/1826247802 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f4680020030 con 0x7f46881518d0 2024-09-21T07:46:43.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:43.720 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:43.720 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4664078b90 msgr2=0x7f466407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4664078b90 0x7f466407b0a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f4678001120 tx=0x7f4678009040 comp rx=0 tx=0).stop 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 msgr2=0x7f4688159820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 0x7f4688159820 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f4680001f40 tx=0x7f468000a430 comp rx=0 tx=0).stop 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 shutdown_connections 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4664078b90 0x7f466407b0a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46880b3520 0x7f46881e6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46881518d0 0x7f4688159820 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 --2- 172.21.15.88:0/1826247802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46880735c0 0x7f468815c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 >> 172.21.15.88:0/1826247802 conn(0x7f46880b1e40 msgr2=0x7f46880b3930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:43.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 shutdown_connections 2024-09-21T07:46:43.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.720+0000 7f4687e00640 1 -- 172.21.15.88:0/1826247802 wait complete. 2024-09-21T07:46:43.747 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 -- 172.21.15.88:0/3251297384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c00af630 msgr2=0x7f62c0073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 --2- 172.21.15.88:0/3251297384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c00af630 0x7f62c0073500 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f62b0005fd0 tx=0x7f62b002a670 comp rx=0 tx=0).stop 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 -- 172.21.15.88:0/3251297384 shutdown_connections 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 --2- 172.21.15.88:0/3251297384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c00af630 0x7f62c0073500 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 -- 172.21.15.88:0/3251297384 >> 172.21.15.88:0/3251297384 conn(0x7f62c006d730 msgr2=0x7f62c00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 -- 172.21.15.88:0/3251297384 shutdown_connections 2024-09-21T07:46:43.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.966+0000 7f62c6a00640 1 -- 172.21.15.88:0/3251297384 wait complete. 2024-09-21T07:46:43.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.969+0000 7f62c6a00640 1 Processor -- start 2024-09-21T07:46:43.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.969+0000 7f62c6a00640 1 -- start start 2024-09-21T07:46:43.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 0x7f62c015e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62c6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62c0073a40 0x7f62c01526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62c6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62c00af630 0x7f62c0159e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62c6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62c00b3d90 con 0x7f62c006f7b0 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62c6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62c00b3c10 con 0x7f62c00af630 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62c6a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62c00b3f10 con 0x7f62c0073a40 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 0x7f62c015e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 0x7f62c015e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58874/0 (socket says 172.21.15.88:58874) 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.970+0000 7f62bfe00640 1 -- 172.21.15.88:0/4190643286 learned_addr learned my addr 172.21.15.88:0/4190643286 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62c4a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62c00af630 0x7f62c0159e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62b7400640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62c0073a40 0x7f62c01526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bfe00640 1 -- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62c0073a40 msgr2=0x7f62c01526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bfe00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62c0073a40 0x7f62c01526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bfe00640 1 -- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62c00af630 msgr2=0x7f62c0159e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bfe00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62c00af630 0x7f62c0159e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bfe00640 1 -- 172.21.15.88:0/4190643286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62c009c8a0 con 0x7f62c006f7b0 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bfe00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 0x7f62c015e3a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f62b0006010 tx=0x7f62b00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b003e030 con 0x7f62c006f7b0 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62b00047a0 con 0x7f62c006f7b0 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.971+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62b0035040 con 0x7f62c006f7b0 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.972+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62c009ce60 con 0x7f62c006f7b0 2024-09-21T07:46:43.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.972+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62c0159640 con 0x7f62c006f7b0 2024-09-21T07:46:43.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.973+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62c00b4400 con 0x7f62c006f7b0 2024-09-21T07:46:43.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.974+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f62b0041030 con 0x7f62c006f7b0 2024-09-21T07:46:43.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.975+0000 7f62bcc00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f629c078b90 0x7f629c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:43.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.975+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7f62b00bb6d0 con 0x7f62c006f7b0 2024-09-21T07:46:43.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.975+0000 7f62b7400640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f629c078b90 0x7f629c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:43.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.976+0000 7f62b7400640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f629c078b90 0x7f629c07b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007fd0 tx=0x7f62a8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:43.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:43.980+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62b0083b00 con 0x7f62c006f7b0 2024-09-21T07:46:44.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.200+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f62c015f6a0 con 0x7f62c006f7b0 2024-09-21T07:46:44.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.200+0000 7f62bcc00640 1 -- 172.21.15.88:0/4190643286 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+5220 (secure 0 0 0) 0x7f62b0083400 con 0x7f62c006f7b0 2024-09-21T07:46:44.203 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:44.203 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":8,"btime":"2024-09-21T07:46:10:212843+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4331,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2139552907","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2139552907},{"type":"v1","addr":"172.21.15.88:6834","nonce":2139552907}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":4334,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/698868905","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":698868905},{"type":"v1","addr":"172.21.15.88:6835","nonce":698868905}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":4395,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2878765392","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2878765392},{"type":"v1","addr":"172.21.15.175:6836","nonce":2878765392}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:06.691548+0000","modified":"2024-09-21T07:46:09.694928+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4423},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4423":{"gid":4423,"name":"b","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/3640011394","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3640011394},{"type":"v1","addr":"172.21.15.175:6837","nonce":3640011394}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4423,"qdb_cluster":[4423]},"id":1}]} 2024-09-21T07:46:44.203 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 8 2024-09-21T07:46:44.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f629c078b90 msgr2=0x7f629c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f629c078b90 0x7f629c07b0a0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f62a8007fd0 tx=0x7f62a8009040 comp rx=0 tx=0).stop 2024-09-21T07:46:44.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 msgr2=0x7f62c015e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 0x7f62c015e3a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f62b0006010 tx=0x7f62b00039e0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 shutdown_connections 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f629c078b90 0x7f629c07b0a0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62c0073a40 0x7f62c01526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62c00af630 0x7f62c0159e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 --2- 172.21.15.88:0/4190643286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62c006f7b0 0x7f62c015e3a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 >> 172.21.15.88:0/4190643286 conn(0x7f62c006d730 msgr2=0x7f62c00aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 shutdown_connections 2024-09-21T07:46:44.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.203+0000 7f62c6a00640 1 -- 172.21.15.88:0/4190643286 wait complete. 2024-09-21T07:46:44.229 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:44.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.446+0000 7fe975400640 1 -- 172.21.15.88:0/4006904286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700aebc0 msgr2=0x7fe970152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.446+0000 7fe975400640 1 --2- 172.21.15.88:0/4006904286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700aebc0 0x7fe970152920 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fe960009830 tx=0x7fe96002a480 comp rx=0 tx=0).stop 2024-09-21T07:46:44.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.447+0000 7fe975400640 1 -- 172.21.15.88:0/4006904286 shutdown_connections 2024-09-21T07:46:44.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.447+0000 7fe975400640 1 --2- 172.21.15.88:0/4006904286 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe97015fae0 0x7fe9700b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.447+0000 7fe975400640 1 --2- 172.21.15.88:0/4006904286 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700aebc0 0x7fe970152920 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.447+0000 7fe975400640 1 -- 172.21.15.88:0/4006904286 >> 172.21.15.88:0/4006904286 conn(0x7fe97006d730 msgr2=0x7fe9700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:44.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.447+0000 7fe975400640 1 -- 172.21.15.88:0/4006904286 shutdown_connections 2024-09-21T07:46:44.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.447+0000 7fe975400640 1 -- 172.21.15.88:0/4006904286 wait complete. 2024-09-21T07:46:44.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.449+0000 7fe975400640 1 Processor -- start 2024-09-21T07:46:44.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.449+0000 7fe975400640 1 -- start start 2024-09-21T07:46:44.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe975400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9700aebc0 0x7fe9701f6510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe975400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700b2750 0x7fe97014fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe975400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 0x7fe9700c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe975400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700b4190 con 0x7fe9700ce6f0 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe975400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9700b4010 con 0x7fe9700b2750 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe975400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9700b4310 con 0x7fe9700aebc0 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 0x7fe9700c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 0x7fe9700c9dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58880/0 (socket says 172.21.15.88:58880) 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 -- 172.21.15.88:0/3256613692 learned_addr learned my addr 172.21.15.88:0/3256613692 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96e800640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9700aebc0 0x7fe9701f6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe965e00640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700b2750 0x7fe97014fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 -- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9700aebc0 msgr2=0x7fe9701f6510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9700aebc0 0x7fe9701f6510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 -- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700b2750 msgr2=0x7fe97014fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700b2750 0x7fe97014fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.451+0000 7fe96f200640 1 -- 172.21.15.88:0/3256613692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9700cee60 con 0x7fe9700ce6f0 2024-09-21T07:46:44.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.452+0000 7fe96f200640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 0x7fe9700c9dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe96800ef00 tx=0x7fe96800c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.452+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968003970 con 0x7fe9700ce6f0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.452+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe968003c00 con 0x7fe9700ce6f0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.452+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe968004020 con 0x7fe9700ce6f0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.452+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9700cf140 con 0x7fe9700ce6f0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.452+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe970150380 con 0x7fe9700ce6f0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.453+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9700cebd0 con 0x7fe9700ce6f0 2024-09-21T07:46:44.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.454+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe9680041c0 con 0x7fe9700ce6f0 2024-09-21T07:46:44.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.455+0000 7fe967400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe94c078bc0 0x7fe94c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.455+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(23..23 src has 1..23) ==== 5709+0+0 (secure 0 0 0) 0x7fe96809b4b0 con 0x7fe9700ce6f0 2024-09-21T07:46:44.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.456+0000 7fe96e800640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe94c078bc0 0x7fe94c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.456+0000 7fe96e800640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe94c078bc0 0x7fe94c07b0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe960034000 tx=0x7fe960004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:44.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.460+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe968063320 con 0x7fe9700ce6f0 2024-09-21T07:46:44.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.647+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe9700b47c0 con 0x7fe9700ce6f0 2024-09-21T07:46:44.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.648+0000 7fe967400640 1 -- 172.21.15.88:0/3256613692 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) ==== 74+0+16681 (secure 0 0 0) 0x7fe968067d50 con 0x7fe9700ce6f0 2024-09-21T07:46:44.651 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:44.651 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":23,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:08.194198+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:04.825550+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:05.649137+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.650+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe94c078bc0 msgr2=0x7fe94c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.650+0000 7fe975400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe94c078bc0 0x7fe94c07b0d0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe960034000 tx=0x7fe960004730 comp rx=0 tx=0).stop 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.650+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 msgr2=0x7fe9700c9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.650+0000 7fe975400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 0x7fe9700c9dd0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe96800ef00 tx=0x7fe96800c740 comp rx=0 tx=0).stop 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 shutdown_connections 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe94c078bc0 0x7fe94c07b0d0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9700ce6f0 0x7fe9700c9dd0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9700b2750 0x7fe97014fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 --2- 172.21.15.88:0/3256613692 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9700aebc0 0x7fe9701f6510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 >> 172.21.15.88:0/3256613692 conn(0x7fe97006d730 msgr2=0x7fe970151c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:44.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 shutdown_connections 2024-09-21T07:46:44.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.651+0000 7fe975400640 1 -- 172.21.15.88:0/3256613692 wait complete. 2024-09-21T07:46:44.675 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:46:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.895+0000 7ffa66800640 1 -- 172.21.15.88:0/2261893199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600705b0 msgr2=0x7ffa600709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.895+0000 7ffa66800640 1 --2- 172.21.15.88:0/2261893199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600705b0 0x7ffa600709c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009830 tx=0x7ffa4802a480 comp rx=0 tx=0).stop 2024-09-21T07:46:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.896+0000 7ffa66800640 1 -- 172.21.15.88:0/2261893199 shutdown_connections 2024-09-21T07:46:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.896+0000 7ffa66800640 1 --2- 172.21.15.88:0/2261893199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600705b0 0x7ffa600709c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.896+0000 7ffa66800640 1 --2- 172.21.15.88:0/2261893199 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600cfc80 0x7ffa60156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.896+0000 7ffa66800640 1 -- 172.21.15.88:0/2261893199 >> 172.21.15.88:0/2261893199 conn(0x7ffa6006d3d0 msgr2=0x7ffa600b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:44.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.896+0000 7ffa66800640 1 -- 172.21.15.88:0/2261893199 shutdown_connections 2024-09-21T07:46:44.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.896+0000 7ffa66800640 1 -- 172.21.15.88:0/2261893199 wait complete. 2024-09-21T07:46:44.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.898+0000 7ffa66800640 1 Processor -- start 2024-09-21T07:46:44.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.898+0000 7ffa66800640 1 -- start start 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.899+0000 7ffa66800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600705b0 0x7ffa600c6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.899+0000 7ffa66800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 0x7ffa600999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.899+0000 7ffa66800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6015ce30 0x7ffa60073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.899+0000 7ffa66800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa600cced0 con 0x7ffa600cfc80 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.899+0000 7ffa66800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa600d0170 con 0x7ffa6015ce30 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.899+0000 7ffa66800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa600cd050 con 0x7ffa600705b0 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 0x7ffa600999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 0x7ffa600999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58898/0 (socket says 172.21.15.88:58898) 2024-09-21T07:46:44.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 -- 172.21.15.88:0/3962313895 learned_addr learned my addr 172.21.15.88:0/3962313895 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa5f400640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600705b0 0x7ffa600c6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa5fe00640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6015ce30 0x7ffa60073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 -- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600705b0 msgr2=0x7ffa600c6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600705b0 0x7ffa600c6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 -- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6015ce30 msgr2=0x7ffa60073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6015ce30 0x7ffa60073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 -- 172.21.15.88:0/3962313895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa601f6f70 con 0x7ffa600cfc80 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa5f400640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600705b0 0x7ffa600c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa5fe00640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6015ce30 0x7ffa60073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.900+0000 7ffa57e00640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 0x7ffa600999a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa4802aa70 tx=0x7ffa48033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.901+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48005c90 con 0x7ffa600cfc80 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.901+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa48031e50 con 0x7ffa600cfc80 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.901+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa48033310 con 0x7ffa600cfc80 2024-09-21T07:46:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.901+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa60099ee0 con 0x7ffa600cfc80 2024-09-21T07:46:44.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.901+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa601f71e0 con 0x7ffa600cfc80 2024-09-21T07:46:44.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.902+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa600cd570 con 0x7ffa600cfc80 2024-09-21T07:46:44.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.903+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7ffa480334b0 con 0x7ffa600cfc80 2024-09-21T07:46:44.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.904+0000 7ffa57400640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa3c078bc0 0x7ffa3c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:44.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.904+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(24..24 src has 1..24) ==== 5767+0+0 (secure 0 0 0) 0x7ffa480bc3b0 con 0x7ffa600cfc80 2024-09-21T07:46:44.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.904+0000 7ffa5f400640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa3c078bc0 0x7ffa3c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:44.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.905+0000 7ffa5f400640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa3c078bc0 0x7ffa3c07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa6009e510 tx=0x7ffa50005dd0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:44.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:44.909+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa60099ee0 con 0x7ffa600cfc80 2024-09-21T07:46:45.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.135+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7ffa600afa70 con 0x7ffa600cfc80 2024-09-21T07:46:45.137 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:46:45.135+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:46:45.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.137+0000 7ffa57400640 1 -- 172.21.15.88:0/3962313895 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) ==== 161+0+0 (secure 0 0 0) 0x7ffa4803f030 con 0x7ffa600cfc80 2024-09-21T07:46:45.139 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:46:45.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.140+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa3c078bc0 msgr2=0x7ffa3c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:45.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.140+0000 7ffa66800640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa3c078bc0 0x7ffa3c07b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7ffa6009e510 tx=0x7ffa50005dd0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.140+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 msgr2=0x7ffa600999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:45.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.140+0000 7ffa66800640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 0x7ffa600999a0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7ffa4802aa70 tx=0x7ffa48033c90 comp rx=0 tx=0).stop 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 shutdown_connections 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffa3c078bc0 0x7ffa3c07b0d0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa6015ce30 0x7ffa60073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa600cfc80 0x7ffa600999a0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 --2- 172.21.15.88:0/3962313895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa600705b0 0x7ffa600c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 >> 172.21.15.88:0/3962313895 conn(0x7ffa6006d3d0 msgr2=0x7ffa6015d240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:45.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 shutdown_connections 2024-09-21T07:46:45.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.141+0000 7ffa66800640 1 -- 172.21.15.88:0/3962313895 wait complete. 2024-09-21T07:46:45.159 INFO:tasks.ceph.mds.b.smithi175.stderr: -7269> 2024-09-21T07:46:12.170+0000 7fd6bee00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:46:45.164 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:46:45.286 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 -- 172.21.15.88:0/1936164349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900b3f30 msgr2=0x7f4490156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 --2- 172.21.15.88:0/1936164349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900b3f30 0x7f4490156ca0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4480005fd0 tx=0x7f448002a830 comp rx=0 tx=0).stop 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 -- 172.21.15.88:0/1936164349 shutdown_connections 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 --2- 172.21.15.88:0/1936164349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f449015d6e0 0x7f44900ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 --2- 172.21.15.88:0/1936164349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900b3f30 0x7f4490156ca0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 -- 172.21.15.88:0/1936164349 >> 172.21.15.88:0/1936164349 conn(0x7f449006d620 msgr2=0x7f44900b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.387+0000 7f448fe00640 1 -- 172.21.15.88:0/1936164349 shutdown_connections 2024-09-21T07:46:45.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.388+0000 7f448fe00640 1 -- 172.21.15.88:0/1936164349 wait complete. 2024-09-21T07:46:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.390+0000 7f448fe00640 1 Processor -- start 2024-09-21T07:46:45.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.390+0000 7f448fe00640 1 -- start start 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.391+0000 7f448fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44900b3f30 0x7f44901f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.391+0000 7f448fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900cd000 0x7f44901f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 0x7f44901f2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44900af960 con 0x7f449015d6e0 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44900af7e0 con 0x7f44900cd000 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f44900afae0 con 0x7f44900b3f30 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 0x7f44901f2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:45.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 0x7f44901f2a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58912/0 (socket says 172.21.15.88:58912) 2024-09-21T07:46:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 -- 172.21.15.88:0/3994091589 learned_addr learned my addr 172.21.15.88:0/3994091589 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448e000640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900cd000 0x7f44901f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:45.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448ea00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44900b3f30 0x7f44901f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 -- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44900b3f30 msgr2=0x7f44901f8c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44900b3f30 0x7f44901f8c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 -- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900cd000 msgr2=0x7f44901f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900cd000 0x7f44901f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.392+0000 7f448f400640 1 -- 172.21.15.88:0/3994091589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44901f6570 con 0x7f449015d6e0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.393+0000 7f448f400640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 0x7f44901f2a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f448800c480 tx=0x7f448800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.393+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488016020 con 0x7f449015d6e0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.393+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4488012940 con 0x7f449015d6e0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.393+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4488012d90 con 0x7f449015d6e0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.393+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44901f9580 con 0x7f449015d6e0 2024-09-21T07:46:45.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.393+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44901f5830 con 0x7f449015d6e0 2024-09-21T07:46:45.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.394+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44901f92f0 con 0x7f449015d6e0 2024-09-21T07:46:45.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.395+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f4488019020 con 0x7f449015d6e0 2024-09-21T07:46:45.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.396+0000 7f4486a00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4470078b90 0x7f447007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:45.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.396+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5853+0+0 (secure 0 0 0) 0x7f448809ad20 con 0x7f449015d6e0 2024-09-21T07:46:45.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.396+0000 7f448ea00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4470078b90 0x7f447007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:45.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.397+0000 7f448ea00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4470078b90 0x7f447007b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4480033000 tx=0x7f4480004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:45.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.400+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4488020020 con 0x7f449015d6e0 2024-09-21T07:46:45.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:45.622+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f4490156ca0 con 0x7f449015d6e0 2024-09-21T07:46:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.139+0000 7f4486a00640 1 -- 172.21.15.88:0/3994091589 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) ==== 104+0+0 (secure 0 0 0) 0x7f4488062b10 con 0x7f449015d6e0 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4470078b90 msgr2=0x7f447007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4470078b90 0x7f447007b0a0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f4480033000 tx=0x7f4480004ab0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 msgr2=0x7f44901f2a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 0x7f44901f2a00 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f448800c480 tx=0x7f448800c940 comp rx=0 tx=0).stop 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 shutdown_connections 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4470078b90 0x7f447007b0a0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f449015d6e0 0x7f44901f2a00 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44900cd000 0x7f44901f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 --2- 172.21.15.88:0/3994091589 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44900b3f30 0x7f44901f8c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 >> 172.21.15.88:0/3994091589 conn(0x7f449006d620 msgr2=0x7f44900cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 shutdown_connections 2024-09-21T07:46:46.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.142+0000 7f448fe00640 1 -- 172.21.15.88:0/3994091589 wait complete. 2024-09-21T07:46:46.167 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.390+0000 7fdff4c00640 1 -- 172.21.15.88:0/2636940411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfd4004d40 msgr2=0x7fdfd4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.390+0000 7fdff4c00640 1 --2- 172.21.15.88:0/2636940411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfd4004d40 0x7fdfd4005150 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fdff00cedc0 tx=0x7fdfd4005830 comp rx=0 tx=0).stop 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.390+0000 7fdfee800640 1 -- 172.21.15.88:0/2636940411 reap_dead start 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.391+0000 7fdff4c00640 1 -- 172.21.15.88:0/2636940411 shutdown_connections 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.391+0000 7fdff4c00640 1 -- 172.21.15.88:0/2636940411 >> 172.21.15.88:0/2636940411 conn(0x7fdff00b2540 msgr2=0x7fdff00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.391+0000 7fdff4c00640 1 -- 172.21.15.88:0/2636940411 shutdown_connections 2024-09-21T07:46:46.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.391+0000 7fdff4c00640 1 -- 172.21.15.88:0/2636940411 wait complete. 2024-09-21T07:46:46.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.393+0000 7fdff4c00640 1 Processor -- start 2024-09-21T07:46:46.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.393+0000 7fdff4c00640 1 -- start start 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdff4c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4004d40 0x7fdff0157580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdff4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 0x7fdff01f3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdff4c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdff00b0fa0 0x7fdff00b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdff4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdff0070a20 con 0x7fdff01e48d0 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdff4c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdff00708a0 con 0x7fdff00b0fa0 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdff4c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdff00d09e0 con 0x7fdfd4004d40 2024-09-21T07:46:46.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 0x7fdff01f3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 0x7fdff01f3090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56442/0 (socket says 172.21.15.88:56442) 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 -- 172.21.15.88:0/1183477813 learned_addr learned my addr 172.21.15.88:0/1183477813 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfede00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4004d40 0x7fdff0157580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfee800640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdff00b0fa0 0x7fdff00b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 -- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4004d40 msgr2=0x7fdff0157580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4004d40 0x7fdff0157580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 -- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdff00b0fa0 msgr2=0x7fdff00b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdff00b0fa0 0x7fdff00b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfed400640 1 -- 172.21.15.88:0/1183477813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdff00b18f0 con 0x7fdff01e48d0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.395+0000 7fdfede00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4004d40 0x7fdff0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.396+0000 7fdfed400640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 0x7fdff01f3090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc001950 tx=0x7fdfdc0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.396+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc015070 con 0x7fdff01e48d0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.396+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfdc011900 con 0x7fdff01e48d0 2024-09-21T07:46:46.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.396+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc011d50 con 0x7fdff01e48d0 2024-09-21T07:46:46.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.396+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdff01f7200 con 0x7fdff01e48d0 2024-09-21T07:46:46.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.396+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdff01f77c0 con 0x7fdff01e48d0 2024-09-21T07:46:46.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.397+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdff00d0f40 con 0x7fdff01e48d0 2024-09-21T07:46:46.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.398+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fdfdc00f430 con 0x7fdff01e48d0 2024-09-21T07:46:46.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.399+0000 7fdfe6000640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:46.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.399+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5853+0+0 (secure 0 0 0) 0x7fdfdc09bb20 con 0x7fdff01e48d0 2024-09-21T07:46:46.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.400+0000 7fdfede00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:46.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.400+0000 7fdfede00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdfd0078b90 0x7fdfd007b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0005fd0 tx=0x7fdfe0009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:46.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.404+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdff00b18f0 con 0x7fdff01e48d0 2024-09-21T07:46:46.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:46.591+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdff01f6f90 con 0x7fdff01e48d0 2024-09-21T07:46:47.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.141+0000 7fdfe6000640 1 -- 172.21.15.88:0/1183477813 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v26) ==== 181+0+0 (secure 0 0 0) 0x7fdff0070bd0 con 0x7fdff01e48d0 2024-09-21T07:46:47.144 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:46:47.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdfd0078b90 msgr2=0x7fdfd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:47.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdfd0078b90 0x7fdfd007b0a0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0005fd0 tx=0x7fdfe0009500 comp rx=0 tx=0).stop 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 msgr2=0x7fdff01f3090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 0x7fdff01f3090 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc001950 tx=0x7fdfdc0039c0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 shutdown_connections 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdfd0078b90 0x7fdfd007b0a0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdff00b0fa0 0x7fdff00b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdff01e48d0 0x7fdff01f3090 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 --2- 172.21.15.88:0/1183477813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4004d40 0x7fdff0157580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.144+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 >> 172.21.15.88:0/1183477813 conn(0x7fdff00b2540 msgr2=0x7fdff0155430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:47.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.145+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 shutdown_connections 2024-09-21T07:46:47.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.145+0000 7fdff4c00640 1 -- 172.21.15.88:0/1183477813 wait complete. 2024-09-21T07:46:47.171 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 -- 172.21.15.88:0/1584348418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1444003a50 msgr2=0x7f1444003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 --2- 172.21.15.88:0/1584348418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1444003a50 0x7f1444003ec0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f1450009b50 tx=0x7f1444005830 comp rx=0 tx=0).stop 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 -- 172.21.15.88:0/1584348418 shutdown_connections 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 --2- 172.21.15.88:0/1584348418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1444003a50 0x7f1444003ec0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 -- 172.21.15.88:0/1584348418 >> 172.21.15.88:0/1584348418 conn(0x7f14600b2540 msgr2=0x7f14600d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 -- 172.21.15.88:0/1584348418 shutdown_connections 2024-09-21T07:46:47.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.396+0000 7f1466e00640 1 -- 172.21.15.88:0/1584348418 wait complete. 2024-09-21T07:46:47.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.399+0000 7f1466e00640 1 Processor -- start 2024-09-21T07:46:47.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.399+0000 7f1466e00640 1 -- start start 2024-09-21T07:46:47.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1466e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1444003a50 0x7f14600cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:47.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1466e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14601f51c0 0x7f14600ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1466e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 0x7f14600c10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1466e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f146006d540 con 0x7f14600ae690 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1466e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f146006d3c0 con 0x7f14601f51c0 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1466e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f146006d6c0 con 0x7f1444003a50 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.400+0000 7f1464e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 0x7f14600c10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 0x7f14600c10a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56470/0 (socket says 172.21.15.88:56470) 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 -- 172.21.15.88:0/3308032437 learned_addr learned my addr 172.21.15.88:0/3308032437 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f145f400640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14601f51c0 0x7f14600ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f145fe00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1444003a50 0x7f14600cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 -- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1444003a50 msgr2=0x7f14600cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1444003a50 0x7f14600cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 -- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14601f51c0 msgr2=0x7f14600ae150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14601f51c0 0x7f14600ae150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 -- 172.21.15.88:0/3308032437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14600aeaa0 con 0x7f14600ae690 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f145fe00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1444003a50 0x7f14600cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f145f400640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14601f51c0 0x7f14600ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1464e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 0x7f14600c10a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14600cac00 tx=0x7f145800eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.401+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f145800c9b0 con 0x7f14600ae690 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.402+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1458018070 con 0x7f14600ae690 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.402+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1458011460 con 0x7f14600ae690 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.402+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14600ce4f0 con 0x7f14600ae690 2024-09-21T07:46:47.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.402+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14600cece0 con 0x7f14600ae690 2024-09-21T07:46:47.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.403+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14600ce770 con 0x7f14600ae690 2024-09-21T07:46:47.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.404+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1458011600 con 0x7f14600ae690 2024-09-21T07:46:47.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.405+0000 7f1457e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1440078bc0 0x7f144007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:47.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.405+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 5415+0+0 (secure 0 0 0) 0x7f145809b130 con 0x7f14600ae690 2024-09-21T07:46:47.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.405+0000 7f145fe00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1440078bc0 0x7f144007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:47.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.406+0000 7f145fe00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1440078bc0 0x7f144007b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1450035000 tx=0x7f14500046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:47.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.409+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f14600aeaa0 con 0x7f14600ae690 2024-09-21T07:46:47.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:47.604+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f14600af800 con 0x7f14600ae690 2024-09-21T07:46:48.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.149+0000 7f1457e00640 1 -- 172.21.15.88:0/3308032437 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v27) ==== 169+0+0 (secure 0 0 0) 0x7f14580635f0 con 0x7f14600ae690 2024-09-21T07:46:48.151 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:46:48.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1440078bc0 msgr2=0x7f144007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1440078bc0 0x7f144007b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f1450035000 tx=0x7f14500046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 msgr2=0x7f14600c10a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 0x7f14600c10a0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f14600cac00 tx=0x7f145800eb60 comp rx=0 tx=0).stop 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 shutdown_connections 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1440078bc0 0x7f144007b0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14600ae690 0x7f14600c10a0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f14601f51c0 0x7f14600ae150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 --2- 172.21.15.88:0/3308032437 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1444003a50 0x7f14600cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 >> 172.21.15.88:0/3308032437 conn(0x7f14600b2540 msgr2=0x7f1460152fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 shutdown_connections 2024-09-21T07:46:48.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.151+0000 7f1466e00640 1 -- 172.21.15.88:0/3308032437 wait complete. 2024-09-21T07:46:48.179 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:46:48.179 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:46:48.180 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:46:48.180 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:48.180 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:46:48.180 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:46:48.180 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:46:48.180 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:48.180 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:46:48.180 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:46:48.181 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:46:48.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:48.181 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:46:48.181 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:46:48.181 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:46:48.181 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:46:48.201 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:46:48.201 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:46:48.205 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:46:48.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:46:48.206 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:46:48.206 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:46:48.207 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:46:48.207 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:46:48.248 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:46:48.249 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:48.251 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:46:48.252 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:48.253 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:46:48.253 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:48.255 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:46:48.255 DEBUG:teuthology.parallel:result is None 2024-09-21T07:46:48.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:46:48.445 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:46:48.445 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:46:48.451 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:46:48.451 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:46:48.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 -- 172.21.15.88:0/162395559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 msgr2=0x7f7490156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 --2- 172.21.15.88:0/162395559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f7490156e40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f74900b4fa0 tx=0x7f748402a530 comp rx=0 tx=0).stop 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 -- 172.21.15.88:0/162395559 shutdown_connections 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 --2- 172.21.15.88:0/162395559 >> v2:172.21.15.175:3300/0 conn(0x7f749015ce30 0x7f749015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 --2- 172.21.15.88:0/162395559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f7490156e40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 -- 172.21.15.88:0/162395559 >> 172.21.15.88:0/162395559 conn(0x7f749006d3d0 msgr2=0x7f74900b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 -- 172.21.15.88:0/162395559 shutdown_connections 2024-09-21T07:46:48.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.489+0000 7f7498800640 1 -- 172.21.15.88:0/162395559 wait complete. 2024-09-21T07:46:48.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.491+0000 7f7498800640 1 Processor -- start 2024-09-21T07:46:48.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.492+0000 7f7498800640 1 -- start start 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7498800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74900705b0 0x7f7490155c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7498800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f74901e8690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7498800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 0x7f74900ade70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7498800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74900cced0 con 0x7f749015ce30 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7498800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f749015d940 con 0x7f74900705b0 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7498800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74900cd050 con 0x7f74900cfc80 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7496800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 0x7f74900ade70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:48.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7496800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 0x7f74900ade70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56492/0 (socket says 172.21.15.88:56492) 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7496800640 1 -- 172.21.15.88:0/268831255 learned_addr learned my addr 172.21.15.88:0/268831255 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.493+0000 7f7495e00640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74900705b0 0x7f7490155c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7495400640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f74901e8690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7496800640 1 -- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 msgr2=0x7f74901e8690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7496800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f74901e8690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7496800640 1 -- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74900705b0 msgr2=0x7f7490155c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7496800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74900705b0 0x7f7490155c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7496800640 1 -- 172.21.15.88:0/268831255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74901f9e10 con 0x7f749015ce30 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7495400640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f74901e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:48.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7496800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 0x7f74900ade70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f748c00ea70 tx=0x7f748c00ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c00ce40 con 0x7f749015ce30 2024-09-21T07:46:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f748c004c10 con 0x7f749015ce30 2024-09-21T07:46:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f748c005020 con 0x7f749015ce30 2024-09-21T07:46:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74901e9200 con 0x7f749015ce30 2024-09-21T07:46:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.494+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74901f9050 con 0x7f749015ce30 2024-09-21T07:46:48.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.495+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74901e8f70 con 0x7f749015ce30 2024-09-21T07:46:48.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.497+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f748c0051c0 con 0x7f749015ce30 2024-09-21T07:46:48.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.498+0000 7f748a000640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7474078b90 0x7f747407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:48.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.498+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 4943+0+0 (secure 0 0 0) 0x7f748c09b870 con 0x7f749015ce30 2024-09-21T07:46:48.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.498+0000 7f7495e00640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7474078b90 0x7f747407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:48.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.499+0000 7f7495e00640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7474078b90 0x7f747407b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7484033000 tx=0x7f7484004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:48.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.503+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f748c063f10 con 0x7f749015ce30 2024-09-21T07:46:48.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:48.711+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f749015d240 con 0x7f749015ce30 2024-09-21T07:46:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.170+0000 7f748a000640 1 -- 172.21.15.88:0/268831255 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v29) ==== 97+0+0 (secure 0 0 0) 0x7f749015d240 con 0x7f749015ce30 2024-09-21T07:46:49.172 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7474078b90 msgr2=0x7f747407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7474078b90 0x7f747407b0a0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7484033000 tx=0x7f7484004730 comp rx=0 tx=0).stop 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 msgr2=0x7f74900ade70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 0x7f74900ade70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f748c00ea70 tx=0x7f748c00ef30 comp rx=0 tx=0).stop 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 shutdown_connections 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7474078b90 0x7f747407b0a0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74900cfc80 0x7f74901e8690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f749015ce30 0x7f74900ade70 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 --2- 172.21.15.88:0/268831255 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74900705b0 0x7f7490155c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.172+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 >> 172.21.15.88:0/268831255 conn(0x7f749006d3d0 msgr2=0x7f749006f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:49.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.173+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 shutdown_connections 2024-09-21T07:46:49.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.173+0000 7f7498800640 1 -- 172.21.15.88:0/268831255 wait complete. 2024-09-21T07:46:49.201 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:46:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 -- 172.21.15.88:0/1664623287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 msgr2=0x7fbd18073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 --2- 172.21.15.88:0/1664623287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 0x7fbd18073500 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009830 tx=0x7fbd0c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 -- 172.21.15.88:0/1664623287 shutdown_connections 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 --2- 172.21.15.88:0/1664623287 >> v2:172.21.15.175:3300/0 conn(0x7fbd1806f7b0 0x7fbd180b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 --2- 172.21.15.88:0/1664623287 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 0x7fbd18073500 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 -- 172.21.15.88:0/1664623287 >> 172.21.15.88:0/1664623287 conn(0x7fbd1806d730 msgr2=0x7fbd180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 -- 172.21.15.88:0/1664623287 shutdown_connections 2024-09-21T07:46:49.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.422+0000 7fbd20200640 1 -- 172.21.15.88:0/1664623287 wait complete. 2024-09-21T07:46:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.424+0000 7fbd20200640 1 Processor -- start 2024-09-21T07:46:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.425+0000 7fbd20200640 1 -- start start 2024-09-21T07:46:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd20200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 0x7fbd18151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd20200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd18073a40 0x7fbd181f9c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd20200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 0x7fbd1809ba10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd20200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180b3c50 con 0x7fbd1806f7b0 2024-09-21T07:46:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd20200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbd180b3ad0 con 0x7fbd18073a40 2024-09-21T07:46:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd20200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbd180b3dd0 con 0x7fbd180af630 2024-09-21T07:46:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd1d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 0x7fbd18151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd1d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 0x7fbd18151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56502/0 (socket says 172.21.15.88:56502) 2024-09-21T07:46:49.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.426+0000 7fbd1d800640 1 -- 172.21.15.88:0/3426910649 learned_addr learned my addr 172.21.15.88:0/3426910649 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd1d800640 1 -- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 msgr2=0x7fbd1809ba10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd1d800640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 0x7fbd1809ba10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd1d800640 1 -- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd18073a40 msgr2=0x7fbd181f9c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd1d800640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd18073a40 0x7fbd181f9c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd1d800640 1 -- 172.21.15.88:0/3426910649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd1815a0e0 con 0x7fbd1806f7b0 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd1d800640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 0x7fbd18151190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009950 tx=0x7fbd0c003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c005c90 con 0x7fbd1806f7b0 2024-09-21T07:46:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd0c005020 con 0x7fbd1806f7b0 2024-09-21T07:46:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd0c005430 con 0x7fbd1806f7b0 2024-09-21T07:46:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd1815a3c0 con 0x7fbd1806f7b0 2024-09-21T07:46:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.427+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd181f9420 con 0x7fbd1806f7b0 2024-09-21T07:46:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.428+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd180b4310 con 0x7fbd1806f7b0 2024-09-21T07:46:49.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.429+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbd0c0317c0 con 0x7fbd1806f7b0 2024-09-21T07:46:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.430+0000 7fbd15600640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbcf8078b90 0x7fbcf807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.431+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fbd0c0bbb30 con 0x7fbd1806f7b0 2024-09-21T07:46:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.431+0000 7fbd1ce00640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbcf8078b90 0x7fbcf807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:49.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.431+0000 7fbd1ce00640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbcf8078b90 0x7fbcf807b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbd1815e8d0 tx=0x7fbd04000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:49.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.435+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbd0c03f030 con 0x7fbd1806f7b0 2024-09-21T07:46:49.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.691+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fbd181592f0 con 0x7fbd1806f7b0 2024-09-21T07:46:49.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.691+0000 7fbd15600640 1 -- 172.21.15.88:0/3426910649 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) ==== 80+0+6645 (secure 0 0 0) 0x7fbd0c084220 con 0x7fbd1806f7b0 2024-09-21T07:46:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:46:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:46:49.696 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.697 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.698 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.699 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.700 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.701 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.702 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:46:49.703 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:46:49.704 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:46:49.705 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-mds", 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCiee5m2ZIVLRAAJfST7muPlx/kH4kzauw+5Q==", 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-mds" 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-mgr", 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCiee5moKQVLRAA9FkWoN6PjYHFxs0dqMp+Fg==", 2024-09-21T07:46:49.706 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-mgr" 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-osd", 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCiee5mpLYVLRAAgYPw4pHnz0rnwuOf6qGyJg==", 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-osd" 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rbd", 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCiee5m5MYVLRAA4kfKKAipavcZq/nLKsKBbQ==", 2024-09-21T07:46:49.707 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rbd" 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rbd-mirror", 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCiee5my9YVLRAALc8WcuJZam4KpZ3/YDaeyA==", 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.708 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.bootstrap-rgw", 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCiee5m4eYVLRAAXefnLQOnBw5lhIxEJtBwyw==", 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile bootstrap-rgw" 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:46:49.709 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:46:49.710 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbcf8078b90 msgr2=0x7fbcf807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbcf8078b90 0x7fbcf807b0a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fbd1815e8d0 tx=0x7fbd04000ed0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 msgr2=0x7fbd18151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 0x7fbd18151190 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fbd0c009950 tx=0x7fbd0c003c10 comp rx=0 tx=0).stop 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 shutdown_connections 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbcf8078b90 0x7fbcf807b0a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbd180af630 0x7fbd1809ba10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd18073a40 0x7fbd181f9c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 --2- 172.21.15.88:0/3426910649 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd1806f7b0 0x7fbd18151190 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 >> 172.21.15.88:0/3426910649 conn(0x7fbd1806d730 msgr2=0x7fbd181522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 shutdown_connections 2024-09-21T07:46:49.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.694+0000 7fbd20200640 1 -- 172.21.15.88:0/3426910649 wait complete. 2024-09-21T07:46:49.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 -- 172.21.15.88:0/2463769202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 msgr2=0x7f879415d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 --2- 172.21.15.88:0/2463769202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415d240 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f87940b4fa0 tx=0x7f878c02aa00 comp rx=0 tx=0).stop 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 -- 172.21.15.88:0/2463769202 shutdown_connections 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 --2- 172.21.15.88:0/2463769202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415d240 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 --2- 172.21.15.88:0/2463769202 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87940cfc80 0x7f8794156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 -- 172.21.15.88:0/2463769202 >> 172.21.15.88:0/2463769202 conn(0x7f879406d3d0 msgr2=0x7f87940b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 -- 172.21.15.88:0/2463769202 shutdown_connections 2024-09-21T07:46:49.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.940+0000 7f879a000640 1 -- 172.21.15.88:0/2463769202 wait complete. 2024-09-21T07:46:49.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.943+0000 7f879a000640 1 Processor -- start 2024-09-21T07:46:49.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.943+0000 7f879a000640 1 -- start start 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.944+0000 7f879a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87940705b0 0x7f87941563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.944+0000 7f879a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87940cfc80 0x7f87940c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.944+0000 7f879a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.944+0000 7f879a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87940cd050 con 0x7f879415ce30 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.944+0000 7f879a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87940cced0 con 0x7f87940cfc80 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.944+0000 7f879a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87940cd1d0 con 0x7f87940705b0 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56508/0 (socket says 172.21.15.88:56508) 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87940705b0 0x7f87941563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 -- 172.21.15.88:0/3487827065 learned_addr learned my addr 172.21.15.88:0/3487827065 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8792a00640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87940cfc80 0x7f87940c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 -- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87940705b0 msgr2=0x7f87941563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87940705b0 0x7f87941563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 -- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87940cfc80 msgr2=0x7f87940c7b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87940cfc80 0x7f87940c7b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 -- 172.21.15.88:0/3487827065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8794156930 con 0x7f879415ce30 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.945+0000 7f8793e00640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415b900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f878c009d60 tx=0x7f878c003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.946+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c033050 con 0x7f879415ce30 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.946+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f878c004450 con 0x7f879415ce30 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.946+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87940c8790 con 0x7f879415ce30 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.946+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878c004860 con 0x7f879415ce30 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.946+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87940cacf0 con 0x7f879415ce30 2024-09-21T07:46:49.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.947+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87940cd6f0 con 0x7f879415ce30 2024-09-21T07:46:49.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.948+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f878c036070 con 0x7f879415ce30 2024-09-21T07:46:49.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.949+0000 7f878b400640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8774078bc0 0x7f877407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:49.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.949+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f878c0bbd70 con 0x7f879415ce30 2024-09-21T07:46:49.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.949+0000 7f8793400640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8774078bc0 0x7f877407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:49.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.950+0000 7f8793400640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8774078bc0 0x7f877407b0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f879409e510 tx=0x7f8784005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:49.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:49.953+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f878c0844f0 con 0x7f879415ce30 2024-09-21T07:46:50.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.212+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) -- 0x7f87940c8470 con 0x7f879415ce30 2024-09-21T07:46:50.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.214+0000 7f878b400640 1 -- 172.21.15.88:0/3487827065 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) ==== 90+0+0 (secure 0 0 0) 0x7f878c083df0 con 0x7f879415ce30 2024-09-21T07:46:50.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8774078bc0 msgr2=0x7f877407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8774078bc0 0x7f877407b0d0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f879409e510 tx=0x7f8784005f90 comp rx=0 tx=0).stop 2024-09-21T07:46:50.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 msgr2=0x7f879415b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415b900 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f878c009d60 tx=0x7f878c003ef0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 shutdown_connections 2024-09-21T07:46:50.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8774078bc0 0x7f877407b0d0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f879415ce30 0x7f879415b900 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f87940cfc80 0x7f87940c7b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 --2- 172.21.15.88:0/3487827065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87940705b0 0x7f87941563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 >> 172.21.15.88:0/3487827065 conn(0x7f879406d3d0 msgr2=0x7f879415dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 shutdown_connections 2024-09-21T07:46:50.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.217+0000 7f879a000640 1 -- 172.21.15.88:0/3487827065 wait complete. 2024-09-21T07:46:50.244 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2024-09-21T07:46:50.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 -- 172.21.15.88:0/3516487514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13140ad9c0 msgr2=0x7f13140af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 --2- 172.21.15.88:0/3516487514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13140ad9c0 0x7f13140af170 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1308009830 tx=0x7f130802a480 comp rx=0 tx=0).stop 2024-09-21T07:46:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 -- 172.21.15.88:0/3516487514 shutdown_connections 2024-09-21T07:46:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 --2- 172.21.15.88:0/3516487514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13140ad9c0 0x7f13140af170 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 -- 172.21.15.88:0/3516487514 >> 172.21.15.88:0/3516487514 conn(0x7f13140b1e40 msgr2=0x7f13140b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 -- 172.21.15.88:0/3516487514 shutdown_connections 2024-09-21T07:46:50.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.461+0000 7f131ae00640 1 -- 172.21.15.88:0/3516487514 wait complete. 2024-09-21T07:46:50.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.463+0000 7f131ae00640 1 Processor -- start 2024-09-21T07:46:50.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.464+0000 7f131ae00640 1 -- start start 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140ad9c0 0x7f1314145ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b2f60 0x7f13140cec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 0x7f13140cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f131406fb20 con 0x7f131415a0f0 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f131406f9a0 con 0x7f13140ad9c0 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f131415d670 con 0x7f13140b2f60 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.465+0000 7f131a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 0x7f13140cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f1319a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140ad9c0 0x7f1314145ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f1319000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b2f60 0x7f13140cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 0x7f13140cf1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56536/0 (socket says 172.21.15.88:56536) 2024-09-21T07:46:50.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 -- 172.21.15.88:0/1501354942 learned_addr learned my addr 172.21.15.88:0/1501354942 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 -- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b2f60 msgr2=0x7f13140cec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b2f60 0x7f13140cec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 -- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140ad9c0 msgr2=0x7f1314145ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140ad9c0 0x7f1314145ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 -- 172.21.15.88:0/1501354942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1314146010 con 0x7f131415a0f0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.466+0000 7f131a400640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 0x7f13140cf1d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f130c00be60 tx=0x7f130c00bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.467+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c019070 con 0x7f131415a0f0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.467+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f130c014440 con 0x7f131415a0f0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.467+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13141f40d0 con 0x7f131415a0f0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.467+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f130c0148d0 con 0x7f131415a0f0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.467+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1314156b50 con 0x7f131415a0f0 2024-09-21T07:46:50.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.468+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1314159d30 con 0x7f131415a0f0 2024-09-21T07:46:50.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.469+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f130c01c070 con 0x7f131415a0f0 2024-09-21T07:46:50.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.470+0000 7f1311600640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12f4078bc0 0x7f12f407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.470+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f130c09aab0 con 0x7f131415a0f0 2024-09-21T07:46:50.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.470+0000 7f1319a00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12f4078bc0 0x7f12f407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.471+0000 7f1319a00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12f4078bc0 0x7f12f407b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1308034000 tx=0x7f1308004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:50.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.474+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1314159d30 con 0x7f131415a0f0 2024-09-21T07:46:50.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.729+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) -- 0x7f13140af170 con 0x7f131415a0f0 2024-09-21T07:46:50.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.732+0000 7f1311600640 1 -- 172.21.15.88:0/1501354942 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) ==== 90+0+0 (secure 0 0 0) 0x7f130c062b50 con 0x7f131415a0f0 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.734+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12f4078bc0 msgr2=0x7f12f407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.734+0000 7f131ae00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12f4078bc0 0x7f12f407b0d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f1308034000 tx=0x7f1308004730 comp rx=0 tx=0).stop 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.734+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 msgr2=0x7f13140cf1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.734+0000 7f131ae00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 0x7f13140cf1d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f130c00be60 tx=0x7f130c00bf50 comp rx=0 tx=0).stop 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 shutdown_connections 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f12f4078bc0 0x7f12f407b0d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13140b2f60 0x7f13140cec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f131415a0f0 0x7f13140cf1d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 --2- 172.21.15.88:0/1501354942 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13140ad9c0 0x7f1314145ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 >> 172.21.15.88:0/1501354942 conn(0x7f13140b1e40 msgr2=0x7f13140b0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 shutdown_connections 2024-09-21T07:46:50.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.735+0000 7f131ae00640 1 -- 172.21.15.88:0/1501354942 wait complete. 2024-09-21T07:46:50.759 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 -- 172.21.15.88:0/558844915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 msgr2=0x7fa2480af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 --2- 172.21.15.88:0/558844915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa2480af8d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa238009830 tx=0x7fa23802a480 comp rx=0 tx=0).stop 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 -- 172.21.15.88:0/558844915 shutdown_connections 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 --2- 172.21.15.88:0/558844915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa248070c80 0x7fa2480b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 --2- 172.21.15.88:0/558844915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa2480af8d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 -- 172.21.15.88:0/558844915 >> 172.21.15.88:0/558844915 conn(0x7fa24806d3d0 msgr2=0x7fa2480b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 -- 172.21.15.88:0/558844915 shutdown_connections 2024-09-21T07:46:50.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.978+0000 7fa24f200640 1 -- 172.21.15.88:0/558844915 wait complete. 2024-09-21T07:46:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.980+0000 7fa24f200640 1 Processor -- start 2024-09-21T07:46:50.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.981+0000 7fa24f200640 1 -- start start 2024-09-21T07:46:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa24f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24806f0d0 0x7fa248099aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa24f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa24815a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa24f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa248158da0 0x7fa248155f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa24f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480b3e20 con 0x7fa2480cbfc0 2024-09-21T07:46:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa24f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2480b3ca0 con 0x7fa24806f0d0 2024-09-21T07:46:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa24f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2480b3fa0 con 0x7fa248158da0 2024-09-21T07:46:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa247400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa24815a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa247400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa24815a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56552/0 (socket says 172.21.15.88:56552) 2024-09-21T07:46:50.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.982+0000 7fa247400640 1 -- 172.21.15.88:0/1072835030 learned_addr learned my addr 172.21.15.88:0/1072835030 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa247400640 1 -- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa248158da0 msgr2=0x7fa248155f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa247400640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa248158da0 0x7fa248155f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa247400640 1 -- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24806f0d0 msgr2=0x7fa248099aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa247400640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24806f0d0 0x7fa248099aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa247400640 1 -- 172.21.15.88:0/1072835030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa2480b10e0 con 0x7fa2480cbfc0 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa247400640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa24815a390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa23400c340 tx=0x7fa23400c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:50.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234017070 con 0x7fa2480cbfc0 2024-09-21T07:46:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2340035f0 con 0x7fa2480cbfc0 2024-09-21T07:46:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa234003a80 con 0x7fa2480cbfc0 2024-09-21T07:46:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa248155440 con 0x7fa2480cbfc0 2024-09-21T07:46:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.983+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa24815a8d0 con 0x7fa2480cbfc0 2024-09-21T07:46:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.984+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2480b3550 con 0x7fa2480cbfc0 2024-09-21T07:46:50.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.986+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fa23401a070 con 0x7fa2480cbfc0 2024-09-21T07:46:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.986+0000 7fa23fe00640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa228078bc0 0x7fa22807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:50.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.987+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fa234099d00 con 0x7fa2480cbfc0 2024-09-21T07:46:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.987+0000 7fa247e00640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa228078bc0 0x7fa22807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:50.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.988+0000 7fa247e00640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa228078bc0 0x7fa22807b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa238034000 tx=0x7fa2380046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:50.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:50.991+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa234062490 con 0x7fa2480cbfc0 2024-09-21T07:46:51.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.244+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) -- 0x7fa2480b0e00 con 0x7fa2480cbfc0 2024-09-21T07:46:51.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.247+0000 7fa23fe00640 1 -- 172.21.15.88:0/1072835030 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) ==== 90+0+0 (secure 0 0 0) 0x7fa234061d90 con 0x7fa2480cbfc0 2024-09-21T07:46:51.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa228078bc0 msgr2=0x7fa22807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa228078bc0 0x7fa22807b0d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa238034000 tx=0x7fa2380046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 msgr2=0x7fa24815a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa24815a390 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fa23400c340 tx=0x7fa23400c800 comp rx=0 tx=0).stop 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 shutdown_connections 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa228078bc0 0x7fa22807b0d0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa248158da0 0x7fa248155f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2480cbfc0 0x7fa24815a390 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 --2- 172.21.15.88:0/1072835030 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa24806f0d0 0x7fa248099aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.249+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 >> 172.21.15.88:0/1072835030 conn(0x7fa24806d3d0 msgr2=0x7fa248068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.250+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 shutdown_connections 2024-09-21T07:46:51.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.250+0000 7fa24f200640 1 -- 172.21.15.88:0/1072835030 wait complete. 2024-09-21T07:46:51.273 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2024-09-21T07:46:51.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07e00640 1 -- 172.21.15.88:0/2241055598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 msgr2=0x7faeec004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07e00640 1 --2- 172.21.15.88:0/2241055598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faeec004800 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7faef000c310 tx=0x7faeec005830 comp rx=0 tx=0).stop 2024-09-21T07:46:51.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07400640 1 -- 172.21.15.88:0/2241055598 reap_dead start 2024-09-21T07:46:51.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07e00640 1 -- 172.21.15.88:0/2241055598 shutdown_connections 2024-09-21T07:46:51.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07e00640 1 -- 172.21.15.88:0/2241055598 >> 172.21.15.88:0/2241055598 conn(0x7faf080b1e40 msgr2=0x7faf080b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:51.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07e00640 1 -- 172.21.15.88:0/2241055598 shutdown_connections 2024-09-21T07:46:51.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.494+0000 7faf07e00640 1 -- 172.21.15.88:0/2241055598 wait complete. 2024-09-21T07:46:51.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.496+0000 7faf07e00640 1 Processor -- start 2024-09-21T07:46:51.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.497+0000 7faf07e00640 1 -- start start 2024-09-21T07:46:51.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf07e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faf081519d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf07e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf08152060 0x7faf08158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf07e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf08158570 0x7faf08156a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf07e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf08155750 con 0x7faeec004070 2024-09-21T07:46:51.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf07e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf081555d0 con 0x7faf08152060 2024-09-21T07:46:51.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf07e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf081558d0 con 0x7faf08158570 2024-09-21T07:46:51.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf06a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faf081519d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf06a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faf081519d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56572/0 (socket says 172.21.15.88:56572) 2024-09-21T07:46:51.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faf06a00640 1 -- 172.21.15.88:0/2089710453 learned_addr learned my addr 172.21.15.88:0/2089710453 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:51.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.498+0000 7faeffe00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf08152060 0x7faf08158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf07400640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf08158570 0x7faf08156a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf06a00640 1 -- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf08158570 msgr2=0x7faf08156a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf06a00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf08158570 0x7faf08156a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf06a00640 1 -- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf08152060 msgr2=0x7faf08158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf06a00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf08152060 0x7faf08158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf06a00640 1 -- 172.21.15.88:0/2089710453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf08158980 con 0x7faeec004070 2024-09-21T07:46:51.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf07400640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf08158570 0x7faf08156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:51.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf06a00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faf081519d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faef8006690 tx=0x7faef80016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:51.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8015020 con 0x7faeec004070 2024-09-21T07:46:51.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faef8011900 con 0x7faeec004070 2024-09-21T07:46:51.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf0809b2f0 con 0x7faeec004070 2024-09-21T07:46:51.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.499+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef8011d50 con 0x7faeec004070 2024-09-21T07:46:51.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.500+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf08155f80 con 0x7faeec004070 2024-09-21T07:46:51.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.501+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf080b4ee0 con 0x7faeec004070 2024-09-21T07:46:51.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.502+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7faef8018020 con 0x7faeec004070 2024-09-21T07:46:51.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.502+0000 7faefea00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faee4078b90 0x7faee407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:51.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.503+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7faef809b5a0 con 0x7faeec004070 2024-09-21T07:46:51.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.503+0000 7faeffe00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faee4078b90 0x7faee407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:51.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.504+0000 7faeffe00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faee4078b90 0x7faee407b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faef0033000 tx=0x7faef0038040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:51.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.507+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf080b4ee0 con 0x7faeec004070 2024-09-21T07:46:51.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.762+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) -- 0x7faf08074ee0 con 0x7faeec004070 2024-09-21T07:46:51.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.765+0000 7faefea00640 1 -- 172.21.15.88:0/2089710453 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) ==== 90+0+0 (secure 0 0 0) 0x7faf08074ee0 con 0x7faeec004070 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.768+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faee4078b90 msgr2=0x7faee407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.768+0000 7faf07e00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faee4078b90 0x7faee407b0a0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faef0033000 tx=0x7faef0038040 comp rx=0 tx=0).stop 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.768+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 msgr2=0x7faf081519d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.768+0000 7faf07e00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faf081519d0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7faef8006690 tx=0x7faef80016f0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 shutdown_connections 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faee4078b90 0x7faee407b0a0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf08158570 0x7faf08156a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf08152060 0x7faf08158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 --2- 172.21.15.88:0/2089710453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faeec004070 0x7faf081519d0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:51.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 >> 172.21.15.88:0/2089710453 conn(0x7faf080b1e40 msgr2=0x7faf081eb110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:51.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 shutdown_connections 2024-09-21T07:46:51.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:51.769+0000 7faf07e00640 1 -- 172.21.15.88:0/2089710453 wait complete. 2024-09-21T07:46:51.791 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2024-09-21T07:46:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 -- 172.21.15.88:0/1217083562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c15d540 msgr2=0x7fef5c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 --2- 172.21.15.88:0/1217083562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c15d540 0x7fef5c156e40 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fef54009be0 tx=0x7fef5402a8d0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 -- 172.21.15.88:0/1217083562 shutdown_connections 2024-09-21T07:46:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 --2- 172.21.15.88:0/1217083562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c15d540 0x7fef5c156e40 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 -- 172.21.15.88:0/1217083562 >> 172.21.15.88:0/1217083562 conn(0x7fef5c06d3d0 msgr2=0x7fef5c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 -- 172.21.15.88:0/1217083562 shutdown_connections 2024-09-21T07:46:52.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.009+0000 7fef62a00640 1 -- 172.21.15.88:0/1217083562 wait complete. 2024-09-21T07:46:52.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.011+0000 7fef62a00640 1 Processor -- start 2024-09-21T07:46:52.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.011+0000 7fef62a00640 1 -- start start 2024-09-21T07:46:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.012+0000 7fef62a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 0x7fef5c154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.012+0000 7fef62a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef5c15d540 0x7fef5c1f9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.012+0000 7fef62a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef5c0b0e70 0x7fef5c0b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.012+0000 7fef62a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c0cd100 con 0x7fef5c0cfc80 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.012+0000 7fef62a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef5c0ccf80 con 0x7fef5c15d540 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.012+0000 7fef62a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef5c0cd280 con 0x7fef5c0b0e70 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 0x7fef5c154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 0x7fef5c154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56580/0 (socket says 172.21.15.88:56580) 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 -- 172.21.15.88:0/3204213228 learned_addr learned my addr 172.21.15.88:0/3204213228 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef53400640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef5c15d540 0x7fef5c1f9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef60a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef5c0b0e70 0x7fef5c0b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 -- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef5c0b0e70 msgr2=0x7fef5c0b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef5c0b0e70 0x7fef5c0b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 -- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef5c15d540 msgr2=0x7fef5c1f9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef5c15d540 0x7fef5c1f9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef5be00640 1 -- 172.21.15.88:0/3204213228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef5c0b17c0 con 0x7fef5c0cfc80 2024-09-21T07:46:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef53400640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef5c15d540 0x7fef5c1f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.013+0000 7fef60a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef5c0b0e70 0x7fef5c0b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.014+0000 7fef5be00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 0x7fef5c154bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c3d0 tx=0x7fef4c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.014+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011790 con 0x7fef5c0cfc80 2024-09-21T07:46:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.014+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef4c017070 con 0x7fef5c0cfc80 2024-09-21T07:46:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.014+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef5c1f5aa0 con 0x7fef5c0cfc80 2024-09-21T07:46:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.014+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef4c011ce0 con 0x7fef5c0cfc80 2024-09-21T07:46:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.014+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef5c1f6060 con 0x7fef5c0cfc80 2024-09-21T07:46:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.015+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef5c0cd770 con 0x7fef5c0cfc80 2024-09-21T07:46:52.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.016+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fef4c01c460 con 0x7fef5c0cfc80 2024-09-21T07:46:52.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.017+0000 7fef58c00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fef38078b90 0x7fef3807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.017+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fef4c09a770 con 0x7fef5c0cfc80 2024-09-21T07:46:52.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.018+0000 7fef53400640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fef38078b90 0x7fef3807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.019+0000 7fef53400640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fef38078b90 0x7fef3807b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0704c0 tx=0x7fef44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.022+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef5c0b17c0 con 0x7fef5c0cfc80 2024-09-21T07:46:52.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.277+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) -- 0x7fef5c0afa70 con 0x7fef5c0cfc80 2024-09-21T07:46:52.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.280+0000 7fef58c00640 1 -- 172.21.15.88:0/3204213228 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) ==== 97+0+0 (secure 0 0 0) 0x7fef4c062f90 con 0x7fef5c0cfc80 2024-09-21T07:46:52.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fef38078b90 msgr2=0x7fef3807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fef38078b90 0x7fef3807b0a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fef5c0704c0 tx=0x7fef44009040 comp rx=0 tx=0).stop 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 msgr2=0x7fef5c154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 0x7fef5c154bf0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fef4c00c3d0 tx=0x7fef4c00c890 comp rx=0 tx=0).stop 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 shutdown_connections 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fef38078b90 0x7fef3807b0a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef5c0b0e70 0x7fef5c0b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef5c15d540 0x7fef5c1f9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 --2- 172.21.15.88:0/3204213228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef5c0cfc80 0x7fef5c154bf0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 >> 172.21.15.88:0/3204213228 conn(0x7fef5c06d3d0 msgr2=0x7fef5c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 shutdown_connections 2024-09-21T07:46:52.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.282+0000 7fef62a00640 1 -- 172.21.15.88:0/3204213228 wait complete. 2024-09-21T07:46:52.306 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.523+0000 7f07a7600640 1 -- 172.21.15.88:0/977328423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a015ce30 msgr2=0x7f07a015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.523+0000 7f07a7600640 1 --2- 172.21.15.88:0/977328423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a015ce30 0x7f07a015d240 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f0798005fd0 tx=0x7f079802a870 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.524+0000 7f07a7600640 1 -- 172.21.15.88:0/977328423 shutdown_connections 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.524+0000 7f07a7600640 1 --2- 172.21.15.88:0/977328423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a015ce30 0x7f07a015d240 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.524+0000 7f07a7600640 1 --2- 172.21.15.88:0/977328423 >> v2:172.21.15.175:3300/0 conn(0x7f07a0070cc0 0x7f07a0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.524+0000 7f07a7600640 1 -- 172.21.15.88:0/977328423 >> 172.21.15.88:0/977328423 conn(0x7f07a006d3d0 msgr2=0x7f07a00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.524+0000 7f07a7600640 1 -- 172.21.15.88:0/977328423 shutdown_connections 2024-09-21T07:46:52.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.524+0000 7f07a7600640 1 -- 172.21.15.88:0/977328423 wait complete. 2024-09-21T07:46:52.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.526+0000 7f07a7600640 1 Processor -- start 2024-09-21T07:46:52.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.526+0000 7f07a7600640 1 -- start start 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.527+0000 7f07a7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a00cfc80 0x7f07a0155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.527+0000 7f07a7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a015ce30 0x7f07a00b0d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.527+0000 7f07a7600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 0x7f07a00b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.527+0000 7f07a7600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07a00cd050 con 0x7f07a01f4060 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.527+0000 7f07a7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f07a00cced0 con 0x7f07a00cfc80 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.527+0000 7f07a7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f07a00cd1d0 con 0x7f07a015ce30 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 0x7f07a00b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 0x7f07a00b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56600/0 (socket says 172.21.15.88:56600) 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 -- 172.21.15.88:0/367642356 learned_addr learned my addr 172.21.15.88:0/367642356 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:52.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a4c00640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a00cfc80 0x7f07a0155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f079fe00640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a015ce30 0x7f07a00b0d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 -- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a015ce30 msgr2=0x7f07a00b0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a015ce30 0x7f07a00b0d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 -- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a00cfc80 msgr2=0x7f07a0155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a00cfc80 0x7f07a0155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.528+0000 7f07a5600640 1 -- 172.21.15.88:0/367642356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07a00af040 con 0x7f07a01f4060 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.529+0000 7f07a5600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 0x7f07a00b1280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0798009cd0 tx=0x7f0798033ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.529+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798042020 con 0x7f07a01f4060 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.529+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0798033db0 con 0x7f07a01f4060 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.529+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07a01f3580 con 0x7f07a01f4060 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.529+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0798037040 con 0x7f07a01f4060 2024-09-21T07:46:52.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.529+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07a01f3b40 con 0x7f07a01f4060 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.530+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07a00aedb0 con 0x7f07a01f4060 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.531+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0798045020 con 0x7f07a01f4060 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.532+0000 7f079cc00640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0780078b90 0x7f078007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.532+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f07980bb830 con 0x7f07a01f4060 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.533+0000 7f07a4c00640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0780078b90 0x7f078007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.533+0000 7f07a4c00640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0780078b90 0x7f078007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07940046b0 tx=0x7f0794007eb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:52.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.536+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f079804c030 con 0x7f07a01f4060 2024-09-21T07:46:52.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.792+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) -- 0x7f07a01f3390 con 0x7f07a01f4060 2024-09-21T07:46:52.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.795+0000 7f079cc00640 1 -- 172.21.15.88:0/367642356 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) ==== 90+0+0 (secure 0 0 0) 0x7f0798083a00 con 0x7f07a01f4060 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0780078b90 msgr2=0x7f078007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0780078b90 0x7f078007b0a0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f07940046b0 tx=0x7f0794007eb0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 msgr2=0x7f07a00b1280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 0x7f07a00b1280 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f0798009cd0 tx=0x7f0798033ad0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 shutdown_connections 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0780078b90 0x7f078007b0a0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f07a015ce30 0x7f07a00b0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07a01f4060 0x7f07a00b1280 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.797+0000 7f07a7600640 1 --2- 172.21.15.88:0/367642356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f07a00cfc80 0x7f07a0155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.798+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 >> 172.21.15.88:0/367642356 conn(0x7f07a006d3d0 msgr2=0x7f07a01603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:52.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.798+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 shutdown_connections 2024-09-21T07:46:52.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:52.798+0000 7f07a7600640 1 -- 172.21.15.88:0/367642356 wait complete. 2024-09-21T07:46:52.821 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb990e00640 1 -- 172.21.15.88:0/3763954471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb970004d40 msgr2=0x7fb970005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb990e00640 1 --2- 172.21.15.88:0/3763954471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb970004d40 0x7fb970005150 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fb984009b20 tx=0x7fb970005830 comp rx=0 tx=0).stop 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb98a800640 1 -- 172.21.15.88:0/3763954471 reap_dead start 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb990e00640 1 -- 172.21.15.88:0/3763954471 shutdown_connections 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb990e00640 1 -- 172.21.15.88:0/3763954471 >> 172.21.15.88:0/3763954471 conn(0x7fb98c06d730 msgr2=0x7fb98c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb990e00640 1 -- 172.21.15.88:0/3763954471 shutdown_connections 2024-09-21T07:46:53.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.037+0000 7fb990e00640 1 -- 172.21.15.88:0/3763954471 wait complete. 2024-09-21T07:46:53.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.040+0000 7fb990e00640 1 Processor -- start 2024-09-21T07:46:53.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.040+0000 7fb990e00640 1 -- start start 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.041+0000 7fb990e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb970004d40 0x7fb98c148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.041+0000 7fb990e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 0x7fb98c1562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.041+0000 7fb990e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb98c15b7f0 0x7fb98c15bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.041+0000 7fb990e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0b4da0 con 0x7fb98c155ee0 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.041+0000 7fb990e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c0b4c20 con 0x7fb98c15b7f0 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.041+0000 7fb990e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb98c0b3960 con 0x7fb970004d40 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 0x7fb98c1562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 0x7fb98c1562f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56614/0 (socket says 172.21.15.88:56614) 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 -- 172.21.15.88:0/2692426785 learned_addr learned my addr 172.21.15.88:0/2692426785 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:53.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb98a800640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb98c15b7f0 0x7fb98c15bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb970004d40 0x7fb98c148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 -- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb970004d40 msgr2=0x7fb98c148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb970004d40 0x7fb98c148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 -- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb98c15b7f0 msgr2=0x7fb98c15bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb98c15b7f0 0x7fb98c15bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 -- 172.21.15.88:0/2692426785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c148e50 con 0x7fb98c155ee0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.042+0000 7fb989400640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 0x7fb98c1562f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e600 tx=0x7fb97800eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.043+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb978015070 con 0x7fb98c155ee0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.043+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb978004310 con 0x7fb98c155ee0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.043+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb97800c9a0 con 0x7fb98c155ee0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.043+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c15c140 con 0x7fb98c155ee0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.043+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c0733c0 con 0x7fb98c155ee0 2024-09-21T07:46:53.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.044+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c1535c0 con 0x7fb98c155ee0 2024-09-21T07:46:53.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.045+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb97800cb40 con 0x7fb98c155ee0 2024-09-21T07:46:53.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.046+0000 7fb982000640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb96c078bc0 0x7fb96c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.047+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7fb97809ae80 con 0x7fb98c155ee0 2024-09-21T07:46:53.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.047+0000 7fb989e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb96c078bc0 0x7fb96c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.048+0000 7fb989e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb96c078bc0 0x7fb96c07b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb97c005fd0 tx=0x7fb97c009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.051+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb98c15c140 con 0x7fb98c155ee0 2024-09-21T07:46:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.237+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb98c0b4610 con 0x7fb98c155ee0 2024-09-21T07:46:53.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.237+0000 7fb982000640 1 -- 172.21.15.88:0/2692426785 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) ==== 84+0+65 (secure 0 0 0) 0x7fb98c148e50 con 0x7fb98c155ee0 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:46:53.240 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:46:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb96c078bc0 msgr2=0x7fb96c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb96c078bc0 0x7fb96c07b0d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fb97c005fd0 tx=0x7fb97c009500 comp rx=0 tx=0).stop 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 msgr2=0x7fb98c1562f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 0x7fb98c1562f0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fb97800e600 tx=0x7fb97800eac0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 shutdown_connections 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb96c078bc0 0x7fb96c07b0d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb98c15b7f0 0x7fb98c15bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb98c155ee0 0x7fb98c1562f0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 --2- 172.21.15.88:0/2692426785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb970004d40 0x7fb98c148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 >> 172.21.15.88:0/2692426785 conn(0x7fb98c06d730 msgr2=0x7fb98c15ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 shutdown_connections 2024-09-21T07:46:53.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.240+0000 7fb990e00640 1 -- 172.21.15.88:0/2692426785 wait complete. 2024-09-21T07:46:53.263 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:46:53.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.485+0000 7f9aeae00640 1 -- 172.21.15.88:0/3161461996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 msgr2=0x7f9ae40b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.485+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3161461996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae40b47c0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0009830 tx=0x7f9ad002a530 comp rx=0 tx=0).stop 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.485+0000 7f9aeae00640 1 -- 172.21.15.88:0/3161461996 shutdown_connections 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.485+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3161461996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae40b47c0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.485+0000 7f9aeae00640 1 -- 172.21.15.88:0/3161461996 >> 172.21.15.88:0/3161461996 conn(0x7f9ae40b1dd0 msgr2=0x7f9ae40b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.486+0000 7f9aeae00640 1 -- 172.21.15.88:0/3161461996 shutdown_connections 2024-09-21T07:46:53.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.486+0000 7f9aeae00640 1 -- 172.21.15.88:0/3161461996 wait complete. 2024-09-21T07:46:53.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.488+0000 7f9aeae00640 1 Processor -- start 2024-09-21T07:46:53.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.488+0000 7f9aeae00640 1 -- start start 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.489+0000 7f9aeae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae4151b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.489+0000 7f9aeae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae40af3c0 0x7f9ae4152820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.489+0000 7f9aeae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ae40b3ec0 0x7f9ae4152d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.489+0000 7f9aeae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4160bd0 con 0x7f9ae40705b0 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.489+0000 7f9aeae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ae4153c60 con 0x7f9ae40af3c0 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.489+0000 7f9aeae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ae4167e20 con 0x7f9ae40b3ec0 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae4151b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae4151b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56624/0 (socket says 172.21.15.88:56624) 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 -- 172.21.15.88:0/3877701767 learned_addr learned my addr 172.21.15.88:0/3877701767 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9000640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae40af3c0 0x7f9ae4152820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9aea400640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ae40b3ec0 0x7f9ae4152d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 -- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ae40b3ec0 msgr2=0x7f9ae4152d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ae40b3ec0 0x7f9ae4152d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 -- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae40af3c0 msgr2=0x7f9ae4152820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae40af3c0 0x7f9ae4152820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 -- 172.21.15.88:0/3877701767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ae41f8fa0 con 0x7f9ae40705b0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.490+0000 7f9ae9a00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae4151b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c3a0 tx=0x7f9ad800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.491+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8017070 con 0x7f9ae40705b0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.491+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ad80035f0 con 0x7f9ae40705b0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.491+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ad8003a40 con 0x7f9ae40705b0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.491+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ae41f9210 con 0x7f9ae40705b0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.491+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ae41f97d0 con 0x7f9ae40705b0 2024-09-21T07:46:53.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.492+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ae40b47c0 con 0x7f9ae40705b0 2024-09-21T07:46:53.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.493+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9ad801a070 con 0x7f9ae40705b0 2024-09-21T07:46:53.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.494+0000 7f9ae1600640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9ac4078bc0 0x7f9ac407b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:53.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.494+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(31..31 src has 1..31) ==== 4857+0+0 (secure 0 0 0) 0x7f9ad8099bf0 con 0x7f9ae40705b0 2024-09-21T07:46:53.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.494+0000 7f9ae9000640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9ac4078bc0 0x7f9ac407b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:53.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.495+0000 7f9ae9000640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9ac4078bc0 0x7f9ac407b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0034000 tx=0x7f9ad0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:53.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.499+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ae40b4ec0 con 0x7f9ae40705b0 2024-09-21T07:46:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:53.685+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f9ae40b2480 con 0x7f9ae40705b0 2024-09-21T07:46:54.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.180+0000 7f9ae1600640 1 -- 172.21.15.88:0/3877701767 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) ==== 138+0+0 (secure 0 0 0) 0x7f9ad8061d20 con 0x7f9ae40705b0 2024-09-21T07:46:54.182 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9ac4078bc0 msgr2=0x7f9ac407b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9ac4078bc0 0x7f9ac407b0d0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f9ad0034000 tx=0x7f9ad0004730 comp rx=0 tx=0).stop 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 msgr2=0x7f9ae4151b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae4151b90 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f9ad800c3a0 tx=0x7f9ad800c860 comp rx=0 tx=0).stop 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 shutdown_connections 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9ac4078bc0 0x7f9ac407b0d0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ae40b3ec0 0x7f9ae4152d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ae40af3c0 0x7f9ae4152820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 --2- 172.21.15.88:0/3877701767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ae40705b0 0x7f9ae4151b90 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.183+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 >> 172.21.15.88:0/3877701767 conn(0x7f9ae40b1dd0 msgr2=0x7f9ae4072e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.184+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 shutdown_connections 2024-09-21T07:46:54.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.184+0000 7f9aeae00640 1 -- 172.21.15.88:0/3877701767 wait complete. 2024-09-21T07:46:54.209 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:46:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.436+0000 7f12af400640 1 -- 172.21.15.88:0/2591246930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 msgr2=0x7f12a80739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.436+0000 7f12af400640 1 --2- 172.21.15.88:0/2591246930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a80739f0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f1294009830 tx=0x7f129402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.437+0000 7f12af400640 1 -- 172.21.15.88:0/2591246930 shutdown_connections 2024-09-21T07:46:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.437+0000 7f12af400640 1 --2- 172.21.15.88:0/2591246930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a80739f0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.437+0000 7f12af400640 1 -- 172.21.15.88:0/2591246930 >> 172.21.15.88:0/2591246930 conn(0x7f12a806d1c0 msgr2=0x7f12a806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:54.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.437+0000 7f12af400640 1 -- 172.21.15.88:0/2591246930 shutdown_connections 2024-09-21T07:46:54.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.437+0000 7f12af400640 1 -- 172.21.15.88:0/2591246930 wait complete. 2024-09-21T07:46:54.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.439+0000 7f12af400640 1 Processor -- start 2024-09-21T07:46:54.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.439+0000 7f12af400640 1 -- start start 2024-09-21T07:46:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12af400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a813da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12af400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12a80cd610 0x7f12a815b500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12af400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12a8145450 0x7f12a815d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12af400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80ce4c0 con 0x7f12a80735e0 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12af400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12a80ce340 con 0x7f12a80cd610 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12af400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f12a80ce640 con 0x7f12a8145450 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a813da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a813da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56638/0 (socket says 172.21.15.88:56638) 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 -- 172.21.15.88:0/3911264841 learned_addr learned my addr 172.21.15.88:0/3911264841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12a7e00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12a80cd610 0x7f12a815b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12ad400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12a8145450 0x7f12a815d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 -- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12a8145450 msgr2=0x7f12a815d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12a8145450 0x7f12a815d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 -- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12a80cd610 msgr2=0x7f12a815b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:54.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12a80cd610 0x7f12a815b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.441+0000 7f12aca00640 1 -- 172.21.15.88:0/3911264841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12a80af100 con 0x7f12a80735e0 2024-09-21T07:46:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.442+0000 7f12aca00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a813da50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f129c00c400 tx=0x7f129c00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.442+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c017070 con 0x7f12a80735e0 2024-09-21T07:46:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.442+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f129c0053c0 con 0x7f12a80735e0 2024-09-21T07:46:54.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.442+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12a80af3e0 con 0x7f12a80735e0 2024-09-21T07:46:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.442+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129c003850 con 0x7f12a80735e0 2024-09-21T07:46:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.442+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12a815e280 con 0x7f12a80735e0 2024-09-21T07:46:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.443+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12a80739f0 con 0x7f12a80735e0 2024-09-21T07:46:54.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.444+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f129c01a070 con 0x7f12a80735e0 2024-09-21T07:46:54.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.445+0000 7f12a4c00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1288078b90 0x7f128807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:54.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.446+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 5227+0+0 (secure 0 0 0) 0x7f129c099f70 con 0x7f12a80735e0 2024-09-21T07:46:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.446+0000 7f12a7e00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1288078b90 0x7f128807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:54.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.447+0000 7f12a7e00640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1288078b90 0x7f128807b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1294034000 tx=0x7f1294004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:54.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.451+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f129c062580 con 0x7f12a80735e0 2024-09-21T07:46:54.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:54.636+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f12a8161000 con 0x7f12a80735e0 2024-09-21T07:46:55.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.183+0000 7f12a4c00640 1 -- 172.21.15.88:0/3911264841 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) ==== 170+0+0 (secure 0 0 0) 0x7f129c061e80 con 0x7f12a80735e0 2024-09-21T07:46:55.186 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:46:55.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1288078b90 msgr2=0x7f128807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1288078b90 0x7f128807b0a0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f1294034000 tx=0x7f1294004730 comp rx=0 tx=0).stop 2024-09-21T07:46:55.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 msgr2=0x7f12a813da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a813da50 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f129c00c400 tx=0x7f129c00c8c0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 shutdown_connections 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1288078b90 0x7f128807b0a0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12a8145450 0x7f12a815d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12a80cd610 0x7f12a815b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 --2- 172.21.15.88:0/3911264841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12a80735e0 0x7f12a813da50 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 >> 172.21.15.88:0/3911264841 conn(0x7f12a806d1c0 msgr2=0x7f12a80cda20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 shutdown_connections 2024-09-21T07:46:55.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.187+0000 7f12af400640 1 -- 172.21.15.88:0/3911264841 wait complete. 2024-09-21T07:46:55.227 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:46:55.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.447+0000 7f8523600640 1 -- 172.21.15.88:0/3010925259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c145550 msgr2=0x7f851c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.447+0000 7f8523600640 1 --2- 172.21.15.88:0/3010925259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c145550 0x7f851c06fb40 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f8510005fd0 tx=0x7f851002a670 comp rx=0 tx=0).stop 2024-09-21T07:46:55.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.448+0000 7f8523600640 1 -- 172.21.15.88:0/3010925259 shutdown_connections 2024-09-21T07:46:55.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.448+0000 7f8523600640 1 --2- 172.21.15.88:0/3010925259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c073c50 0x7f851c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.448+0000 7f8523600640 1 --2- 172.21.15.88:0/3010925259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c145550 0x7f851c06fb40 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.448+0000 7f8523600640 1 -- 172.21.15.88:0/3010925259 >> 172.21.15.88:0/3010925259 conn(0x7f851c06d1c0 msgr2=0x7f851c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:55.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.448+0000 7f8523600640 1 -- 172.21.15.88:0/3010925259 shutdown_connections 2024-09-21T07:46:55.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.448+0000 7f8523600640 1 -- 172.21.15.88:0/3010925259 wait complete. 2024-09-21T07:46:55.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.450+0000 7f8523600640 1 Processor -- start 2024-09-21T07:46:55.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.450+0000 7f8523600640 1 -- start start 2024-09-21T07:46:55.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.451+0000 7f8523600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c145550 0x7f851c132180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.451+0000 7f8523600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851c145a40 0x7f851c15bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.451+0000 7f8523600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 0x7f851c1f4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.451+0000 7f8523600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0ce150 con 0x7f851c130720 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.451+0000 7f8523600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f851c0cdfd0 con 0x7f851c145a40 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.451+0000 7f8523600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f851c0ce2d0 con 0x7f851c145550 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 0x7f851c1f4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 0x7f851c1f4550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56666/0 (socket says 172.21.15.88:56666) 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 -- 172.21.15.88:0/306277587 learned_addr learned my addr 172.21.15.88:0/306277587 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f851be00640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851c145a40 0x7f851c15bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8520c00640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c145550 0x7f851c132180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 -- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c145550 msgr2=0x7f851c132180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c145550 0x7f851c132180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 -- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851c145a40 msgr2=0x7f851c15bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851c145a40 0x7f851c15bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:55.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8521600640 1 -- 172.21.15.88:0/306277587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f851c15c4e0 con 0x7f851c130720 2024-09-21T07:46:55.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.452+0000 7f8520c00640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c145550 0x7f851c132180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:46:55.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.453+0000 7f8521600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 0x7f851c1f4550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f851400e960 tx=0x7f851400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:55.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.453+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514004d20 con 0x7f851c130720 2024-09-21T07:46:55.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.453+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f851400cac0 con 0x7f851c130720 2024-09-21T07:46:55.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.453+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f851c1310e0 con 0x7f851c130720 2024-09-21T07:46:55.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.453+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8514005020 con 0x7f851c130720 2024-09-21T07:46:55.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.453+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f851c1316a0 con 0x7f851c130720 2024-09-21T07:46:55.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.454+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f851c0ce7f0 con 0x7f851c130720 2024-09-21T07:46:55.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.455+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f85140051c0 con 0x7f851c130720 2024-09-21T07:46:55.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.456+0000 7f8518c00640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f84fc078bc0 0x7f84fc07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:55.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.456+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(33..33 src has 1..33) ==== 5609+0+0 (secure 0 0 0) 0x7f851409b5c0 con 0x7f851c130720 2024-09-21T07:46:55.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.457+0000 7f8520c00640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f84fc078bc0 0x7f84fc07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:55.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.458+0000 7f8520c00640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f84fc078bc0 0x7f84fc07b0d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:55.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.461+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8514016030 con 0x7f851c130720 2024-09-21T07:46:55.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:55.680+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f851c1412f0 con 0x7f851c130720 2024-09-21T07:46:55.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:55.884 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:46:56.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.206+0000 7f8518c00640 1 -- 172.21.15.88:0/306277587 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v13) ==== 172+0+0 (secure 0 0 0) 0x7f8514063a50 con 0x7f851c130720 2024-09-21T07:46:56.211 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 4 and data pool 5 2024-09-21T07:46:56.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.212+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f84fc078bc0 msgr2=0x7f84fc07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.212+0000 7f8523600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f84fc078bc0 0x7f84fc07b0d0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f8510033000 tx=0x7f8510004730 comp rx=0 tx=0).stop 2024-09-21T07:46:56.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.212+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 msgr2=0x7f851c1f4550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.212+0000 7f8523600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 0x7f851c1f4550 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f851400e960 tx=0x7f851400ee20 comp rx=0 tx=0).stop 2024-09-21T07:46:56.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 shutdown_connections 2024-09-21T07:46:56.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f84fc078bc0 0x7f84fc07b0d0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f851c130720 0x7f851c1f4550 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f851c145a40 0x7f851c15bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 --2- 172.21.15.88:0/306277587 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f851c145550 0x7f851c132180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 >> 172.21.15.88:0/306277587 conn(0x7f851c06d1c0 msgr2=0x7f851c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 shutdown_connections 2024-09-21T07:46:56.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.214+0000 7f8523600640 1 -- 172.21.15.88:0/306277587 wait complete. 2024-09-21T07:46:56.258 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:56.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 -- 172.21.15.88:0/1651763560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce4070000 msgr2=0x7f8ce4070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 --2- 172.21.15.88:0/1651763560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce4070000 0x7f8ce4070410 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f8ce40d1190 tx=0x7f8cdc02aa50 comp rx=0 tx=0).stop 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 -- 172.21.15.88:0/1651763560 shutdown_connections 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 --2- 172.21.15.88:0/1651763560 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce4070000 0x7f8ce4070410 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 --2- 172.21.15.88:0/1651763560 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce4072e60 0x7f8ce4073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 --2- 172.21.15.88:0/1651763560 >> v2:172.21.15.175:3300/0 conn(0x7f8ce40af050 0x7f8ce40ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 -- 172.21.15.88:0/1651763560 >> 172.21.15.88:0/1651763560 conn(0x7f8ce40b1e40 msgr2=0x7f8ce40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 -- 172.21.15.88:0/1651763560 shutdown_connections 2024-09-21T07:46:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.483+0000 7f8ce3200640 1 -- 172.21.15.88:0/1651763560 wait complete. 2024-09-21T07:46:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.485+0000 7f8ce3200640 1 Processor -- start 2024-09-21T07:46:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.485+0000 7f8ce3200640 1 -- start start 2024-09-21T07:46:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce3200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce4070000 0x7f8ce4142970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce3200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce4072e60 0x7f8ce414bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce3200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 0x7f8ce414eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce3200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce415c0b0 con 0x7f8ce40af050 2024-09-21T07:46:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce3200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ce415bf30 con 0x7f8ce4072e60 2024-09-21T07:46:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce3200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ce415c230 con 0x7f8ce4070000 2024-09-21T07:46:56.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 0x7f8ce414eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 0x7f8ce414eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58362/0 (socket says 172.21.15.88:58362) 2024-09-21T07:46:56.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.487+0000 7f8ce2800640 1 -- 172.21.15.88:0/2125835205 learned_addr learned my addr 172.21.15.88:0/2125835205 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:56.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce1e00640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce4070000 0x7f8ce4142970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce1400640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce4072e60 0x7f8ce414bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce2800640 1 -- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce4070000 msgr2=0x7f8ce4142970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce2800640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce4070000 0x7f8ce4142970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce2800640 1 -- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce4072e60 msgr2=0x7f8ce414bfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce2800640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce4072e60 0x7f8ce414bfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce2800640 1 -- 172.21.15.88:0/2125835205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ce414a630 con 0x7f8ce40af050 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce2800640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 0x7f8ce414eed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc02b010 tx=0x7f8cdc003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc033050 con 0x7f8ce40af050 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8cdc004250 con 0x7f8ce40af050 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cdc004680 con 0x7f8ce40af050 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ce414a910 con 0x7f8ce40af050 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.488+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ce414aed0 con 0x7f8ce40af050 2024-09-21T07:46:56.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.490+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ce415c750 con 0x7f8ce40af050 2024-09-21T07:46:56.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.491+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8cdc036070 con 0x7f8ce40af050 2024-09-21T07:46:56.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.492+0000 7f8cda000640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cc4078b90 0x7f8cc407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.492+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7f8cdc0bbf10 con 0x7f8ce40af050 2024-09-21T07:46:56.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.492+0000 7f8ce1e00640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cc4078b90 0x7f8cc407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.493+0000 7f8ce1e00640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cc4078b90 0x7f8cc407b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4005fd0 tx=0x7f8cd4005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:56.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.497+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8cdc0843e0 con 0x7f8ce40af050 2024-09-21T07:46:56.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.686+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8ce40ae3a0 con 0x7f8ce40af050 2024-09-21T07:46:56.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.687+0000 7f8cda000640 1 -- 172.21.15.88:0/2125835205 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16694 (secure 0 0 0) 0x7f8cdc004820 con 0x7f8ce40af050 2024-09-21T07:46:56.690 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:56.690 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":34,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:55.682699+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:53.697978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:54.649546+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:56.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cc4078b90 msgr2=0x7f8cc407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cc4078b90 0x7f8cc407b0a0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8cd4005fd0 tx=0x7f8cd4005850 comp rx=0 tx=0).stop 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 msgr2=0x7f8ce414eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 0x7f8ce414eed0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f8cdc02b010 tx=0x7f8cdc003f10 comp rx=0 tx=0).stop 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 shutdown_connections 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cc4078b90 0x7f8cc407b0a0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ce40af050 0x7f8ce414eed0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ce4072e60 0x7f8ce414bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 --2- 172.21.15.88:0/2125835205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ce4070000 0x7f8ce4142970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 >> 172.21.15.88:0/2125835205 conn(0x7f8ce40b1e40 msgr2=0x7f8ce40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 shutdown_connections 2024-09-21T07:46:56.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.690+0000 7f8ce3200640 1 -- 172.21.15.88:0/2125835205 wait complete. 2024-09-21T07:46:56.714 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:56.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 -- 172.21.15.88:0/3539445311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d4fc0 msgr2=0x7fec600d2310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 --2- 172.21.15.88:0/3539445311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d4fc0 0x7fec600d2310 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005fd0 tx=0x7fec4c02a540 comp rx=0 tx=0).stop 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 -- 172.21.15.88:0/3539445311 shutdown_connections 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 --2- 172.21.15.88:0/3539445311 >> v2:172.21.15.175:3300/0 conn(0x7fec600d2850 0x7fec600cfd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 --2- 172.21.15.88:0/3539445311 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d4fc0 0x7fec600d2310 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 -- 172.21.15.88:0/3539445311 >> 172.21.15.88:0/3539445311 conn(0x7fec600d44a0 msgr2=0x7fec600d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 -- 172.21.15.88:0/3539445311 shutdown_connections 2024-09-21T07:46:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.953+0000 7fec5fe00640 1 -- 172.21.15.88:0/3539445311 wait complete. 2024-09-21T07:46:56.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.955+0000 7fec5fe00640 1 Processor -- start 2024-09-21T07:46:56.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.955+0000 7fec5fe00640 1 -- start start 2024-09-21T07:46:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec600b9840 0x7fec600ca9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d2850 0x7fec601f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 0x7fec600b6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec600d0e90 con 0x7fec600d4fc0 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec600d0d10 con 0x7fec600b9840 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec600d1010 con 0x7fec600d2850 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 0x7fec600b6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 0x7fec600b6270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58370/0 (socket says 172.21.15.88:58370) 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5f400640 1 -- 172.21.15.88:0/2884153477 learned_addr learned my addr 172.21.15.88:0/2884153477 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5e000640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d2850 0x7fec601f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5ea00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec600b9840 0x7fec600ca9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.957+0000 7fec5f400640 1 -- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d2850 msgr2=0x7fec601f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5f400640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d2850 0x7fec601f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5f400640 1 -- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec600b9840 msgr2=0x7fec600ca9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5f400640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec600b9840 0x7fec600ca9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5f400640 1 -- 172.21.15.88:0/2884153477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec600cd190 con 0x7fec600d4fc0 2024-09-21T07:46:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5f400640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 0x7fec600b6270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec5800ea80 tx=0x7fec5800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5800ce40 con 0x7fec600d4fc0 2024-09-21T07:46:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec58004b20 con 0x7fec600d4fc0 2024-09-21T07:46:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec600cd470 con 0x7fec600d4fc0 2024-09-21T07:46:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec58005020 con 0x7fec600d4fc0 2024-09-21T07:46:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.958+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec600c96d0 con 0x7fec600d4fc0 2024-09-21T07:46:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.959+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec6008cc70 con 0x7fec600d4fc0 2024-09-21T07:46:56.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.962+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fec580051c0 con 0x7fec600d4fc0 2024-09-21T07:46:56.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.963+0000 7fec56a00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078bc0 0x7fec4007b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:56.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.963+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 5709+0+0 (secure 0 0 0) 0x7fec5809b850 con 0x7fec600d4fc0 2024-09-21T07:46:56.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.963+0000 7fec5ea00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078bc0 0x7fec4007b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:56.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.964+0000 7fec5ea00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078bc0 0x7fec4007b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec600b3ed0 tx=0x7fec50009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:56.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:56.967+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec6008cc70 con 0x7fec600d4fc0 2024-09-21T07:46:57.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.162+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fec600d1510 con 0x7fec600d4fc0 2024-09-21T07:46:57.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.163+0000 7fec56a00640 1 -- 172.21.15.88:0/2884153477 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) ==== 74+0+16694 (secure 0 0 0) 0x7fec58063d30 con 0x7fec600d4fc0 2024-09-21T07:46:57.166 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:57.166 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":34,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:55.682699+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:53.697978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:54.649546+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:46:57.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078bc0 msgr2=0x7fec4007b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078bc0 0x7fec4007b0d0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fec600b3ed0 tx=0x7fec50009040 comp rx=0 tx=0).stop 2024-09-21T07:46:57.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 msgr2=0x7fec600b6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 0x7fec600b6270 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fec5800ea80 tx=0x7fec5800ef40 comp rx=0 tx=0).stop 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 shutdown_connections 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078bc0 0x7fec4007b0d0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec600d2850 0x7fec601f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600d4fc0 0x7fec600b6270 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 --2- 172.21.15.88:0/2884153477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec600b9840 0x7fec600ca9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 >> 172.21.15.88:0/2884153477 conn(0x7fec600d44a0 msgr2=0x7fec600b1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 shutdown_connections 2024-09-21T07:46:57.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.166+0000 7fec5fe00640 1 -- 172.21.15.88:0/2884153477 wait complete. 2024-09-21T07:46:57.190 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:46:57.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 -- 172.21.15.88:0/3350807556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 msgr2=0x7fe634161f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 --2- 172.21.15.88:0/3350807556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 0x7fe634161f90 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe628009860 tx=0x7fe62802a5a0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 -- 172.21.15.88:0/3350807556 shutdown_connections 2024-09-21T07:46:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 --2- 172.21.15.88:0/3350807556 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 0x7fe634161f90 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 -- 172.21.15.88:0/3350807556 >> 172.21.15.88:0/3350807556 conn(0x7fe63409d900 msgr2=0x7fe63408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 -- 172.21.15.88:0/3350807556 shutdown_connections 2024-09-21T07:46:57.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.417+0000 7fe63a800640 1 -- 172.21.15.88:0/3350807556 wait complete. 2024-09-21T07:46:57.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.419+0000 7fe63a800640 1 Processor -- start 2024-09-21T07:46:57.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.419+0000 7fe63a800640 1 -- start start 2024-09-21T07:46:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.420+0000 7fe63a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 0x7fe63415afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.420+0000 7fe63a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 0x7fe6340b49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.420+0000 7fe63a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6340d43d0 0x7fe6341ec610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe63a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6340b8d90 con 0x7fe6340b7ee0 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe63a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6340b8c10 con 0x7fe6340d43d0 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe63a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6340b8f10 con 0x7fe63406de60 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 0x7fe6340b49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 0x7fe6340b49f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58382/0 (socket says 172.21.15.88:58382) 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 -- 172.21.15.88:0/1402893736 learned_addr learned my addr 172.21.15.88:0/1402893736 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe639400640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 0x7fe63415afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe639e00640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6340d43d0 0x7fe6341ec610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 -- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 msgr2=0x7fe63415afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 0x7fe63415afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 -- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6340d43d0 msgr2=0x7fe6341ec610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6340d43d0 0x7fe6341ec610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.421+0000 7fe638a00640 1 -- 172.21.15.88:0/1402893736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6340a2220 con 0x7fe6340b7ee0 2024-09-21T07:46:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.422+0000 7fe638a00640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 0x7fe6340b49f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe62000c390 tx=0x7fe62000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.422+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620017070 con 0x7fe6340b7ee0 2024-09-21T07:46:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.422+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6200053c0 con 0x7fe6340b7ee0 2024-09-21T07:46:57.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.422+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6340b5d60 con 0x7fe6340b7ee0 2024-09-21T07:46:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.422+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe620003850 con 0x7fe6340b7ee0 2024-09-21T07:46:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.422+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6340b6320 con 0x7fe6340b7ee0 2024-09-21T07:46:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.423+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6340a1f90 con 0x7fe6340b7ee0 2024-09-21T07:46:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.425+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fe62001a070 con 0x7fe6340b7ee0 2024-09-21T07:46:57.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.425+0000 7fe631600640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe614078b90 0x7fe61407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:57.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.426+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe620099fe0 con 0x7fe6340b7ee0 2024-09-21T07:46:57.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.426+0000 7fe639400640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe614078b90 0x7fe61407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.427+0000 7fe639400640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe614078b90 0x7fe61407b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe628034000 tx=0x7fe628004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:57.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.430+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe620061da0 con 0x7fe6340b7ee0 2024-09-21T07:46:57.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:57.657+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fe634161f90 con 0x7fe6340b7ee0 2024-09-21T07:46:58.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.219+0000 7fe631600640 1 -- 172.21.15.88:0/1402893736 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) ==== 118+0+0 (secure 0 0 0) 0x7fe6200667d0 con 0x7fe6340b7ee0 2024-09-21T07:46:58.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe614078b90 msgr2=0x7fe61407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe614078b90 0x7fe61407b0a0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fe628034000 tx=0x7fe628004730 comp rx=0 tx=0).stop 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 msgr2=0x7fe6340b49f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 0x7fe6340b49f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe62000c390 tx=0x7fe62000c850 comp rx=0 tx=0).stop 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 shutdown_connections 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe614078b90 0x7fe61407b0a0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6340d43d0 0x7fe6341ec610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6340b7ee0 0x7fe6340b49f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 --2- 172.21.15.88:0/1402893736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe63406de60 0x7fe63415afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 >> 172.21.15.88:0/1402893736 conn(0x7fe63409d900 msgr2=0x7fe63408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 shutdown_connections 2024-09-21T07:46:58.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.222+0000 7fe63a800640 1 -- 172.21.15.88:0/1402893736 wait complete. 2024-09-21T07:46:58.248 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:46:58.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.470+0000 7f09a9c00640 1 -- 172.21.15.88:0/4005041668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0988003a50 msgr2=0x7f0988003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.470+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4005041668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0988003a50 0x7f0988003ec0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f0994009b50 tx=0x7f0988005830 comp rx=0 tx=0).stop 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.470+0000 7f09a9c00640 1 -- 172.21.15.88:0/4005041668 shutdown_connections 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.470+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4005041668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0988003a50 0x7f0988003ec0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.470+0000 7f09a9c00640 1 -- 172.21.15.88:0/4005041668 >> 172.21.15.88:0/4005041668 conn(0x7f09a406d1c0 msgr2=0x7f09a406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.471+0000 7f09a9c00640 1 -- 172.21.15.88:0/4005041668 shutdown_connections 2024-09-21T07:46:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.471+0000 7f09a9c00640 1 -- 172.21.15.88:0/4005041668 wait complete. 2024-09-21T07:46:58.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.473+0000 7f09a9c00640 1 Processor -- start 2024-09-21T07:46:58.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.473+0000 7f09a9c00640 1 -- start start 2024-09-21T07:46:58.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0988003a50 0x7f09a415bec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a40b3e50 0x7f09a40b4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 0x7f09a41e8730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a9c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4073540 con 0x7f09a41e8320 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09a4070180 con 0x7f09a40b3e50 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f09a40736c0 con 0x7f0988003a50 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.474+0000 7f09a3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 0x7f09a41e8730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 0x7f09a41e8730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58408/0 (socket says 172.21.15.88:58408) 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 -- 172.21.15.88:0/4099182772 learned_addr learned my addr 172.21.15.88:0/4099182772 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a2a00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0988003a50 0x7f09a415bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a2000640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a40b3e50 0x7f09a40b4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 -- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0988003a50 msgr2=0x7f09a415bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0988003a50 0x7f09a415bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 -- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a40b3e50 msgr2=0x7f09a40b4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a40b3e50 0x7f09a40b4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 -- 172.21.15.88:0/4099182772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09a41e8c70 con 0x7f09a41e8320 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a2a00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0988003a50 0x7f09a415bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a2000640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a40b3e50 0x7f09a40b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f09a3400640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 0x7f09a41e8730 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f09a40730d0 tx=0x7f099c00eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.475+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00c910 con 0x7f09a41e8320 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.476+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f099c017070 con 0x7f09a41e8320 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.476+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09a41f5bd0 con 0x7f09a41e8320 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.476+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f099c00ce40 con 0x7f09a41e8320 2024-09-21T07:46:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.476+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09a41f6190 con 0x7f09a41e8320 2024-09-21T07:46:58.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.477+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09a4073c20 con 0x7f09a41e8320 2024-09-21T07:46:58.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.478+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f099c01c460 con 0x7f09a41e8320 2024-09-21T07:46:58.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.479+0000 7f099aa00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0984078b90 0x7f098407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:58.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.479+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f099c09b1a0 con 0x7f09a41e8320 2024-09-21T07:46:58.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.479+0000 7f09a2a00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0984078b90 0x7f098407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:58.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.480+0000 7f09a2a00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0984078b90 0x7f098407b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0994035000 tx=0x7f09940046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:58.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.483+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09a41e8c70 con 0x7f09a41e8320 2024-09-21T07:46:58.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:58.707+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f09a40b18d0 con 0x7f09a41e8320 2024-09-21T07:46:59.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.221+0000 7f099aa00640 1 -- 172.21.15.88:0/4099182772 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f099c0635e0 con 0x7f09a41e8320 2024-09-21T07:46:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0984078b90 msgr2=0x7f098407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0984078b90 0x7f098407b0a0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f0994035000 tx=0x7f09940046c0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 msgr2=0x7f09a41e8730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 0x7f09a41e8730 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f09a40730d0 tx=0x7f099c00eae0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 shutdown_connections 2024-09-21T07:46:59.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0984078b90 0x7f098407b0a0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09a41e8320 0x7f09a41e8730 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09a40b3e50 0x7f09a40b4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 --2- 172.21.15.88:0/4099182772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0988003a50 0x7f09a415bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 >> 172.21.15.88:0/4099182772 conn(0x7f09a406d1c0 msgr2=0x7f09a40b2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 shutdown_connections 2024-09-21T07:46:59.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.224+0000 7f09a9c00640 1 -- 172.21.15.88:0/4099182772 wait complete. 2024-09-21T07:46:59.248 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:46:59.249 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:46:59.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 -- 172.21.15.88:0/910415334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c0b2f60 msgr2=0x7f918c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 --2- 172.21.15.88:0/910415334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c0b2f60 0x7f918c15a0f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9174009830 tx=0x7f917402a480 comp rx=0 tx=0).stop 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 -- 172.21.15.88:0/910415334 shutdown_connections 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 --2- 172.21.15.88:0/910415334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c0b2f60 0x7f918c15a0f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 --2- 172.21.15.88:0/910415334 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f918c0ad9c0 0x7f918c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 -- 172.21.15.88:0/910415334 >> 172.21.15.88:0/910415334 conn(0x7f918c0b1e40 msgr2=0x7f918c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 -- 172.21.15.88:0/910415334 shutdown_connections 2024-09-21T07:46:59.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.467+0000 7f9191600640 1 -- 172.21.15.88:0/910415334 wait complete. 2024-09-21T07:46:59.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.469+0000 7f9191600640 1 Processor -- start 2024-09-21T07:46:59.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.469+0000 7f9191600640 1 -- start start 2024-09-21T07:46:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9191600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f918c0ad9c0 0x7f918c149aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9191600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f918c0b2f60 0x7f918c158650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9191600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 0x7f918c158b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9191600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f918c06f410 con 0x7f918c15d670 2024-09-21T07:46:59.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9191600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f918c06f290 con 0x7f918c0b2f60 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9191600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f918c06f590 con 0x7f918c0ad9c0 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 0x7f918c158b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 0x7f918c158b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58424/0 (socket says 172.21.15.88:58424) 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 -- 172.21.15.88:0/2576753343 learned_addr learned my addr 172.21.15.88:0/2576753343 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f918be00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f918c0ad9c0 0x7f918c149aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9183400640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f918c0b2f60 0x7f918c158650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 -- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f918c0ad9c0 msgr2=0x7f918c149aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f918c0ad9c0 0x7f918c149aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 -- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f918c0b2f60 msgr2=0x7f918c158650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f918c0b2f60 0x7f918c158650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.471+0000 7f9190c00640 1 -- 172.21.15.88:0/2576753343 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f918c1573f0 con 0x7f918c15d670 2024-09-21T07:46:59.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.472+0000 7f9190c00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 0x7f918c158b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f918400cae0 tx=0x7f918400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:59.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.472+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184016020 con 0x7f918c15d670 2024-09-21T07:46:59.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.472+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f918400ad30 con 0x7f918c15d670 2024-09-21T07:46:59.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.472+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f918c1576d0 con 0x7f918c15d670 2024-09-21T07:46:59.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.472+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9184012680 con 0x7f918c15d670 2024-09-21T07:46:59.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.472+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f918c157c90 con 0x7f918c15d670 2024-09-21T07:46:59.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.473+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f918c06fa60 con 0x7f918c15d670 2024-09-21T07:46:59.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.474+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9184019020 con 0x7f918c15d670 2024-09-21T07:46:59.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.475+0000 7f9188c00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9168078b90 0x7f916807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.475+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f918409b3c0 con 0x7f918c15d670 2024-09-21T07:46:59.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.476+0000 7f918be00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9168078b90 0x7f916807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.476+0000 7f918be00640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9168078b90 0x7f916807b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f917c00b810 tx=0x7f917c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:59.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.480+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9184020020 con 0x7f918c15d670 2024-09-21T07:46:59.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.700+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f918c0af260 con 0x7f918c15d670 2024-09-21T07:46:59.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.700+0000 7f9188c00640 1 -- 172.21.15.88:0/2576753343 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f91840637f0 con 0x7f918c15d670 2024-09-21T07:46:59.703 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:46:59.703 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":17,"btime":"2024-09-21T07:46:59:220696+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:55.682651+0000","modified":"2024-09-21T07:46:58.709003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1785638532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1785638532},{"type":"v1","addr":"172.21.15.88:6835","nonce":1785638532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4899,"qdb_cluster":[4899]},"id":2}]} 2024-09-21T07:46:59.703 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 17 2024-09-21T07:46:59.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9168078b90 msgr2=0x7f916807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9168078b90 0x7f916807b0a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f917c00b810 tx=0x7f917c008040 comp rx=0 tx=0).stop 2024-09-21T07:46:59.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 msgr2=0x7f918c158b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 0x7f918c158b90 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f918400cae0 tx=0x7f918400a060 comp rx=0 tx=0).stop 2024-09-21T07:46:59.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 shutdown_connections 2024-09-21T07:46:59.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9168078b90 0x7f916807b0a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f918c15d670 0x7f918c158b90 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f918c0b2f60 0x7f918c158650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 --2- 172.21.15.88:0/2576753343 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f918c0ad9c0 0x7f918c149aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 >> 172.21.15.88:0/2576753343 conn(0x7f918c0b1e40 msgr2=0x7f918c15c1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 shutdown_connections 2024-09-21T07:46:59.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.703+0000 7f9191600640 1 -- 172.21.15.88:0/2576753343 wait complete. 2024-09-21T07:46:59.726 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:46:59.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.945+0000 7fbac1800640 1 -- 172.21.15.88:0/3539292589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaa0003c60 msgr2=0x7fbaa00040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.945+0000 7fbac1800640 1 --2- 172.21.15.88:0/3539292589 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaa0003c60 0x7fbaa00040b0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbaac005fd0 tx=0x7fbaa0005830 comp rx=0 tx=0).stop 2024-09-21T07:46:59.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.945+0000 7fbabb400640 1 -- 172.21.15.88:0/3539292589 reap_dead start 2024-09-21T07:46:59.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.946+0000 7fbac1800640 1 -- 172.21.15.88:0/3539292589 shutdown_connections 2024-09-21T07:46:59.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.946+0000 7fbac1800640 1 -- 172.21.15.88:0/3539292589 >> 172.21.15.88:0/3539292589 conn(0x7fbabc06d730 msgr2=0x7fbabc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:46:59.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.946+0000 7fbac1800640 1 -- 172.21.15.88:0/3539292589 shutdown_connections 2024-09-21T07:46:59.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.946+0000 7fbac1800640 1 -- 172.21.15.88:0/3539292589 wait complete. 2024-09-21T07:46:59.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.948+0000 7fbac1800640 1 Processor -- start 2024-09-21T07:46:59.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.948+0000 7fbac1800640 1 -- start start 2024-09-21T07:46:59.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.949+0000 7fbac1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa0003c60 0x7fbabc14c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.949+0000 7fbac1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 0x7fbabc15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.949+0000 7fbac1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbabc15bd30 0x7fbabc073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.949+0000 7fbac1800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc0b41f0 con 0x7fbabc1f5400 2024-09-21T07:46:59.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.949+0000 7fbac1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbabc0b4070 con 0x7fbaa0003c60 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.949+0000 7fbac1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbabc0b4370 con 0x7fbabc15bd30 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 0x7fbabc15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 0x7fbabc15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58452/0 (socket says 172.21.15.88:58452) 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 -- 172.21.15.88:0/3693153217 learned_addr learned my addr 172.21.15.88:0/3693153217 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbabaa00640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa0003c60 0x7fbabc14c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbabb400640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbabc15bd30 0x7fbabc073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 -- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbabc15bd30 msgr2=0x7fbabc073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbabc15bd30 0x7fbabc073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 -- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa0003c60 msgr2=0x7fbabc14c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:46:59.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa0003c60 0x7fbabc14c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:46:59.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 -- 172.21.15.88:0/3693153217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbabc15c140 con 0x7fbabc1f5400 2024-09-21T07:46:59.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.950+0000 7fbaba000640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 0x7fbabc15b7f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa80039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:59.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.951+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8015070 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.951+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaa8011840 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.951+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaa8011c90 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.951+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbabc14a130 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.951+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbabc14e350 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.952+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbabc149ea0 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.953+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fbaa8018070 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.954+0000 7fbab2a00640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fba9c078b90 0x7fba9c07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.954+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fbaa809b940 con 0x7fbabc1f5400 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.954+0000 7fbabaa00640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fba9c078b90 0x7fba9c07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:46:59.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.955+0000 7fbabaa00640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fba9c078b90 0x7fba9c07b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbaac035000 tx=0x7fbaac004a40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:46:59.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:46:59.958+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbabc15c140 con 0x7fbabc1f5400 2024-09-21T07:47:00.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.142+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fbabc153780 con 0x7fbabc1f5400 2024-09-21T07:47:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.143+0000 7fbab2a00640 1 -- 172.21.15.88:0/3693153217 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7fbaa8063730 con 0x7fbabc1f5400 2024-09-21T07:47:00.146 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:00.146 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":35,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:57.207741+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:53.697978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:54.649546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fba9c078b90 msgr2=0x7fba9c07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fba9c078b90 0x7fba9c07b0a0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbaac035000 tx=0x7fbaac004a40 comp rx=0 tx=0).stop 2024-09-21T07:47:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 msgr2=0x7fbabc15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 0x7fbabc15b7f0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fbaa8006fd0 tx=0x7fbaa80039c0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 shutdown_connections 2024-09-21T07:47:00.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fba9c078b90 0x7fba9c07b0a0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbabc15bd30 0x7fbabc073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbabc1f5400 0x7fbabc15b7f0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 --2- 172.21.15.88:0/3693153217 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaa0003c60 0x7fbabc14c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 >> 172.21.15.88:0/3693153217 conn(0x7fbabc06d730 msgr2=0x7fbabc15eb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 shutdown_connections 2024-09-21T07:47:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.146+0000 7fbac1800640 1 -- 172.21.15.88:0/3693153217 wait complete. 2024-09-21T07:47:00.173 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:47:00.173 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:00.178 INFO:tasks.ceph.mds.c.smithi088.stderr:2024-09-21T07:47:00.176+0000 7f90f8000640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.391+0000 7f9ba5e00640 1 -- 172.21.15.88:0/1993786169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba015d760 msgr2=0x7f9ba01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.391+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/1993786169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba015d760 0x7f9ba01535c0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009830 tx=0x7f9b9002a480 comp rx=0 tx=0).stop 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.391+0000 7f9ba5e00640 1 -- 172.21.15.88:0/1993786169 shutdown_connections 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.391+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/1993786169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba015d760 0x7f9ba01535c0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.391+0000 7f9ba5e00640 1 -- 172.21.15.88:0/1993786169 >> 172.21.15.88:0/1993786169 conn(0x7f9ba006d730 msgr2=0x7f9ba00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.392+0000 7f9ba5e00640 1 -- 172.21.15.88:0/1993786169 shutdown_connections 2024-09-21T07:47:00.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.392+0000 7f9ba5e00640 1 -- 172.21.15.88:0/1993786169 wait complete. 2024-09-21T07:47:00.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.394+0000 7f9ba5e00640 1 Processor -- start 2024-09-21T07:47:00.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.394+0000 7f9ba5e00640 1 -- start start 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.395+0000 7f9ba5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 0x7f9ba009de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.395+0000 7f9ba5e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba00b3550 0x7f9ba014a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.395+0000 7f9ba5e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba015d760 0x7f9ba01f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.395+0000 7f9ba5e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba006f240 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.395+0000 7f9ba5e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba00b4e30 con 0x7f9ba015d760 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.395+0000 7f9ba5e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba00b4070 con 0x7f9ba00b3550 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 0x7f9ba009de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 0x7f9ba009de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58464/0 (socket says 172.21.15.88:58464) 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 -- 172.21.15.88:0/833830821 learned_addr learned my addr 172.21.15.88:0/833830821 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9e000640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba00b3550 0x7f9ba014a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9f400640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba015d760 0x7f9ba01f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 -- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba00b3550 msgr2=0x7f9ba014a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba00b3550 0x7f9ba014a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 -- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba015d760 msgr2=0x7f9ba01f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba015d760 0x7f9ba01f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 -- 172.21.15.88:0/833830821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba0151bd0 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.396+0000 7f9b9ea00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 0x7f9ba009de00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009950 tx=0x7f9b90033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.397+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b90031d60 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.397+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9b90033070 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.397+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b900334c0 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.397+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba0156b30 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.397+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba01f5920 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.398+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba0070880 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.399+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f9b90033660 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.400+0000 7f9b97400640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9b80078b90 0x7f9b8007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.400+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f9b900bc320 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.401+0000 7f9b9e000640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9b80078b90 0x7f9b8007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.401+0000 7f9b9e000640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9b80078b90 0x7f9b8007b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00b2080 tx=0x7f9b8c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.405+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ba0070880 con 0x7f9ba00ad0e0 2024-09-21T07:47:00.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.590+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9ba01f5710 con 0x7f9b80078b90 2024-09-21T07:47:00.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.596+0000 7f9b97400640 1 -- 172.21.15.88:0/833830821 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249152 (secure 0 0 0) 0x7f9ba01f5710 con 0x7f9b80078b90 2024-09-21T07:47:00.601 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:00.603 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9b80078b90 msgr2=0x7f9b8007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9b80078b90 0x7f9b8007b0a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9ba00b2080 tx=0x7f9b8c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 msgr2=0x7f9ba009de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 0x7f9ba009de00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9b90009950 tx=0x7f9b90033d60 comp rx=0 tx=0).stop 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 shutdown_connections 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9b80078b90 0x7f9b8007b0a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba00b3550 0x7f9ba014a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9ba015d760 0x7f9ba01f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 --2- 172.21.15.88:0/833830821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba00ad0e0 0x7f9ba009de00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 >> 172.21.15.88:0/833830821 conn(0x7f9ba006d730 msgr2=0x7f9ba0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 shutdown_connections 2024-09-21T07:47:00.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.600+0000 7f9ba5e00640 1 -- 172.21.15.88:0/833830821 wait complete. 2024-09-21T07:47:00.627 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:46:59.866867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220348,"kb_used_data":4096,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754372,"statfs":{"total":773094113280,"available":772868476928,"internally_reserved":0,"allocated":4194304,"data_stored":2319819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.679739"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196875+0000","last_change":"2024-09-21T07:46:55.196875+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196875+0000","last_undegraded":"2024-09-21T07:46:55.196875+0000","last_fullsized":"2024-09-21T07:46:55.196875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228903+0000","last_change":"2024-09-21T07:46:56.229990+0000","last_active":"2024-09-21T07:46:57.228903+0000","last_peered":"2024-09-21T07:46:57.228903+0000","last_clean":"2024-09-21T07:46:57.228903+0000","last_became_active":"2024-09-21T07:46:56.229655+0000","last_became_peered":"2024-09-21T07:46:56.229655+0000","last_unstale":"2024-09-21T07:46:57.228903+0000","last_undegraded":"2024-09-21T07:46:57.228903+0000","last_fullsized":"2024-09-21T07:46:57.228903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.209740+0000","last_change":"2024-09-21T07:46:55.209740+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.209740+0000","last_undegraded":"2024-09-21T07:46:55.209740+0000","last_fullsized":"2024-09-21T07:46:55.209740+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219416+0000","last_change":"2024-09-21T07:46:56.235757+0000","last_active":"2024-09-21T07:46:57.219416+0000","last_peered":"2024-09-21T07:46:57.219416+0000","last_clean":"2024-09-21T07:46:57.219416+0000","last_became_active":"2024-09-21T07:46:56.228680+0000","last_became_peered":"2024-09-21T07:46:56.228680+0000","last_unstale":"2024-09-21T07:46:57.219416+0000","last_undegraded":"2024-09-21T07:46:57.219416+0000","last_fullsized":"2024-09-21T07:46:57.219416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228641+0000","last_change":"2024-09-21T07:46:56.229963+0000","last_active":"2024-09-21T07:46:57.228641+0000","last_peered":"2024-09-21T07:46:57.228641+0000","last_clean":"2024-09-21T07:46:57.228641+0000","last_became_active":"2024-09-21T07:46:56.229617+0000","last_became_peered":"2024-09-21T07:46:56.229617+0000","last_unstale":"2024-09-21T07:46:57.228641+0000","last_undegraded":"2024-09-21T07:46:57.228641+0000","last_fullsized":"2024-09-21T07:46:57.228641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219492+0000","last_change":"2024-09-21T07:46:56.229684+0000","last_active":"2024-09-21T07:46:57.219492+0000","last_peered":"2024-09-21T07:46:57.219492+0000","last_clean":"2024-09-21T07:46:57.219492+0000","last_became_active":"2024-09-21T07:46:56.229339+0000","last_became_peered":"2024-09-21T07:46:56.229339+0000","last_unstale":"2024-09-21T07:46:57.219492+0000","last_undegraded":"2024-09-21T07:46:57.219492+0000","last_fullsized":"2024-09-21T07:46:57.219492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:21.621424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198590+0000","last_change":"2024-09-21T07:46:55.198590+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198590+0000","last_undegraded":"2024-09-21T07:46:55.198590+0000","last_fullsized":"2024-09-21T07:46:55.198590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195067+0000","last_change":"2024-09-21T07:46:55.195067+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195067+0000","last_undegraded":"2024-09-21T07:46:55.195067+0000","last_fullsized":"2024-09-21T07:46:55.195067+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.210023+0000","last_change":"2024-09-21T07:46:55.210023+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.210023+0000","last_undegraded":"2024-09-21T07:46:55.210023+0000","last_fullsized":"2024-09-21T07:46:55.210023+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220083+0000","last_change":"2024-09-21T07:46:56.231062+0000","last_active":"2024-09-21T07:46:57.220083+0000","last_peered":"2024-09-21T07:46:57.220083+0000","last_clean":"2024-09-21T07:46:57.220083+0000","last_became_active":"2024-09-21T07:46:56.229427+0000","last_became_peered":"2024-09-21T07:46:56.229427+0000","last_unstale":"2024-09-21T07:46:57.220083+0000","last_undegraded":"2024-09-21T07:46:57.220083+0000","last_fullsized":"2024-09-21T07:46:57.220083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:47.565734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198313+0000","last_change":"2024-09-21T07:46:55.198313+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198313+0000","last_undegraded":"2024-09-21T07:46:55.198313+0000","last_fullsized":"2024-09-21T07:46:55.198313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.199137+0000","last_change":"2024-09-21T07:46:55.199137+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.199137+0000","last_undegraded":"2024-09-21T07:46:55.199137+0000","last_fullsized":"2024-09-21T07:46:55.199137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707947+0000","last_change":"2024-09-21T07:46:56.237624+0000","last_active":"2024-09-21T07:46:57.707947+0000","last_peered":"2024-09-21T07:46:57.707947+0000","last_clean":"2024-09-21T07:46:57.707947+0000","last_became_active":"2024-09-21T07:46:56.237037+0000","last_became_peered":"2024-09-21T07:46:56.237037+0000","last_unstale":"2024-09-21T07:46:57.707947+0000","last_undegraded":"2024-09-21T07:46:57.707947+0000","last_fullsized":"2024-09-21T07:46:57.707947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:43.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219004+0000","last_change":"2024-09-21T07:46:56.239271+0000","last_active":"2024-09-21T07:46:57.219004+0000","last_peered":"2024-09-21T07:46:57.219004+0000","last_clean":"2024-09-21T07:46:57.219004+0000","last_became_active":"2024-09-21T07:46:56.238936+0000","last_became_peered":"2024-09-21T07:46:56.238936+0000","last_unstale":"2024-09-21T07:46:57.219004+0000","last_undegraded":"2024-09-21T07:46:57.219004+0000","last_fullsized":"2024-09-21T07:46:57.219004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194649+0000","last_change":"2024-09-21T07:46:55.194649+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194649+0000","last_undegraded":"2024-09-21T07:46:55.194649+0000","last_fullsized":"2024-09-21T07:46:55.194649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194072+0000","last_change":"2024-09-21T07:46:55.194072+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194072+0000","last_undegraded":"2024-09-21T07:46:55.194072+0000","last_fullsized":"2024-09-21T07:46:55.194072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198999+0000","last_change":"2024-09-21T07:46:55.198999+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198999+0000","last_undegraded":"2024-09-21T07:46:55.198999+0000","last_fullsized":"2024-09-21T07:46:55.198999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220865+0000","last_change":"2024-09-21T07:46:56.224534+0000","last_active":"2024-09-21T07:46:57.220865+0000","last_peered":"2024-09-21T07:46:57.220865+0000","last_clean":"2024-09-21T07:46:57.220865+0000","last_became_active":"2024-09-21T07:46:56.224167+0000","last_became_peered":"2024-09-21T07:46:56.224167+0000","last_unstale":"2024-09-21T07:46:57.220865+0000","last_undegraded":"2024-09-21T07:46:57.220865+0000","last_fullsized":"2024-09-21T07:46:57.220865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:18.755342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220883+0000","last_change":"2024-09-21T07:46:56.228375+0000","last_active":"2024-09-21T07:46:57.220883+0000","last_peered":"2024-09-21T07:46:57.220883+0000","last_clean":"2024-09-21T07:46:57.220883+0000","last_became_active":"2024-09-21T07:46:56.225793+0000","last_became_peered":"2024-09-21T07:46:56.225793+0000","last_unstale":"2024-09-21T07:46:57.220883+0000","last_undegraded":"2024-09-21T07:46:57.220883+0000","last_fullsized":"2024-09-21T07:46:57.220883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:46.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708077+0000","last_change":"2024-09-21T07:46:56.226928+0000","last_active":"2024-09-21T07:46:57.708077+0000","last_peered":"2024-09-21T07:46:57.708077+0000","last_clean":"2024-09-21T07:46:57.708077+0000","last_became_active":"2024-09-21T07:46:56.226560+0000","last_became_peered":"2024-09-21T07:46:56.226560+0000","last_unstale":"2024-09-21T07:46:57.708077+0000","last_undegraded":"2024-09-21T07:46:57.708077+0000","last_fullsized":"2024-09-21T07:46:57.708077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.197713+0000","last_change":"2024-09-21T07:46:55.197713+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.197713+0000","last_undegraded":"2024-09-21T07:46:55.197713+0000","last_fullsized":"2024-09-21T07:46:55.197713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195512+0000","last_change":"2024-09-21T07:46:55.195512+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195512+0000","last_undegraded":"2024-09-21T07:46:55.195512+0000","last_fullsized":"2024-09-21T07:46:55.195512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707640+0000","last_change":"2024-09-21T07:46:56.227950+0000","last_active":"2024-09-21T07:46:57.707640+0000","last_peered":"2024-09-21T07:46:57.707640+0000","last_clean":"2024-09-21T07:46:57.707640+0000","last_became_active":"2024-09-21T07:46:56.227647+0000","last_became_peered":"2024-09-21T07:46:56.227647+0000","last_unstale":"2024-09-21T07:46:57.707640+0000","last_undegraded":"2024-09-21T07:46:57.707640+0000","last_fullsized":"2024-09-21T07:46:57.707640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:08.480356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194574+0000","last_change":"2024-09-21T07:46:55.194574+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194574+0000","last_undegraded":"2024-09-21T07:46:55.194574+0000","last_fullsized":"2024-09-21T07:46:55.194574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219395+0000","last_change":"2024-09-21T07:46:56.217732+0000","last_active":"2024-09-21T07:46:57.219395+0000","last_peered":"2024-09-21T07:46:57.219395+0000","last_clean":"2024-09-21T07:46:57.219395+0000","last_became_active":"2024-09-21T07:46:56.217410+0000","last_became_peered":"2024-09-21T07:46:56.217410+0000","last_unstale":"2024-09-21T07:46:57.219395+0000","last_undegraded":"2024-09-21T07:46:57.219395+0000","last_fullsized":"2024-09-21T07:46:57.219395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219810+0000","last_change":"2024-09-21T07:46:56.220112+0000","last_active":"2024-09-21T07:46:57.219810+0000","last_peered":"2024-09-21T07:46:57.219810+0000","last_clean":"2024-09-21T07:46:57.219810+0000","last_became_active":"2024-09-21T07:46:56.217329+0000","last_became_peered":"2024-09-21T07:46:56.217329+0000","last_unstale":"2024-09-21T07:46:57.219810+0000","last_undegraded":"2024-09-21T07:46:57.219810+0000","last_fullsized":"2024-09-21T07:46:57.219810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:04.144314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228789+0000","last_change":"2024-09-21T07:46:56.223494+0000","last_active":"2024-09-21T07:46:57.228789+0000","last_peered":"2024-09-21T07:46:57.228789+0000","last_clean":"2024-09-21T07:46:57.228789+0000","last_became_active":"2024-09-21T07:46:56.223149+0000","last_became_peered":"2024-09-21T07:46:56.223149+0000","last_unstale":"2024-09-21T07:46:57.228789+0000","last_undegraded":"2024-09-21T07:46:57.228789+0000","last_fullsized":"2024-09-21T07:46:57.228789+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198069+0000","last_change":"2024-09-21T07:46:55.198069+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198069+0000","last_undegraded":"2024-09-21T07:46:55.198069+0000","last_fullsized":"2024-09-21T07:46:55.198069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.200475+0000","last_change":"2024-09-21T07:46:55.200475+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.200475+0000","last_undegraded":"2024-09-21T07:46:55.200475+0000","last_fullsized":"2024-09-21T07:46:55.200475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196887+0000","last_change":"2024-09-21T07:46:55.196887+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196887+0000","last_undegraded":"2024-09-21T07:46:55.196887+0000","last_fullsized":"2024-09-21T07:46:55.196887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.192541+0000","last_change":"2024-09-21T07:46:55.192541+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.192541+0000","last_undegraded":"2024-09-21T07:46:55.192541+0000","last_fullsized":"2024-09-21T07:46:55.192541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198692+0000","last_change":"2024-09-21T07:46:55.198692+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198692+0000","last_undegraded":"2024-09-21T07:46:55.198692+0000","last_fullsized":"2024-09-21T07:46:55.198692+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708018+0000","last_change":"2024-09-21T07:46:56.231049+0000","last_active":"2024-09-21T07:46:57.708018+0000","last_peered":"2024-09-21T07:46:57.708018+0000","last_clean":"2024-09-21T07:46:57.708018+0000","last_became_active":"2024-09-21T07:46:56.230778+0000","last_became_peered":"2024-09-21T07:46:56.230778+0000","last_unstale":"2024-09-21T07:46:57.708018+0000","last_undegraded":"2024-09-21T07:46:57.708018+0000","last_fullsized":"2024-09-21T07:46:57.708018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.211797+0000","last_change":"2024-09-21T07:46:55.204066+0000","last_active":"2024-09-21T07:46:55.211797+0000","last_peered":"2024-09-21T07:46:55.211797+0000","last_clean":"2024-09-21T07:46:55.211797+0000","last_became_active":"2024-09-21T07:46:55.203753+0000","last_became_peered":"2024-09-21T07:46:55.203753+0000","last_unstale":"2024-09-21T07:46:55.211797+0000","last_undegraded":"2024-09-21T07:46:55.211797+0000","last_fullsized":"2024-09-21T07:46:55.211797+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:20.693634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196356+0000","last_change":"2024-09-21T07:46:55.196356+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196356+0000","last_undegraded":"2024-09-21T07:46:55.196356+0000","last_fullsized":"2024-09-21T07:46:55.196356+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219963+0000","last_change":"2024-09-21T07:46:55.212364+0000","last_active":"2024-09-21T07:46:57.219963+0000","last_peered":"2024-09-21T07:46:57.219963+0000","last_clean":"2024-09-21T07:46:57.219963+0000","last_became_active":"2024-09-21T07:46:55.211980+0000","last_became_peered":"2024-09-21T07:46:55.211980+0000","last_unstale":"2024-09-21T07:46:57.219963+0000","last_undegraded":"2024-09-21T07:46:57.219963+0000","last_fullsized":"2024-09-21T07:46:57.219963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:40.287704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.197598+0000","last_change":"2024-09-21T07:46:55.197598+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.197598+0000","last_undegraded":"2024-09-21T07:46:55.197598+0000","last_fullsized":"2024-09-21T07:46:55.197598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221045+0000","last_change":"2024-09-21T07:46:55.216737+0000","last_active":"2024-09-21T07:46:57.221045+0000","last_peered":"2024-09-21T07:46:57.221045+0000","last_clean":"2024-09-21T07:46:57.221045+0000","last_became_active":"2024-09-21T07:46:55.216402+0000","last_became_peered":"2024-09-21T07:46:55.216402+0000","last_unstale":"2024-09-21T07:46:57.221045+0000","last_undegraded":"2024-09-21T07:46:57.221045+0000","last_fullsized":"2024-09-21T07:46:57.221045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:40.966275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219312+0000","last_change":"2024-09-21T07:46:56.237682+0000","last_active":"2024-09-21T07:46:57.219312+0000","last_peered":"2024-09-21T07:46:57.219312+0000","last_clean":"2024-09-21T07:46:57.219312+0000","last_became_active":"2024-09-21T07:46:56.237340+0000","last_became_peered":"2024-09-21T07:46:56.237340+0000","last_unstale":"2024-09-21T07:46:57.219312+0000","last_undegraded":"2024-09-21T07:46:57.219312+0000","last_fullsized":"2024-09-21T07:46:57.219312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219765+0000","last_change":"2024-09-21T07:46:55.211939+0000","last_active":"2024-09-21T07:46:57.219765+0000","last_peered":"2024-09-21T07:46:57.219765+0000","last_clean":"2024-09-21T07:46:57.219765+0000","last_became_active":"2024-09-21T07:46:55.211170+0000","last_became_peered":"2024-09-21T07:46:55.211170+0000","last_unstale":"2024-09-21T07:46:57.219765+0000","last_undegraded":"2024-09-21T07:46:57.219765+0000","last_fullsized":"2024-09-21T07:46:57.219765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194935+0000","last_change":"2024-09-21T07:46:55.194935+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194935+0000","last_undegraded":"2024-09-21T07:46:55.194935+0000","last_fullsized":"2024-09-21T07:46:55.194935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.210107+0000","last_change":"2024-09-21T07:46:55.204599+0000","last_active":"2024-09-21T07:46:55.210107+0000","last_peered":"2024-09-21T07:46:55.210107+0000","last_clean":"2024-09-21T07:46:55.210107+0000","last_became_active":"2024-09-21T07:46:55.204186+0000","last_became_peered":"2024-09-21T07:46:55.204186+0000","last_unstale":"2024-09-21T07:46:55.210107+0000","last_undegraded":"2024-09-21T07:46:55.210107+0000","last_fullsized":"2024-09-21T07:46:55.210107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:36.808640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193937+0000","last_change":"2024-09-21T07:46:55.193937+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193937+0000","last_undegraded":"2024-09-21T07:46:55.193937+0000","last_fullsized":"2024-09-21T07:46:55.193937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.210087+0000","last_change":"2024-09-21T07:46:55.205089+0000","last_active":"2024-09-21T07:46:55.210087+0000","last_peered":"2024-09-21T07:46:55.210087+0000","last_clean":"2024-09-21T07:46:55.210087+0000","last_became_active":"2024-09-21T07:46:55.204642+0000","last_became_peered":"2024-09-21T07:46:55.204642+0000","last_unstale":"2024-09-21T07:46:55.210087+0000","last_undegraded":"2024-09-21T07:46:55.210087+0000","last_fullsized":"2024-09-21T07:46:55.210087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:12.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193798+0000","last_change":"2024-09-21T07:46:55.193798+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193798+0000","last_undegraded":"2024-09-21T07:46:55.193798+0000","last_fullsized":"2024-09-21T07:46:55.193798+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218717+0000","last_change":"2024-09-21T07:46:55.218472+0000","last_active":"2024-09-21T07:46:57.218717+0000","last_peered":"2024-09-21T07:46:57.218717+0000","last_clean":"2024-09-21T07:46:57.218717+0000","last_became_active":"2024-09-21T07:46:55.218044+0000","last_became_peered":"2024-09-21T07:46:55.218044+0000","last_unstale":"2024-09-21T07:46:57.218717+0000","last_undegraded":"2024-09-21T07:46:57.218717+0000","last_fullsized":"2024-09-21T07:46:57.218717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:18.860660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218669+0000","last_change":"2024-09-21T07:46:56.221089+0000","last_active":"2024-09-21T07:46:57.218669+0000","last_peered":"2024-09-21T07:46:57.218669+0000","last_clean":"2024-09-21T07:46:57.218669+0000","last_became_active":"2024-09-21T07:46:56.220746+0000","last_became_peered":"2024-09-21T07:46:56.220746+0000","last_unstale":"2024-09-21T07:46:57.218669+0000","last_undegraded":"2024-09-21T07:46:57.218669+0000","last_fullsized":"2024-09-21T07:46:57.218669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.213270+0000","last_change":"2024-09-21T07:46:55.206317+0000","last_active":"2024-09-21T07:46:55.213270+0000","last_peered":"2024-09-21T07:46:55.213270+0000","last_clean":"2024-09-21T07:46:55.213270+0000","last_became_active":"2024-09-21T07:46:55.206000+0000","last_became_peered":"2024-09-21T07:46:55.206000+0000","last_unstale":"2024-09-21T07:46:55.213270+0000","last_undegraded":"2024-09-21T07:46:55.213270+0000","last_fullsized":"2024-09-21T07:46:55.213270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:35.469725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198239+0000","last_change":"2024-09-21T07:46:55.198239+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198239+0000","last_undegraded":"2024-09-21T07:46:55.198239+0000","last_fullsized":"2024-09-21T07:46:55.198239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228976+0000","last_change":"2024-09-21T07:46:55.213497+0000","last_active":"2024-09-21T07:46:57.228976+0000","last_peered":"2024-09-21T07:46:57.228976+0000","last_clean":"2024-09-21T07:46:57.228976+0000","last_became_active":"2024-09-21T07:46:55.213173+0000","last_became_peered":"2024-09-21T07:46:55.213173+0000","last_unstale":"2024-09-21T07:46:57.228976+0000","last_undegraded":"2024-09-21T07:46:57.228976+0000","last_fullsized":"2024-09-21T07:46:57.228976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.908077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219686+0000","last_change":"2024-09-21T07:46:56.229383+0000","last_active":"2024-09-21T07:46:57.219686+0000","last_peered":"2024-09-21T07:46:57.219686+0000","last_clean":"2024-09-21T07:46:57.219686+0000","last_became_active":"2024-09-21T07:46:56.229012+0000","last_became_peered":"2024-09-21T07:46:56.229012+0000","last_unstale":"2024-09-21T07:46:57.219686+0000","last_undegraded":"2024-09-21T07:46:57.219686+0000","last_fullsized":"2024-09-21T07:46:57.219686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:56.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228698+0000","last_change":"2024-09-21T07:46:55.212681+0000","last_active":"2024-09-21T07:46:57.228698+0000","last_peered":"2024-09-21T07:46:57.228698+0000","last_clean":"2024-09-21T07:46:57.228698+0000","last_became_active":"2024-09-21T07:46:55.211907+0000","last_became_peered":"2024-09-21T07:46:55.211907+0000","last_unstale":"2024-09-21T07:46:57.228698+0000","last_undegraded":"2024-09-21T07:46:57.228698+0000","last_fullsized":"2024-09-21T07:46:57.228698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:30.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219680+0000","last_change":"2024-09-21T07:46:56.224792+0000","last_active":"2024-09-21T07:46:57.219680+0000","last_peered":"2024-09-21T07:46:57.219680+0000","last_clean":"2024-09-21T07:46:57.219680+0000","last_became_active":"2024-09-21T07:46:56.224554+0000","last_became_peered":"2024-09-21T07:46:56.224554+0000","last_unstale":"2024-09-21T07:46:57.219680+0000","last_undegraded":"2024-09-21T07:46:57.219680+0000","last_fullsized":"2024-09-21T07:46:57.219680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:31.069660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.214576+0000","last_change":"2024-09-21T07:46:55.207287+0000","last_active":"2024-09-21T07:46:55.214576+0000","last_peered":"2024-09-21T07:46:55.214576+0000","last_clean":"2024-09-21T07:46:55.214576+0000","last_became_active":"2024-09-21T07:46:55.207008+0000","last_became_peered":"2024-09-21T07:46:55.207008+0000","last_unstale":"2024-09-21T07:46:55.214576+0000","last_undegraded":"2024-09-21T07:46:55.214576+0000","last_fullsized":"2024-09-21T07:46:55.214576+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:31.482910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196425+0000","last_change":"2024-09-21T07:46:55.196425+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196425+0000","last_undegraded":"2024-09-21T07:46:55.196425+0000","last_fullsized":"2024-09-21T07:46:55.196425+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707883+0000","last_change":"2024-09-21T07:46:55.209727+0000","last_active":"2024-09-21T07:46:57.707883+0000","last_peered":"2024-09-21T07:46:57.707883+0000","last_clean":"2024-09-21T07:46:57.707883+0000","last_became_active":"2024-09-21T07:46:55.209437+0000","last_became_peered":"2024-09-21T07:46:55.209437+0000","last_unstale":"2024-09-21T07:46:57.707883+0000","last_undegraded":"2024-09-21T07:46:57.707883+0000","last_fullsized":"2024-09-21T07:46:57.707883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707613+0000","last_change":"2024-09-21T07:46:56.234403+0000","last_active":"2024-09-21T07:46:57.707613+0000","last_peered":"2024-09-21T07:46:57.707613+0000","last_clean":"2024-09-21T07:46:57.707613+0000","last_became_active":"2024-09-21T07:46:56.233998+0000","last_became_peered":"2024-09-21T07:46:56.233998+0000","last_unstale":"2024-09-21T07:46:57.707613+0000","last_undegraded":"2024-09-21T07:46:57.707613+0000","last_fullsized":"2024-09-21T07:46:57.707613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:52.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217390+0000","last_change":"2024-09-21T07:46:55.212435+0000","last_active":"2024-09-21T07:46:55.217390+0000","last_peered":"2024-09-21T07:46:55.217390+0000","last_clean":"2024-09-21T07:46:55.217390+0000","last_became_active":"2024-09-21T07:46:55.212093+0000","last_became_peered":"2024-09-21T07:46:55.212093+0000","last_unstale":"2024-09-21T07:46:55.217390+0000","last_undegraded":"2024-09-21T07:46:55.217390+0000","last_fullsized":"2024-09-21T07:46:55.217390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:44.281725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219594+0000","last_change":"2024-09-21T07:46:56.236480+0000","last_active":"2024-09-21T07:46:57.219594+0000","last_peered":"2024-09-21T07:46:57.219594+0000","last_clean":"2024-09-21T07:46:57.219594+0000","last_became_active":"2024-09-21T07:46:56.236111+0000","last_became_peered":"2024-09-21T07:46:56.236111+0000","last_unstale":"2024-09-21T07:46:57.219594+0000","last_undegraded":"2024-09-21T07:46:57.219594+0000","last_fullsized":"2024-09-21T07:46:57.219594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219132+0000","last_change":"2024-09-21T07:46:55.211160+0000","last_active":"2024-09-21T07:46:57.219132+0000","last_peered":"2024-09-21T07:46:57.219132+0000","last_clean":"2024-09-21T07:46:57.219132+0000","last_became_active":"2024-09-21T07:46:55.210797+0000","last_became_peered":"2024-09-21T07:46:55.210797+0000","last_unstale":"2024-09-21T07:46:57.219132+0000","last_undegraded":"2024-09-21T07:46:57.219132+0000","last_fullsized":"2024-09-21T07:46:57.219132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:33.882042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.192745+0000","last_change":"2024-09-21T07:46:55.192745+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.192745+0000","last_undegraded":"2024-09-21T07:46:55.192745+0000","last_fullsized":"2024-09-21T07:46:55.192745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218445+0000","last_change":"2024-09-21T07:46:55.205503+0000","last_active":"2024-09-21T07:46:57.218445+0000","last_peered":"2024-09-21T07:46:57.218445+0000","last_clean":"2024-09-21T07:46:57.218445+0000","last_became_active":"2024-09-21T07:46:55.204821+0000","last_became_peered":"2024-09-21T07:46:55.204821+0000","last_unstale":"2024-09-21T07:46:57.218445+0000","last_undegraded":"2024-09-21T07:46:57.218445+0000","last_fullsized":"2024-09-21T07:46:57.218445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195717+0000","last_change":"2024-09-21T07:46:55.195717+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195717+0000","last_undegraded":"2024-09-21T07:46:55.195717+0000","last_fullsized":"2024-09-21T07:46:55.195717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218410+0000","last_change":"2024-09-21T07:46:55.216736+0000","last_active":"2024-09-21T07:46:57.218410+0000","last_peered":"2024-09-21T07:46:57.218410+0000","last_clean":"2024-09-21T07:46:57.218410+0000","last_became_active":"2024-09-21T07:46:55.216312+0000","last_became_peered":"2024-09-21T07:46:55.216312+0000","last_unstale":"2024-09-21T07:46:57.218410+0000","last_undegraded":"2024-09-21T07:46:57.218410+0000","last_fullsized":"2024-09-21T07:46:57.218410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.195061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193288+0000","last_change":"2024-09-21T07:46:55.193288+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193288+0000","last_undegraded":"2024-09-21T07:46:55.193288+0000","last_fullsized":"2024-09-21T07:46:55.193288+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220824+0000","last_change":"2024-09-21T07:46:55.218143+0000","last_active":"2024-09-21T07:46:57.220824+0000","last_peered":"2024-09-21T07:46:57.220824+0000","last_clean":"2024-09-21T07:46:57.220824+0000","last_became_active":"2024-09-21T07:46:55.217800+0000","last_became_peered":"2024-09-21T07:46:55.217800+0000","last_unstale":"2024-09-21T07:46:57.220824+0000","last_undegraded":"2024-09-21T07:46:57.220824+0000","last_fullsized":"2024-09-21T07:46:57.220824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:57.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.205097+0000","last_change":"2024-09-21T07:46:55.205097+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.205097+0000","last_undegraded":"2024-09-21T07:46:55.205097+0000","last_fullsized":"2024-09-21T07:46:55.205097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217443+0000","last_change":"2024-09-21T07:46:55.212733+0000","last_active":"2024-09-21T07:46:55.217443+0000","last_peered":"2024-09-21T07:46:55.217443+0000","last_clean":"2024-09-21T07:46:55.217443+0000","last_became_active":"2024-09-21T07:46:55.212478+0000","last_became_peered":"2024-09-21T07:46:55.212478+0000","last_unstale":"2024-09-21T07:46:55.217443+0000","last_undegraded":"2024-09-21T07:46:55.217443+0000","last_fullsized":"2024-09-21T07:46:55.217443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.222888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228788+0000","last_change":"2024-09-21T07:46:56.223493+0000","last_active":"2024-09-21T07:46:57.228788+0000","last_peered":"2024-09-21T07:46:57.228788+0000","last_clean":"2024-09-21T07:46:57.228788+0000","last_became_active":"2024-09-21T07:46:56.223060+0000","last_became_peered":"2024-09-21T07:46:56.223060+0000","last_unstale":"2024-09-21T07:46:57.228788+0000","last_undegraded":"2024-09-21T07:46:57.228788+0000","last_fullsized":"2024-09-21T07:46:57.228788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217612+0000","last_change":"2024-09-21T07:46:55.213737+0000","last_active":"2024-09-21T07:46:55.217612+0000","last_peered":"2024-09-21T07:46:55.217612+0000","last_clean":"2024-09-21T07:46:55.217612+0000","last_became_active":"2024-09-21T07:46:55.213452+0000","last_became_peered":"2024-09-21T07:46:55.213452+0000","last_unstale":"2024-09-21T07:46:55.217612+0000","last_undegraded":"2024-09-21T07:46:55.217612+0000","last_fullsized":"2024-09-21T07:46:55.217612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:10.689164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220927+0000","last_change":"2024-09-21T07:46:56.232119+0000","last_active":"2024-09-21T07:46:57.220927+0000","last_peered":"2024-09-21T07:46:57.220927+0000","last_clean":"2024-09-21T07:46:57.220927+0000","last_became_active":"2024-09-21T07:46:56.231764+0000","last_became_peered":"2024-09-21T07:46:56.231764+0000","last_unstale":"2024-09-21T07:46:57.220927+0000","last_undegraded":"2024-09-21T07:46:57.220927+0000","last_fullsized":"2024-09-21T07:46:57.220927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:35.746093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.208347+0000","last_change":"2024-09-21T07:46:55.206900+0000","last_active":"2024-09-21T07:46:55.208347+0000","last_peered":"2024-09-21T07:46:55.208347+0000","last_clean":"2024-09-21T07:46:55.208347+0000","last_became_active":"2024-09-21T07:46:55.206441+0000","last_became_peered":"2024-09-21T07:46:55.206441+0000","last_unstale":"2024-09-21T07:46:55.208347+0000","last_undegraded":"2024-09-21T07:46:55.208347+0000","last_fullsized":"2024-09-21T07:46:55.208347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.920219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707559+0000","last_change":"2024-09-21T07:46:56.233293+0000","last_active":"2024-09-21T07:46:57.707559+0000","last_peered":"2024-09-21T07:46:57.707559+0000","last_clean":"2024-09-21T07:46:57.707559+0000","last_became_active":"2024-09-21T07:46:56.232951+0000","last_became_peered":"2024-09-21T07:46:56.232951+0000","last_unstale":"2024-09-21T07:46:57.707559+0000","last_undegraded":"2024-09-21T07:46:57.707559+0000","last_fullsized":"2024-09-21T07:46:57.707559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:58.570223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228732+0000","last_change":"2024-09-21T07:46:55.206362+0000","last_active":"2024-09-21T07:46:57.228732+0000","last_peered":"2024-09-21T07:46:57.228732+0000","last_clean":"2024-09-21T07:46:57.228732+0000","last_became_active":"2024-09-21T07:46:55.206044+0000","last_became_peered":"2024-09-21T07:46:55.206044+0000","last_unstale":"2024-09-21T07:46:57.228732+0000","last_undegraded":"2024-09-21T07:46:57.228732+0000","last_fullsized":"2024-09-21T07:46:57.228732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:20.127719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228770+0000","last_change":"2024-09-21T07:46:56.221448+0000","last_active":"2024-09-21T07:46:57.228770+0000","last_peered":"2024-09-21T07:46:57.228770+0000","last_clean":"2024-09-21T07:46:57.228770+0000","last_became_active":"2024-09-21T07:46:56.221116+0000","last_became_peered":"2024-09-21T07:46:56.221116+0000","last_unstale":"2024-09-21T07:46:57.228770+0000","last_undegraded":"2024-09-21T07:46:57.228770+0000","last_fullsized":"2024-09-21T07:46:57.228770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217541+0000","last_change":"2024-09-21T07:46:55.213426+0000","last_active":"2024-09-21T07:46:55.217541+0000","last_peered":"2024-09-21T07:46:55.217541+0000","last_clean":"2024-09-21T07:46:55.217541+0000","last_became_active":"2024-09-21T07:46:55.211244+0000","last_became_peered":"2024-09-21T07:46:55.211244+0000","last_unstale":"2024-09-21T07:46:55.217541+0000","last_undegraded":"2024-09-21T07:46:55.217541+0000","last_fullsized":"2024-09-21T07:46:55.217541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:04.764546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707540+0000","last_change":"2024-09-21T07:46:56.232335+0000","last_active":"2024-09-21T07:46:57.707540+0000","last_peered":"2024-09-21T07:46:57.707540+0000","last_clean":"2024-09-21T07:46:57.707540+0000","last_became_active":"2024-09-21T07:46:56.231953+0000","last_became_peered":"2024-09-21T07:46:56.231953+0000","last_unstale":"2024-09-21T07:46:57.707540+0000","last_undegraded":"2024-09-21T07:46:57.707540+0000","last_fullsized":"2024-09-21T07:46:57.707540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:04.369378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.219860+0000","last_change":"2024-09-21T07:46:55.214923+0000","last_active":"2024-09-21T07:46:55.219860+0000","last_peered":"2024-09-21T07:46:55.219860+0000","last_clean":"2024-09-21T07:46:55.219860+0000","last_became_active":"2024-09-21T07:46:55.214607+0000","last_became_peered":"2024-09-21T07:46:55.214607+0000","last_unstale":"2024-09-21T07:46:55.219860+0000","last_undegraded":"2024-09-21T07:46:55.219860+0000","last_fullsized":"2024-09-21T07:46:55.219860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:00.922921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195966+0000","last_change":"2024-09-21T07:46:55.195966+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195966+0000","last_undegraded":"2024-09-21T07:46:55.195966+0000","last_fullsized":"2024-09-21T07:46:55.195966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.215316+0000","last_change":"2024-09-21T07:46:55.208755+0000","last_active":"2024-09-21T07:46:55.215316+0000","last_peered":"2024-09-21T07:46:55.215316+0000","last_clean":"2024-09-21T07:46:55.215316+0000","last_became_active":"2024-09-21T07:46:55.207806+0000","last_became_peered":"2024-09-21T07:46:55.207806+0000","last_unstale":"2024-09-21T07:46:55.215316+0000","last_undegraded":"2024-09-21T07:46:55.215316+0000","last_fullsized":"2024-09-21T07:46:55.215316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:36.972091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220825+0000","last_change":"2024-09-21T07:46:56.232337+0000","last_active":"2024-09-21T07:46:57.220825+0000","last_peered":"2024-09-21T07:46:57.220825+0000","last_clean":"2024-09-21T07:46:57.220825+0000","last_became_active":"2024-09-21T07:46:56.230029+0000","last_became_peered":"2024-09-21T07:46:56.230029+0000","last_unstale":"2024-09-21T07:46:57.220825+0000","last_undegraded":"2024-09-21T07:46:57.220825+0000","last_fullsized":"2024-09-21T07:46:57.220825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.727543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708019+0000","last_change":"2024-09-21T07:46:55.211058+0000","last_active":"2024-09-21T07:46:57.708019+0000","last_peered":"2024-09-21T07:46:57.708019+0000","last_clean":"2024-09-21T07:46:57.708019+0000","last_became_active":"2024-09-21T07:46:55.210625+0000","last_became_peered":"2024-09-21T07:46:55.210625+0000","last_unstale":"2024-09-21T07:46:57.708019+0000","last_undegraded":"2024-09-21T07:46:57.708019+0000","last_fullsized":"2024-09-21T07:46:57.708019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.191906+0000","last_change":"2024-09-21T07:46:55.191906+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.191906+0000","last_undegraded":"2024-09-21T07:46:55.191906+0000","last_fullsized":"2024-09-21T07:46:55.191906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228909+0000","last_change":"2024-09-21T07:46:55.209507+0000","last_active":"2024-09-21T07:46:57.228909+0000","last_peered":"2024-09-21T07:46:57.228909+0000","last_clean":"2024-09-21T07:46:57.228909+0000","last_became_active":"2024-09-21T07:46:55.209003+0000","last_became_peered":"2024-09-21T07:46:55.209003+0000","last_unstale":"2024-09-21T07:46:57.228909+0000","last_undegraded":"2024-09-21T07:46:57.228909+0000","last_fullsized":"2024-09-21T07:46:57.228909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:25.605174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221110+0000","last_change":"2024-09-21T07:46:56.241826+0000","last_active":"2024-09-21T07:46:57.221110+0000","last_peered":"2024-09-21T07:46:57.221110+0000","last_clean":"2024-09-21T07:46:57.221110+0000","last_became_active":"2024-09-21T07:46:56.241465+0000","last_became_peered":"2024-09-21T07:46:56.241465+0000","last_unstale":"2024-09-21T07:46:57.221110+0000","last_undegraded":"2024-09-21T07:46:57.221110+0000","last_fullsized":"2024-09-21T07:46:57.221110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:13.411156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708039+0000","last_change":"2024-09-21T07:46:55.209001+0000","last_active":"2024-09-21T07:46:57.708039+0000","last_peered":"2024-09-21T07:46:57.708039+0000","last_clean":"2024-09-21T07:46:57.708039+0000","last_became_active":"2024-09-21T07:46:55.208596+0000","last_became_peered":"2024-09-21T07:46:55.208596+0000","last_unstale":"2024-09-21T07:46:57.708039+0000","last_undegraded":"2024-09-21T07:46:57.708039+0000","last_fullsized":"2024-09-21T07:46:57.708039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196436+0000","last_change":"2024-09-21T07:46:55.196436+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196436+0000","last_undegraded":"2024-09-21T07:46:55.196436+0000","last_fullsized":"2024-09-21T07:46:55.196436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707914+0000","last_change":"2024-09-21T07:46:55.210484+0000","last_active":"2024-09-21T07:46:57.707914+0000","last_peered":"2024-09-21T07:46:57.707914+0000","last_clean":"2024-09-21T07:46:57.707914+0000","last_became_active":"2024-09-21T07:46:55.210140+0000","last_became_peered":"2024-09-21T07:46:55.210140+0000","last_unstale":"2024-09-21T07:46:57.707914+0000","last_undegraded":"2024-09-21T07:46:57.707914+0000","last_fullsized":"2024-09-21T07:46:57.707914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219755+0000","last_change":"2024-09-21T07:46:56.233200+0000","last_active":"2024-09-21T07:46:57.219755+0000","last_peered":"2024-09-21T07:46:57.219755+0000","last_clean":"2024-09-21T07:46:57.219755+0000","last_became_active":"2024-09-21T07:46:56.232818+0000","last_became_peered":"2024-09-21T07:46:56.232818+0000","last_unstale":"2024-09-21T07:46:57.219755+0000","last_undegraded":"2024-09-21T07:46:57.219755+0000","last_fullsized":"2024-09-21T07:46:57.219755+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.180535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707865+0000","last_change":"2024-09-21T07:46:55.215083+0000","last_active":"2024-09-21T07:46:57.707865+0000","last_peered":"2024-09-21T07:46:57.707865+0000","last_clean":"2024-09-21T07:46:57.707865+0000","last_became_active":"2024-09-21T07:46:55.214531+0000","last_became_peered":"2024-09-21T07:46:55.214531+0000","last_unstale":"2024-09-21T07:46:57.707865+0000","last_undegraded":"2024-09-21T07:46:57.707865+0000","last_fullsized":"2024-09-21T07:46:57.707865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218812+0000","last_change":"2024-09-21T07:46:56.220432+0000","last_active":"2024-09-21T07:46:57.218812+0000","last_peered":"2024-09-21T07:46:57.218812+0000","last_clean":"2024-09-21T07:46:57.218812+0000","last_became_active":"2024-09-21T07:46:56.219527+0000","last_became_peered":"2024-09-21T07:46:56.219527+0000","last_unstale":"2024-09-21T07:46:57.218812+0000","last_undegraded":"2024-09-21T07:46:57.218812+0000","last_fullsized":"2024-09-21T07:46:57.218812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:30.402592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217461+0000","last_change":"2024-09-21T07:46:55.212708+0000","last_active":"2024-09-21T07:46:55.217461+0000","last_peered":"2024-09-21T07:46:55.217461+0000","last_clean":"2024-09-21T07:46:55.217461+0000","last_became_active":"2024-09-21T07:46:55.212115+0000","last_became_peered":"2024-09-21T07:46:55.212115+0000","last_unstale":"2024-09-21T07:46:55.217461+0000","last_undegraded":"2024-09-21T07:46:55.217461+0000","last_fullsized":"2024-09-21T07:46:55.217461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:26.943301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.191831+0000","last_change":"2024-09-21T07:46:55.191831+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.191831+0000","last_undegraded":"2024-09-21T07:46:55.191831+0000","last_fullsized":"2024-09-21T07:46:55.191831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219828+0000","last_change":"2024-09-21T07:46:55.214726+0000","last_active":"2024-09-21T07:46:57.219828+0000","last_peered":"2024-09-21T07:46:57.219828+0000","last_clean":"2024-09-21T07:46:57.219828+0000","last_became_active":"2024-09-21T07:46:55.214331+0000","last_became_peered":"2024-09-21T07:46:55.214331+0000","last_unstale":"2024-09-21T07:46:57.219828+0000","last_undegraded":"2024-09-21T07:46:57.219828+0000","last_fullsized":"2024-09-21T07:46:57.219828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198188+0000","last_change":"2024-09-21T07:46:55.198188+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198188+0000","last_undegraded":"2024-09-21T07:46:55.198188+0000","last_fullsized":"2024-09-21T07:46:55.198188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.222778+0000","last_change":"2024-09-21T07:46:55.218103+0000","last_active":"2024-09-21T07:46:55.222778+0000","last_peered":"2024-09-21T07:46:55.222778+0000","last_clean":"2024-09-21T07:46:55.222778+0000","last_became_active":"2024-09-21T07:46:55.217796+0000","last_became_peered":"2024-09-21T07:46:55.217796+0000","last_unstale":"2024-09-21T07:46:55.222778+0000","last_undegraded":"2024-09-21T07:46:55.222778+0000","last_fullsized":"2024-09-21T07:46:55.222778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:07.250160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.211593+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:55.211593+0000","last_peered":"2024-09-21T07:46:55.211593+0000","last_clean":"2024-09-21T07:46:55.211593+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:55.211593+0000","last_undegraded":"2024-09-21T07:46:55.211593+0000","last_fullsized":"2024-09-21T07:46:55.211593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":720244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":397312,"data_stored":150838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":740176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":150860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":149248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:00.632 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.887+0000 7f1ef0600640 1 -- 172.21.15.88:0/779139501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee815d760 msgr2=0x7f1ee81535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.887+0000 7f1ef0600640 1 --2- 172.21.15.88:0/779139501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee815d760 0x7f1ee81535c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f1ee8072d60 tx=0x7f1edc02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.887+0000 7f1ef0600640 1 -- 172.21.15.88:0/779139501 shutdown_connections 2024-09-21T07:47:00.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.887+0000 7f1ef0600640 1 --2- 172.21.15.88:0/779139501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8160310 0x7f1ee8160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.888+0000 7f1ef0600640 1 --2- 172.21.15.88:0/779139501 >> v2:172.21.15.175:3300/0 conn(0x7f1ee80ad0e0 0x7f1ee80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.888+0000 7f1ef0600640 1 --2- 172.21.15.88:0/779139501 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee815d760 0x7f1ee81535c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.888+0000 7f1ef0600640 1 -- 172.21.15.88:0/779139501 >> 172.21.15.88:0/779139501 conn(0x7f1ee806d730 msgr2=0x7f1ee80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.888+0000 7f1ef0600640 1 -- 172.21.15.88:0/779139501 shutdown_connections 2024-09-21T07:47:00.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.888+0000 7f1ef0600640 1 -- 172.21.15.88:0/779139501 wait complete. 2024-09-21T07:47:00.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.890+0000 7f1ef0600640 1 Processor -- start 2024-09-21T07:47:00.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.890+0000 7f1ef0600640 1 -- start start 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.891+0000 7f1ef0600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 0x7f1ee809a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1ef0600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee815d760 0x7f1ee8148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1ef0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee8160310 0x7f1ee815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1ef0600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee80b4da0 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1ef0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee80b4c20 con 0x7f1ee8160310 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1ef0600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee8160c60 con 0x7f1ee815d760 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 0x7f1ee809a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 0x7f1ee809a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58476/0 (socket says 172.21.15.88:58476) 2024-09-21T07:47:00.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 -- 172.21.15.88:0/4101598867 learned_addr learned my addr 172.21.15.88:0/4101598867 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eed200640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee815d760 0x7f1ee8148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eee600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee8160310 0x7f1ee815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 -- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee815d760 msgr2=0x7f1ee8148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee815d760 0x7f1ee8148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 -- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee8160310 msgr2=0x7f1ee815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee8160310 0x7f1ee815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.892+0000 7f1eedc00640 1 -- 172.21.15.88:0/4101598867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee809a810 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.893+0000 7f1eedc00640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 0x7f1ee809a2d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000c00 tx=0x7f1edc003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.893+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc03e030 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.893+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1edc003e00 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.893+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1edc035040 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.893+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee81f9a50 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.893+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee8156b90 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.894+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee8074b80 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.895+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f1edc041030 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.896+0000 7f1ee5600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ec8078b90 0x7f1ec807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.896+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f1edc0bb4d0 con 0x7f1ee80ad0e0 2024-09-21T07:47:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.897+0000 7f1eed200640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ec8078b90 0x7f1ec807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.898+0000 7f1eed200640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ec8078b90 0x7f1ec807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4005fd0 tx=0x7f1ed4005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:00.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:00.901+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1edc0839a0 con 0x7f1ee80ad0e0 2024-09-21T07:47:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.081+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1ee81f9800 con 0x7f1ec8078b90 2024-09-21T07:47:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.086+0000 7f1ee5600640 1 -- 172.21.15.88:0/4101598867 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249152 (secure 0 0 0) 0x7f1ee81f9800 con 0x7f1ec8078b90 2024-09-21T07:47:01.091 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:01.092 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ec8078b90 msgr2=0x7f1ec807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ec8078b90 0x7f1ec807b0a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1ed4005fd0 tx=0x7f1ed4005850 comp rx=0 tx=0).stop 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 msgr2=0x7f1ee809a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 0x7f1ee809a2d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1edc000c00 tx=0x7f1edc003910 comp rx=0 tx=0).stop 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 shutdown_connections 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1ec8078b90 0x7f1ec807b0a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee815d760 0x7f1ee8148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee8160310 0x7f1ee815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 --2- 172.21.15.88:0/4101598867 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80ad0e0 0x7f1ee809a2d0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 >> 172.21.15.88:0/4101598867 conn(0x7f1ee806d730 msgr2=0x7f1ee815ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 shutdown_connections 2024-09-21T07:47:01.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.091+0000 7f1ef0600640 1 -- 172.21.15.88:0/4101598867 wait complete. 2024-09-21T07:47:01.118 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:46:59.866867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220348,"kb_used_data":4096,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754372,"statfs":{"total":773094113280,"available":772868476928,"internally_reserved":0,"allocated":4194304,"data_stored":2319819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.679739"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196875+0000","last_change":"2024-09-21T07:46:55.196875+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196875+0000","last_undegraded":"2024-09-21T07:46:55.196875+0000","last_fullsized":"2024-09-21T07:46:55.196875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228903+0000","last_change":"2024-09-21T07:46:56.229990+0000","last_active":"2024-09-21T07:46:57.228903+0000","last_peered":"2024-09-21T07:46:57.228903+0000","last_clean":"2024-09-21T07:46:57.228903+0000","last_became_active":"2024-09-21T07:46:56.229655+0000","last_became_peered":"2024-09-21T07:46:56.229655+0000","last_unstale":"2024-09-21T07:46:57.228903+0000","last_undegraded":"2024-09-21T07:46:57.228903+0000","last_fullsized":"2024-09-21T07:46:57.228903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.209740+0000","last_change":"2024-09-21T07:46:55.209740+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.209740+0000","last_undegraded":"2024-09-21T07:46:55.209740+0000","last_fullsized":"2024-09-21T07:46:55.209740+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219416+0000","last_change":"2024-09-21T07:46:56.235757+0000","last_active":"2024-09-21T07:46:57.219416+0000","last_peered":"2024-09-21T07:46:57.219416+0000","last_clean":"2024-09-21T07:46:57.219416+0000","last_became_active":"2024-09-21T07:46:56.228680+0000","last_became_peered":"2024-09-21T07:46:56.228680+0000","last_unstale":"2024-09-21T07:46:57.219416+0000","last_undegraded":"2024-09-21T07:46:57.219416+0000","last_fullsized":"2024-09-21T07:46:57.219416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228641+0000","last_change":"2024-09-21T07:46:56.229963+0000","last_active":"2024-09-21T07:46:57.228641+0000","last_peered":"2024-09-21T07:46:57.228641+0000","last_clean":"2024-09-21T07:46:57.228641+0000","last_became_active":"2024-09-21T07:46:56.229617+0000","last_became_peered":"2024-09-21T07:46:56.229617+0000","last_unstale":"2024-09-21T07:46:57.228641+0000","last_undegraded":"2024-09-21T07:46:57.228641+0000","last_fullsized":"2024-09-21T07:46:57.228641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219492+0000","last_change":"2024-09-21T07:46:56.229684+0000","last_active":"2024-09-21T07:46:57.219492+0000","last_peered":"2024-09-21T07:46:57.219492+0000","last_clean":"2024-09-21T07:46:57.219492+0000","last_became_active":"2024-09-21T07:46:56.229339+0000","last_became_peered":"2024-09-21T07:46:56.229339+0000","last_unstale":"2024-09-21T07:46:57.219492+0000","last_undegraded":"2024-09-21T07:46:57.219492+0000","last_fullsized":"2024-09-21T07:46:57.219492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:21.621424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198590+0000","last_change":"2024-09-21T07:46:55.198590+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198590+0000","last_undegraded":"2024-09-21T07:46:55.198590+0000","last_fullsized":"2024-09-21T07:46:55.198590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195067+0000","last_change":"2024-09-21T07:46:55.195067+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195067+0000","last_undegraded":"2024-09-21T07:46:55.195067+0000","last_fullsized":"2024-09-21T07:46:55.195067+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.210023+0000","last_change":"2024-09-21T07:46:55.210023+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.210023+0000","last_undegraded":"2024-09-21T07:46:55.210023+0000","last_fullsized":"2024-09-21T07:46:55.210023+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220083+0000","last_change":"2024-09-21T07:46:56.231062+0000","last_active":"2024-09-21T07:46:57.220083+0000","last_peered":"2024-09-21T07:46:57.220083+0000","last_clean":"2024-09-21T07:46:57.220083+0000","last_became_active":"2024-09-21T07:46:56.229427+0000","last_became_peered":"2024-09-21T07:46:56.229427+0000","last_unstale":"2024-09-21T07:46:57.220083+0000","last_undegraded":"2024-09-21T07:46:57.220083+0000","last_fullsized":"2024-09-21T07:46:57.220083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:47.565734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198313+0000","last_change":"2024-09-21T07:46:55.198313+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198313+0000","last_undegraded":"2024-09-21T07:46:55.198313+0000","last_fullsized":"2024-09-21T07:46:55.198313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.199137+0000","last_change":"2024-09-21T07:46:55.199137+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.199137+0000","last_undegraded":"2024-09-21T07:46:55.199137+0000","last_fullsized":"2024-09-21T07:46:55.199137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707947+0000","last_change":"2024-09-21T07:46:56.237624+0000","last_active":"2024-09-21T07:46:57.707947+0000","last_peered":"2024-09-21T07:46:57.707947+0000","last_clean":"2024-09-21T07:46:57.707947+0000","last_became_active":"2024-09-21T07:46:56.237037+0000","last_became_peered":"2024-09-21T07:46:56.237037+0000","last_unstale":"2024-09-21T07:46:57.707947+0000","last_undegraded":"2024-09-21T07:46:57.707947+0000","last_fullsized":"2024-09-21T07:46:57.707947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:43.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219004+0000","last_change":"2024-09-21T07:46:56.239271+0000","last_active":"2024-09-21T07:46:57.219004+0000","last_peered":"2024-09-21T07:46:57.219004+0000","last_clean":"2024-09-21T07:46:57.219004+0000","last_became_active":"2024-09-21T07:46:56.238936+0000","last_became_peered":"2024-09-21T07:46:56.238936+0000","last_unstale":"2024-09-21T07:46:57.219004+0000","last_undegraded":"2024-09-21T07:46:57.219004+0000","last_fullsized":"2024-09-21T07:46:57.219004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194649+0000","last_change":"2024-09-21T07:46:55.194649+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194649+0000","last_undegraded":"2024-09-21T07:46:55.194649+0000","last_fullsized":"2024-09-21T07:46:55.194649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194072+0000","last_change":"2024-09-21T07:46:55.194072+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194072+0000","last_undegraded":"2024-09-21T07:46:55.194072+0000","last_fullsized":"2024-09-21T07:46:55.194072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198999+0000","last_change":"2024-09-21T07:46:55.198999+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198999+0000","last_undegraded":"2024-09-21T07:46:55.198999+0000","last_fullsized":"2024-09-21T07:46:55.198999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220865+0000","last_change":"2024-09-21T07:46:56.224534+0000","last_active":"2024-09-21T07:46:57.220865+0000","last_peered":"2024-09-21T07:46:57.220865+0000","last_clean":"2024-09-21T07:46:57.220865+0000","last_became_active":"2024-09-21T07:46:56.224167+0000","last_became_peered":"2024-09-21T07:46:56.224167+0000","last_unstale":"2024-09-21T07:46:57.220865+0000","last_undegraded":"2024-09-21T07:46:57.220865+0000","last_fullsized":"2024-09-21T07:46:57.220865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:18.755342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220883+0000","last_change":"2024-09-21T07:46:56.228375+0000","last_active":"2024-09-21T07:46:57.220883+0000","last_peered":"2024-09-21T07:46:57.220883+0000","last_clean":"2024-09-21T07:46:57.220883+0000","last_became_active":"2024-09-21T07:46:56.225793+0000","last_became_peered":"2024-09-21T07:46:56.225793+0000","last_unstale":"2024-09-21T07:46:57.220883+0000","last_undegraded":"2024-09-21T07:46:57.220883+0000","last_fullsized":"2024-09-21T07:46:57.220883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:46.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708077+0000","last_change":"2024-09-21T07:46:56.226928+0000","last_active":"2024-09-21T07:46:57.708077+0000","last_peered":"2024-09-21T07:46:57.708077+0000","last_clean":"2024-09-21T07:46:57.708077+0000","last_became_active":"2024-09-21T07:46:56.226560+0000","last_became_peered":"2024-09-21T07:46:56.226560+0000","last_unstale":"2024-09-21T07:46:57.708077+0000","last_undegraded":"2024-09-21T07:46:57.708077+0000","last_fullsized":"2024-09-21T07:46:57.708077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.197713+0000","last_change":"2024-09-21T07:46:55.197713+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.197713+0000","last_undegraded":"2024-09-21T07:46:55.197713+0000","last_fullsized":"2024-09-21T07:46:55.197713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195512+0000","last_change":"2024-09-21T07:46:55.195512+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195512+0000","last_undegraded":"2024-09-21T07:46:55.195512+0000","last_fullsized":"2024-09-21T07:46:55.195512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707640+0000","last_change":"2024-09-21T07:46:56.227950+0000","last_active":"2024-09-21T07:46:57.707640+0000","last_peered":"2024-09-21T07:46:57.707640+0000","last_clean":"2024-09-21T07:46:57.707640+0000","last_became_active":"2024-09-21T07:46:56.227647+0000","last_became_peered":"2024-09-21T07:46:56.227647+0000","last_unstale":"2024-09-21T07:46:57.707640+0000","last_undegraded":"2024-09-21T07:46:57.707640+0000","last_fullsized":"2024-09-21T07:46:57.707640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:08.480356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194574+0000","last_change":"2024-09-21T07:46:55.194574+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194574+0000","last_undegraded":"2024-09-21T07:46:55.194574+0000","last_fullsized":"2024-09-21T07:46:55.194574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219395+0000","last_change":"2024-09-21T07:46:56.217732+0000","last_active":"2024-09-21T07:46:57.219395+0000","last_peered":"2024-09-21T07:46:57.219395+0000","last_clean":"2024-09-21T07:46:57.219395+0000","last_became_active":"2024-09-21T07:46:56.217410+0000","last_became_peered":"2024-09-21T07:46:56.217410+0000","last_unstale":"2024-09-21T07:46:57.219395+0000","last_undegraded":"2024-09-21T07:46:57.219395+0000","last_fullsized":"2024-09-21T07:46:57.219395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219810+0000","last_change":"2024-09-21T07:46:56.220112+0000","last_active":"2024-09-21T07:46:57.219810+0000","last_peered":"2024-09-21T07:46:57.219810+0000","last_clean":"2024-09-21T07:46:57.219810+0000","last_became_active":"2024-09-21T07:46:56.217329+0000","last_became_peered":"2024-09-21T07:46:56.217329+0000","last_unstale":"2024-09-21T07:46:57.219810+0000","last_undegraded":"2024-09-21T07:46:57.219810+0000","last_fullsized":"2024-09-21T07:46:57.219810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:04.144314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228789+0000","last_change":"2024-09-21T07:46:56.223494+0000","last_active":"2024-09-21T07:46:57.228789+0000","last_peered":"2024-09-21T07:46:57.228789+0000","last_clean":"2024-09-21T07:46:57.228789+0000","last_became_active":"2024-09-21T07:46:56.223149+0000","last_became_peered":"2024-09-21T07:46:56.223149+0000","last_unstale":"2024-09-21T07:46:57.228789+0000","last_undegraded":"2024-09-21T07:46:57.228789+0000","last_fullsized":"2024-09-21T07:46:57.228789+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198069+0000","last_change":"2024-09-21T07:46:55.198069+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198069+0000","last_undegraded":"2024-09-21T07:46:55.198069+0000","last_fullsized":"2024-09-21T07:46:55.198069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.200475+0000","last_change":"2024-09-21T07:46:55.200475+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.200475+0000","last_undegraded":"2024-09-21T07:46:55.200475+0000","last_fullsized":"2024-09-21T07:46:55.200475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196887+0000","last_change":"2024-09-21T07:46:55.196887+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196887+0000","last_undegraded":"2024-09-21T07:46:55.196887+0000","last_fullsized":"2024-09-21T07:46:55.196887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.192541+0000","last_change":"2024-09-21T07:46:55.192541+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.192541+0000","last_undegraded":"2024-09-21T07:46:55.192541+0000","last_fullsized":"2024-09-21T07:46:55.192541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198692+0000","last_change":"2024-09-21T07:46:55.198692+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198692+0000","last_undegraded":"2024-09-21T07:46:55.198692+0000","last_fullsized":"2024-09-21T07:46:55.198692+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708018+0000","last_change":"2024-09-21T07:46:56.231049+0000","last_active":"2024-09-21T07:46:57.708018+0000","last_peered":"2024-09-21T07:46:57.708018+0000","last_clean":"2024-09-21T07:46:57.708018+0000","last_became_active":"2024-09-21T07:46:56.230778+0000","last_became_peered":"2024-09-21T07:46:56.230778+0000","last_unstale":"2024-09-21T07:46:57.708018+0000","last_undegraded":"2024-09-21T07:46:57.708018+0000","last_fullsized":"2024-09-21T07:46:57.708018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.211797+0000","last_change":"2024-09-21T07:46:55.204066+0000","last_active":"2024-09-21T07:46:55.211797+0000","last_peered":"2024-09-21T07:46:55.211797+0000","last_clean":"2024-09-21T07:46:55.211797+0000","last_became_active":"2024-09-21T07:46:55.203753+0000","last_became_peered":"2024-09-21T07:46:55.203753+0000","last_unstale":"2024-09-21T07:46:55.211797+0000","last_undegraded":"2024-09-21T07:46:55.211797+0000","last_fullsized":"2024-09-21T07:46:55.211797+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:20.693634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196356+0000","last_change":"2024-09-21T07:46:55.196356+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196356+0000","last_undegraded":"2024-09-21T07:46:55.196356+0000","last_fullsized":"2024-09-21T07:46:55.196356+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219963+0000","last_change":"2024-09-21T07:46:55.212364+0000","last_active":"2024-09-21T07:46:57.219963+0000","last_peered":"2024-09-21T07:46:57.219963+0000","last_clean":"2024-09-21T07:46:57.219963+0000","last_became_active":"2024-09-21T07:46:55.211980+0000","last_became_peered":"2024-09-21T07:46:55.211980+0000","last_unstale":"2024-09-21T07:46:57.219963+0000","last_undegraded":"2024-09-21T07:46:57.219963+0000","last_fullsized":"2024-09-21T07:46:57.219963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:40.287704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.197598+0000","last_change":"2024-09-21T07:46:55.197598+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.197598+0000","last_undegraded":"2024-09-21T07:46:55.197598+0000","last_fullsized":"2024-09-21T07:46:55.197598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221045+0000","last_change":"2024-09-21T07:46:55.216737+0000","last_active":"2024-09-21T07:46:57.221045+0000","last_peered":"2024-09-21T07:46:57.221045+0000","last_clean":"2024-09-21T07:46:57.221045+0000","last_became_active":"2024-09-21T07:46:55.216402+0000","last_became_peered":"2024-09-21T07:46:55.216402+0000","last_unstale":"2024-09-21T07:46:57.221045+0000","last_undegraded":"2024-09-21T07:46:57.221045+0000","last_fullsized":"2024-09-21T07:46:57.221045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:40.966275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219312+0000","last_change":"2024-09-21T07:46:56.237682+0000","last_active":"2024-09-21T07:46:57.219312+0000","last_peered":"2024-09-21T07:46:57.219312+0000","last_clean":"2024-09-21T07:46:57.219312+0000","last_became_active":"2024-09-21T07:46:56.237340+0000","last_became_peered":"2024-09-21T07:46:56.237340+0000","last_unstale":"2024-09-21T07:46:57.219312+0000","last_undegraded":"2024-09-21T07:46:57.219312+0000","last_fullsized":"2024-09-21T07:46:57.219312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219765+0000","last_change":"2024-09-21T07:46:55.211939+0000","last_active":"2024-09-21T07:46:57.219765+0000","last_peered":"2024-09-21T07:46:57.219765+0000","last_clean":"2024-09-21T07:46:57.219765+0000","last_became_active":"2024-09-21T07:46:55.211170+0000","last_became_peered":"2024-09-21T07:46:55.211170+0000","last_unstale":"2024-09-21T07:46:57.219765+0000","last_undegraded":"2024-09-21T07:46:57.219765+0000","last_fullsized":"2024-09-21T07:46:57.219765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194935+0000","last_change":"2024-09-21T07:46:55.194935+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194935+0000","last_undegraded":"2024-09-21T07:46:55.194935+0000","last_fullsized":"2024-09-21T07:46:55.194935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.210107+0000","last_change":"2024-09-21T07:46:55.204599+0000","last_active":"2024-09-21T07:46:55.210107+0000","last_peered":"2024-09-21T07:46:55.210107+0000","last_clean":"2024-09-21T07:46:55.210107+0000","last_became_active":"2024-09-21T07:46:55.204186+0000","last_became_peered":"2024-09-21T07:46:55.204186+0000","last_unstale":"2024-09-21T07:46:55.210107+0000","last_undegraded":"2024-09-21T07:46:55.210107+0000","last_fullsized":"2024-09-21T07:46:55.210107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:36.808640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193937+0000","last_change":"2024-09-21T07:46:55.193937+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193937+0000","last_undegraded":"2024-09-21T07:46:55.193937+0000","last_fullsized":"2024-09-21T07:46:55.193937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.210087+0000","last_change":"2024-09-21T07:46:55.205089+0000","last_active":"2024-09-21T07:46:55.210087+0000","last_peered":"2024-09-21T07:46:55.210087+0000","last_clean":"2024-09-21T07:46:55.210087+0000","last_became_active":"2024-09-21T07:46:55.204642+0000","last_became_peered":"2024-09-21T07:46:55.204642+0000","last_unstale":"2024-09-21T07:46:55.210087+0000","last_undegraded":"2024-09-21T07:46:55.210087+0000","last_fullsized":"2024-09-21T07:46:55.210087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:12.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193798+0000","last_change":"2024-09-21T07:46:55.193798+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193798+0000","last_undegraded":"2024-09-21T07:46:55.193798+0000","last_fullsized":"2024-09-21T07:46:55.193798+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218717+0000","last_change":"2024-09-21T07:46:55.218472+0000","last_active":"2024-09-21T07:46:57.218717+0000","last_peered":"2024-09-21T07:46:57.218717+0000","last_clean":"2024-09-21T07:46:57.218717+0000","last_became_active":"2024-09-21T07:46:55.218044+0000","last_became_peered":"2024-09-21T07:46:55.218044+0000","last_unstale":"2024-09-21T07:46:57.218717+0000","last_undegraded":"2024-09-21T07:46:57.218717+0000","last_fullsized":"2024-09-21T07:46:57.218717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:18.860660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218669+0000","last_change":"2024-09-21T07:46:56.221089+0000","last_active":"2024-09-21T07:46:57.218669+0000","last_peered":"2024-09-21T07:46:57.218669+0000","last_clean":"2024-09-21T07:46:57.218669+0000","last_became_active":"2024-09-21T07:46:56.220746+0000","last_became_peered":"2024-09-21T07:46:56.220746+0000","last_unstale":"2024-09-21T07:46:57.218669+0000","last_undegraded":"2024-09-21T07:46:57.218669+0000","last_fullsized":"2024-09-21T07:46:57.218669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.213270+0000","last_change":"2024-09-21T07:46:55.206317+0000","last_active":"2024-09-21T07:46:55.213270+0000","last_peered":"2024-09-21T07:46:55.213270+0000","last_clean":"2024-09-21T07:46:55.213270+0000","last_became_active":"2024-09-21T07:46:55.206000+0000","last_became_peered":"2024-09-21T07:46:55.206000+0000","last_unstale":"2024-09-21T07:46:55.213270+0000","last_undegraded":"2024-09-21T07:46:55.213270+0000","last_fullsized":"2024-09-21T07:46:55.213270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:35.469725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198239+0000","last_change":"2024-09-21T07:46:55.198239+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198239+0000","last_undegraded":"2024-09-21T07:46:55.198239+0000","last_fullsized":"2024-09-21T07:46:55.198239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228976+0000","last_change":"2024-09-21T07:46:55.213497+0000","last_active":"2024-09-21T07:46:57.228976+0000","last_peered":"2024-09-21T07:46:57.228976+0000","last_clean":"2024-09-21T07:46:57.228976+0000","last_became_active":"2024-09-21T07:46:55.213173+0000","last_became_peered":"2024-09-21T07:46:55.213173+0000","last_unstale":"2024-09-21T07:46:57.228976+0000","last_undegraded":"2024-09-21T07:46:57.228976+0000","last_fullsized":"2024-09-21T07:46:57.228976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.908077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219686+0000","last_change":"2024-09-21T07:46:56.229383+0000","last_active":"2024-09-21T07:46:57.219686+0000","last_peered":"2024-09-21T07:46:57.219686+0000","last_clean":"2024-09-21T07:46:57.219686+0000","last_became_active":"2024-09-21T07:46:56.229012+0000","last_became_peered":"2024-09-21T07:46:56.229012+0000","last_unstale":"2024-09-21T07:46:57.219686+0000","last_undegraded":"2024-09-21T07:46:57.219686+0000","last_fullsized":"2024-09-21T07:46:57.219686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:56.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228698+0000","last_change":"2024-09-21T07:46:55.212681+0000","last_active":"2024-09-21T07:46:57.228698+0000","last_peered":"2024-09-21T07:46:57.228698+0000","last_clean":"2024-09-21T07:46:57.228698+0000","last_became_active":"2024-09-21T07:46:55.211907+0000","last_became_peered":"2024-09-21T07:46:55.211907+0000","last_unstale":"2024-09-21T07:46:57.228698+0000","last_undegraded":"2024-09-21T07:46:57.228698+0000","last_fullsized":"2024-09-21T07:46:57.228698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:30.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219680+0000","last_change":"2024-09-21T07:46:56.224792+0000","last_active":"2024-09-21T07:46:57.219680+0000","last_peered":"2024-09-21T07:46:57.219680+0000","last_clean":"2024-09-21T07:46:57.219680+0000","last_became_active":"2024-09-21T07:46:56.224554+0000","last_became_peered":"2024-09-21T07:46:56.224554+0000","last_unstale":"2024-09-21T07:46:57.219680+0000","last_undegraded":"2024-09-21T07:46:57.219680+0000","last_fullsized":"2024-09-21T07:46:57.219680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:31.069660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.214576+0000","last_change":"2024-09-21T07:46:55.207287+0000","last_active":"2024-09-21T07:46:55.214576+0000","last_peered":"2024-09-21T07:46:55.214576+0000","last_clean":"2024-09-21T07:46:55.214576+0000","last_became_active":"2024-09-21T07:46:55.207008+0000","last_became_peered":"2024-09-21T07:46:55.207008+0000","last_unstale":"2024-09-21T07:46:55.214576+0000","last_undegraded":"2024-09-21T07:46:55.214576+0000","last_fullsized":"2024-09-21T07:46:55.214576+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:31.482910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196425+0000","last_change":"2024-09-21T07:46:55.196425+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196425+0000","last_undegraded":"2024-09-21T07:46:55.196425+0000","last_fullsized":"2024-09-21T07:46:55.196425+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707883+0000","last_change":"2024-09-21T07:46:55.209727+0000","last_active":"2024-09-21T07:46:57.707883+0000","last_peered":"2024-09-21T07:46:57.707883+0000","last_clean":"2024-09-21T07:46:57.707883+0000","last_became_active":"2024-09-21T07:46:55.209437+0000","last_became_peered":"2024-09-21T07:46:55.209437+0000","last_unstale":"2024-09-21T07:46:57.707883+0000","last_undegraded":"2024-09-21T07:46:57.707883+0000","last_fullsized":"2024-09-21T07:46:57.707883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707613+0000","last_change":"2024-09-21T07:46:56.234403+0000","last_active":"2024-09-21T07:46:57.707613+0000","last_peered":"2024-09-21T07:46:57.707613+0000","last_clean":"2024-09-21T07:46:57.707613+0000","last_became_active":"2024-09-21T07:46:56.233998+0000","last_became_peered":"2024-09-21T07:46:56.233998+0000","last_unstale":"2024-09-21T07:46:57.707613+0000","last_undegraded":"2024-09-21T07:46:57.707613+0000","last_fullsized":"2024-09-21T07:46:57.707613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:52.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217390+0000","last_change":"2024-09-21T07:46:55.212435+0000","last_active":"2024-09-21T07:46:55.217390+0000","last_peered":"2024-09-21T07:46:55.217390+0000","last_clean":"2024-09-21T07:46:55.217390+0000","last_became_active":"2024-09-21T07:46:55.212093+0000","last_became_peered":"2024-09-21T07:46:55.212093+0000","last_unstale":"2024-09-21T07:46:55.217390+0000","last_undegraded":"2024-09-21T07:46:55.217390+0000","last_fullsized":"2024-09-21T07:46:55.217390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:44.281725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219594+0000","last_change":"2024-09-21T07:46:56.236480+0000","last_active":"2024-09-21T07:46:57.219594+0000","last_peered":"2024-09-21T07:46:57.219594+0000","last_clean":"2024-09-21T07:46:57.219594+0000","last_became_active":"2024-09-21T07:46:56.236111+0000","last_became_peered":"2024-09-21T07:46:56.236111+0000","last_unstale":"2024-09-21T07:46:57.219594+0000","last_undegraded":"2024-09-21T07:46:57.219594+0000","last_fullsized":"2024-09-21T07:46:57.219594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219132+0000","last_change":"2024-09-21T07:46:55.211160+0000","last_active":"2024-09-21T07:46:57.219132+0000","last_peered":"2024-09-21T07:46:57.219132+0000","last_clean":"2024-09-21T07:46:57.219132+0000","last_became_active":"2024-09-21T07:46:55.210797+0000","last_became_peered":"2024-09-21T07:46:55.210797+0000","last_unstale":"2024-09-21T07:46:57.219132+0000","last_undegraded":"2024-09-21T07:46:57.219132+0000","last_fullsized":"2024-09-21T07:46:57.219132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:33.882042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.192745+0000","last_change":"2024-09-21T07:46:55.192745+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.192745+0000","last_undegraded":"2024-09-21T07:46:55.192745+0000","last_fullsized":"2024-09-21T07:46:55.192745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218445+0000","last_change":"2024-09-21T07:46:55.205503+0000","last_active":"2024-09-21T07:46:57.218445+0000","last_peered":"2024-09-21T07:46:57.218445+0000","last_clean":"2024-09-21T07:46:57.218445+0000","last_became_active":"2024-09-21T07:46:55.204821+0000","last_became_peered":"2024-09-21T07:46:55.204821+0000","last_unstale":"2024-09-21T07:46:57.218445+0000","last_undegraded":"2024-09-21T07:46:57.218445+0000","last_fullsized":"2024-09-21T07:46:57.218445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195717+0000","last_change":"2024-09-21T07:46:55.195717+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195717+0000","last_undegraded":"2024-09-21T07:46:55.195717+0000","last_fullsized":"2024-09-21T07:46:55.195717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218410+0000","last_change":"2024-09-21T07:46:55.216736+0000","last_active":"2024-09-21T07:46:57.218410+0000","last_peered":"2024-09-21T07:46:57.218410+0000","last_clean":"2024-09-21T07:46:57.218410+0000","last_became_active":"2024-09-21T07:46:55.216312+0000","last_became_peered":"2024-09-21T07:46:55.216312+0000","last_unstale":"2024-09-21T07:46:57.218410+0000","last_undegraded":"2024-09-21T07:46:57.218410+0000","last_fullsized":"2024-09-21T07:46:57.218410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.195061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193288+0000","last_change":"2024-09-21T07:46:55.193288+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193288+0000","last_undegraded":"2024-09-21T07:46:55.193288+0000","last_fullsized":"2024-09-21T07:46:55.193288+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220824+0000","last_change":"2024-09-21T07:46:55.218143+0000","last_active":"2024-09-21T07:46:57.220824+0000","last_peered":"2024-09-21T07:46:57.220824+0000","last_clean":"2024-09-21T07:46:57.220824+0000","last_became_active":"2024-09-21T07:46:55.217800+0000","last_became_peered":"2024-09-21T07:46:55.217800+0000","last_unstale":"2024-09-21T07:46:57.220824+0000","last_undegraded":"2024-09-21T07:46:57.220824+0000","last_fullsized":"2024-09-21T07:46:57.220824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:57.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.205097+0000","last_change":"2024-09-21T07:46:55.205097+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.205097+0000","last_undegraded":"2024-09-21T07:46:55.205097+0000","last_fullsized":"2024-09-21T07:46:55.205097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217443+0000","last_change":"2024-09-21T07:46:55.212733+0000","last_active":"2024-09-21T07:46:55.217443+0000","last_peered":"2024-09-21T07:46:55.217443+0000","last_clean":"2024-09-21T07:46:55.217443+0000","last_became_active":"2024-09-21T07:46:55.212478+0000","last_became_peered":"2024-09-21T07:46:55.212478+0000","last_unstale":"2024-09-21T07:46:55.217443+0000","last_undegraded":"2024-09-21T07:46:55.217443+0000","last_fullsized":"2024-09-21T07:46:55.217443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.222888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228788+0000","last_change":"2024-09-21T07:46:56.223493+0000","last_active":"2024-09-21T07:46:57.228788+0000","last_peered":"2024-09-21T07:46:57.228788+0000","last_clean":"2024-09-21T07:46:57.228788+0000","last_became_active":"2024-09-21T07:46:56.223060+0000","last_became_peered":"2024-09-21T07:46:56.223060+0000","last_unstale":"2024-09-21T07:46:57.228788+0000","last_undegraded":"2024-09-21T07:46:57.228788+0000","last_fullsized":"2024-09-21T07:46:57.228788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217612+0000","last_change":"2024-09-21T07:46:55.213737+0000","last_active":"2024-09-21T07:46:55.217612+0000","last_peered":"2024-09-21T07:46:55.217612+0000","last_clean":"2024-09-21T07:46:55.217612+0000","last_became_active":"2024-09-21T07:46:55.213452+0000","last_became_peered":"2024-09-21T07:46:55.213452+0000","last_unstale":"2024-09-21T07:46:55.217612+0000","last_undegraded":"2024-09-21T07:46:55.217612+0000","last_fullsized":"2024-09-21T07:46:55.217612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:10.689164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220927+0000","last_change":"2024-09-21T07:46:56.232119+0000","last_active":"2024-09-21T07:46:57.220927+0000","last_peered":"2024-09-21T07:46:57.220927+0000","last_clean":"2024-09-21T07:46:57.220927+0000","last_became_active":"2024-09-21T07:46:56.231764+0000","last_became_peered":"2024-09-21T07:46:56.231764+0000","last_unstale":"2024-09-21T07:46:57.220927+0000","last_undegraded":"2024-09-21T07:46:57.220927+0000","last_fullsized":"2024-09-21T07:46:57.220927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:35.746093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.208347+0000","last_change":"2024-09-21T07:46:55.206900+0000","last_active":"2024-09-21T07:46:55.208347+0000","last_peered":"2024-09-21T07:46:55.208347+0000","last_clean":"2024-09-21T07:46:55.208347+0000","last_became_active":"2024-09-21T07:46:55.206441+0000","last_became_peered":"2024-09-21T07:46:55.206441+0000","last_unstale":"2024-09-21T07:46:55.208347+0000","last_undegraded":"2024-09-21T07:46:55.208347+0000","last_fullsized":"2024-09-21T07:46:55.208347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.920219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707559+0000","last_change":"2024-09-21T07:46:56.233293+0000","last_active":"2024-09-21T07:46:57.707559+0000","last_peered":"2024-09-21T07:46:57.707559+0000","last_clean":"2024-09-21T07:46:57.707559+0000","last_became_active":"2024-09-21T07:46:56.232951+0000","last_became_peered":"2024-09-21T07:46:56.232951+0000","last_unstale":"2024-09-21T07:46:57.707559+0000","last_undegraded":"2024-09-21T07:46:57.707559+0000","last_fullsized":"2024-09-21T07:46:57.707559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:58.570223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228732+0000","last_change":"2024-09-21T07:46:55.206362+0000","last_active":"2024-09-21T07:46:57.228732+0000","last_peered":"2024-09-21T07:46:57.228732+0000","last_clean":"2024-09-21T07:46:57.228732+0000","last_became_active":"2024-09-21T07:46:55.206044+0000","last_became_peered":"2024-09-21T07:46:55.206044+0000","last_unstale":"2024-09-21T07:46:57.228732+0000","last_undegraded":"2024-09-21T07:46:57.228732+0000","last_fullsized":"2024-09-21T07:46:57.228732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:20.127719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228770+0000","last_change":"2024-09-21T07:46:56.221448+0000","last_active":"2024-09-21T07:46:57.228770+0000","last_peered":"2024-09-21T07:46:57.228770+0000","last_clean":"2024-09-21T07:46:57.228770+0000","last_became_active":"2024-09-21T07:46:56.221116+0000","last_became_peered":"2024-09-21T07:46:56.221116+0000","last_unstale":"2024-09-21T07:46:57.228770+0000","last_undegraded":"2024-09-21T07:46:57.228770+0000","last_fullsized":"2024-09-21T07:46:57.228770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217541+0000","last_change":"2024-09-21T07:46:55.213426+0000","last_active":"2024-09-21T07:46:55.217541+0000","last_peered":"2024-09-21T07:46:55.217541+0000","last_clean":"2024-09-21T07:46:55.217541+0000","last_became_active":"2024-09-21T07:46:55.211244+0000","last_became_peered":"2024-09-21T07:46:55.211244+0000","last_unstale":"2024-09-21T07:46:55.217541+0000","last_undegraded":"2024-09-21T07:46:55.217541+0000","last_fullsized":"2024-09-21T07:46:55.217541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:04.764546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707540+0000","last_change":"2024-09-21T07:46:56.232335+0000","last_active":"2024-09-21T07:46:57.707540+0000","last_peered":"2024-09-21T07:46:57.707540+0000","last_clean":"2024-09-21T07:46:57.707540+0000","last_became_active":"2024-09-21T07:46:56.231953+0000","last_became_peered":"2024-09-21T07:46:56.231953+0000","last_unstale":"2024-09-21T07:46:57.707540+0000","last_undegraded":"2024-09-21T07:46:57.707540+0000","last_fullsized":"2024-09-21T07:46:57.707540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:04.369378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.219860+0000","last_change":"2024-09-21T07:46:55.214923+0000","last_active":"2024-09-21T07:46:55.219860+0000","last_peered":"2024-09-21T07:46:55.219860+0000","last_clean":"2024-09-21T07:46:55.219860+0000","last_became_active":"2024-09-21T07:46:55.214607+0000","last_became_peered":"2024-09-21T07:46:55.214607+0000","last_unstale":"2024-09-21T07:46:55.219860+0000","last_undegraded":"2024-09-21T07:46:55.219860+0000","last_fullsized":"2024-09-21T07:46:55.219860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:00.922921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195966+0000","last_change":"2024-09-21T07:46:55.195966+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195966+0000","last_undegraded":"2024-09-21T07:46:55.195966+0000","last_fullsized":"2024-09-21T07:46:55.195966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.215316+0000","last_change":"2024-09-21T07:46:55.208755+0000","last_active":"2024-09-21T07:46:55.215316+0000","last_peered":"2024-09-21T07:46:55.215316+0000","last_clean":"2024-09-21T07:46:55.215316+0000","last_became_active":"2024-09-21T07:46:55.207806+0000","last_became_peered":"2024-09-21T07:46:55.207806+0000","last_unstale":"2024-09-21T07:46:55.215316+0000","last_undegraded":"2024-09-21T07:46:55.215316+0000","last_fullsized":"2024-09-21T07:46:55.215316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:36.972091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220825+0000","last_change":"2024-09-21T07:46:56.232337+0000","last_active":"2024-09-21T07:46:57.220825+0000","last_peered":"2024-09-21T07:46:57.220825+0000","last_clean":"2024-09-21T07:46:57.220825+0000","last_became_active":"2024-09-21T07:46:56.230029+0000","last_became_peered":"2024-09-21T07:46:56.230029+0000","last_unstale":"2024-09-21T07:46:57.220825+0000","last_undegraded":"2024-09-21T07:46:57.220825+0000","last_fullsized":"2024-09-21T07:46:57.220825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.727543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708019+0000","last_change":"2024-09-21T07:46:55.211058+0000","last_active":"2024-09-21T07:46:57.708019+0000","last_peered":"2024-09-21T07:46:57.708019+0000","last_clean":"2024-09-21T07:46:57.708019+0000","last_became_active":"2024-09-21T07:46:55.210625+0000","last_became_peered":"2024-09-21T07:46:55.210625+0000","last_unstale":"2024-09-21T07:46:57.708019+0000","last_undegraded":"2024-09-21T07:46:57.708019+0000","last_fullsized":"2024-09-21T07:46:57.708019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.191906+0000","last_change":"2024-09-21T07:46:55.191906+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.191906+0000","last_undegraded":"2024-09-21T07:46:55.191906+0000","last_fullsized":"2024-09-21T07:46:55.191906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228909+0000","last_change":"2024-09-21T07:46:55.209507+0000","last_active":"2024-09-21T07:46:57.228909+0000","last_peered":"2024-09-21T07:46:57.228909+0000","last_clean":"2024-09-21T07:46:57.228909+0000","last_became_active":"2024-09-21T07:46:55.209003+0000","last_became_peered":"2024-09-21T07:46:55.209003+0000","last_unstale":"2024-09-21T07:46:57.228909+0000","last_undegraded":"2024-09-21T07:46:57.228909+0000","last_fullsized":"2024-09-21T07:46:57.228909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:25.605174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221110+0000","last_change":"2024-09-21T07:46:56.241826+0000","last_active":"2024-09-21T07:46:57.221110+0000","last_peered":"2024-09-21T07:46:57.221110+0000","last_clean":"2024-09-21T07:46:57.221110+0000","last_became_active":"2024-09-21T07:46:56.241465+0000","last_became_peered":"2024-09-21T07:46:56.241465+0000","last_unstale":"2024-09-21T07:46:57.221110+0000","last_undegraded":"2024-09-21T07:46:57.221110+0000","last_fullsized":"2024-09-21T07:46:57.221110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:13.411156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708039+0000","last_change":"2024-09-21T07:46:55.209001+0000","last_active":"2024-09-21T07:46:57.708039+0000","last_peered":"2024-09-21T07:46:57.708039+0000","last_clean":"2024-09-21T07:46:57.708039+0000","last_became_active":"2024-09-21T07:46:55.208596+0000","last_became_peered":"2024-09-21T07:46:55.208596+0000","last_unstale":"2024-09-21T07:46:57.708039+0000","last_undegraded":"2024-09-21T07:46:57.708039+0000","last_fullsized":"2024-09-21T07:46:57.708039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196436+0000","last_change":"2024-09-21T07:46:55.196436+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196436+0000","last_undegraded":"2024-09-21T07:46:55.196436+0000","last_fullsized":"2024-09-21T07:46:55.196436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707914+0000","last_change":"2024-09-21T07:46:55.210484+0000","last_active":"2024-09-21T07:46:57.707914+0000","last_peered":"2024-09-21T07:46:57.707914+0000","last_clean":"2024-09-21T07:46:57.707914+0000","last_became_active":"2024-09-21T07:46:55.210140+0000","last_became_peered":"2024-09-21T07:46:55.210140+0000","last_unstale":"2024-09-21T07:46:57.707914+0000","last_undegraded":"2024-09-21T07:46:57.707914+0000","last_fullsized":"2024-09-21T07:46:57.707914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219755+0000","last_change":"2024-09-21T07:46:56.233200+0000","last_active":"2024-09-21T07:46:57.219755+0000","last_peered":"2024-09-21T07:46:57.219755+0000","last_clean":"2024-09-21T07:46:57.219755+0000","last_became_active":"2024-09-21T07:46:56.232818+0000","last_became_peered":"2024-09-21T07:46:56.232818+0000","last_unstale":"2024-09-21T07:46:57.219755+0000","last_undegraded":"2024-09-21T07:46:57.219755+0000","last_fullsized":"2024-09-21T07:46:57.219755+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.180535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707865+0000","last_change":"2024-09-21T07:46:55.215083+0000","last_active":"2024-09-21T07:46:57.707865+0000","last_peered":"2024-09-21T07:46:57.707865+0000","last_clean":"2024-09-21T07:46:57.707865+0000","last_became_active":"2024-09-21T07:46:55.214531+0000","last_became_peered":"2024-09-21T07:46:55.214531+0000","last_unstale":"2024-09-21T07:46:57.707865+0000","last_undegraded":"2024-09-21T07:46:57.707865+0000","last_fullsized":"2024-09-21T07:46:57.707865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218812+0000","last_change":"2024-09-21T07:46:56.220432+0000","last_active":"2024-09-21T07:46:57.218812+0000","last_peered":"2024-09-21T07:46:57.218812+0000","last_clean":"2024-09-21T07:46:57.218812+0000","last_became_active":"2024-09-21T07:46:56.219527+0000","last_became_peered":"2024-09-21T07:46:56.219527+0000","last_unstale":"2024-09-21T07:46:57.218812+0000","last_undegraded":"2024-09-21T07:46:57.218812+0000","last_fullsized":"2024-09-21T07:46:57.218812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:30.402592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217461+0000","last_change":"2024-09-21T07:46:55.212708+0000","last_active":"2024-09-21T07:46:55.217461+0000","last_peered":"2024-09-21T07:46:55.217461+0000","last_clean":"2024-09-21T07:46:55.217461+0000","last_became_active":"2024-09-21T07:46:55.212115+0000","last_became_peered":"2024-09-21T07:46:55.212115+0000","last_unstale":"2024-09-21T07:46:55.217461+0000","last_undegraded":"2024-09-21T07:46:55.217461+0000","last_fullsized":"2024-09-21T07:46:55.217461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:26.943301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.191831+0000","last_change":"2024-09-21T07:46:55.191831+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.191831+0000","last_undegraded":"2024-09-21T07:46:55.191831+0000","last_fullsized":"2024-09-21T07:46:55.191831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219828+0000","last_change":"2024-09-21T07:46:55.214726+0000","last_active":"2024-09-21T07:46:57.219828+0000","last_peered":"2024-09-21T07:46:57.219828+0000","last_clean":"2024-09-21T07:46:57.219828+0000","last_became_active":"2024-09-21T07:46:55.214331+0000","last_became_peered":"2024-09-21T07:46:55.214331+0000","last_unstale":"2024-09-21T07:46:57.219828+0000","last_undegraded":"2024-09-21T07:46:57.219828+0000","last_fullsized":"2024-09-21T07:46:57.219828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198188+0000","last_change":"2024-09-21T07:46:55.198188+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198188+0000","last_undegraded":"2024-09-21T07:46:55.198188+0000","last_fullsized":"2024-09-21T07:46:55.198188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.222778+0000","last_change":"2024-09-21T07:46:55.218103+0000","last_active":"2024-09-21T07:46:55.222778+0000","last_peered":"2024-09-21T07:46:55.222778+0000","last_clean":"2024-09-21T07:46:55.222778+0000","last_became_active":"2024-09-21T07:46:55.217796+0000","last_became_peered":"2024-09-21T07:46:55.217796+0000","last_unstale":"2024-09-21T07:46:55.222778+0000","last_undegraded":"2024-09-21T07:46:55.222778+0000","last_fullsized":"2024-09-21T07:46:55.222778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:07.250160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.211593+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:55.211593+0000","last_peered":"2024-09-21T07:46:55.211593+0000","last_clean":"2024-09-21T07:46:55.211593+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:55.211593+0000","last_undegraded":"2024-09-21T07:46:55.211593+0000","last_fullsized":"2024-09-21T07:46:55.211593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":720244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":397312,"data_stored":150838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":740176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":150860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":149248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:01.122 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 -- 172.21.15.88:0/3626982989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c015d760 msgr2=0x7f98c01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3626982989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c015d760 0x7f98c01535c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98b0005fd0 tx=0x7f98b002a540 comp rx=0 tx=0).stop 2024-09-21T07:47:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 -- 172.21.15.88:0/3626982989 shutdown_connections 2024-09-21T07:47:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3626982989 >> v2:172.21.15.175:3300/0 conn(0x7f98c00ad0e0 0x7f98c00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3626982989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c015d760 0x7f98c01535c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 -- 172.21.15.88:0/3626982989 >> 172.21.15.88:0/3626982989 conn(0x7f98c006d730 msgr2=0x7f98c00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 -- 172.21.15.88:0/3626982989 shutdown_connections 2024-09-21T07:47:01.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.378+0000 7f98c6e00640 1 -- 172.21.15.88:0/3626982989 wait complete. 2024-09-21T07:47:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.380+0000 7f98c6e00640 1 Processor -- start 2024-09-21T07:47:01.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.381+0000 7f98c6e00640 1 -- start start 2024-09-21T07:47:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c00ad0e0 0x7f98c0149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 0x7f98c0156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0160310 0x7f98c00741c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98c006f220 con 0x7f98c015d760 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98c006f0a0 con 0x7f98c0160310 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98c00b4c20 con 0x7f98c00ad0e0 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 0x7f98c0156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 0x7f98c0156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58482/0 (socket says 172.21.15.88:58482) 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98bf400640 1 -- 172.21.15.88:0/3672223672 learned_addr learned my addr 172.21.15.88:0/3672223672 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98c4e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0160310 0x7f98c00741c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.382+0000 7f98bfe00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c00ad0e0 0x7f98c0149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98bf400640 1 -- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c00ad0e0 msgr2=0x7f98c0149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98bf400640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c00ad0e0 0x7f98c0149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98bf400640 1 -- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0160310 msgr2=0x7f98c00741c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98bf400640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0160310 0x7f98c00741c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98bf400640 1 -- 172.21.15.88:0/3672223672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c0156b60 con 0x7f98c015d760 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98bf400640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 0x7f98c0156620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009fd0 tx=0x7f98ac00ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac00cc60 con 0x7f98c015d760 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c01f9110 con 0x7f98c015d760 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98ac00ce00 con 0x7f98c015d760 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98ac004280 con 0x7f98c015d760 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.383+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c01f96d0 con 0x7f98c015d760 2024-09-21T07:47:01.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.385+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c0149dd0 con 0x7f98c015d760 2024-09-21T07:47:01.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.386+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f98ac0035f0 con 0x7f98c015d760 2024-09-21T07:47:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.387+0000 7f98b7e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f98a0078b90 0x7f98a007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.387+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f98ac09b7f0 con 0x7f98c015d760 2024-09-21T07:47:01.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.387+0000 7f98bfe00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f98a0078b90 0x7f98a007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.388+0000 7f98bfe00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f98a0078b90 0x7f98a007b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98b0033000 tx=0x7f98b0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.391+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98ac0634c0 con 0x7f98c015d760 2024-09-21T07:47:01.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.572+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f98c0156d90 con 0x7f98a0078b90 2024-09-21T07:47:01.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.577+0000 7f98b7e00640 1 -- 172.21.15.88:0/3672223672 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+249152 (secure 0 0 0) 0x7f98c0156d90 con 0x7f98a0078b90 2024-09-21T07:47:01.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:01.583 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f98a0078b90 msgr2=0x7f98a007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f98a0078b90 0x7f98a007b0a0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f98b0033000 tx=0x7f98b0004730 comp rx=0 tx=0).stop 2024-09-21T07:47:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 msgr2=0x7f98c0156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 0x7f98c0156620 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009fd0 tx=0x7f98ac00ec20 comp rx=0 tx=0).stop 2024-09-21T07:47:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 shutdown_connections 2024-09-21T07:47:01.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f98a0078b90 0x7f98a007b0a0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98c0160310 0x7f98c00741c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98c015d760 0x7f98c0156620 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 --2- 172.21.15.88:0/3672223672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98c00ad0e0 0x7f98c0149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 >> 172.21.15.88:0/3672223672 conn(0x7f98c006d730 msgr2=0x7f98c00b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 shutdown_connections 2024-09-21T07:47:01.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.581+0000 7f98c6e00640 1 -- 172.21.15.88:0/3672223672 wait complete. 2024-09-21T07:47:01.604 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:46:59.866867+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592002,"num_objects":16,"num_object_clones":0,"num_object_copies":32,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":16,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":131,"num_write_kb":1632,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":97,"ondisk_log_size":97,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":105,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220348,"kb_used_data":4096,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754372,"statfs":{"total":773094113280,"available":772868476928,"internally_reserved":0,"allocated":4194304,"data_stored":2319819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.679739"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196875+0000","last_change":"2024-09-21T07:46:55.196875+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196875+0000","last_undegraded":"2024-09-21T07:46:55.196875+0000","last_fullsized":"2024-09-21T07:46:55.196875+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228903+0000","last_change":"2024-09-21T07:46:56.229990+0000","last_active":"2024-09-21T07:46:57.228903+0000","last_peered":"2024-09-21T07:46:57.228903+0000","last_clean":"2024-09-21T07:46:57.228903+0000","last_became_active":"2024-09-21T07:46:56.229655+0000","last_became_peered":"2024-09-21T07:46:56.229655+0000","last_unstale":"2024-09-21T07:46:57.228903+0000","last_undegraded":"2024-09-21T07:46:57.228903+0000","last_fullsized":"2024-09-21T07:46:57.228903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.209740+0000","last_change":"2024-09-21T07:46:55.209740+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.209740+0000","last_undegraded":"2024-09-21T07:46:55.209740+0000","last_fullsized":"2024-09-21T07:46:55.209740+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219416+0000","last_change":"2024-09-21T07:46:56.235757+0000","last_active":"2024-09-21T07:46:57.219416+0000","last_peered":"2024-09-21T07:46:57.219416+0000","last_clean":"2024-09-21T07:46:57.219416+0000","last_became_active":"2024-09-21T07:46:56.228680+0000","last_became_peered":"2024-09-21T07:46:56.228680+0000","last_unstale":"2024-09-21T07:46:57.219416+0000","last_undegraded":"2024-09-21T07:46:57.219416+0000","last_fullsized":"2024-09-21T07:46:57.219416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228641+0000","last_change":"2024-09-21T07:46:56.229963+0000","last_active":"2024-09-21T07:46:57.228641+0000","last_peered":"2024-09-21T07:46:57.228641+0000","last_clean":"2024-09-21T07:46:57.228641+0000","last_became_active":"2024-09-21T07:46:56.229617+0000","last_became_peered":"2024-09-21T07:46:56.229617+0000","last_unstale":"2024-09-21T07:46:57.228641+0000","last_undegraded":"2024-09-21T07:46:57.228641+0000","last_fullsized":"2024-09-21T07:46:57.228641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219492+0000","last_change":"2024-09-21T07:46:56.229684+0000","last_active":"2024-09-21T07:46:57.219492+0000","last_peered":"2024-09-21T07:46:57.219492+0000","last_clean":"2024-09-21T07:46:57.219492+0000","last_became_active":"2024-09-21T07:46:56.229339+0000","last_became_peered":"2024-09-21T07:46:56.229339+0000","last_unstale":"2024-09-21T07:46:57.219492+0000","last_undegraded":"2024-09-21T07:46:57.219492+0000","last_fullsized":"2024-09-21T07:46:57.219492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:21.621424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198590+0000","last_change":"2024-09-21T07:46:55.198590+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198590+0000","last_undegraded":"2024-09-21T07:46:55.198590+0000","last_fullsized":"2024-09-21T07:46:55.198590+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195067+0000","last_change":"2024-09-21T07:46:55.195067+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195067+0000","last_undegraded":"2024-09-21T07:46:55.195067+0000","last_fullsized":"2024-09-21T07:46:55.195067+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.210023+0000","last_change":"2024-09-21T07:46:55.210023+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.210023+0000","last_undegraded":"2024-09-21T07:46:55.210023+0000","last_fullsized":"2024-09-21T07:46:55.210023+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220083+0000","last_change":"2024-09-21T07:46:56.231062+0000","last_active":"2024-09-21T07:46:57.220083+0000","last_peered":"2024-09-21T07:46:57.220083+0000","last_clean":"2024-09-21T07:46:57.220083+0000","last_became_active":"2024-09-21T07:46:56.229427+0000","last_became_peered":"2024-09-21T07:46:56.229427+0000","last_unstale":"2024-09-21T07:46:57.220083+0000","last_undegraded":"2024-09-21T07:46:57.220083+0000","last_fullsized":"2024-09-21T07:46:57.220083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:47.565734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198313+0000","last_change":"2024-09-21T07:46:55.198313+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198313+0000","last_undegraded":"2024-09-21T07:46:55.198313+0000","last_fullsized":"2024-09-21T07:46:55.198313+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.199137+0000","last_change":"2024-09-21T07:46:55.199137+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.199137+0000","last_undegraded":"2024-09-21T07:46:55.199137+0000","last_fullsized":"2024-09-21T07:46:55.199137+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707947+0000","last_change":"2024-09-21T07:46:56.237624+0000","last_active":"2024-09-21T07:46:57.707947+0000","last_peered":"2024-09-21T07:46:57.707947+0000","last_clean":"2024-09-21T07:46:57.707947+0000","last_became_active":"2024-09-21T07:46:56.237037+0000","last_became_peered":"2024-09-21T07:46:56.237037+0000","last_unstale":"2024-09-21T07:46:57.707947+0000","last_undegraded":"2024-09-21T07:46:57.707947+0000","last_fullsized":"2024-09-21T07:46:57.707947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:43.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219004+0000","last_change":"2024-09-21T07:46:56.239271+0000","last_active":"2024-09-21T07:46:57.219004+0000","last_peered":"2024-09-21T07:46:57.219004+0000","last_clean":"2024-09-21T07:46:57.219004+0000","last_became_active":"2024-09-21T07:46:56.238936+0000","last_became_peered":"2024-09-21T07:46:56.238936+0000","last_unstale":"2024-09-21T07:46:57.219004+0000","last_undegraded":"2024-09-21T07:46:57.219004+0000","last_fullsized":"2024-09-21T07:46:57.219004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194649+0000","last_change":"2024-09-21T07:46:55.194649+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194649+0000","last_undegraded":"2024-09-21T07:46:55.194649+0000","last_fullsized":"2024-09-21T07:46:55.194649+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194072+0000","last_change":"2024-09-21T07:46:55.194072+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194072+0000","last_undegraded":"2024-09-21T07:46:55.194072+0000","last_fullsized":"2024-09-21T07:46:55.194072+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198999+0000","last_change":"2024-09-21T07:46:55.198999+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198999+0000","last_undegraded":"2024-09-21T07:46:55.198999+0000","last_fullsized":"2024-09-21T07:46:55.198999+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220865+0000","last_change":"2024-09-21T07:46:56.224534+0000","last_active":"2024-09-21T07:46:57.220865+0000","last_peered":"2024-09-21T07:46:57.220865+0000","last_clean":"2024-09-21T07:46:57.220865+0000","last_became_active":"2024-09-21T07:46:56.224167+0000","last_became_peered":"2024-09-21T07:46:56.224167+0000","last_unstale":"2024-09-21T07:46:57.220865+0000","last_undegraded":"2024-09-21T07:46:57.220865+0000","last_fullsized":"2024-09-21T07:46:57.220865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:18.755342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220883+0000","last_change":"2024-09-21T07:46:56.228375+0000","last_active":"2024-09-21T07:46:57.220883+0000","last_peered":"2024-09-21T07:46:57.220883+0000","last_clean":"2024-09-21T07:46:57.220883+0000","last_became_active":"2024-09-21T07:46:56.225793+0000","last_became_peered":"2024-09-21T07:46:56.225793+0000","last_unstale":"2024-09-21T07:46:57.220883+0000","last_undegraded":"2024-09-21T07:46:57.220883+0000","last_fullsized":"2024-09-21T07:46:57.220883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:46.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708077+0000","last_change":"2024-09-21T07:46:56.226928+0000","last_active":"2024-09-21T07:46:57.708077+0000","last_peered":"2024-09-21T07:46:57.708077+0000","last_clean":"2024-09-21T07:46:57.708077+0000","last_became_active":"2024-09-21T07:46:56.226560+0000","last_became_peered":"2024-09-21T07:46:56.226560+0000","last_unstale":"2024-09-21T07:46:57.708077+0000","last_undegraded":"2024-09-21T07:46:57.708077+0000","last_fullsized":"2024-09-21T07:46:57.708077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.197713+0000","last_change":"2024-09-21T07:46:55.197713+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.197713+0000","last_undegraded":"2024-09-21T07:46:55.197713+0000","last_fullsized":"2024-09-21T07:46:55.197713+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195512+0000","last_change":"2024-09-21T07:46:55.195512+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195512+0000","last_undegraded":"2024-09-21T07:46:55.195512+0000","last_fullsized":"2024-09-21T07:46:55.195512+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707640+0000","last_change":"2024-09-21T07:46:56.227950+0000","last_active":"2024-09-21T07:46:57.707640+0000","last_peered":"2024-09-21T07:46:57.707640+0000","last_clean":"2024-09-21T07:46:57.707640+0000","last_became_active":"2024-09-21T07:46:56.227647+0000","last_became_peered":"2024-09-21T07:46:56.227647+0000","last_unstale":"2024-09-21T07:46:57.707640+0000","last_undegraded":"2024-09-21T07:46:57.707640+0000","last_fullsized":"2024-09-21T07:46:57.707640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:08.480356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194574+0000","last_change":"2024-09-21T07:46:55.194574+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194574+0000","last_undegraded":"2024-09-21T07:46:55.194574+0000","last_fullsized":"2024-09-21T07:46:55.194574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219395+0000","last_change":"2024-09-21T07:46:56.217732+0000","last_active":"2024-09-21T07:46:57.219395+0000","last_peered":"2024-09-21T07:46:57.219395+0000","last_clean":"2024-09-21T07:46:57.219395+0000","last_became_active":"2024-09-21T07:46:56.217410+0000","last_became_peered":"2024-09-21T07:46:56.217410+0000","last_unstale":"2024-09-21T07:46:57.219395+0000","last_undegraded":"2024-09-21T07:46:57.219395+0000","last_fullsized":"2024-09-21T07:46:57.219395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219810+0000","last_change":"2024-09-21T07:46:56.220112+0000","last_active":"2024-09-21T07:46:57.219810+0000","last_peered":"2024-09-21T07:46:57.219810+0000","last_clean":"2024-09-21T07:46:57.219810+0000","last_became_active":"2024-09-21T07:46:56.217329+0000","last_became_peered":"2024-09-21T07:46:56.217329+0000","last_unstale":"2024-09-21T07:46:57.219810+0000","last_undegraded":"2024-09-21T07:46:57.219810+0000","last_fullsized":"2024-09-21T07:46:57.219810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:04.144314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228789+0000","last_change":"2024-09-21T07:46:56.223494+0000","last_active":"2024-09-21T07:46:57.228789+0000","last_peered":"2024-09-21T07:46:57.228789+0000","last_clean":"2024-09-21T07:46:57.228789+0000","last_became_active":"2024-09-21T07:46:56.223149+0000","last_became_peered":"2024-09-21T07:46:56.223149+0000","last_unstale":"2024-09-21T07:46:57.228789+0000","last_undegraded":"2024-09-21T07:46:57.228789+0000","last_fullsized":"2024-09-21T07:46:57.228789+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198069+0000","last_change":"2024-09-21T07:46:55.198069+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198069+0000","last_undegraded":"2024-09-21T07:46:55.198069+0000","last_fullsized":"2024-09-21T07:46:55.198069+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.200475+0000","last_change":"2024-09-21T07:46:55.200475+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.200475+0000","last_undegraded":"2024-09-21T07:46:55.200475+0000","last_fullsized":"2024-09-21T07:46:55.200475+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196887+0000","last_change":"2024-09-21T07:46:55.196887+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196887+0000","last_undegraded":"2024-09-21T07:46:55.196887+0000","last_fullsized":"2024-09-21T07:46:55.196887+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.192541+0000","last_change":"2024-09-21T07:46:55.192541+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.192541+0000","last_undegraded":"2024-09-21T07:46:55.192541+0000","last_fullsized":"2024-09-21T07:46:55.192541+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198692+0000","last_change":"2024-09-21T07:46:55.198692+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198692+0000","last_undegraded":"2024-09-21T07:46:55.198692+0000","last_fullsized":"2024-09-21T07:46:55.198692+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708018+0000","last_change":"2024-09-21T07:46:56.231049+0000","last_active":"2024-09-21T07:46:57.708018+0000","last_peered":"2024-09-21T07:46:57.708018+0000","last_clean":"2024-09-21T07:46:57.708018+0000","last_became_active":"2024-09-21T07:46:56.230778+0000","last_became_peered":"2024-09-21T07:46:56.230778+0000","last_unstale":"2024-09-21T07:46:57.708018+0000","last_undegraded":"2024-09-21T07:46:57.708018+0000","last_fullsized":"2024-09-21T07:46:57.708018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.211797+0000","last_change":"2024-09-21T07:46:55.204066+0000","last_active":"2024-09-21T07:46:55.211797+0000","last_peered":"2024-09-21T07:46:55.211797+0000","last_clean":"2024-09-21T07:46:55.211797+0000","last_became_active":"2024-09-21T07:46:55.203753+0000","last_became_peered":"2024-09-21T07:46:55.203753+0000","last_unstale":"2024-09-21T07:46:55.211797+0000","last_undegraded":"2024-09-21T07:46:55.211797+0000","last_fullsized":"2024-09-21T07:46:55.211797+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:20.693634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196356+0000","last_change":"2024-09-21T07:46:55.196356+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196356+0000","last_undegraded":"2024-09-21T07:46:55.196356+0000","last_fullsized":"2024-09-21T07:46:55.196356+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219963+0000","last_change":"2024-09-21T07:46:55.212364+0000","last_active":"2024-09-21T07:46:57.219963+0000","last_peered":"2024-09-21T07:46:57.219963+0000","last_clean":"2024-09-21T07:46:57.219963+0000","last_became_active":"2024-09-21T07:46:55.211980+0000","last_became_peered":"2024-09-21T07:46:55.211980+0000","last_unstale":"2024-09-21T07:46:57.219963+0000","last_undegraded":"2024-09-21T07:46:57.219963+0000","last_fullsized":"2024-09-21T07:46:57.219963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:40.287704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.197598+0000","last_change":"2024-09-21T07:46:55.197598+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.197598+0000","last_undegraded":"2024-09-21T07:46:55.197598+0000","last_fullsized":"2024-09-21T07:46:55.197598+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221045+0000","last_change":"2024-09-21T07:46:55.216737+0000","last_active":"2024-09-21T07:46:57.221045+0000","last_peered":"2024-09-21T07:46:57.221045+0000","last_clean":"2024-09-21T07:46:57.221045+0000","last_became_active":"2024-09-21T07:46:55.216402+0000","last_became_peered":"2024-09-21T07:46:55.216402+0000","last_unstale":"2024-09-21T07:46:57.221045+0000","last_undegraded":"2024-09-21T07:46:57.221045+0000","last_fullsized":"2024-09-21T07:46:57.221045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:40.966275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219312+0000","last_change":"2024-09-21T07:46:56.237682+0000","last_active":"2024-09-21T07:46:57.219312+0000","last_peered":"2024-09-21T07:46:57.219312+0000","last_clean":"2024-09-21T07:46:57.219312+0000","last_became_active":"2024-09-21T07:46:56.237340+0000","last_became_peered":"2024-09-21T07:46:56.237340+0000","last_unstale":"2024-09-21T07:46:57.219312+0000","last_undegraded":"2024-09-21T07:46:57.219312+0000","last_fullsized":"2024-09-21T07:46:57.219312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219765+0000","last_change":"2024-09-21T07:46:55.211939+0000","last_active":"2024-09-21T07:46:57.219765+0000","last_peered":"2024-09-21T07:46:57.219765+0000","last_clean":"2024-09-21T07:46:57.219765+0000","last_became_active":"2024-09-21T07:46:55.211170+0000","last_became_peered":"2024-09-21T07:46:55.211170+0000","last_unstale":"2024-09-21T07:46:57.219765+0000","last_undegraded":"2024-09-21T07:46:57.219765+0000","last_fullsized":"2024-09-21T07:46:57.219765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.194935+0000","last_change":"2024-09-21T07:46:55.194935+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.194935+0000","last_undegraded":"2024-09-21T07:46:55.194935+0000","last_fullsized":"2024-09-21T07:46:55.194935+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.210107+0000","last_change":"2024-09-21T07:46:55.204599+0000","last_active":"2024-09-21T07:46:55.210107+0000","last_peered":"2024-09-21T07:46:55.210107+0000","last_clean":"2024-09-21T07:46:55.210107+0000","last_became_active":"2024-09-21T07:46:55.204186+0000","last_became_peered":"2024-09-21T07:46:55.204186+0000","last_unstale":"2024-09-21T07:46:55.210107+0000","last_undegraded":"2024-09-21T07:46:55.210107+0000","last_fullsized":"2024-09-21T07:46:55.210107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:36.808640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193937+0000","last_change":"2024-09-21T07:46:55.193937+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193937+0000","last_undegraded":"2024-09-21T07:46:55.193937+0000","last_fullsized":"2024-09-21T07:46:55.193937+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.210087+0000","last_change":"2024-09-21T07:46:55.205089+0000","last_active":"2024-09-21T07:46:55.210087+0000","last_peered":"2024-09-21T07:46:55.210087+0000","last_clean":"2024-09-21T07:46:55.210087+0000","last_became_active":"2024-09-21T07:46:55.204642+0000","last_became_peered":"2024-09-21T07:46:55.204642+0000","last_unstale":"2024-09-21T07:46:55.210087+0000","last_undegraded":"2024-09-21T07:46:55.210087+0000","last_fullsized":"2024-09-21T07:46:55.210087+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:45:12.864094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193798+0000","last_change":"2024-09-21T07:46:55.193798+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193798+0000","last_undegraded":"2024-09-21T07:46:55.193798+0000","last_fullsized":"2024-09-21T07:46:55.193798+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218717+0000","last_change":"2024-09-21T07:46:55.218472+0000","last_active":"2024-09-21T07:46:57.218717+0000","last_peered":"2024-09-21T07:46:57.218717+0000","last_clean":"2024-09-21T07:46:57.218717+0000","last_became_active":"2024-09-21T07:46:55.218044+0000","last_became_peered":"2024-09-21T07:46:55.218044+0000","last_unstale":"2024-09-21T07:46:57.218717+0000","last_undegraded":"2024-09-21T07:46:57.218717+0000","last_fullsized":"2024-09-21T07:46:57.218717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:18.860660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218669+0000","last_change":"2024-09-21T07:46:56.221089+0000","last_active":"2024-09-21T07:46:57.218669+0000","last_peered":"2024-09-21T07:46:57.218669+0000","last_clean":"2024-09-21T07:46:57.218669+0000","last_became_active":"2024-09-21T07:46:56.220746+0000","last_became_peered":"2024-09-21T07:46:56.220746+0000","last_unstale":"2024-09-21T07:46:57.218669+0000","last_undegraded":"2024-09-21T07:46:57.218669+0000","last_fullsized":"2024-09-21T07:46:57.218669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.213270+0000","last_change":"2024-09-21T07:46:55.206317+0000","last_active":"2024-09-21T07:46:55.213270+0000","last_peered":"2024-09-21T07:46:55.213270+0000","last_clean":"2024-09-21T07:46:55.213270+0000","last_became_active":"2024-09-21T07:46:55.206000+0000","last_became_peered":"2024-09-21T07:46:55.206000+0000","last_unstale":"2024-09-21T07:46:55.213270+0000","last_undegraded":"2024-09-21T07:46:55.213270+0000","last_fullsized":"2024-09-21T07:46:55.213270+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:35.469725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198239+0000","last_change":"2024-09-21T07:46:55.198239+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198239+0000","last_undegraded":"2024-09-21T07:46:55.198239+0000","last_fullsized":"2024-09-21T07:46:55.198239+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228976+0000","last_change":"2024-09-21T07:46:55.213497+0000","last_active":"2024-09-21T07:46:57.228976+0000","last_peered":"2024-09-21T07:46:57.228976+0000","last_clean":"2024-09-21T07:46:57.228976+0000","last_became_active":"2024-09-21T07:46:55.213173+0000","last_became_peered":"2024-09-21T07:46:55.213173+0000","last_unstale":"2024-09-21T07:46:57.228976+0000","last_undegraded":"2024-09-21T07:46:57.228976+0000","last_fullsized":"2024-09-21T07:46:57.228976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.908077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219686+0000","last_change":"2024-09-21T07:46:56.229383+0000","last_active":"2024-09-21T07:46:57.219686+0000","last_peered":"2024-09-21T07:46:57.219686+0000","last_clean":"2024-09-21T07:46:57.219686+0000","last_became_active":"2024-09-21T07:46:56.229012+0000","last_became_peered":"2024-09-21T07:46:56.229012+0000","last_unstale":"2024-09-21T07:46:57.219686+0000","last_undegraded":"2024-09-21T07:46:57.219686+0000","last_fullsized":"2024-09-21T07:46:57.219686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:56.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228698+0000","last_change":"2024-09-21T07:46:55.212681+0000","last_active":"2024-09-21T07:46:57.228698+0000","last_peered":"2024-09-21T07:46:57.228698+0000","last_clean":"2024-09-21T07:46:57.228698+0000","last_became_active":"2024-09-21T07:46:55.211907+0000","last_became_peered":"2024-09-21T07:46:55.211907+0000","last_unstale":"2024-09-21T07:46:57.228698+0000","last_undegraded":"2024-09-21T07:46:57.228698+0000","last_fullsized":"2024-09-21T07:46:57.228698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:30.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219680+0000","last_change":"2024-09-21T07:46:56.224792+0000","last_active":"2024-09-21T07:46:57.219680+0000","last_peered":"2024-09-21T07:46:57.219680+0000","last_clean":"2024-09-21T07:46:57.219680+0000","last_became_active":"2024-09-21T07:46:56.224554+0000","last_became_peered":"2024-09-21T07:46:56.224554+0000","last_unstale":"2024-09-21T07:46:57.219680+0000","last_undegraded":"2024-09-21T07:46:57.219680+0000","last_fullsized":"2024-09-21T07:46:57.219680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:31.069660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.214576+0000","last_change":"2024-09-21T07:46:55.207287+0000","last_active":"2024-09-21T07:46:55.214576+0000","last_peered":"2024-09-21T07:46:55.214576+0000","last_clean":"2024-09-21T07:46:55.214576+0000","last_became_active":"2024-09-21T07:46:55.207008+0000","last_became_peered":"2024-09-21T07:46:55.207008+0000","last_unstale":"2024-09-21T07:46:55.214576+0000","last_undegraded":"2024-09-21T07:46:55.214576+0000","last_fullsized":"2024-09-21T07:46:55.214576+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:31.482910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196425+0000","last_change":"2024-09-21T07:46:55.196425+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196425+0000","last_undegraded":"2024-09-21T07:46:55.196425+0000","last_fullsized":"2024-09-21T07:46:55.196425+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707883+0000","last_change":"2024-09-21T07:46:55.209727+0000","last_active":"2024-09-21T07:46:57.707883+0000","last_peered":"2024-09-21T07:46:57.707883+0000","last_clean":"2024-09-21T07:46:57.707883+0000","last_became_active":"2024-09-21T07:46:55.209437+0000","last_became_peered":"2024-09-21T07:46:55.209437+0000","last_unstale":"2024-09-21T07:46:57.707883+0000","last_undegraded":"2024-09-21T07:46:57.707883+0000","last_fullsized":"2024-09-21T07:46:57.707883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707613+0000","last_change":"2024-09-21T07:46:56.234403+0000","last_active":"2024-09-21T07:46:57.707613+0000","last_peered":"2024-09-21T07:46:57.707613+0000","last_clean":"2024-09-21T07:46:57.707613+0000","last_became_active":"2024-09-21T07:46:56.233998+0000","last_became_peered":"2024-09-21T07:46:56.233998+0000","last_unstale":"2024-09-21T07:46:57.707613+0000","last_undegraded":"2024-09-21T07:46:57.707613+0000","last_fullsized":"2024-09-21T07:46:57.707613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:52.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217390+0000","last_change":"2024-09-21T07:46:55.212435+0000","last_active":"2024-09-21T07:46:55.217390+0000","last_peered":"2024-09-21T07:46:55.217390+0000","last_clean":"2024-09-21T07:46:55.217390+0000","last_became_active":"2024-09-21T07:46:55.212093+0000","last_became_peered":"2024-09-21T07:46:55.212093+0000","last_unstale":"2024-09-21T07:46:55.217390+0000","last_undegraded":"2024-09-21T07:46:55.217390+0000","last_fullsized":"2024-09-21T07:46:55.217390+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:32:44.281725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219594+0000","last_change":"2024-09-21T07:46:56.236480+0000","last_active":"2024-09-21T07:46:57.219594+0000","last_peered":"2024-09-21T07:46:57.219594+0000","last_clean":"2024-09-21T07:46:57.219594+0000","last_became_active":"2024-09-21T07:46:56.236111+0000","last_became_peered":"2024-09-21T07:46:56.236111+0000","last_unstale":"2024-09-21T07:46:57.219594+0000","last_undegraded":"2024-09-21T07:46:57.219594+0000","last_fullsized":"2024-09-21T07:46:57.219594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219132+0000","last_change":"2024-09-21T07:46:55.211160+0000","last_active":"2024-09-21T07:46:57.219132+0000","last_peered":"2024-09-21T07:46:57.219132+0000","last_clean":"2024-09-21T07:46:57.219132+0000","last_became_active":"2024-09-21T07:46:55.210797+0000","last_became_peered":"2024-09-21T07:46:55.210797+0000","last_unstale":"2024-09-21T07:46:57.219132+0000","last_undegraded":"2024-09-21T07:46:57.219132+0000","last_fullsized":"2024-09-21T07:46:57.219132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:33.882042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.192745+0000","last_change":"2024-09-21T07:46:55.192745+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.192745+0000","last_undegraded":"2024-09-21T07:46:55.192745+0000","last_fullsized":"2024-09-21T07:46:55.192745+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218445+0000","last_change":"2024-09-21T07:46:55.205503+0000","last_active":"2024-09-21T07:46:57.218445+0000","last_peered":"2024-09-21T07:46:57.218445+0000","last_clean":"2024-09-21T07:46:57.218445+0000","last_became_active":"2024-09-21T07:46:55.204821+0000","last_became_peered":"2024-09-21T07:46:55.204821+0000","last_unstale":"2024-09-21T07:46:57.218445+0000","last_undegraded":"2024-09-21T07:46:57.218445+0000","last_fullsized":"2024-09-21T07:46:57.218445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195717+0000","last_change":"2024-09-21T07:46:55.195717+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195717+0000","last_undegraded":"2024-09-21T07:46:55.195717+0000","last_fullsized":"2024-09-21T07:46:55.195717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218410+0000","last_change":"2024-09-21T07:46:55.216736+0000","last_active":"2024-09-21T07:46:57.218410+0000","last_peered":"2024-09-21T07:46:57.218410+0000","last_clean":"2024-09-21T07:46:57.218410+0000","last_became_active":"2024-09-21T07:46:55.216312+0000","last_became_peered":"2024-09-21T07:46:55.216312+0000","last_unstale":"2024-09-21T07:46:57.218410+0000","last_undegraded":"2024-09-21T07:46:57.218410+0000","last_fullsized":"2024-09-21T07:46:57.218410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.195061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.193288+0000","last_change":"2024-09-21T07:46:55.193288+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.193288+0000","last_undegraded":"2024-09-21T07:46:55.193288+0000","last_fullsized":"2024-09-21T07:46:55.193288+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220824+0000","last_change":"2024-09-21T07:46:55.218143+0000","last_active":"2024-09-21T07:46:57.220824+0000","last_peered":"2024-09-21T07:46:57.220824+0000","last_clean":"2024-09-21T07:46:57.220824+0000","last_became_active":"2024-09-21T07:46:55.217800+0000","last_became_peered":"2024-09-21T07:46:55.217800+0000","last_unstale":"2024-09-21T07:46:57.220824+0000","last_undegraded":"2024-09-21T07:46:57.220824+0000","last_fullsized":"2024-09-21T07:46:57.220824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:57.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.205097+0000","last_change":"2024-09-21T07:46:55.205097+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.205097+0000","last_undegraded":"2024-09-21T07:46:55.205097+0000","last_fullsized":"2024-09-21T07:46:55.205097+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217443+0000","last_change":"2024-09-21T07:46:55.212733+0000","last_active":"2024-09-21T07:46:55.217443+0000","last_peered":"2024-09-21T07:46:55.217443+0000","last_clean":"2024-09-21T07:46:55.217443+0000","last_became_active":"2024-09-21T07:46:55.212478+0000","last_became_peered":"2024-09-21T07:46:55.212478+0000","last_unstale":"2024-09-21T07:46:55.217443+0000","last_undegraded":"2024-09-21T07:46:55.217443+0000","last_fullsized":"2024-09-21T07:46:55.217443+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:42.222888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228788+0000","last_change":"2024-09-21T07:46:56.223493+0000","last_active":"2024-09-21T07:46:57.228788+0000","last_peered":"2024-09-21T07:46:57.228788+0000","last_clean":"2024-09-21T07:46:57.228788+0000","last_became_active":"2024-09-21T07:46:56.223060+0000","last_became_peered":"2024-09-21T07:46:56.223060+0000","last_unstale":"2024-09-21T07:46:57.228788+0000","last_undegraded":"2024-09-21T07:46:57.228788+0000","last_fullsized":"2024-09-21T07:46:57.228788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217612+0000","last_change":"2024-09-21T07:46:55.213737+0000","last_active":"2024-09-21T07:46:55.217612+0000","last_peered":"2024-09-21T07:46:55.217612+0000","last_clean":"2024-09-21T07:46:55.217612+0000","last_became_active":"2024-09-21T07:46:55.213452+0000","last_became_peered":"2024-09-21T07:46:55.213452+0000","last_unstale":"2024-09-21T07:46:55.217612+0000","last_undegraded":"2024-09-21T07:46:55.217612+0000","last_fullsized":"2024-09-21T07:46:55.217612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:10.689164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220927+0000","last_change":"2024-09-21T07:46:56.232119+0000","last_active":"2024-09-21T07:46:57.220927+0000","last_peered":"2024-09-21T07:46:57.220927+0000","last_clean":"2024-09-21T07:46:57.220927+0000","last_became_active":"2024-09-21T07:46:56.231764+0000","last_became_peered":"2024-09-21T07:46:56.231764+0000","last_unstale":"2024-09-21T07:46:57.220927+0000","last_undegraded":"2024-09-21T07:46:57.220927+0000","last_fullsized":"2024-09-21T07:46:57.220927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:35.746093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.208347+0000","last_change":"2024-09-21T07:46:55.206900+0000","last_active":"2024-09-21T07:46:55.208347+0000","last_peered":"2024-09-21T07:46:55.208347+0000","last_clean":"2024-09-21T07:46:55.208347+0000","last_became_active":"2024-09-21T07:46:55.206441+0000","last_became_peered":"2024-09-21T07:46:55.206441+0000","last_unstale":"2024-09-21T07:46:55.208347+0000","last_undegraded":"2024-09-21T07:46:55.208347+0000","last_fullsized":"2024-09-21T07:46:55.208347+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:22.920219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707559+0000","last_change":"2024-09-21T07:46:56.233293+0000","last_active":"2024-09-21T07:46:57.707559+0000","last_peered":"2024-09-21T07:46:57.707559+0000","last_clean":"2024-09-21T07:46:57.707559+0000","last_became_active":"2024-09-21T07:46:56.232951+0000","last_became_peered":"2024-09-21T07:46:56.232951+0000","last_unstale":"2024-09-21T07:46:57.707559+0000","last_undegraded":"2024-09-21T07:46:57.707559+0000","last_fullsized":"2024-09-21T07:46:57.707559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:58.570223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228732+0000","last_change":"2024-09-21T07:46:55.206362+0000","last_active":"2024-09-21T07:46:57.228732+0000","last_peered":"2024-09-21T07:46:57.228732+0000","last_clean":"2024-09-21T07:46:57.228732+0000","last_became_active":"2024-09-21T07:46:55.206044+0000","last_became_peered":"2024-09-21T07:46:55.206044+0000","last_unstale":"2024-09-21T07:46:57.228732+0000","last_undegraded":"2024-09-21T07:46:57.228732+0000","last_fullsized":"2024-09-21T07:46:57.228732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:20.127719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228770+0000","last_change":"2024-09-21T07:46:56.221448+0000","last_active":"2024-09-21T07:46:57.228770+0000","last_peered":"2024-09-21T07:46:57.228770+0000","last_clean":"2024-09-21T07:46:57.228770+0000","last_became_active":"2024-09-21T07:46:56.221116+0000","last_became_peered":"2024-09-21T07:46:56.221116+0000","last_unstale":"2024-09-21T07:46:57.228770+0000","last_undegraded":"2024-09-21T07:46:57.228770+0000","last_fullsized":"2024-09-21T07:46:57.228770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217541+0000","last_change":"2024-09-21T07:46:55.213426+0000","last_active":"2024-09-21T07:46:55.217541+0000","last_peered":"2024-09-21T07:46:55.217541+0000","last_clean":"2024-09-21T07:46:55.217541+0000","last_became_active":"2024-09-21T07:46:55.211244+0000","last_became_peered":"2024-09-21T07:46:55.211244+0000","last_unstale":"2024-09-21T07:46:55.217541+0000","last_undegraded":"2024-09-21T07:46:55.217541+0000","last_fullsized":"2024-09-21T07:46:55.217541+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:04.764546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707540+0000","last_change":"2024-09-21T07:46:56.232335+0000","last_active":"2024-09-21T07:46:57.707540+0000","last_peered":"2024-09-21T07:46:57.707540+0000","last_clean":"2024-09-21T07:46:57.707540+0000","last_became_active":"2024-09-21T07:46:56.231953+0000","last_became_peered":"2024-09-21T07:46:56.231953+0000","last_unstale":"2024-09-21T07:46:57.707540+0000","last_undegraded":"2024-09-21T07:46:57.707540+0000","last_fullsized":"2024-09-21T07:46:57.707540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:04.369378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.219860+0000","last_change":"2024-09-21T07:46:55.214923+0000","last_active":"2024-09-21T07:46:55.219860+0000","last_peered":"2024-09-21T07:46:55.219860+0000","last_clean":"2024-09-21T07:46:55.219860+0000","last_became_active":"2024-09-21T07:46:55.214607+0000","last_became_peered":"2024-09-21T07:46:55.214607+0000","last_unstale":"2024-09-21T07:46:55.219860+0000","last_undegraded":"2024-09-21T07:46:55.219860+0000","last_fullsized":"2024-09-21T07:46:55.219860+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:39:00.922921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.195966+0000","last_change":"2024-09-21T07:46:55.195966+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.195966+0000","last_undegraded":"2024-09-21T07:46:55.195966+0000","last_fullsized":"2024-09-21T07:46:55.195966+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.215316+0000","last_change":"2024-09-21T07:46:55.208755+0000","last_active":"2024-09-21T07:46:55.215316+0000","last_peered":"2024-09-21T07:46:55.215316+0000","last_clean":"2024-09-21T07:46:55.215316+0000","last_became_active":"2024-09-21T07:46:55.207806+0000","last_became_peered":"2024-09-21T07:46:55.207806+0000","last_unstale":"2024-09-21T07:46:55.215316+0000","last_undegraded":"2024-09-21T07:46:55.215316+0000","last_fullsized":"2024-09-21T07:46:55.215316+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:36.972091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220825+0000","last_change":"2024-09-21T07:46:56.232337+0000","last_active":"2024-09-21T07:46:57.220825+0000","last_peered":"2024-09-21T07:46:57.220825+0000","last_clean":"2024-09-21T07:46:57.220825+0000","last_became_active":"2024-09-21T07:46:56.230029+0000","last_became_peered":"2024-09-21T07:46:56.230029+0000","last_unstale":"2024-09-21T07:46:57.220825+0000","last_undegraded":"2024-09-21T07:46:57.220825+0000","last_fullsized":"2024-09-21T07:46:57.220825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.727543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708019+0000","last_change":"2024-09-21T07:46:55.211058+0000","last_active":"2024-09-21T07:46:57.708019+0000","last_peered":"2024-09-21T07:46:57.708019+0000","last_clean":"2024-09-21T07:46:57.708019+0000","last_became_active":"2024-09-21T07:46:55.210625+0000","last_became_peered":"2024-09-21T07:46:55.210625+0000","last_unstale":"2024-09-21T07:46:57.708019+0000","last_undegraded":"2024-09-21T07:46:57.708019+0000","last_fullsized":"2024-09-21T07:46:57.708019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.191906+0000","last_change":"2024-09-21T07:46:55.191906+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.191906+0000","last_undegraded":"2024-09-21T07:46:55.191906+0000","last_fullsized":"2024-09-21T07:46:55.191906+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228909+0000","last_change":"2024-09-21T07:46:55.209507+0000","last_active":"2024-09-21T07:46:57.228909+0000","last_peered":"2024-09-21T07:46:57.228909+0000","last_clean":"2024-09-21T07:46:57.228909+0000","last_became_active":"2024-09-21T07:46:55.209003+0000","last_became_peered":"2024-09-21T07:46:55.209003+0000","last_unstale":"2024-09-21T07:46:57.228909+0000","last_undegraded":"2024-09-21T07:46:57.228909+0000","last_fullsized":"2024-09-21T07:46:57.228909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:25.605174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221110+0000","last_change":"2024-09-21T07:46:56.241826+0000","last_active":"2024-09-21T07:46:57.221110+0000","last_peered":"2024-09-21T07:46:57.221110+0000","last_clean":"2024-09-21T07:46:57.221110+0000","last_became_active":"2024-09-21T07:46:56.241465+0000","last_became_peered":"2024-09-21T07:46:56.241465+0000","last_unstale":"2024-09-21T07:46:57.221110+0000","last_undegraded":"2024-09-21T07:46:57.221110+0000","last_fullsized":"2024-09-21T07:46:57.221110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:13.411156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708039+0000","last_change":"2024-09-21T07:46:55.209001+0000","last_active":"2024-09-21T07:46:57.708039+0000","last_peered":"2024-09-21T07:46:57.708039+0000","last_clean":"2024-09-21T07:46:57.708039+0000","last_became_active":"2024-09-21T07:46:55.208596+0000","last_became_peered":"2024-09-21T07:46:55.208596+0000","last_unstale":"2024-09-21T07:46:57.708039+0000","last_undegraded":"2024-09-21T07:46:57.708039+0000","last_fullsized":"2024-09-21T07:46:57.708039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.196436+0000","last_change":"2024-09-21T07:46:55.196436+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.196436+0000","last_undegraded":"2024-09-21T07:46:55.196436+0000","last_fullsized":"2024-09-21T07:46:55.196436+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707914+0000","last_change":"2024-09-21T07:46:55.210484+0000","last_active":"2024-09-21T07:46:57.707914+0000","last_peered":"2024-09-21T07:46:57.707914+0000","last_clean":"2024-09-21T07:46:57.707914+0000","last_became_active":"2024-09-21T07:46:55.210140+0000","last_became_peered":"2024-09-21T07:46:55.210140+0000","last_unstale":"2024-09-21T07:46:57.707914+0000","last_undegraded":"2024-09-21T07:46:57.707914+0000","last_fullsized":"2024-09-21T07:46:57.707914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219755+0000","last_change":"2024-09-21T07:46:56.233200+0000","last_active":"2024-09-21T07:46:57.219755+0000","last_peered":"2024-09-21T07:46:57.219755+0000","last_clean":"2024-09-21T07:46:57.219755+0000","last_became_active":"2024-09-21T07:46:56.232818+0000","last_became_peered":"2024-09-21T07:46:56.232818+0000","last_unstale":"2024-09-21T07:46:57.219755+0000","last_undegraded":"2024-09-21T07:46:57.219755+0000","last_fullsized":"2024-09-21T07:46:57.219755+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.180535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707865+0000","last_change":"2024-09-21T07:46:55.215083+0000","last_active":"2024-09-21T07:46:57.707865+0000","last_peered":"2024-09-21T07:46:57.707865+0000","last_clean":"2024-09-21T07:46:57.707865+0000","last_became_active":"2024-09-21T07:46:55.214531+0000","last_became_peered":"2024-09-21T07:46:55.214531+0000","last_unstale":"2024-09-21T07:46:57.707865+0000","last_undegraded":"2024-09-21T07:46:57.707865+0000","last_fullsized":"2024-09-21T07:46:57.707865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218812+0000","last_change":"2024-09-21T07:46:56.220432+0000","last_active":"2024-09-21T07:46:57.218812+0000","last_peered":"2024-09-21T07:46:57.218812+0000","last_clean":"2024-09-21T07:46:57.218812+0000","last_became_active":"2024-09-21T07:46:56.219527+0000","last_became_peered":"2024-09-21T07:46:56.219527+0000","last_unstale":"2024-09-21T07:46:57.218812+0000","last_undegraded":"2024-09-21T07:46:57.218812+0000","last_fullsized":"2024-09-21T07:46:57.218812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:30.402592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.217461+0000","last_change":"2024-09-21T07:46:55.212708+0000","last_active":"2024-09-21T07:46:55.217461+0000","last_peered":"2024-09-21T07:46:55.217461+0000","last_clean":"2024-09-21T07:46:55.217461+0000","last_became_active":"2024-09-21T07:46:55.212115+0000","last_became_peered":"2024-09-21T07:46:55.212115+0000","last_unstale":"2024-09-21T07:46:55.217461+0000","last_undegraded":"2024-09-21T07:46:55.217461+0000","last_fullsized":"2024-09-21T07:46:55.217461+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:26.943301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.191831+0000","last_change":"2024-09-21T07:46:55.191831+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.191831+0000","last_undegraded":"2024-09-21T07:46:55.191831+0000","last_fullsized":"2024-09-21T07:46:55.191831+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219828+0000","last_change":"2024-09-21T07:46:55.214726+0000","last_active":"2024-09-21T07:46:57.219828+0000","last_peered":"2024-09-21T07:46:57.219828+0000","last_clean":"2024-09-21T07:46:57.219828+0000","last_became_active":"2024-09-21T07:46:55.214331+0000","last_became_peered":"2024-09-21T07:46:55.214331+0000","last_unstale":"2024-09-21T07:46:57.219828+0000","last_undegraded":"2024-09-21T07:46:57.219828+0000","last_fullsized":"2024-09-21T07:46:57.219828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":1,"reported_epoch":33,"state":"creating+peering","last_fresh":"2024-09-21T07:46:55.198188+0000","last_change":"2024-09-21T07:46:55.198188+0000","last_active":"2024-09-21T07:46:55.181730+0000","last_peered":"2024-09-21T07:46:55.181730+0000","last_clean":"2024-09-21T07:46:55.181730+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:46:55.198188+0000","last_undegraded":"2024-09-21T07:46:55.198188+0000","last_fullsized":"2024-09-21T07:46:55.198188+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.222778+0000","last_change":"2024-09-21T07:46:55.218103+0000","last_active":"2024-09-21T07:46:55.222778+0000","last_peered":"2024-09-21T07:46:55.222778+0000","last_clean":"2024-09-21T07:46:55.222778+0000","last_became_active":"2024-09-21T07:46:55.217796+0000","last_became_peered":"2024-09-21T07:46:55.217796+0000","last_unstale":"2024-09-21T07:46:55.222778+0000","last_undegraded":"2024-09-21T07:46:55.222778+0000","last_fullsized":"2024-09-21T07:46:55.222778+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:07.250160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":129,"reported_epoch":33,"state":"active+clean","last_fresh":"2024-09-21T07:46:55.211593+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:55.211593+0000","last_peered":"2024-09-21T07:46:55.211593+0000","last_clean":"2024-09-21T07:46:55.211593+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:55.211593+0000","last_undegraded":"2024-09-21T07:46:55.211593+0000","last_fullsized":"2024-09-21T07:46:55.211593+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1634,"num_objects":14,"num_object_clones":0,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":30,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":10,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":3850,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25,"ondisk_log_size":25,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476754,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476754,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476751,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27948,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343892,"statfs":{"total":96636764160,"available":96608145408,"internally_reserved":0,"allocated":942080,"data_stored":720244,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27384,"kb_used_data":352,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344456,"statfs":{"total":96636764160,"available":96608722944,"internally_reserved":0,"allocated":360448,"data_stored":136151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":397312,"data_stored":150838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":740176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":150860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":149248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3f is not active+clean 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.196875+0000', 'last_change': '2024-09-21T07:46:55.196875+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.196875+0000', 'last_undegraded': '2024-09-21T07:46:55.196875+0000', 'last_fullsized': '2024-09-21T07:46:55.196875+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:PG 5.3d is not active+clean 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.209740+0000', 'last_change': '2024-09-21T07:46:55.209740+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.209740+0000', 'last_undegraded': '2024-09-21T07:46:55.209740+0000', 'last_fullsized': '2024-09-21T07:46:55.209740+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:PG 5.39 is not active+clean 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198590+0000', 'last_change': '2024-09-21T07:46:55.198590+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198590+0000', 'last_undegraded': '2024-09-21T07:46:55.198590+0000', 'last_fullsized': '2024-09-21T07:46:55.198590+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.608 INFO:tasks.ceph.ceph_manager.ceph:PG 5.38 is not active+clean 2024-09-21T07:47:01.609 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.195067+0000', 'last_change': '2024-09-21T07:46:55.195067+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.195067+0000', 'last_undegraded': '2024-09-21T07:46:55.195067+0000', 'last_fullsized': '2024-09-21T07:46:55.195067+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.609 INFO:tasks.ceph.ceph_manager.ceph:PG 5.37 is not active+clean 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.210023+0000', 'last_change': '2024-09-21T07:46:55.210023+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.210023+0000', 'last_undegraded': '2024-09-21T07:46:55.210023+0000', 'last_fullsized': '2024-09-21T07:46:55.210023+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:PG 5.35 is not active+clean 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198313+0000', 'last_change': '2024-09-21T07:46:55.198313+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198313+0000', 'last_undegraded': '2024-09-21T07:46:55.198313+0000', 'last_fullsized': '2024-09-21T07:46:55.198313+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:PG 5.34 is not active+clean 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.199137+0000', 'last_change': '2024-09-21T07:46:55.199137+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.199137+0000', 'last_undegraded': '2024-09-21T07:46:55.199137+0000', 'last_fullsized': '2024-09-21T07:46:55.199137+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:PG 5.31 is not active+clean 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.194649+0000', 'last_change': '2024-09-21T07:46:55.194649+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.194649+0000', 'last_undegraded': '2024-09-21T07:46:55.194649+0000', 'last_fullsized': '2024-09-21T07:46:55.194649+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:PG 5.30 is not active+clean 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.194072+0000', 'last_change': '2024-09-21T07:46:55.194072+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.194072+0000', 'last_undegraded': '2024-09-21T07:46:55.194072+0000', 'last_fullsized': '2024-09-21T07:46:55.194072+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.610 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2f is not active+clean 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198999+0000', 'last_change': '2024-09-21T07:46:55.198999+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198999+0000', 'last_undegraded': '2024-09-21T07:46:55.198999+0000', 'last_fullsized': '2024-09-21T07:46:55.198999+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2b is not active+clean 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.197713+0000', 'last_change': '2024-09-21T07:46:55.197713+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.197713+0000', 'last_undegraded': '2024-09-21T07:46:55.197713+0000', 'last_fullsized': '2024-09-21T07:46:55.197713+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2a is not active+clean 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.195512+0000', 'last_change': '2024-09-21T07:46:55.195512+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.195512+0000', 'last_undegraded': '2024-09-21T07:46:55.195512+0000', 'last_fullsized': '2024-09-21T07:46:55.195512+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:PG 5.28 is not active+clean 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.194574+0000', 'last_change': '2024-09-21T07:46:55.194574+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.194574+0000', 'last_undegraded': '2024-09-21T07:46:55.194574+0000', 'last_fullsized': '2024-09-21T07:46:55.194574+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:PG 5.24 is not active+clean 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198069+0000', 'last_change': '2024-09-21T07:46:55.198069+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198069+0000', 'last_undegraded': '2024-09-21T07:46:55.198069+0000', 'last_fullsized': '2024-09-21T07:46:55.198069+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.611 INFO:tasks.ceph.ceph_manager.ceph:PG 5.23 is not active+clean 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.200475+0000', 'last_change': '2024-09-21T07:46:55.200475+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.200475+0000', 'last_undegraded': '2024-09-21T07:46:55.200475+0000', 'last_fullsized': '2024-09-21T07:46:55.200475+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:PG 5.22 is not active+clean 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.196887+0000', 'last_change': '2024-09-21T07:46:55.196887+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.196887+0000', 'last_undegraded': '2024-09-21T07:46:55.196887+0000', 'last_fullsized': '2024-09-21T07:46:55.196887+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:PG 5.21 is not active+clean 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.192541+0000', 'last_change': '2024-09-21T07:46:55.192541+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.192541+0000', 'last_undegraded': '2024-09-21T07:46:55.192541+0000', 'last_fullsized': '2024-09-21T07:46:55.192541+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:PG 5.20 is not active+clean 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198692+0000', 'last_change': '2024-09-21T07:46:55.198692+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198692+0000', 'last_undegraded': '2024-09-21T07:46:55.198692+0000', 'last_fullsized': '2024-09-21T07:46:55.198692+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1f is not active+clean 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.196356+0000', 'last_change': '2024-09-21T07:46:55.196356+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.196356+0000', 'last_undegraded': '2024-09-21T07:46:55.196356+0000', 'last_fullsized': '2024-09-21T07:46:55.196356+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1c is not active+clean 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.197598+0000', 'last_change': '2024-09-21T07:46:55.197598+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.197598+0000', 'last_undegraded': '2024-09-21T07:46:55.197598+0000', 'last_fullsized': '2024-09-21T07:46:55.197598+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.612 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1a is not active+clean 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.194935+0000', 'last_change': '2024-09-21T07:46:55.194935+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.194935+0000', 'last_undegraded': '2024-09-21T07:46:55.194935+0000', 'last_fullsized': '2024-09-21T07:46:55.194935+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 4], 'acting': [1, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1b is not active+clean 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.193937+0000', 'last_change': '2024-09-21T07:46:55.193937+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.193937+0000', 'last_undegraded': '2024-09-21T07:46:55.193937+0000', 'last_fullsized': '2024-09-21T07:46:55.193937+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:PG 5.18 is not active+clean 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.193798+0000', 'last_change': '2024-09-21T07:46:55.193798+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.193798+0000', 'last_undegraded': '2024-09-21T07:46:55.193798+0000', 'last_fullsized': '2024-09-21T07:46:55.193798+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:PG 5.16 is not active+clean 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198239+0000', 'last_change': '2024-09-21T07:46:55.198239+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198239+0000', 'last_undegraded': '2024-09-21T07:46:55.198239+0000', 'last_fullsized': '2024-09-21T07:46:55.198239+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 6], 'acting': [3, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:PG 5.15 is not active+clean 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.196425+0000', 'last_change': '2024-09-21T07:46:55.196425+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.196425+0000', 'last_undegraded': '2024-09-21T07:46:55.196425+0000', 'last_fullsized': '2024-09-21T07:46:55.196425+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:PG 5.10 is not active+clean 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.192745+0000', 'last_change': '2024-09-21T07:46:55.192745+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.192745+0000', 'last_undegraded': '2024-09-21T07:46:55.192745+0000', 'last_fullsized': '2024-09-21T07:46:55.192745+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:47:01.613 INFO:tasks.ceph.ceph_manager.ceph:PG 5.11 is not active+clean 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.195717+0000', 'last_change': '2024-09-21T07:46:55.195717+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.195717+0000', 'last_undegraded': '2024-09-21T07:46:55.195717+0000', 'last_fullsized': '2024-09-21T07:46:55.195717+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:PG 5.e is not active+clean 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.193288+0000', 'last_change': '2024-09-21T07:46:55.193288+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.193288+0000', 'last_undegraded': '2024-09-21T07:46:55.193288+0000', 'last_fullsized': '2024-09-21T07:46:55.193288+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:PG 5.f is not active+clean 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.205097+0000', 'last_change': '2024-09-21T07:46:55.205097+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.205097+0000', 'last_undegraded': '2024-09-21T07:46:55.205097+0000', 'last_fullsized': '2024-09-21T07:46:55.205097+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:PG 5.9 is not active+clean 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.195966+0000', 'last_change': '2024-09-21T07:46:55.195966+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.195966+0000', 'last_undegraded': '2024-09-21T07:46:55.195966+0000', 'last_fullsized': '2024-09-21T07:46:55.195966+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:PG 5.7 is not active+clean 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.191906+0000', 'last_change': '2024-09-21T07:46:55.191906+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.191906+0000', 'last_undegraded': '2024-09-21T07:46:55.191906+0000', 'last_fullsized': '2024-09-21T07:46:55.191906+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:PG 5.2 is not active+clean 2024-09-21T07:47:01.614 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.196436+0000', 'last_change': '2024-09-21T07:46:55.196436+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.196436+0000', 'last_undegraded': '2024-09-21T07:46:55.196436+0000', 'last_fullsized': '2024-09-21T07:46:55.196436+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.615 INFO:tasks.ceph.ceph_manager.ceph:PG 5.1 is not active+clean 2024-09-21T07:47:01.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.191831+0000', 'last_change': '2024-09-21T07:46:55.191831+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.191831+0000', 'last_undegraded': '2024-09-21T07:46:55.191831+0000', 'last_fullsized': '2024-09-21T07:46:55.191831+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:47:01.615 INFO:tasks.ceph.ceph_manager.ceph:PG 5.4 is not active+clean 2024-09-21T07:47:01.615 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 33, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:46:55.198188+0000', 'last_change': '2024-09-21T07:46:55.198188+0000', 'last_active': '2024-09-21T07:46:55.181730+0000', 'last_peered': '2024-09-21T07:46:55.181730+0000', 'last_clean': '2024-09-21T07:46:55.181730+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:46:55.198188+0000', 'last_undegraded': '2024-09-21T07:46:55.198188+0000', 'last_fullsized': '2024-09-21T07:46:55.198188+0000', 'mapping_epoch': 33, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 33, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'last_clean_scrub_stamp': '2024-09-21T07:46:55.181730+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:47:01.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:47:01.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 -- 172.21.15.88:0/1675816740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 msgr2=0x7f8b3c00ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 --2- 172.21.15.88:0/1675816740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b3c00ba50 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00c770 tx=0x7f8b3c005830 comp rx=0 tx=0).stop 2024-09-21T07:47:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 -- 172.21.15.88:0/1675816740 shutdown_connections 2024-09-21T07:47:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 --2- 172.21.15.88:0/1675816740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b3c00ba50 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 -- 172.21.15.88:0/1675816740 >> 172.21.15.88:0/1675816740 conn(0x7f8b54071070 msgr2=0x7f8b54071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 -- 172.21.15.88:0/1675816740 shutdown_connections 2024-09-21T07:47:01.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.863+0000 7f8b5a800640 1 -- 172.21.15.88:0/1675816740 wait complete. 2024-09-21T07:47:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.865+0000 7f8b5a800640 1 Processor -- start 2024-09-21T07:47:01.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.865+0000 7f8b5a800640 1 -- start start 2024-09-21T07:47:01.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b5a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b541f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b5a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b540cb180 0x7f8b540b7430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b5a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b540b7970 0x7f8b5414ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b5a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b540cca70 con 0x7f8b3c0053a0 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b5a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8b540b5770 con 0x7f8b540b7970 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b5a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8b540cecf0 con 0x7f8b540cb180 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b541f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b541f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58498/0 (socket says 172.21.15.88:58498) 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b59400640 1 -- 172.21.15.88:0/3174609988 learned_addr learned my addr 172.21.15.88:0/3174609988 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b59e00640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b540b7970 0x7f8b5414ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.867+0000 7f8b58a00640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b540cb180 0x7f8b540b7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b59400640 1 -- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b540cb180 msgr2=0x7f8b540b7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b59400640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b540cb180 0x7f8b540b7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b59400640 1 -- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b540b7970 msgr2=0x7f8b5414ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b59400640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b540b7970 0x7f8b5414ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b59400640 1 -- 172.21.15.88:0/3174609988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b540b7d80 con 0x7f8b3c0053a0 2024-09-21T07:47:01.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b59400640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b541f41e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b48004ed0 tx=0x7f8b48001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48015020 con 0x7f8b3c0053a0 2024-09-21T07:47:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b48011960 con 0x7f8b3c0053a0 2024-09-21T07:47:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b48011d90 con 0x7f8b3c0053a0 2024-09-21T07:47:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b540c6120 con 0x7f8b3c0053a0 2024-09-21T07:47:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.868+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b541f4aa0 con 0x7f8b3c0053a0 2024-09-21T07:47:01.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.869+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8b540c6640 con 0x7f8b3c0053a0 2024-09-21T07:47:01.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.871+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8b48018020 con 0x7f8b3c0053a0 2024-09-21T07:47:01.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.872+0000 7f8b51600640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8b34078b90 0x7f8b3407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:01.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.872+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8b4809b690 con 0x7f8b3c0053a0 2024-09-21T07:47:01.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.872+0000 7f8b58a00640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8b34078b90 0x7f8b3407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:01.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.873+0000 7f8b58a00640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8b34078b90 0x7f8b3407b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b54051010 tx=0x7f8b40005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:01.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:01.876+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8b4801f020 con 0x7f8b3c0053a0 2024-09-21T07:47:02.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.138+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f8b541f4890 con 0x7f8b3c0053a0 2024-09-21T07:47:02.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.139+0000 7f8b51600640 1 -- 172.21.15.88:0/3174609988 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1686 (secure 0 0 0) 0x7f8b540b7d80 con 0x7f8b3c0053a0 2024-09-21T07:47:02.141 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:02.141 INFO:teuthology.orchestra.run.smithi088.stdout:{"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1726904755,"num_in_osds":8,"osd_in_since":1726904744,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":63},{"state_name":"creating+peering","count":34}],"num_pgs":97,"num_pools":3,"num_objects":16,"data_bytes":592002,"bytes_used":225636352,"bytes_avail":772868476928,"bytes_total":773094113280,"inactive_pgs_ratio":0.3505154550075531,"write_bytes_sec":721,"read_op_per_sec":0,"write_op_per_sec":5},"fsmap":{"epoch":17,"btime":"2024-09-21T07:46:59:220696+0000","id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"c","status":"up:active","gid":4899}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":5,"modified":"2024-09-21T07:46:55.866211+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2024-09-21T07:47:02.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8b34078b90 msgr2=0x7f8b3407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8b34078b90 0x7f8b3407b0a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f8b54051010 tx=0x7f8b40005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 msgr2=0x7f8b541f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b541f41e0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f8b48004ed0 tx=0x7f8b48001630 comp rx=0 tx=0).stop 2024-09-21T07:47:02.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 shutdown_connections 2024-09-21T07:47:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8b34078b90 0x7f8b3407b0a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8b540cb180 0x7f8b540b7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8b540b7970 0x7f8b5414ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 --2- 172.21.15.88:0/3174609988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b3c0053a0 0x7f8b541f41e0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 >> 172.21.15.88:0/3174609988 conn(0x7f8b54071070 msgr2=0x7f8b540b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:02.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 shutdown_connections 2024-09-21T07:47:02.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.141+0000 7f8b5a800640 1 -- 172.21.15.88:0/3174609988 wait complete. 2024-09-21T07:47:02.164 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:47:02.165 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.383+0000 7f21c3600640 1 -- 172.21.15.88:0/4100728475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21a4004d40 msgr2=0x7f21a4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.383+0000 7f21c3600640 1 --2- 172.21.15.88:0/4100728475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21a4004d40 0x7f21a4005150 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f21bc0aea60 tx=0x7f21a4005830 comp rx=0 tx=0).stop 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.383+0000 7f21c1600640 1 -- 172.21.15.88:0/4100728475 reap_dead start 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.383+0000 7f21c3600640 1 -- 172.21.15.88:0/4100728475 shutdown_connections 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.383+0000 7f21c3600640 1 -- 172.21.15.88:0/4100728475 >> 172.21.15.88:0/4100728475 conn(0x7f21bc06d730 msgr2=0x7f21bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.384+0000 7f21c3600640 1 -- 172.21.15.88:0/4100728475 shutdown_connections 2024-09-21T07:47:02.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.384+0000 7f21c3600640 1 -- 172.21.15.88:0/4100728475 wait complete. 2024-09-21T07:47:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.386+0000 7f21c3600640 1 Processor -- start 2024-09-21T07:47:02.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.386+0000 7f21c3600640 1 -- start start 2024-09-21T07:47:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.387+0000 7f21c3600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21a4004d40 0x7f21bc150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.387+0000 7f21c3600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21bc151680 0x7f21bc06faf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.387+0000 7f21c3600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 0x7f21bc0743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.387+0000 7f21c3600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc0b4da0 con 0x7f21bc070030 2024-09-21T07:47:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.387+0000 7f21c3600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21bc0b4c20 con 0x7f21a4004d40 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.387+0000 7f21c3600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21bc0ad4f0 con 0x7f21bc151680 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 0x7f21bc0743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 0x7f21bc0743c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58526/0 (socket says 172.21.15.88:58526) 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 -- 172.21.15.88:0/590694838 learned_addr learned my addr 172.21.15.88:0/590694838 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21bbe00640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21bc151680 0x7f21bc06faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c0c00640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21a4004d40 0x7f21bc150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 -- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21bc151680 msgr2=0x7f21bc06faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21bc151680 0x7f21bc06faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 -- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21a4004d40 msgr2=0x7f21bc150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21a4004d40 0x7f21bc150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 -- 172.21.15.88:0/590694838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21bc070440 con 0x7f21bc070030 2024-09-21T07:47:02.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.388+0000 7f21c1600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 0x7f21bc0743c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f21b4036040 tx=0x7f21b4033b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.389+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4043070 con 0x7f21bc070030 2024-09-21T07:47:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.389+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21b4033cf0 con 0x7f21bc070030 2024-09-21T07:47:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.389+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b4038040 con 0x7f21bc070030 2024-09-21T07:47:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.389+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21bc15a640 con 0x7f21bc070030 2024-09-21T07:47:02.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.389+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21bc1584c0 con 0x7f21bc070030 2024-09-21T07:47:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.390+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21bc15a3b0 con 0x7f21bc070030 2024-09-21T07:47:02.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.391+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f21b4003bd0 con 0x7f21bc070030 2024-09-21T07:47:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.392+0000 7f21b8c00640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f219c078bc0 0x7f219c07b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.392+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f21b40bae80 con 0x7f21bc070030 2024-09-21T07:47:02.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.393+0000 7f21c0c00640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f219c078bc0 0x7f219c07b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:02.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.393+0000 7f21c0c00640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f219c078bc0 0x7f219c07b0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21b0000fb0 tx=0x7f21b00011c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:02.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.396+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21bc070440 con 0x7f21bc070030 2024-09-21T07:47:02.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.577+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21bc1514b0 con 0x7f219c078bc0 2024-09-21T07:47:02.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.583+0000 7f21b8c00640 1 -- 172.21.15.88:0/590694838 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251944 (secure 0 0 0) 0x7f21bc1514b0 con 0x7f219c078bc0 2024-09-21T07:47:02.588 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:02.589 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f219c078bc0 msgr2=0x7f219c07b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f219c078bc0 0x7f219c07b0d0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21b0000fb0 tx=0x7f21b00011c0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 msgr2=0x7f21bc0743c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 0x7f21bc0743c0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f21b4036040 tx=0x7f21b4033b80 comp rx=0 tx=0).stop 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 shutdown_connections 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f219c078bc0 0x7f219c07b0d0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21bc151680 0x7f21bc06faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21bc070030 0x7f21bc0743c0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 --2- 172.21.15.88:0/590694838 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21a4004d40 0x7f21bc150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:02.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 >> 172.21.15.88:0/590694838 conn(0x7f21bc06d730 msgr2=0x7f21bc1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:02.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 shutdown_connections 2024-09-21T07:47:02.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:02.587+0000 7f21c3600640 1 -- 172.21.15.88:0/590694838 wait complete. 2024-09-21T07:47:02.610 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2024-09-21T07:47:01.867353+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220480,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754240,"statfs":{"total":773094113280,"available":772868341760,"internally_reserved":0,"allocated":4333568,"data_stored":2379364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.680071"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709175+0000","last_change":"2024-09-21T07:46:56.222670+0000","last_active":"2024-09-21T07:46:57.709175+0000","last_peered":"2024-09-21T07:46:57.709175+0000","last_clean":"2024-09-21T07:46:57.709175+0000","last_became_active":"2024-09-21T07:46:56.220432+0000","last_became_peered":"2024-09-21T07:46:56.220432+0000","last_unstale":"2024-09-21T07:46:57.709175+0000","last_undegraded":"2024-09-21T07:46:57.709175+0000","last_fullsized":"2024-09-21T07:46:57.709175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:19.419293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228903+0000","last_change":"2024-09-21T07:46:56.229990+0000","last_active":"2024-09-21T07:46:57.228903+0000","last_peered":"2024-09-21T07:46:57.228903+0000","last_clean":"2024-09-21T07:46:57.228903+0000","last_became_active":"2024-09-21T07:46:56.229655+0000","last_became_peered":"2024-09-21T07:46:56.229655+0000","last_unstale":"2024-09-21T07:46:57.228903+0000","last_undegraded":"2024-09-21T07:46:57.228903+0000","last_fullsized":"2024-09-21T07:46:57.228903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707653+0000","last_change":"2024-09-21T07:46:56.209979+0000","last_active":"2024-09-21T07:46:57.707653+0000","last_peered":"2024-09-21T07:46:57.707653+0000","last_clean":"2024-09-21T07:46:57.707653+0000","last_became_active":"2024-09-21T07:46:56.209344+0000","last_became_peered":"2024-09-21T07:46:56.209344+0000","last_unstale":"2024-09-21T07:46:57.707653+0000","last_undegraded":"2024-09-21T07:46:57.707653+0000","last_fullsized":"2024-09-21T07:46:57.707653+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219416+0000","last_change":"2024-09-21T07:46:56.235757+0000","last_active":"2024-09-21T07:46:57.219416+0000","last_peered":"2024-09-21T07:46:57.219416+0000","last_clean":"2024-09-21T07:46:57.219416+0000","last_became_active":"2024-09-21T07:46:56.228680+0000","last_became_peered":"2024-09-21T07:46:56.228680+0000","last_unstale":"2024-09-21T07:46:57.219416+0000","last_undegraded":"2024-09-21T07:46:57.219416+0000","last_fullsized":"2024-09-21T07:46:57.219416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228641+0000","last_change":"2024-09-21T07:46:56.229963+0000","last_active":"2024-09-21T07:46:57.228641+0000","last_peered":"2024-09-21T07:46:57.228641+0000","last_clean":"2024-09-21T07:46:57.228641+0000","last_became_active":"2024-09-21T07:46:56.229617+0000","last_became_peered":"2024-09-21T07:46:56.229617+0000","last_unstale":"2024-09-21T07:46:57.228641+0000","last_undegraded":"2024-09-21T07:46:57.228641+0000","last_fullsized":"2024-09-21T07:46:57.228641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219492+0000","last_change":"2024-09-21T07:46:56.229684+0000","last_active":"2024-09-21T07:46:57.219492+0000","last_peered":"2024-09-21T07:46:57.219492+0000","last_clean":"2024-09-21T07:46:57.219492+0000","last_became_active":"2024-09-21T07:46:56.229339+0000","last_became_peered":"2024-09-21T07:46:56.229339+0000","last_unstale":"2024-09-21T07:46:57.219492+0000","last_undegraded":"2024-09-21T07:46:57.219492+0000","last_fullsized":"2024-09-21T07:46:57.219492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:21.621424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708409+0000","last_change":"2024-09-21T07:46:56.212908+0000","last_active":"2024-09-21T07:46:57.708409+0000","last_peered":"2024-09-21T07:46:57.708409+0000","last_clean":"2024-09-21T07:46:57.708409+0000","last_became_active":"2024-09-21T07:46:56.210441+0000","last_became_peered":"2024-09-21T07:46:56.210441+0000","last_unstale":"2024-09-21T07:46:57.708409+0000","last_undegraded":"2024-09-21T07:46:57.708409+0000","last_fullsized":"2024-09-21T07:46:57.708409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708011+0000","last_change":"2024-09-21T07:46:56.221463+0000","last_active":"2024-09-21T07:46:57.708011+0000","last_peered":"2024-09-21T07:46:57.708011+0000","last_clean":"2024-09-21T07:46:57.708011+0000","last_became_active":"2024-09-21T07:46:56.221112+0000","last_became_peered":"2024-09-21T07:46:56.221112+0000","last_unstale":"2024-09-21T07:46:57.708011+0000","last_undegraded":"2024-09-21T07:46:57.708011+0000","last_fullsized":"2024-09-21T07:46:57.708011+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.488514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707286+0000","last_change":"2024-09-21T07:46:56.210833+0000","last_active":"2024-09-21T07:46:57.707286+0000","last_peered":"2024-09-21T07:46:57.707286+0000","last_clean":"2024-09-21T07:46:57.707286+0000","last_became_active":"2024-09-21T07:46:56.209936+0000","last_became_peered":"2024-09-21T07:46:56.209936+0000","last_unstale":"2024-09-21T07:46:57.707286+0000","last_undegraded":"2024-09-21T07:46:57.707286+0000","last_fullsized":"2024-09-21T07:46:57.707286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:48.704288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220083+0000","last_change":"2024-09-21T07:46:56.231062+0000","last_active":"2024-09-21T07:46:57.220083+0000","last_peered":"2024-09-21T07:46:57.220083+0000","last_clean":"2024-09-21T07:46:57.220083+0000","last_became_active":"2024-09-21T07:46:56.229427+0000","last_became_peered":"2024-09-21T07:46:56.229427+0000","last_unstale":"2024-09-21T07:46:57.220083+0000","last_undegraded":"2024-09-21T07:46:57.220083+0000","last_fullsized":"2024-09-21T07:46:57.220083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:47.565734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221307+0000","last_change":"2024-09-21T07:46:56.215059+0000","last_active":"2024-09-21T07:46:57.221307+0000","last_peered":"2024-09-21T07:46:57.221307+0000","last_clean":"2024-09-21T07:46:57.221307+0000","last_became_active":"2024-09-21T07:46:56.212314+0000","last_became_peered":"2024-09-21T07:46:56.212314+0000","last_unstale":"2024-09-21T07:46:57.221307+0000","last_undegraded":"2024-09-21T07:46:57.221307+0000","last_fullsized":"2024-09-21T07:46:57.221307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:32.152592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707294+0000","last_change":"2024-09-21T07:46:56.222911+0000","last_active":"2024-09-21T07:46:57.707294+0000","last_peered":"2024-09-21T07:46:57.707294+0000","last_clean":"2024-09-21T07:46:57.707294+0000","last_became_active":"2024-09-21T07:46:56.222587+0000","last_became_peered":"2024-09-21T07:46:56.222587+0000","last_unstale":"2024-09-21T07:46:57.707294+0000","last_undegraded":"2024-09-21T07:46:57.707294+0000","last_fullsized":"2024-09-21T07:46:57.707294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:18.632830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707947+0000","last_change":"2024-09-21T07:46:56.237624+0000","last_active":"2024-09-21T07:46:57.707947+0000","last_peered":"2024-09-21T07:46:57.707947+0000","last_clean":"2024-09-21T07:46:57.707947+0000","last_became_active":"2024-09-21T07:46:56.237037+0000","last_became_peered":"2024-09-21T07:46:56.237037+0000","last_unstale":"2024-09-21T07:46:57.707947+0000","last_undegraded":"2024-09-21T07:46:57.707947+0000","last_fullsized":"2024-09-21T07:46:57.707947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:43.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219004+0000","last_change":"2024-09-21T07:46:56.239271+0000","last_active":"2024-09-21T07:46:57.219004+0000","last_peered":"2024-09-21T07:46:57.219004+0000","last_clean":"2024-09-21T07:46:57.219004+0000","last_became_active":"2024-09-21T07:46:56.238936+0000","last_became_peered":"2024-09-21T07:46:56.238936+0000","last_unstale":"2024-09-21T07:46:57.219004+0000","last_undegraded":"2024-09-21T07:46:57.219004+0000","last_fullsized":"2024-09-21T07:46:57.219004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706826+0000","last_change":"2024-09-21T07:46:56.220351+0000","last_active":"2024-09-21T07:46:57.706826+0000","last_peered":"2024-09-21T07:46:57.706826+0000","last_clean":"2024-09-21T07:46:57.706826+0000","last_became_active":"2024-09-21T07:46:56.220039+0000","last_became_peered":"2024-09-21T07:46:56.220039+0000","last_unstale":"2024-09-21T07:46:57.706826+0000","last_undegraded":"2024-09-21T07:46:57.706826+0000","last_fullsized":"2024-09-21T07:46:57.706826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707569+0000","last_change":"2024-09-21T07:46:56.219494+0000","last_active":"2024-09-21T07:46:57.707569+0000","last_peered":"2024-09-21T07:46:57.707569+0000","last_clean":"2024-09-21T07:46:57.707569+0000","last_became_active":"2024-09-21T07:46:56.219194+0000","last_became_peered":"2024-09-21T07:46:56.219194+0000","last_unstale":"2024-09-21T07:46:57.707569+0000","last_undegraded":"2024-09-21T07:46:57.707569+0000","last_fullsized":"2024-09-21T07:46:57.707569+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:54.406099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221559+0000","last_change":"2024-09-21T07:46:56.222353+0000","last_active":"2024-09-21T07:46:57.221559+0000","last_peered":"2024-09-21T07:46:57.221559+0000","last_clean":"2024-09-21T07:46:57.221559+0000","last_became_active":"2024-09-21T07:46:56.222045+0000","last_became_peered":"2024-09-21T07:46:56.222045+0000","last_unstale":"2024-09-21T07:46:57.221559+0000","last_undegraded":"2024-09-21T07:46:57.221559+0000","last_fullsized":"2024-09-21T07:46:57.221559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:43.027555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220865+0000","last_change":"2024-09-21T07:46:56.224534+0000","last_active":"2024-09-21T07:46:57.220865+0000","last_peered":"2024-09-21T07:46:57.220865+0000","last_clean":"2024-09-21T07:46:57.220865+0000","last_became_active":"2024-09-21T07:46:56.224167+0000","last_became_peered":"2024-09-21T07:46:56.224167+0000","last_unstale":"2024-09-21T07:46:57.220865+0000","last_undegraded":"2024-09-21T07:46:57.220865+0000","last_fullsized":"2024-09-21T07:46:57.220865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:18.755342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220883+0000","last_change":"2024-09-21T07:46:56.228375+0000","last_active":"2024-09-21T07:46:57.220883+0000","last_peered":"2024-09-21T07:46:57.220883+0000","last_clean":"2024-09-21T07:46:57.220883+0000","last_became_active":"2024-09-21T07:46:56.225793+0000","last_became_peered":"2024-09-21T07:46:56.225793+0000","last_unstale":"2024-09-21T07:46:57.220883+0000","last_undegraded":"2024-09-21T07:46:57.220883+0000","last_fullsized":"2024-09-21T07:46:57.220883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:46.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708077+0000","last_change":"2024-09-21T07:46:56.226928+0000","last_active":"2024-09-21T07:46:57.708077+0000","last_peered":"2024-09-21T07:46:57.708077+0000","last_clean":"2024-09-21T07:46:57.708077+0000","last_became_active":"2024-09-21T07:46:56.226560+0000","last_became_peered":"2024-09-21T07:46:56.226560+0000","last_unstale":"2024-09-21T07:46:57.708077+0000","last_undegraded":"2024-09-21T07:46:57.708077+0000","last_fullsized":"2024-09-21T07:46:57.708077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707252+0000","last_change":"2024-09-21T07:46:56.225740+0000","last_active":"2024-09-21T07:46:57.707252+0000","last_peered":"2024-09-21T07:46:57.707252+0000","last_clean":"2024-09-21T07:46:57.707252+0000","last_became_active":"2024-09-21T07:46:56.225360+0000","last_became_peered":"2024-09-21T07:46:56.225360+0000","last_unstale":"2024-09-21T07:46:57.707252+0000","last_undegraded":"2024-09-21T07:46:57.707252+0000","last_fullsized":"2024-09-21T07:46:57.707252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:44.291034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708446+0000","last_change":"2024-09-21T07:46:56.230572+0000","last_active":"2024-09-21T07:46:57.708446+0000","last_peered":"2024-09-21T07:46:57.708446+0000","last_clean":"2024-09-21T07:46:57.708446+0000","last_became_active":"2024-09-21T07:46:56.230237+0000","last_became_peered":"2024-09-21T07:46:56.230237+0000","last_unstale":"2024-09-21T07:46:57.708446+0000","last_undegraded":"2024-09-21T07:46:57.708446+0000","last_fullsized":"2024-09-21T07:46:57.708446+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707640+0000","last_change":"2024-09-21T07:46:56.227950+0000","last_active":"2024-09-21T07:46:57.707640+0000","last_peered":"2024-09-21T07:46:57.707640+0000","last_clean":"2024-09-21T07:46:57.707640+0000","last_became_active":"2024-09-21T07:46:56.227647+0000","last_became_peered":"2024-09-21T07:46:56.227647+0000","last_unstale":"2024-09-21T07:46:57.707640+0000","last_undegraded":"2024-09-21T07:46:57.707640+0000","last_fullsized":"2024-09-21T07:46:57.707640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:08.480356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708053+0000","last_change":"2024-09-21T07:46:56.211905+0000","last_active":"2024-09-21T07:46:57.708053+0000","last_peered":"2024-09-21T07:46:57.708053+0000","last_clean":"2024-09-21T07:46:57.708053+0000","last_became_active":"2024-09-21T07:46:56.211721+0000","last_became_peered":"2024-09-21T07:46:56.211721+0000","last_unstale":"2024-09-21T07:46:57.708053+0000","last_undegraded":"2024-09-21T07:46:57.708053+0000","last_fullsized":"2024-09-21T07:46:57.708053+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219395+0000","last_change":"2024-09-21T07:46:56.217732+0000","last_active":"2024-09-21T07:46:57.219395+0000","last_peered":"2024-09-21T07:46:57.219395+0000","last_clean":"2024-09-21T07:46:57.219395+0000","last_became_active":"2024-09-21T07:46:56.217410+0000","last_became_peered":"2024-09-21T07:46:56.217410+0000","last_unstale":"2024-09-21T07:46:57.219395+0000","last_undegraded":"2024-09-21T07:46:57.219395+0000","last_fullsized":"2024-09-21T07:46:57.219395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219810+0000","last_change":"2024-09-21T07:46:56.220112+0000","last_active":"2024-09-21T07:46:57.219810+0000","last_peered":"2024-09-21T07:46:57.219810+0000","last_clean":"2024-09-21T07:46:57.219810+0000","last_became_active":"2024-09-21T07:46:56.217329+0000","last_became_peered":"2024-09-21T07:46:56.217329+0000","last_unstale":"2024-09-21T07:46:57.219810+0000","last_undegraded":"2024-09-21T07:46:57.219810+0000","last_fullsized":"2024-09-21T07:46:57.219810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:04.144314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228789+0000","last_change":"2024-09-21T07:46:56.223494+0000","last_active":"2024-09-21T07:46:57.228789+0000","last_peered":"2024-09-21T07:46:57.228789+0000","last_clean":"2024-09-21T07:46:57.228789+0000","last_became_active":"2024-09-21T07:46:56.223149+0000","last_became_peered":"2024-09-21T07:46:56.223149+0000","last_unstale":"2024-09-21T07:46:57.228789+0000","last_undegraded":"2024-09-21T07:46:57.228789+0000","last_fullsized":"2024-09-21T07:46:57.228789+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708836+0000","last_change":"2024-09-21T07:46:56.228315+0000","last_active":"2024-09-21T07:46:57.708836+0000","last_peered":"2024-09-21T07:46:57.708836+0000","last_clean":"2024-09-21T07:46:57.708836+0000","last_became_active":"2024-09-21T07:46:56.224550+0000","last_became_peered":"2024-09-21T07:46:56.224550+0000","last_unstale":"2024-09-21T07:46:57.708836+0000","last_undegraded":"2024-09-21T07:46:57.708836+0000","last_fullsized":"2024-09-21T07:46:57.708836+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706953+0000","last_change":"2024-09-21T07:46:56.235164+0000","last_active":"2024-09-21T07:46:57.706953+0000","last_peered":"2024-09-21T07:46:57.706953+0000","last_clean":"2024-09-21T07:46:57.706953+0000","last_became_active":"2024-09-21T07:46:56.234854+0000","last_became_peered":"2024-09-21T07:46:56.234854+0000","last_unstale":"2024-09-21T07:46:57.706953+0000","last_undegraded":"2024-09-21T07:46:57.706953+0000","last_fullsized":"2024-09-21T07:46:57.706953+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707171+0000","last_change":"2024-09-21T07:46:56.216052+0000","last_active":"2024-09-21T07:46:57.707171+0000","last_peered":"2024-09-21T07:46:57.707171+0000","last_clean":"2024-09-21T07:46:57.707171+0000","last_became_active":"2024-09-21T07:46:56.215728+0000","last_became_peered":"2024-09-21T07:46:56.215728+0000","last_unstale":"2024-09-21T07:46:57.707171+0000","last_undegraded":"2024-09-21T07:46:57.707171+0000","last_fullsized":"2024-09-21T07:46:57.707171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:15.115400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708408+0000","last_change":"2024-09-21T07:46:56.218473+0000","last_active":"2024-09-21T07:46:57.708408+0000","last_peered":"2024-09-21T07:46:57.708408+0000","last_clean":"2024-09-21T07:46:57.708408+0000","last_became_active":"2024-09-21T07:46:56.217355+0000","last_became_peered":"2024-09-21T07:46:56.217355+0000","last_unstale":"2024-09-21T07:46:57.708408+0000","last_undegraded":"2024-09-21T07:46:57.708408+0000","last_fullsized":"2024-09-21T07:46:57.708408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220525+0000","last_change":"2024-09-21T07:46:56.209321+0000","last_active":"2024-09-21T07:46:57.220525+0000","last_peered":"2024-09-21T07:46:57.220525+0000","last_clean":"2024-09-21T07:46:57.220525+0000","last_became_active":"2024-09-21T07:46:56.208580+0000","last_became_peered":"2024-09-21T07:46:56.208580+0000","last_unstale":"2024-09-21T07:46:57.220525+0000","last_undegraded":"2024-09-21T07:46:57.220525+0000","last_fullsized":"2024-09-21T07:46:57.220525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:44:35.986725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708018+0000","last_change":"2024-09-21T07:46:56.231049+0000","last_active":"2024-09-21T07:46:57.708018+0000","last_peered":"2024-09-21T07:46:57.708018+0000","last_clean":"2024-09-21T07:46:57.708018+0000","last_became_active":"2024-09-21T07:46:56.230778+0000","last_became_peered":"2024-09-21T07:46:56.230778+0000","last_unstale":"2024-09-21T07:46:57.708018+0000","last_undegraded":"2024-09-21T07:46:57.708018+0000","last_fullsized":"2024-09-21T07:46:57.708018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221717+0000","last_change":"2024-09-21T07:46:55.204066+0000","last_active":"2024-09-21T07:46:57.221717+0000","last_peered":"2024-09-21T07:46:57.221717+0000","last_clean":"2024-09-21T07:46:57.221717+0000","last_became_active":"2024-09-21T07:46:55.203753+0000","last_became_peered":"2024-09-21T07:46:55.203753+0000","last_unstale":"2024-09-21T07:46:57.221717+0000","last_undegraded":"2024-09-21T07:46:57.221717+0000","last_fullsized":"2024-09-21T07:46:57.221717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:59.681758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709170+0000","last_change":"2024-09-21T07:46:56.228457+0000","last_active":"2024-09-21T07:46:57.709170+0000","last_peered":"2024-09-21T07:46:57.709170+0000","last_clean":"2024-09-21T07:46:57.709170+0000","last_became_active":"2024-09-21T07:46:56.228095+0000","last_became_peered":"2024-09-21T07:46:56.228095+0000","last_unstale":"2024-09-21T07:46:57.709170+0000","last_undegraded":"2024-09-21T07:46:57.709170+0000","last_fullsized":"2024-09-21T07:46:57.709170+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219963+0000","last_change":"2024-09-21T07:46:55.212364+0000","last_active":"2024-09-21T07:46:57.219963+0000","last_peered":"2024-09-21T07:46:57.219963+0000","last_clean":"2024-09-21T07:46:57.219963+0000","last_became_active":"2024-09-21T07:46:55.211980+0000","last_became_peered":"2024-09-21T07:46:55.211980+0000","last_unstale":"2024-09-21T07:46:57.219963+0000","last_undegraded":"2024-09-21T07:46:57.219963+0000","last_fullsized":"2024-09-21T07:46:57.219963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:40.287704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709015+0000","last_change":"2024-09-21T07:46:56.230650+0000","last_active":"2024-09-21T07:46:57.709015+0000","last_peered":"2024-09-21T07:46:57.709015+0000","last_clean":"2024-09-21T07:46:57.709015+0000","last_became_active":"2024-09-21T07:46:56.230348+0000","last_became_peered":"2024-09-21T07:46:56.230348+0000","last_unstale":"2024-09-21T07:46:57.709015+0000","last_undegraded":"2024-09-21T07:46:57.709015+0000","last_fullsized":"2024-09-21T07:46:57.709015+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221045+0000","last_change":"2024-09-21T07:46:55.216737+0000","last_active":"2024-09-21T07:46:57.221045+0000","last_peered":"2024-09-21T07:46:57.221045+0000","last_clean":"2024-09-21T07:46:57.221045+0000","last_became_active":"2024-09-21T07:46:55.216402+0000","last_became_peered":"2024-09-21T07:46:55.216402+0000","last_unstale":"2024-09-21T07:46:57.221045+0000","last_undegraded":"2024-09-21T07:46:57.221045+0000","last_fullsized":"2024-09-21T07:46:57.221045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:40.966275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219312+0000","last_change":"2024-09-21T07:46:56.237682+0000","last_active":"2024-09-21T07:46:57.219312+0000","last_peered":"2024-09-21T07:46:57.219312+0000","last_clean":"2024-09-21T07:46:57.219312+0000","last_became_active":"2024-09-21T07:46:56.237340+0000","last_became_peered":"2024-09-21T07:46:56.237340+0000","last_unstale":"2024-09-21T07:46:57.219312+0000","last_undegraded":"2024-09-21T07:46:57.219312+0000","last_fullsized":"2024-09-21T07:46:57.219312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219765+0000","last_change":"2024-09-21T07:46:55.211939+0000","last_active":"2024-09-21T07:46:57.219765+0000","last_peered":"2024-09-21T07:46:57.219765+0000","last_clean":"2024-09-21T07:46:57.219765+0000","last_became_active":"2024-09-21T07:46:55.211170+0000","last_became_peered":"2024-09-21T07:46:55.211170+0000","last_unstale":"2024-09-21T07:46:57.219765+0000","last_undegraded":"2024-09-21T07:46:57.219765+0000","last_fullsized":"2024-09-21T07:46:57.219765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707334+0000","last_change":"2024-09-21T07:46:56.208982+0000","last_active":"2024-09-21T07:46:57.707334+0000","last_peered":"2024-09-21T07:46:57.707334+0000","last_clean":"2024-09-21T07:46:57.707334+0000","last_became_active":"2024-09-21T07:46:56.208692+0000","last_became_peered":"2024-09-21T07:46:56.208692+0000","last_unstale":"2024-09-21T07:46:57.707334+0000","last_undegraded":"2024-09-21T07:46:57.707334+0000","last_fullsized":"2024-09-21T07:46:57.707334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:22.862558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708963+0000","last_change":"2024-09-21T07:46:55.204599+0000","last_active":"2024-09-21T07:46:57.708963+0000","last_peered":"2024-09-21T07:46:57.708963+0000","last_clean":"2024-09-21T07:46:57.708963+0000","last_became_active":"2024-09-21T07:46:55.204186+0000","last_became_peered":"2024-09-21T07:46:55.204186+0000","last_unstale":"2024-09-21T07:46:57.708963+0000","last_undegraded":"2024-09-21T07:46:57.708963+0000","last_fullsized":"2024-09-21T07:46:57.708963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708759+0000","last_change":"2024-09-21T07:46:56.221721+0000","last_active":"2024-09-21T07:46:57.708759+0000","last_peered":"2024-09-21T07:46:57.708759+0000","last_clean":"2024-09-21T07:46:57.708759+0000","last_became_active":"2024-09-21T07:46:56.221359+0000","last_became_peered":"2024-09-21T07:46:56.221359+0000","last_unstale":"2024-09-21T07:46:57.708759+0000","last_undegraded":"2024-09-21T07:46:57.708759+0000","last_fullsized":"2024-09-21T07:46:57.708759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708468+0000","last_change":"2024-09-21T07:46:55.205089+0000","last_active":"2024-09-21T07:46:57.708468+0000","last_peered":"2024-09-21T07:46:57.708468+0000","last_clean":"2024-09-21T07:46:57.708468+0000","last_became_active":"2024-09-21T07:46:55.204642+0000","last_became_peered":"2024-09-21T07:46:55.204642+0000","last_unstale":"2024-09-21T07:46:57.708468+0000","last_undegraded":"2024-09-21T07:46:57.708468+0000","last_fullsized":"2024-09-21T07:46:57.708468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:26.924492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706743+0000","last_change":"2024-09-21T07:46:56.214234+0000","last_active":"2024-09-21T07:46:57.706743+0000","last_peered":"2024-09-21T07:46:57.706743+0000","last_clean":"2024-09-21T07:46:57.706743+0000","last_became_active":"2024-09-21T07:46:56.213914+0000","last_became_peered":"2024-09-21T07:46:56.213914+0000","last_unstale":"2024-09-21T07:46:57.706743+0000","last_undegraded":"2024-09-21T07:46:57.706743+0000","last_fullsized":"2024-09-21T07:46:57.706743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:05.770150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218717+0000","last_change":"2024-09-21T07:46:55.218472+0000","last_active":"2024-09-21T07:46:57.218717+0000","last_peered":"2024-09-21T07:46:57.218717+0000","last_clean":"2024-09-21T07:46:57.218717+0000","last_became_active":"2024-09-21T07:46:55.218044+0000","last_became_peered":"2024-09-21T07:46:55.218044+0000","last_unstale":"2024-09-21T07:46:57.218717+0000","last_undegraded":"2024-09-21T07:46:57.218717+0000","last_fullsized":"2024-09-21T07:46:57.218717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:18.860660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218669+0000","last_change":"2024-09-21T07:46:56.221089+0000","last_active":"2024-09-21T07:46:57.218669+0000","last_peered":"2024-09-21T07:46:57.218669+0000","last_clean":"2024-09-21T07:46:57.218669+0000","last_became_active":"2024-09-21T07:46:56.220746+0000","last_became_peered":"2024-09-21T07:46:56.220746+0000","last_unstale":"2024-09-21T07:46:57.218669+0000","last_undegraded":"2024-09-21T07:46:57.218669+0000","last_fullsized":"2024-09-21T07:46:57.218669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708222+0000","last_change":"2024-09-21T07:46:55.206317+0000","last_active":"2024-09-21T07:46:57.708222+0000","last_peered":"2024-09-21T07:46:57.708222+0000","last_clean":"2024-09-21T07:46:57.708222+0000","last_became_active":"2024-09-21T07:46:55.206000+0000","last_became_peered":"2024-09-21T07:46:55.206000+0000","last_unstale":"2024-09-21T07:46:57.708222+0000","last_undegraded":"2024-09-21T07:46:57.708222+0000","last_fullsized":"2024-09-21T07:46:57.708222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220947+0000","last_change":"2024-09-21T07:46:56.224113+0000","last_active":"2024-09-21T07:46:57.220947+0000","last_peered":"2024-09-21T07:46:57.220947+0000","last_clean":"2024-09-21T07:46:57.220947+0000","last_became_active":"2024-09-21T07:46:56.223793+0000","last_became_peered":"2024-09-21T07:46:56.223793+0000","last_unstale":"2024-09-21T07:46:57.220947+0000","last_undegraded":"2024-09-21T07:46:57.220947+0000","last_fullsized":"2024-09-21T07:46:57.220947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:15.172501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228976+0000","last_change":"2024-09-21T07:46:55.213497+0000","last_active":"2024-09-21T07:46:57.228976+0000","last_peered":"2024-09-21T07:46:57.228976+0000","last_clean":"2024-09-21T07:46:57.228976+0000","last_became_active":"2024-09-21T07:46:55.213173+0000","last_became_peered":"2024-09-21T07:46:55.213173+0000","last_unstale":"2024-09-21T07:46:57.228976+0000","last_undegraded":"2024-09-21T07:46:57.228976+0000","last_fullsized":"2024-09-21T07:46:57.228976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.908077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219686+0000","last_change":"2024-09-21T07:46:56.229383+0000","last_active":"2024-09-21T07:46:57.219686+0000","last_peered":"2024-09-21T07:46:57.219686+0000","last_clean":"2024-09-21T07:46:57.219686+0000","last_became_active":"2024-09-21T07:46:56.229012+0000","last_became_peered":"2024-09-21T07:46:56.229012+0000","last_unstale":"2024-09-21T07:46:57.219686+0000","last_undegraded":"2024-09-21T07:46:57.219686+0000","last_fullsized":"2024-09-21T07:46:57.219686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:56.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228698+0000","last_change":"2024-09-21T07:46:55.212681+0000","last_active":"2024-09-21T07:46:57.228698+0000","last_peered":"2024-09-21T07:46:57.228698+0000","last_clean":"2024-09-21T07:46:57.228698+0000","last_became_active":"2024-09-21T07:46:55.211907+0000","last_became_peered":"2024-09-21T07:46:55.211907+0000","last_unstale":"2024-09-21T07:46:57.228698+0000","last_undegraded":"2024-09-21T07:46:57.228698+0000","last_fullsized":"2024-09-21T07:46:57.228698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:30.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219680+0000","last_change":"2024-09-21T07:46:56.224792+0000","last_active":"2024-09-21T07:46:57.219680+0000","last_peered":"2024-09-21T07:46:57.219680+0000","last_clean":"2024-09-21T07:46:57.219680+0000","last_became_active":"2024-09-21T07:46:56.224554+0000","last_became_peered":"2024-09-21T07:46:56.224554+0000","last_unstale":"2024-09-21T07:46:57.219680+0000","last_undegraded":"2024-09-21T07:46:57.219680+0000","last_fullsized":"2024-09-21T07:46:57.219680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:31.069660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221096+0000","last_change":"2024-09-21T07:46:55.207287+0000","last_active":"2024-09-21T07:46:57.221096+0000","last_peered":"2024-09-21T07:46:57.221096+0000","last_clean":"2024-09-21T07:46:57.221096+0000","last_became_active":"2024-09-21T07:46:55.207008+0000","last_became_peered":"2024-09-21T07:46:55.207008+0000","last_unstale":"2024-09-21T07:46:57.221096+0000","last_undegraded":"2024-09-21T07:46:57.221096+0000","last_fullsized":"2024-09-21T07:46:57.221096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.602685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221088+0000","last_change":"2024-09-21T07:46:56.212672+0000","last_active":"2024-09-21T07:46:57.221088+0000","last_peered":"2024-09-21T07:46:57.221088+0000","last_clean":"2024-09-21T07:46:57.221088+0000","last_became_active":"2024-09-21T07:46:56.212383+0000","last_became_peered":"2024-09-21T07:46:56.212383+0000","last_unstale":"2024-09-21T07:46:57.221088+0000","last_undegraded":"2024-09-21T07:46:57.221088+0000","last_fullsized":"2024-09-21T07:46:57.221088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:13.284361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707883+0000","last_change":"2024-09-21T07:46:55.209727+0000","last_active":"2024-09-21T07:46:57.707883+0000","last_peered":"2024-09-21T07:46:57.707883+0000","last_clean":"2024-09-21T07:46:57.707883+0000","last_became_active":"2024-09-21T07:46:55.209437+0000","last_became_peered":"2024-09-21T07:46:55.209437+0000","last_unstale":"2024-09-21T07:46:57.707883+0000","last_undegraded":"2024-09-21T07:46:57.707883+0000","last_fullsized":"2024-09-21T07:46:57.707883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707613+0000","last_change":"2024-09-21T07:46:56.234403+0000","last_active":"2024-09-21T07:46:57.707613+0000","last_peered":"2024-09-21T07:46:57.707613+0000","last_clean":"2024-09-21T07:46:57.707613+0000","last_became_active":"2024-09-21T07:46:56.233998+0000","last_became_peered":"2024-09-21T07:46:56.233998+0000","last_unstale":"2024-09-21T07:46:57.707613+0000","last_undegraded":"2024-09-21T07:46:57.707613+0000","last_fullsized":"2024-09-21T07:46:57.707613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:52.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708952+0000","last_change":"2024-09-21T07:46:55.212435+0000","last_active":"2024-09-21T07:46:57.708952+0000","last_peered":"2024-09-21T07:46:57.708952+0000","last_clean":"2024-09-21T07:46:57.708952+0000","last_became_active":"2024-09-21T07:46:55.212093+0000","last_became_peered":"2024-09-21T07:46:55.212093+0000","last_unstale":"2024-09-21T07:46:57.708952+0000","last_undegraded":"2024-09-21T07:46:57.708952+0000","last_fullsized":"2024-09-21T07:46:57.708952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219594+0000","last_change":"2024-09-21T07:46:56.236480+0000","last_active":"2024-09-21T07:46:57.219594+0000","last_peered":"2024-09-21T07:46:57.219594+0000","last_clean":"2024-09-21T07:46:57.219594+0000","last_became_active":"2024-09-21T07:46:56.236111+0000","last_became_peered":"2024-09-21T07:46:56.236111+0000","last_unstale":"2024-09-21T07:46:57.219594+0000","last_undegraded":"2024-09-21T07:46:57.219594+0000","last_fullsized":"2024-09-21T07:46:57.219594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219132+0000","last_change":"2024-09-21T07:46:55.211160+0000","last_active":"2024-09-21T07:46:57.219132+0000","last_peered":"2024-09-21T07:46:57.219132+0000","last_clean":"2024-09-21T07:46:57.219132+0000","last_became_active":"2024-09-21T07:46:55.210797+0000","last_became_peered":"2024-09-21T07:46:55.210797+0000","last_unstale":"2024-09-21T07:46:57.219132+0000","last_undegraded":"2024-09-21T07:46:57.219132+0000","last_fullsized":"2024-09-21T07:46:57.219132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:33.882042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708245+0000","last_change":"2024-09-21T07:46:56.208519+0000","last_active":"2024-09-21T07:46:57.708245+0000","last_peered":"2024-09-21T07:46:57.708245+0000","last_clean":"2024-09-21T07:46:57.708245+0000","last_became_active":"2024-09-21T07:46:56.207695+0000","last_became_peered":"2024-09-21T07:46:56.207695+0000","last_unstale":"2024-09-21T07:46:57.708245+0000","last_undegraded":"2024-09-21T07:46:57.708245+0000","last_fullsized":"2024-09-21T07:46:57.708245+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218445+0000","last_change":"2024-09-21T07:46:55.205503+0000","last_active":"2024-09-21T07:46:57.218445+0000","last_peered":"2024-09-21T07:46:57.218445+0000","last_clean":"2024-09-21T07:46:57.218445+0000","last_became_active":"2024-09-21T07:46:55.204821+0000","last_became_peered":"2024-09-21T07:46:55.204821+0000","last_unstale":"2024-09-21T07:46:57.218445+0000","last_undegraded":"2024-09-21T07:46:57.218445+0000","last_fullsized":"2024-09-21T07:46:57.218445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220676+0000","last_change":"2024-09-21T07:46:56.212672+0000","last_active":"2024-09-21T07:46:57.220676+0000","last_peered":"2024-09-21T07:46:57.220676+0000","last_clean":"2024-09-21T07:46:57.220676+0000","last_became_active":"2024-09-21T07:46:56.212349+0000","last_became_peered":"2024-09-21T07:46:56.212349+0000","last_unstale":"2024-09-21T07:46:57.220676+0000","last_undegraded":"2024-09-21T07:46:57.220676+0000","last_fullsized":"2024-09-21T07:46:57.220676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:02.037379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218410+0000","last_change":"2024-09-21T07:46:55.216736+0000","last_active":"2024-09-21T07:46:57.218410+0000","last_peered":"2024-09-21T07:46:57.218410+0000","last_clean":"2024-09-21T07:46:57.218410+0000","last_became_active":"2024-09-21T07:46:55.216312+0000","last_became_peered":"2024-09-21T07:46:55.216312+0000","last_unstale":"2024-09-21T07:46:57.218410+0000","last_undegraded":"2024-09-21T07:46:57.218410+0000","last_fullsized":"2024-09-21T07:46:57.218410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.195061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221175+0000","last_change":"2024-09-21T07:46:56.209062+0000","last_active":"2024-09-21T07:46:57.221175+0000","last_peered":"2024-09-21T07:46:57.221175+0000","last_clean":"2024-09-21T07:46:57.221175+0000","last_became_active":"2024-09-21T07:46:56.208769+0000","last_became_peered":"2024-09-21T07:46:56.208769+0000","last_unstale":"2024-09-21T07:46:57.221175+0000","last_undegraded":"2024-09-21T07:46:57.221175+0000","last_fullsized":"2024-09-21T07:46:57.221175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:50.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220824+0000","last_change":"2024-09-21T07:46:55.218143+0000","last_active":"2024-09-21T07:46:57.220824+0000","last_peered":"2024-09-21T07:46:57.220824+0000","last_clean":"2024-09-21T07:46:57.220824+0000","last_became_active":"2024-09-21T07:46:55.217800+0000","last_became_peered":"2024-09-21T07:46:55.217800+0000","last_unstale":"2024-09-21T07:46:57.220824+0000","last_undegraded":"2024-09-21T07:46:57.220824+0000","last_fullsized":"2024-09-21T07:46:57.220824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:57.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707549+0000","last_change":"2024-09-21T07:46:56.218140+0000","last_active":"2024-09-21T07:46:57.707549+0000","last_peered":"2024-09-21T07:46:57.707549+0000","last_clean":"2024-09-21T07:46:57.707549+0000","last_became_active":"2024-09-21T07:46:56.217824+0000","last_became_peered":"2024-09-21T07:46:56.217824+0000","last_unstale":"2024-09-21T07:46:57.707549+0000","last_undegraded":"2024-09-21T07:46:57.707549+0000","last_fullsized":"2024-09-21T07:46:57.707549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:46.806098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709120+0000","last_change":"2024-09-21T07:46:55.212733+0000","last_active":"2024-09-21T07:46:57.709120+0000","last_peered":"2024-09-21T07:46:57.709120+0000","last_clean":"2024-09-21T07:46:57.709120+0000","last_became_active":"2024-09-21T07:46:55.212478+0000","last_became_peered":"2024-09-21T07:46:55.212478+0000","last_unstale":"2024-09-21T07:46:57.709120+0000","last_undegraded":"2024-09-21T07:46:57.709120+0000","last_fullsized":"2024-09-21T07:46:57.709120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228788+0000","last_change":"2024-09-21T07:46:56.223493+0000","last_active":"2024-09-21T07:46:57.228788+0000","last_peered":"2024-09-21T07:46:57.228788+0000","last_clean":"2024-09-21T07:46:57.228788+0000","last_became_active":"2024-09-21T07:46:56.223060+0000","last_became_peered":"2024-09-21T07:46:56.223060+0000","last_unstale":"2024-09-21T07:46:57.228788+0000","last_undegraded":"2024-09-21T07:46:57.228788+0000","last_fullsized":"2024-09-21T07:46:57.228788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221247+0000","last_change":"2024-09-21T07:46:55.213737+0000","last_active":"2024-09-21T07:46:57.221247+0000","last_peered":"2024-09-21T07:46:57.221247+0000","last_clean":"2024-09-21T07:46:57.221247+0000","last_became_active":"2024-09-21T07:46:55.213452+0000","last_became_peered":"2024-09-21T07:46:55.213452+0000","last_unstale":"2024-09-21T07:46:57.221247+0000","last_undegraded":"2024-09-21T07:46:57.221247+0000","last_fullsized":"2024-09-21T07:46:57.221247+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:38.058509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220927+0000","last_change":"2024-09-21T07:46:56.232119+0000","last_active":"2024-09-21T07:46:57.220927+0000","last_peered":"2024-09-21T07:46:57.220927+0000","last_clean":"2024-09-21T07:46:57.220927+0000","last_became_active":"2024-09-21T07:46:56.231764+0000","last_became_peered":"2024-09-21T07:46:56.231764+0000","last_unstale":"2024-09-21T07:46:57.220927+0000","last_undegraded":"2024-09-21T07:46:57.220927+0000","last_fullsized":"2024-09-21T07:46:57.220927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:35.746093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707452+0000","last_change":"2024-09-21T07:46:55.206900+0000","last_active":"2024-09-21T07:46:57.707452+0000","last_peered":"2024-09-21T07:46:57.707452+0000","last_clean":"2024-09-21T07:46:57.707452+0000","last_became_active":"2024-09-21T07:46:55.206441+0000","last_became_peered":"2024-09-21T07:46:55.206441+0000","last_unstale":"2024-09-21T07:46:57.707452+0000","last_undegraded":"2024-09-21T07:46:57.707452+0000","last_fullsized":"2024-09-21T07:46:57.707452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:52.569990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707559+0000","last_change":"2024-09-21T07:46:56.233293+0000","last_active":"2024-09-21T07:46:57.707559+0000","last_peered":"2024-09-21T07:46:57.707559+0000","last_clean":"2024-09-21T07:46:57.707559+0000","last_became_active":"2024-09-21T07:46:56.232951+0000","last_became_peered":"2024-09-21T07:46:56.232951+0000","last_unstale":"2024-09-21T07:46:57.707559+0000","last_undegraded":"2024-09-21T07:46:57.707559+0000","last_fullsized":"2024-09-21T07:46:57.707559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:58.570223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228732+0000","last_change":"2024-09-21T07:46:55.206362+0000","last_active":"2024-09-21T07:46:57.228732+0000","last_peered":"2024-09-21T07:46:57.228732+0000","last_clean":"2024-09-21T07:46:57.228732+0000","last_became_active":"2024-09-21T07:46:55.206044+0000","last_became_peered":"2024-09-21T07:46:55.206044+0000","last_unstale":"2024-09-21T07:46:57.228732+0000","last_undegraded":"2024-09-21T07:46:57.228732+0000","last_fullsized":"2024-09-21T07:46:57.228732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:20.127719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228770+0000","last_change":"2024-09-21T07:46:56.221448+0000","last_active":"2024-09-21T07:46:57.228770+0000","last_peered":"2024-09-21T07:46:57.228770+0000","last_clean":"2024-09-21T07:46:57.228770+0000","last_became_active":"2024-09-21T07:46:56.221116+0000","last_became_peered":"2024-09-21T07:46:56.221116+0000","last_unstale":"2024-09-21T07:46:57.228770+0000","last_undegraded":"2024-09-21T07:46:57.228770+0000","last_fullsized":"2024-09-21T07:46:57.228770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708582+0000","last_change":"2024-09-21T07:46:55.213426+0000","last_active":"2024-09-21T07:46:57.708582+0000","last_peered":"2024-09-21T07:46:57.708582+0000","last_clean":"2024-09-21T07:46:57.708582+0000","last_became_active":"2024-09-21T07:46:55.211244+0000","last_became_peered":"2024-09-21T07:46:55.211244+0000","last_unstale":"2024-09-21T07:46:57.708582+0000","last_undegraded":"2024-09-21T07:46:57.708582+0000","last_fullsized":"2024-09-21T07:46:57.708582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707540+0000","last_change":"2024-09-21T07:46:56.232335+0000","last_active":"2024-09-21T07:46:57.707540+0000","last_peered":"2024-09-21T07:46:57.707540+0000","last_clean":"2024-09-21T07:46:57.707540+0000","last_became_active":"2024-09-21T07:46:56.231953+0000","last_became_peered":"2024-09-21T07:46:56.231953+0000","last_unstale":"2024-09-21T07:46:57.707540+0000","last_undegraded":"2024-09-21T07:46:57.707540+0000","last_fullsized":"2024-09-21T07:46:57.707540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:04.369378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707946+0000","last_change":"2024-09-21T07:46:55.214923+0000","last_active":"2024-09-21T07:46:57.707946+0000","last_peered":"2024-09-21T07:46:57.707946+0000","last_clean":"2024-09-21T07:46:57.707946+0000","last_became_active":"2024-09-21T07:46:55.214607+0000","last_became_peered":"2024-09-21T07:46:55.214607+0000","last_unstale":"2024-09-21T07:46:57.707946+0000","last_undegraded":"2024-09-21T07:46:57.707946+0000","last_fullsized":"2024-09-21T07:46:57.707946+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.689624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706862+0000","last_change":"2024-09-21T07:46:56.214731+0000","last_active":"2024-09-21T07:46:57.706862+0000","last_peered":"2024-09-21T07:46:57.706862+0000","last_clean":"2024-09-21T07:46:57.706862+0000","last_became_active":"2024-09-21T07:46:56.214437+0000","last_became_peered":"2024-09-21T07:46:56.214437+0000","last_unstale":"2024-09-21T07:46:57.706862+0000","last_undegraded":"2024-09-21T07:46:57.706862+0000","last_fullsized":"2024-09-21T07:46:57.706862+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:39.838252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220502+0000","last_change":"2024-09-21T07:46:55.208755+0000","last_active":"2024-09-21T07:46:57.220502+0000","last_peered":"2024-09-21T07:46:57.220502+0000","last_clean":"2024-09-21T07:46:57.220502+0000","last_became_active":"2024-09-21T07:46:55.207806+0000","last_became_peered":"2024-09-21T07:46:55.207806+0000","last_unstale":"2024-09-21T07:46:57.220502+0000","last_undegraded":"2024-09-21T07:46:57.220502+0000","last_fullsized":"2024-09-21T07:46:57.220502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:29.396988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220825+0000","last_change":"2024-09-21T07:46:56.232337+0000","last_active":"2024-09-21T07:46:57.220825+0000","last_peered":"2024-09-21T07:46:57.220825+0000","last_clean":"2024-09-21T07:46:57.220825+0000","last_became_active":"2024-09-21T07:46:56.230029+0000","last_became_peered":"2024-09-21T07:46:56.230029+0000","last_unstale":"2024-09-21T07:46:57.220825+0000","last_undegraded":"2024-09-21T07:46:57.220825+0000","last_fullsized":"2024-09-21T07:46:57.220825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.727543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708019+0000","last_change":"2024-09-21T07:46:55.211058+0000","last_active":"2024-09-21T07:46:57.708019+0000","last_peered":"2024-09-21T07:46:57.708019+0000","last_clean":"2024-09-21T07:46:57.708019+0000","last_became_active":"2024-09-21T07:46:55.210625+0000","last_became_peered":"2024-09-21T07:46:55.210625+0000","last_unstale":"2024-09-21T07:46:57.708019+0000","last_undegraded":"2024-09-21T07:46:57.708019+0000","last_fullsized":"2024-09-21T07:46:57.708019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221610+0000","last_change":"2024-09-21T07:46:56.229165+0000","last_active":"2024-09-21T07:46:57.221610+0000","last_peered":"2024-09-21T07:46:57.221610+0000","last_clean":"2024-09-21T07:46:57.221610+0000","last_became_active":"2024-09-21T07:46:56.228862+0000","last_became_peered":"2024-09-21T07:46:56.228862+0000","last_unstale":"2024-09-21T07:46:57.221610+0000","last_undegraded":"2024-09-21T07:46:57.221610+0000","last_fullsized":"2024-09-21T07:46:57.221610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:22.071500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228909+0000","last_change":"2024-09-21T07:46:55.209507+0000","last_active":"2024-09-21T07:46:57.228909+0000","last_peered":"2024-09-21T07:46:57.228909+0000","last_clean":"2024-09-21T07:46:57.228909+0000","last_became_active":"2024-09-21T07:46:55.209003+0000","last_became_peered":"2024-09-21T07:46:55.209003+0000","last_unstale":"2024-09-21T07:46:57.228909+0000","last_undegraded":"2024-09-21T07:46:57.228909+0000","last_fullsized":"2024-09-21T07:46:57.228909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:25.605174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221110+0000","last_change":"2024-09-21T07:46:56.241826+0000","last_active":"2024-09-21T07:46:57.221110+0000","last_peered":"2024-09-21T07:46:57.221110+0000","last_clean":"2024-09-21T07:46:57.221110+0000","last_became_active":"2024-09-21T07:46:56.241465+0000","last_became_peered":"2024-09-21T07:46:56.241465+0000","last_unstale":"2024-09-21T07:46:57.221110+0000","last_undegraded":"2024-09-21T07:46:57.221110+0000","last_fullsized":"2024-09-21T07:46:57.221110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:13.411156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708039+0000","last_change":"2024-09-21T07:46:55.209001+0000","last_active":"2024-09-21T07:46:57.708039+0000","last_peered":"2024-09-21T07:46:57.708039+0000","last_clean":"2024-09-21T07:46:57.708039+0000","last_became_active":"2024-09-21T07:46:55.208596+0000","last_became_peered":"2024-09-21T07:46:55.208596+0000","last_unstale":"2024-09-21T07:46:57.708039+0000","last_undegraded":"2024-09-21T07:46:57.708039+0000","last_fullsized":"2024-09-21T07:46:57.708039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707408+0000","last_change":"2024-09-21T07:46:56.227045+0000","last_active":"2024-09-21T07:46:57.707408+0000","last_peered":"2024-09-21T07:46:57.707408+0000","last_clean":"2024-09-21T07:46:57.707408+0000","last_became_active":"2024-09-21T07:46:56.220245+0000","last_became_peered":"2024-09-21T07:46:56.220245+0000","last_unstale":"2024-09-21T07:46:57.707408+0000","last_undegraded":"2024-09-21T07:46:57.707408+0000","last_fullsized":"2024-09-21T07:46:57.707408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707914+0000","last_change":"2024-09-21T07:46:55.210484+0000","last_active":"2024-09-21T07:46:57.707914+0000","last_peered":"2024-09-21T07:46:57.707914+0000","last_clean":"2024-09-21T07:46:57.707914+0000","last_became_active":"2024-09-21T07:46:55.210140+0000","last_became_peered":"2024-09-21T07:46:55.210140+0000","last_unstale":"2024-09-21T07:46:57.707914+0000","last_undegraded":"2024-09-21T07:46:57.707914+0000","last_fullsized":"2024-09-21T07:46:57.707914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219755+0000","last_change":"2024-09-21T07:46:56.233200+0000","last_active":"2024-09-21T07:46:57.219755+0000","last_peered":"2024-09-21T07:46:57.219755+0000","last_clean":"2024-09-21T07:46:57.219755+0000","last_became_active":"2024-09-21T07:46:56.232818+0000","last_became_peered":"2024-09-21T07:46:56.232818+0000","last_unstale":"2024-09-21T07:46:57.219755+0000","last_undegraded":"2024-09-21T07:46:57.219755+0000","last_fullsized":"2024-09-21T07:46:57.219755+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.180535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707865+0000","last_change":"2024-09-21T07:46:55.215083+0000","last_active":"2024-09-21T07:46:57.707865+0000","last_peered":"2024-09-21T07:46:57.707865+0000","last_clean":"2024-09-21T07:46:57.707865+0000","last_became_active":"2024-09-21T07:46:55.214531+0000","last_became_peered":"2024-09-21T07:46:55.214531+0000","last_unstale":"2024-09-21T07:46:57.707865+0000","last_undegraded":"2024-09-21T07:46:57.707865+0000","last_fullsized":"2024-09-21T07:46:57.707865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218812+0000","last_change":"2024-09-21T07:46:56.220432+0000","last_active":"2024-09-21T07:46:57.218812+0000","last_peered":"2024-09-21T07:46:57.218812+0000","last_clean":"2024-09-21T07:46:57.218812+0000","last_became_active":"2024-09-21T07:46:56.219527+0000","last_became_peered":"2024-09-21T07:46:56.219527+0000","last_unstale":"2024-09-21T07:46:57.218812+0000","last_undegraded":"2024-09-21T07:46:57.218812+0000","last_fullsized":"2024-09-21T07:46:57.218812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:30.402592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708593+0000","last_change":"2024-09-21T07:46:55.212708+0000","last_active":"2024-09-21T07:46:57.708593+0000","last_peered":"2024-09-21T07:46:57.708593+0000","last_clean":"2024-09-21T07:46:57.708593+0000","last_became_active":"2024-09-21T07:46:55.212115+0000","last_became_peered":"2024-09-21T07:46:55.212115+0000","last_unstale":"2024-09-21T07:46:57.708593+0000","last_undegraded":"2024-09-21T07:46:57.708593+0000","last_fullsized":"2024-09-21T07:46:57.708593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220717+0000","last_change":"2024-09-21T07:46:56.218638+0000","last_active":"2024-09-21T07:46:57.220717+0000","last_peered":"2024-09-21T07:46:57.220717+0000","last_clean":"2024-09-21T07:46:57.220717+0000","last_became_active":"2024-09-21T07:46:56.218288+0000","last_became_peered":"2024-09-21T07:46:56.218288+0000","last_unstale":"2024-09-21T07:46:57.220717+0000","last_undegraded":"2024-09-21T07:46:57.220717+0000","last_fullsized":"2024-09-21T07:46:57.220717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:57.104592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219828+0000","last_change":"2024-09-21T07:46:55.214726+0000","last_active":"2024-09-21T07:46:57.219828+0000","last_peered":"2024-09-21T07:46:57.219828+0000","last_clean":"2024-09-21T07:46:57.219828+0000","last_became_active":"2024-09-21T07:46:55.214331+0000","last_became_peered":"2024-09-21T07:46:55.214331+0000","last_unstale":"2024-09-21T07:46:57.219828+0000","last_undegraded":"2024-09-21T07:46:57.219828+0000","last_fullsized":"2024-09-21T07:46:57.219828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707852+0000","last_change":"2024-09-21T07:46:56.220549+0000","last_active":"2024-09-21T07:46:57.707852+0000","last_peered":"2024-09-21T07:46:57.707852+0000","last_clean":"2024-09-21T07:46:57.707852+0000","last_became_active":"2024-09-21T07:46:56.220238+0000","last_became_peered":"2024-09-21T07:46:56.220238+0000","last_unstale":"2024-09-21T07:46:57.707852+0000","last_undegraded":"2024-09-21T07:46:57.707852+0000","last_fullsized":"2024-09-21T07:46:57.707852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:59.146665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221352+0000","last_change":"2024-09-21T07:46:55.218103+0000","last_active":"2024-09-21T07:46:57.221352+0000","last_peered":"2024-09-21T07:46:57.221352+0000","last_clean":"2024-09-21T07:46:57.221352+0000","last_became_active":"2024-09-21T07:46:55.217796+0000","last_became_peered":"2024-09-21T07:46:55.217796+0000","last_unstale":"2024-09-21T07:46:57.221352+0000","last_undegraded":"2024-09-21T07:46:57.221352+0000","last_fullsized":"2024-09-21T07:46:57.221352+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:44.455935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":133,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706779+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:57.706779+0000","last_peered":"2024-09-21T07:46:57.706779+0000","last_clean":"2024-09-21T07:46:57.706779+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:57.706779+0000","last_undegraded":"2024-09-21T07:46:57.706779+0000","last_fullsized":"2024-09-21T07:46:57.706779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476755,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":150002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476755,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":149260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476752,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":739664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476752,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":149316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509455,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":397312,"data_stored":150838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509456,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":740176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":150860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542160,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":149248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:05.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.844+0000 7f0f61800640 1 -- 172.21.15.88:0/1095687240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c15fae0 msgr2=0x7f0f5c166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.844+0000 7f0f61800640 1 --2- 172.21.15.88:0/1095687240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c15fae0 0x7f0f5c166550 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f0f54009830 tx=0x7f0f5402a530 comp rx=0 tx=0).stop 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.844+0000 7f0f61800640 1 -- 172.21.15.88:0/1095687240 shutdown_connections 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.844+0000 7f0f61800640 1 --2- 172.21.15.88:0/1095687240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c15fae0 0x7f0f5c166550 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.844+0000 7f0f61800640 1 -- 172.21.15.88:0/1095687240 >> 172.21.15.88:0/1095687240 conn(0x7f0f5c06d730 msgr2=0x7f0f5c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.844+0000 7f0f61800640 1 -- 172.21.15.88:0/1095687240 shutdown_connections 2024-09-21T07:47:05.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.845+0000 7f0f61800640 1 -- 172.21.15.88:0/1095687240 wait complete. 2024-09-21T07:47:05.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.847+0000 7f0f61800640 1 Processor -- start 2024-09-21T07:47:05.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.847+0000 7f0f61800640 1 -- start start 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f61800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c0aebc0 0x7f0f5c149220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f61800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 0x7f0f5c099020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f61800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c0cedf0 0x7f0f5c15e0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f61800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c166e20 con 0x7f0f5c152920 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f61800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f5c166ca0 con 0x7f0f5c0aebc0 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f61800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f5c166fa0 con 0x7f0f5c0cedf0 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f52000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 0x7f0f5c099020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f52000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 0x7f0f5c099020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48340/0 (socket says 172.21.15.88:48340) 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.848+0000 7f0f52000640 1 -- 172.21.15.88:0/2683455582 learned_addr learned my addr 172.21.15.88:0/2683455582 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f5aa00640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c0aebc0 0x7f0f5c149220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f5b400640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c0cedf0 0x7f0f5c15e0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f52000640 1 -- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c0cedf0 msgr2=0x7f0f5c15e0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f52000640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c0cedf0 0x7f0f5c15e0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f52000640 1 -- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c0aebc0 msgr2=0x7f0f5c149220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:05.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f52000640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c0aebc0 0x7f0f5c149220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f52000640 1 -- 172.21.15.88:0/2683455582 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5c099560 con 0x7f0f5c152920 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f52000640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 0x7f0f5c099020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c3a0 tx=0x7f0f4400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44017070 con 0x7f0f5c152920 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f440035f0 con 0x7f0f5c152920 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5c14e6f0 con 0x7f0f5c152920 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.849+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f44003a40 con 0x7f0f5c152920 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.850+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f5c148460 con 0x7f0f5c152920 2024-09-21T07:47:05.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.851+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f5c0b2750 con 0x7f0f5c152920 2024-09-21T07:47:05.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.852+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0f4401a070 con 0x7f0f5c152920 2024-09-21T07:47:05.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.853+0000 7f0f53400640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f38078bc0 0x7f0f3807b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:05.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.853+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0f4409a090 con 0x7f0f5c152920 2024-09-21T07:47:05.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.854+0000 7f0f5aa00640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f38078bc0 0x7f0f3807b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:05.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.855+0000 7f0f5aa00640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f38078bc0 0x7f0f3807b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c007fd0 tx=0x7f0f4c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:05.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:05.858+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f5c099560 con 0x7f0f5c152920 2024-09-21T07:47:06.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.046+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f5c15c4c0 con 0x7f0f38078bc0 2024-09-21T07:47:06.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.051+0000 7f0f53400640 1 -- 172.21.15.88:0/2683455582 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251832 (secure 0 0 0) 0x7f0f5c15c4c0 con 0x7f0f38078bc0 2024-09-21T07:47:06.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:06.057 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f38078bc0 msgr2=0x7f0f3807b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f38078bc0 0x7f0f3807b0d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c007fd0 tx=0x7f0f4c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 msgr2=0x7f0f5c099020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 0x7f0f5c099020 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f0f4400c3a0 tx=0x7f0f4400c860 comp rx=0 tx=0).stop 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 shutdown_connections 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f38078bc0 0x7f0f3807b0d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f5c0cedf0 0x7f0f5c15e0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f5c152920 0x7f0f5c099020 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 --2- 172.21.15.88:0/2683455582 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f5c0aebc0 0x7f0f5c149220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 >> 172.21.15.88:0/2683455582 conn(0x7f0f5c06d730 msgr2=0x7f0f5c15f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 shutdown_connections 2024-09-21T07:47:06.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.055+0000 7f0f61800640 1 -- 172.21.15.88:0/2683455582 wait complete. 2024-09-21T07:47:06.079 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":67,"stamp":"2024-09-21T07:47:05.868427+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":220480,"kb_used_data":4232,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754240,"statfs":{"total":773094113280,"available":772868341760,"internally_reserved":0,"allocated":4333568,"data_stored":2379364,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.667060"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709175+0000","last_change":"2024-09-21T07:46:56.222670+0000","last_active":"2024-09-21T07:46:57.709175+0000","last_peered":"2024-09-21T07:46:57.709175+0000","last_clean":"2024-09-21T07:46:57.709175+0000","last_became_active":"2024-09-21T07:46:56.220432+0000","last_became_peered":"2024-09-21T07:46:56.220432+0000","last_unstale":"2024-09-21T07:46:57.709175+0000","last_undegraded":"2024-09-21T07:46:57.709175+0000","last_fullsized":"2024-09-21T07:46:57.709175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:19.419293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228903+0000","last_change":"2024-09-21T07:46:56.229990+0000","last_active":"2024-09-21T07:46:57.228903+0000","last_peered":"2024-09-21T07:46:57.228903+0000","last_clean":"2024-09-21T07:46:57.228903+0000","last_became_active":"2024-09-21T07:46:56.229655+0000","last_became_peered":"2024-09-21T07:46:56.229655+0000","last_unstale":"2024-09-21T07:46:57.228903+0000","last_undegraded":"2024-09-21T07:46:57.228903+0000","last_fullsized":"2024-09-21T07:46:57.228903+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707653+0000","last_change":"2024-09-21T07:46:56.209979+0000","last_active":"2024-09-21T07:46:57.707653+0000","last_peered":"2024-09-21T07:46:57.707653+0000","last_clean":"2024-09-21T07:46:57.707653+0000","last_became_active":"2024-09-21T07:46:56.209344+0000","last_became_peered":"2024-09-21T07:46:56.209344+0000","last_unstale":"2024-09-21T07:46:57.707653+0000","last_undegraded":"2024-09-21T07:46:57.707653+0000","last_fullsized":"2024-09-21T07:46:57.707653+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:37.808616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219416+0000","last_change":"2024-09-21T07:46:56.235757+0000","last_active":"2024-09-21T07:46:57.219416+0000","last_peered":"2024-09-21T07:46:57.219416+0000","last_clean":"2024-09-21T07:46:57.219416+0000","last_became_active":"2024-09-21T07:46:56.228680+0000","last_became_peered":"2024-09-21T07:46:56.228680+0000","last_unstale":"2024-09-21T07:46:57.219416+0000","last_undegraded":"2024-09-21T07:46:57.219416+0000","last_fullsized":"2024-09-21T07:46:57.219416+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228641+0000","last_change":"2024-09-21T07:46:56.229963+0000","last_active":"2024-09-21T07:46:57.228641+0000","last_peered":"2024-09-21T07:46:57.228641+0000","last_clean":"2024-09-21T07:46:57.228641+0000","last_became_active":"2024-09-21T07:46:56.229617+0000","last_became_peered":"2024-09-21T07:46:56.229617+0000","last_unstale":"2024-09-21T07:46:57.228641+0000","last_undegraded":"2024-09-21T07:46:57.228641+0000","last_fullsized":"2024-09-21T07:46:57.228641+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219492+0000","last_change":"2024-09-21T07:46:56.229684+0000","last_active":"2024-09-21T07:46:57.219492+0000","last_peered":"2024-09-21T07:46:57.219492+0000","last_clean":"2024-09-21T07:46:57.219492+0000","last_became_active":"2024-09-21T07:46:56.229339+0000","last_became_peered":"2024-09-21T07:46:56.229339+0000","last_unstale":"2024-09-21T07:46:57.219492+0000","last_undegraded":"2024-09-21T07:46:57.219492+0000","last_fullsized":"2024-09-21T07:46:57.219492+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:21.621424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708409+0000","last_change":"2024-09-21T07:46:56.212908+0000","last_active":"2024-09-21T07:46:57.708409+0000","last_peered":"2024-09-21T07:46:57.708409+0000","last_clean":"2024-09-21T07:46:57.708409+0000","last_became_active":"2024-09-21T07:46:56.210441+0000","last_became_peered":"2024-09-21T07:46:56.210441+0000","last_unstale":"2024-09-21T07:46:57.708409+0000","last_undegraded":"2024-09-21T07:46:57.708409+0000","last_fullsized":"2024-09-21T07:46:57.708409+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708011+0000","last_change":"2024-09-21T07:46:56.221463+0000","last_active":"2024-09-21T07:46:57.708011+0000","last_peered":"2024-09-21T07:46:57.708011+0000","last_clean":"2024-09-21T07:46:57.708011+0000","last_became_active":"2024-09-21T07:46:56.221112+0000","last_became_peered":"2024-09-21T07:46:56.221112+0000","last_unstale":"2024-09-21T07:46:57.708011+0000","last_undegraded":"2024-09-21T07:46:57.708011+0000","last_fullsized":"2024-09-21T07:46:57.708011+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:24:32.488514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707286+0000","last_change":"2024-09-21T07:46:56.210833+0000","last_active":"2024-09-21T07:46:57.707286+0000","last_peered":"2024-09-21T07:46:57.707286+0000","last_clean":"2024-09-21T07:46:57.707286+0000","last_became_active":"2024-09-21T07:46:56.209936+0000","last_became_peered":"2024-09-21T07:46:56.209936+0000","last_unstale":"2024-09-21T07:46:57.707286+0000","last_undegraded":"2024-09-21T07:46:57.707286+0000","last_fullsized":"2024-09-21T07:46:57.707286+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:48.704288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220083+0000","last_change":"2024-09-21T07:46:56.231062+0000","last_active":"2024-09-21T07:46:57.220083+0000","last_peered":"2024-09-21T07:46:57.220083+0000","last_clean":"2024-09-21T07:46:57.220083+0000","last_became_active":"2024-09-21T07:46:56.229427+0000","last_became_peered":"2024-09-21T07:46:56.229427+0000","last_unstale":"2024-09-21T07:46:57.220083+0000","last_undegraded":"2024-09-21T07:46:57.220083+0000","last_fullsized":"2024-09-21T07:46:57.220083+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:47.565734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221307+0000","last_change":"2024-09-21T07:46:56.215059+0000","last_active":"2024-09-21T07:46:57.221307+0000","last_peered":"2024-09-21T07:46:57.221307+0000","last_clean":"2024-09-21T07:46:57.221307+0000","last_became_active":"2024-09-21T07:46:56.212314+0000","last_became_peered":"2024-09-21T07:46:56.212314+0000","last_unstale":"2024-09-21T07:46:57.221307+0000","last_undegraded":"2024-09-21T07:46:57.221307+0000","last_fullsized":"2024-09-21T07:46:57.221307+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:32.152592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707294+0000","last_change":"2024-09-21T07:46:56.222911+0000","last_active":"2024-09-21T07:46:57.707294+0000","last_peered":"2024-09-21T07:46:57.707294+0000","last_clean":"2024-09-21T07:46:57.707294+0000","last_became_active":"2024-09-21T07:46:56.222587+0000","last_became_peered":"2024-09-21T07:46:56.222587+0000","last_unstale":"2024-09-21T07:46:57.707294+0000","last_undegraded":"2024-09-21T07:46:57.707294+0000","last_fullsized":"2024-09-21T07:46:57.707294+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:18.632830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707947+0000","last_change":"2024-09-21T07:46:56.237624+0000","last_active":"2024-09-21T07:46:57.707947+0000","last_peered":"2024-09-21T07:46:57.707947+0000","last_clean":"2024-09-21T07:46:57.707947+0000","last_became_active":"2024-09-21T07:46:56.237037+0000","last_became_peered":"2024-09-21T07:46:56.237037+0000","last_unstale":"2024-09-21T07:46:57.707947+0000","last_undegraded":"2024-09-21T07:46:57.707947+0000","last_fullsized":"2024-09-21T07:46:57.707947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:43.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219004+0000","last_change":"2024-09-21T07:46:56.239271+0000","last_active":"2024-09-21T07:46:57.219004+0000","last_peered":"2024-09-21T07:46:57.219004+0000","last_clean":"2024-09-21T07:46:57.219004+0000","last_became_active":"2024-09-21T07:46:56.238936+0000","last_became_peered":"2024-09-21T07:46:56.238936+0000","last_unstale":"2024-09-21T07:46:57.219004+0000","last_undegraded":"2024-09-21T07:46:57.219004+0000","last_fullsized":"2024-09-21T07:46:57.219004+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706826+0000","last_change":"2024-09-21T07:46:56.220351+0000","last_active":"2024-09-21T07:46:57.706826+0000","last_peered":"2024-09-21T07:46:57.706826+0000","last_clean":"2024-09-21T07:46:57.706826+0000","last_became_active":"2024-09-21T07:46:56.220039+0000","last_became_peered":"2024-09-21T07:46:56.220039+0000","last_unstale":"2024-09-21T07:46:57.706826+0000","last_undegraded":"2024-09-21T07:46:57.706826+0000","last_fullsized":"2024-09-21T07:46:57.706826+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707569+0000","last_change":"2024-09-21T07:46:56.219494+0000","last_active":"2024-09-21T07:46:57.707569+0000","last_peered":"2024-09-21T07:46:57.707569+0000","last_clean":"2024-09-21T07:46:57.707569+0000","last_became_active":"2024-09-21T07:46:56.219194+0000","last_became_peered":"2024-09-21T07:46:56.219194+0000","last_unstale":"2024-09-21T07:46:57.707569+0000","last_undegraded":"2024-09-21T07:46:57.707569+0000","last_fullsized":"2024-09-21T07:46:57.707569+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:54.406099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221559+0000","last_change":"2024-09-21T07:46:56.222353+0000","last_active":"2024-09-21T07:46:57.221559+0000","last_peered":"2024-09-21T07:46:57.221559+0000","last_clean":"2024-09-21T07:46:57.221559+0000","last_became_active":"2024-09-21T07:46:56.222045+0000","last_became_peered":"2024-09-21T07:46:56.222045+0000","last_unstale":"2024-09-21T07:46:57.221559+0000","last_undegraded":"2024-09-21T07:46:57.221559+0000","last_fullsized":"2024-09-21T07:46:57.221559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:44:43.027555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220865+0000","last_change":"2024-09-21T07:46:56.224534+0000","last_active":"2024-09-21T07:46:57.220865+0000","last_peered":"2024-09-21T07:46:57.220865+0000","last_clean":"2024-09-21T07:46:57.220865+0000","last_became_active":"2024-09-21T07:46:56.224167+0000","last_became_peered":"2024-09-21T07:46:56.224167+0000","last_unstale":"2024-09-21T07:46:57.220865+0000","last_undegraded":"2024-09-21T07:46:57.220865+0000","last_fullsized":"2024-09-21T07:46:57.220865+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:45:18.755342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220883+0000","last_change":"2024-09-21T07:46:56.228375+0000","last_active":"2024-09-21T07:46:57.220883+0000","last_peered":"2024-09-21T07:46:57.220883+0000","last_clean":"2024-09-21T07:46:57.220883+0000","last_became_active":"2024-09-21T07:46:56.225793+0000","last_became_peered":"2024-09-21T07:46:56.225793+0000","last_unstale":"2024-09-21T07:46:57.220883+0000","last_undegraded":"2024-09-21T07:46:57.220883+0000","last_fullsized":"2024-09-21T07:46:57.220883+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:20:46.376207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708077+0000","last_change":"2024-09-21T07:46:56.226928+0000","last_active":"2024-09-21T07:46:57.708077+0000","last_peered":"2024-09-21T07:46:57.708077+0000","last_clean":"2024-09-21T07:46:57.708077+0000","last_became_active":"2024-09-21T07:46:56.226560+0000","last_became_peered":"2024-09-21T07:46:56.226560+0000","last_unstale":"2024-09-21T07:46:57.708077+0000","last_undegraded":"2024-09-21T07:46:57.708077+0000","last_fullsized":"2024-09-21T07:46:57.708077+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707252+0000","last_change":"2024-09-21T07:46:56.225740+0000","last_active":"2024-09-21T07:46:57.707252+0000","last_peered":"2024-09-21T07:46:57.707252+0000","last_clean":"2024-09-21T07:46:57.707252+0000","last_became_active":"2024-09-21T07:46:56.225360+0000","last_became_peered":"2024-09-21T07:46:56.225360+0000","last_unstale":"2024-09-21T07:46:57.707252+0000","last_undegraded":"2024-09-21T07:46:57.707252+0000","last_fullsized":"2024-09-21T07:46:57.707252+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:44.291034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708446+0000","last_change":"2024-09-21T07:46:56.230572+0000","last_active":"2024-09-21T07:46:57.708446+0000","last_peered":"2024-09-21T07:46:57.708446+0000","last_clean":"2024-09-21T07:46:57.708446+0000","last_became_active":"2024-09-21T07:46:56.230237+0000","last_became_peered":"2024-09-21T07:46:56.230237+0000","last_unstale":"2024-09-21T07:46:57.708446+0000","last_undegraded":"2024-09-21T07:46:57.708446+0000","last_fullsized":"2024-09-21T07:46:57.708446+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.608289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707640+0000","last_change":"2024-09-21T07:46:56.227950+0000","last_active":"2024-09-21T07:46:57.707640+0000","last_peered":"2024-09-21T07:46:57.707640+0000","last_clean":"2024-09-21T07:46:57.707640+0000","last_became_active":"2024-09-21T07:46:56.227647+0000","last_became_peered":"2024-09-21T07:46:56.227647+0000","last_unstale":"2024-09-21T07:46:57.707640+0000","last_undegraded":"2024-09-21T07:46:57.707640+0000","last_fullsized":"2024-09-21T07:46:57.707640+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:08.480356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708053+0000","last_change":"2024-09-21T07:46:56.211905+0000","last_active":"2024-09-21T07:46:57.708053+0000","last_peered":"2024-09-21T07:46:57.708053+0000","last_clean":"2024-09-21T07:46:57.708053+0000","last_became_active":"2024-09-21T07:46:56.211721+0000","last_became_peered":"2024-09-21T07:46:56.211721+0000","last_unstale":"2024-09-21T07:46:57.708053+0000","last_undegraded":"2024-09-21T07:46:57.708053+0000","last_fullsized":"2024-09-21T07:46:57.708053+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:41:37.977695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219395+0000","last_change":"2024-09-21T07:46:56.217732+0000","last_active":"2024-09-21T07:46:57.219395+0000","last_peered":"2024-09-21T07:46:57.219395+0000","last_clean":"2024-09-21T07:46:57.219395+0000","last_became_active":"2024-09-21T07:46:56.217410+0000","last_became_peered":"2024-09-21T07:46:56.217410+0000","last_unstale":"2024-09-21T07:46:57.219395+0000","last_undegraded":"2024-09-21T07:46:57.219395+0000","last_fullsized":"2024-09-21T07:46:57.219395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219810+0000","last_change":"2024-09-21T07:46:56.220112+0000","last_active":"2024-09-21T07:46:57.219810+0000","last_peered":"2024-09-21T07:46:57.219810+0000","last_clean":"2024-09-21T07:46:57.219810+0000","last_became_active":"2024-09-21T07:46:56.217329+0000","last_became_peered":"2024-09-21T07:46:56.217329+0000","last_unstale":"2024-09-21T07:46:57.219810+0000","last_undegraded":"2024-09-21T07:46:57.219810+0000","last_fullsized":"2024-09-21T07:46:57.219810+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:04.144314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228789+0000","last_change":"2024-09-21T07:46:56.223494+0000","last_active":"2024-09-21T07:46:57.228789+0000","last_peered":"2024-09-21T07:46:57.228789+0000","last_clean":"2024-09-21T07:46:57.228789+0000","last_became_active":"2024-09-21T07:46:56.223149+0000","last_became_peered":"2024-09-21T07:46:56.223149+0000","last_unstale":"2024-09-21T07:46:57.228789+0000","last_undegraded":"2024-09-21T07:46:57.228789+0000","last_fullsized":"2024-09-21T07:46:57.228789+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708836+0000","last_change":"2024-09-21T07:46:56.228315+0000","last_active":"2024-09-21T07:46:57.708836+0000","last_peered":"2024-09-21T07:46:57.708836+0000","last_clean":"2024-09-21T07:46:57.708836+0000","last_became_active":"2024-09-21T07:46:56.224550+0000","last_became_peered":"2024-09-21T07:46:56.224550+0000","last_unstale":"2024-09-21T07:46:57.708836+0000","last_undegraded":"2024-09-21T07:46:57.708836+0000","last_fullsized":"2024-09-21T07:46:57.708836+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:45.461539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706953+0000","last_change":"2024-09-21T07:46:56.235164+0000","last_active":"2024-09-21T07:46:57.706953+0000","last_peered":"2024-09-21T07:46:57.706953+0000","last_clean":"2024-09-21T07:46:57.706953+0000","last_became_active":"2024-09-21T07:46:56.234854+0000","last_became_peered":"2024-09-21T07:46:56.234854+0000","last_unstale":"2024-09-21T07:46:57.706953+0000","last_undegraded":"2024-09-21T07:46:57.706953+0000","last_fullsized":"2024-09-21T07:46:57.706953+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:13.323626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707171+0000","last_change":"2024-09-21T07:46:56.216052+0000","last_active":"2024-09-21T07:46:57.707171+0000","last_peered":"2024-09-21T07:46:57.707171+0000","last_clean":"2024-09-21T07:46:57.707171+0000","last_became_active":"2024-09-21T07:46:56.215728+0000","last_became_peered":"2024-09-21T07:46:56.215728+0000","last_unstale":"2024-09-21T07:46:57.707171+0000","last_undegraded":"2024-09-21T07:46:57.707171+0000","last_fullsized":"2024-09-21T07:46:57.707171+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:15.115400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708408+0000","last_change":"2024-09-21T07:46:56.218473+0000","last_active":"2024-09-21T07:46:57.708408+0000","last_peered":"2024-09-21T07:46:57.708408+0000","last_clean":"2024-09-21T07:46:57.708408+0000","last_became_active":"2024-09-21T07:46:56.217355+0000","last_became_peered":"2024-09-21T07:46:56.217355+0000","last_unstale":"2024-09-21T07:46:57.708408+0000","last_undegraded":"2024-09-21T07:46:57.708408+0000","last_fullsized":"2024-09-21T07:46:57.708408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220525+0000","last_change":"2024-09-21T07:46:56.209321+0000","last_active":"2024-09-21T07:46:57.220525+0000","last_peered":"2024-09-21T07:46:57.220525+0000","last_clean":"2024-09-21T07:46:57.220525+0000","last_became_active":"2024-09-21T07:46:56.208580+0000","last_became_peered":"2024-09-21T07:46:56.208580+0000","last_unstale":"2024-09-21T07:46:57.220525+0000","last_undegraded":"2024-09-21T07:46:57.220525+0000","last_fullsized":"2024-09-21T07:46:57.220525+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:44:35.986725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708018+0000","last_change":"2024-09-21T07:46:56.231049+0000","last_active":"2024-09-21T07:46:57.708018+0000","last_peered":"2024-09-21T07:46:57.708018+0000","last_clean":"2024-09-21T07:46:57.708018+0000","last_became_active":"2024-09-21T07:46:56.230778+0000","last_became_peered":"2024-09-21T07:46:56.230778+0000","last_unstale":"2024-09-21T07:46:57.708018+0000","last_undegraded":"2024-09-21T07:46:57.708018+0000","last_fullsized":"2024-09-21T07:46:57.708018+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:43.972634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":19,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221717+0000","last_change":"2024-09-21T07:46:55.204066+0000","last_active":"2024-09-21T07:46:57.221717+0000","last_peered":"2024-09-21T07:46:57.221717+0000","last_clean":"2024-09-21T07:46:57.221717+0000","last_became_active":"2024-09-21T07:46:55.203753+0000","last_became_peered":"2024-09-21T07:46:55.203753+0000","last_unstale":"2024-09-21T07:46:57.221717+0000","last_undegraded":"2024-09-21T07:46:57.221717+0000","last_fullsized":"2024-09-21T07:46:57.221717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:59.681758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709170+0000","last_change":"2024-09-21T07:46:56.228457+0000","last_active":"2024-09-21T07:46:57.709170+0000","last_peered":"2024-09-21T07:46:57.709170+0000","last_clean":"2024-09-21T07:46:57.709170+0000","last_became_active":"2024-09-21T07:46:56.228095+0000","last_became_peered":"2024-09-21T07:46:56.228095+0000","last_unstale":"2024-09-21T07:46:57.709170+0000","last_undegraded":"2024-09-21T07:46:57.709170+0000","last_fullsized":"2024-09-21T07:46:57.709170+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219963+0000","last_change":"2024-09-21T07:46:55.212364+0000","last_active":"2024-09-21T07:46:57.219963+0000","last_peered":"2024-09-21T07:46:57.219963+0000","last_clean":"2024-09-21T07:46:57.219963+0000","last_became_active":"2024-09-21T07:46:55.211980+0000","last_became_peered":"2024-09-21T07:46:55.211980+0000","last_unstale":"2024-09-21T07:46:57.219963+0000","last_undegraded":"2024-09-21T07:46:57.219963+0000","last_fullsized":"2024-09-21T07:46:57.219963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:40.287704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709015+0000","last_change":"2024-09-21T07:46:56.230650+0000","last_active":"2024-09-21T07:46:57.709015+0000","last_peered":"2024-09-21T07:46:57.709015+0000","last_clean":"2024-09-21T07:46:57.709015+0000","last_became_active":"2024-09-21T07:46:56.230348+0000","last_became_peered":"2024-09-21T07:46:56.230348+0000","last_unstale":"2024-09-21T07:46:57.709015+0000","last_undegraded":"2024-09-21T07:46:57.709015+0000","last_fullsized":"2024-09-21T07:46:57.709015+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:00.687362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221045+0000","last_change":"2024-09-21T07:46:55.216737+0000","last_active":"2024-09-21T07:46:57.221045+0000","last_peered":"2024-09-21T07:46:57.221045+0000","last_clean":"2024-09-21T07:46:57.221045+0000","last_became_active":"2024-09-21T07:46:55.216402+0000","last_became_peered":"2024-09-21T07:46:55.216402+0000","last_unstale":"2024-09-21T07:46:57.221045+0000","last_undegraded":"2024-09-21T07:46:57.221045+0000","last_fullsized":"2024-09-21T07:46:57.221045+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:05:40.966275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219312+0000","last_change":"2024-09-21T07:46:56.237682+0000","last_active":"2024-09-21T07:46:57.219312+0000","last_peered":"2024-09-21T07:46:57.219312+0000","last_clean":"2024-09-21T07:46:57.219312+0000","last_became_active":"2024-09-21T07:46:56.237340+0000","last_became_peered":"2024-09-21T07:46:56.237340+0000","last_unstale":"2024-09-21T07:46:57.219312+0000","last_undegraded":"2024-09-21T07:46:57.219312+0000","last_fullsized":"2024-09-21T07:46:57.219312+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219765+0000","last_change":"2024-09-21T07:46:55.211939+0000","last_active":"2024-09-21T07:46:57.219765+0000","last_peered":"2024-09-21T07:46:57.219765+0000","last_clean":"2024-09-21T07:46:57.219765+0000","last_became_active":"2024-09-21T07:46:55.211170+0000","last_became_peered":"2024-09-21T07:46:55.211170+0000","last_unstale":"2024-09-21T07:46:57.219765+0000","last_undegraded":"2024-09-21T07:46:57.219765+0000","last_fullsized":"2024-09-21T07:46:57.219765+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707334+0000","last_change":"2024-09-21T07:46:56.208982+0000","last_active":"2024-09-21T07:46:57.707334+0000","last_peered":"2024-09-21T07:46:57.707334+0000","last_clean":"2024-09-21T07:46:57.707334+0000","last_became_active":"2024-09-21T07:46:56.208692+0000","last_became_peered":"2024-09-21T07:46:56.208692+0000","last_unstale":"2024-09-21T07:46:57.707334+0000","last_undegraded":"2024-09-21T07:46:57.707334+0000","last_fullsized":"2024-09-21T07:46:57.707334+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:22.862558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708963+0000","last_change":"2024-09-21T07:46:55.204599+0000","last_active":"2024-09-21T07:46:57.708963+0000","last_peered":"2024-09-21T07:46:57.708963+0000","last_clean":"2024-09-21T07:46:57.708963+0000","last_became_active":"2024-09-21T07:46:55.204186+0000","last_became_peered":"2024-09-21T07:46:55.204186+0000","last_unstale":"2024-09-21T07:46:57.708963+0000","last_undegraded":"2024-09-21T07:46:57.708963+0000","last_fullsized":"2024-09-21T07:46:57.708963+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708759+0000","last_change":"2024-09-21T07:46:56.221721+0000","last_active":"2024-09-21T07:46:57.708759+0000","last_peered":"2024-09-21T07:46:57.708759+0000","last_clean":"2024-09-21T07:46:57.708759+0000","last_became_active":"2024-09-21T07:46:56.221359+0000","last_became_peered":"2024-09-21T07:46:56.221359+0000","last_unstale":"2024-09-21T07:46:57.708759+0000","last_undegraded":"2024-09-21T07:46:57.708759+0000","last_fullsized":"2024-09-21T07:46:57.708759+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:39.064113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708468+0000","last_change":"2024-09-21T07:46:55.205089+0000","last_active":"2024-09-21T07:46:57.708468+0000","last_peered":"2024-09-21T07:46:57.708468+0000","last_clean":"2024-09-21T07:46:57.708468+0000","last_became_active":"2024-09-21T07:46:55.204642+0000","last_became_peered":"2024-09-21T07:46:55.204642+0000","last_unstale":"2024-09-21T07:46:57.708468+0000","last_undegraded":"2024-09-21T07:46:57.708468+0000","last_fullsized":"2024-09-21T07:46:57.708468+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:26.924492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706743+0000","last_change":"2024-09-21T07:46:56.214234+0000","last_active":"2024-09-21T07:46:57.706743+0000","last_peered":"2024-09-21T07:46:57.706743+0000","last_clean":"2024-09-21T07:46:57.706743+0000","last_became_active":"2024-09-21T07:46:56.213914+0000","last_became_peered":"2024-09-21T07:46:56.213914+0000","last_unstale":"2024-09-21T07:46:57.706743+0000","last_undegraded":"2024-09-21T07:46:57.706743+0000","last_fullsized":"2024-09-21T07:46:57.706743+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:05.770150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218717+0000","last_change":"2024-09-21T07:46:55.218472+0000","last_active":"2024-09-21T07:46:57.218717+0000","last_peered":"2024-09-21T07:46:57.218717+0000","last_clean":"2024-09-21T07:46:57.218717+0000","last_became_active":"2024-09-21T07:46:55.218044+0000","last_became_peered":"2024-09-21T07:46:55.218044+0000","last_unstale":"2024-09-21T07:46:57.218717+0000","last_undegraded":"2024-09-21T07:46:57.218717+0000","last_fullsized":"2024-09-21T07:46:57.218717+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:48:18.860660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218669+0000","last_change":"2024-09-21T07:46:56.221089+0000","last_active":"2024-09-21T07:46:57.218669+0000","last_peered":"2024-09-21T07:46:57.218669+0000","last_clean":"2024-09-21T07:46:57.218669+0000","last_became_active":"2024-09-21T07:46:56.220746+0000","last_became_peered":"2024-09-21T07:46:56.220746+0000","last_unstale":"2024-09-21T07:46:57.218669+0000","last_undegraded":"2024-09-21T07:46:57.218669+0000","last_fullsized":"2024-09-21T07:46:57.218669+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708222+0000","last_change":"2024-09-21T07:46:55.206317+0000","last_active":"2024-09-21T07:46:57.708222+0000","last_peered":"2024-09-21T07:46:57.708222+0000","last_clean":"2024-09-21T07:46:57.708222+0000","last_became_active":"2024-09-21T07:46:55.206000+0000","last_became_peered":"2024-09-21T07:46:55.206000+0000","last_unstale":"2024-09-21T07:46:57.708222+0000","last_undegraded":"2024-09-21T07:46:57.708222+0000","last_fullsized":"2024-09-21T07:46:57.708222+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220947+0000","last_change":"2024-09-21T07:46:56.224113+0000","last_active":"2024-09-21T07:46:57.220947+0000","last_peered":"2024-09-21T07:46:57.220947+0000","last_clean":"2024-09-21T07:46:57.220947+0000","last_became_active":"2024-09-21T07:46:56.223793+0000","last_became_peered":"2024-09-21T07:46:56.223793+0000","last_unstale":"2024-09-21T07:46:57.220947+0000","last_undegraded":"2024-09-21T07:46:57.220947+0000","last_fullsized":"2024-09-21T07:46:57.220947+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:15.172501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228976+0000","last_change":"2024-09-21T07:46:55.213497+0000","last_active":"2024-09-21T07:46:57.228976+0000","last_peered":"2024-09-21T07:46:57.228976+0000","last_clean":"2024-09-21T07:46:57.228976+0000","last_became_active":"2024-09-21T07:46:55.213173+0000","last_became_peered":"2024-09-21T07:46:55.213173+0000","last_unstale":"2024-09-21T07:46:57.228976+0000","last_undegraded":"2024-09-21T07:46:57.228976+0000","last_fullsized":"2024-09-21T07:46:57.228976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.908077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219686+0000","last_change":"2024-09-21T07:46:56.229383+0000","last_active":"2024-09-21T07:46:57.219686+0000","last_peered":"2024-09-21T07:46:57.219686+0000","last_clean":"2024-09-21T07:46:57.219686+0000","last_became_active":"2024-09-21T07:46:56.229012+0000","last_became_peered":"2024-09-21T07:46:56.229012+0000","last_unstale":"2024-09-21T07:46:57.219686+0000","last_undegraded":"2024-09-21T07:46:57.219686+0000","last_fullsized":"2024-09-21T07:46:57.219686+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:22:56.521559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228698+0000","last_change":"2024-09-21T07:46:55.212681+0000","last_active":"2024-09-21T07:46:57.228698+0000","last_peered":"2024-09-21T07:46:57.228698+0000","last_clean":"2024-09-21T07:46:57.228698+0000","last_became_active":"2024-09-21T07:46:55.211907+0000","last_became_peered":"2024-09-21T07:46:55.211907+0000","last_unstale":"2024-09-21T07:46:57.228698+0000","last_undegraded":"2024-09-21T07:46:57.228698+0000","last_fullsized":"2024-09-21T07:46:57.228698+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:30.596619+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219680+0000","last_change":"2024-09-21T07:46:56.224792+0000","last_active":"2024-09-21T07:46:57.219680+0000","last_peered":"2024-09-21T07:46:57.219680+0000","last_clean":"2024-09-21T07:46:57.219680+0000","last_became_active":"2024-09-21T07:46:56.224554+0000","last_became_peered":"2024-09-21T07:46:56.224554+0000","last_unstale":"2024-09-21T07:46:57.219680+0000","last_undegraded":"2024-09-21T07:46:57.219680+0000","last_fullsized":"2024-09-21T07:46:57.219680+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:31.069660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":18,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221096+0000","last_change":"2024-09-21T07:46:55.207287+0000","last_active":"2024-09-21T07:46:57.221096+0000","last_peered":"2024-09-21T07:46:57.221096+0000","last_clean":"2024-09-21T07:46:57.221096+0000","last_became_active":"2024-09-21T07:46:55.207008+0000","last_became_peered":"2024-09-21T07:46:55.207008+0000","last_unstale":"2024-09-21T07:46:57.221096+0000","last_undegraded":"2024-09-21T07:46:57.221096+0000","last_fullsized":"2024-09-21T07:46:57.221096+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:30.602685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221088+0000","last_change":"2024-09-21T07:46:56.212672+0000","last_active":"2024-09-21T07:46:57.221088+0000","last_peered":"2024-09-21T07:46:57.221088+0000","last_clean":"2024-09-21T07:46:57.221088+0000","last_became_active":"2024-09-21T07:46:56.212383+0000","last_became_peered":"2024-09-21T07:46:56.212383+0000","last_unstale":"2024-09-21T07:46:57.221088+0000","last_undegraded":"2024-09-21T07:46:57.221088+0000","last_fullsized":"2024-09-21T07:46:57.221088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:13.284361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":21,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707883+0000","last_change":"2024-09-21T07:46:55.209727+0000","last_active":"2024-09-21T07:46:57.707883+0000","last_peered":"2024-09-21T07:46:57.707883+0000","last_clean":"2024-09-21T07:46:57.707883+0000","last_became_active":"2024-09-21T07:46:55.209437+0000","last_became_peered":"2024-09-21T07:46:55.209437+0000","last_unstale":"2024-09-21T07:46:57.707883+0000","last_undegraded":"2024-09-21T07:46:57.707883+0000","last_fullsized":"2024-09-21T07:46:57.707883+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707613+0000","last_change":"2024-09-21T07:46:56.234403+0000","last_active":"2024-09-21T07:46:57.707613+0000","last_peered":"2024-09-21T07:46:57.707613+0000","last_clean":"2024-09-21T07:46:57.707613+0000","last_became_active":"2024-09-21T07:46:56.233998+0000","last_became_peered":"2024-09-21T07:46:56.233998+0000","last_unstale":"2024-09-21T07:46:57.707613+0000","last_undegraded":"2024-09-21T07:46:57.707613+0000","last_fullsized":"2024-09-21T07:46:57.707613+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:21:52.840706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708952+0000","last_change":"2024-09-21T07:46:55.212435+0000","last_active":"2024-09-21T07:46:57.708952+0000","last_peered":"2024-09-21T07:46:57.708952+0000","last_clean":"2024-09-21T07:46:57.708952+0000","last_became_active":"2024-09-21T07:46:55.212093+0000","last_became_peered":"2024-09-21T07:46:55.212093+0000","last_unstale":"2024-09-21T07:46:57.708952+0000","last_undegraded":"2024-09-21T07:46:57.708952+0000","last_fullsized":"2024-09-21T07:46:57.708952+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219594+0000","last_change":"2024-09-21T07:46:56.236480+0000","last_active":"2024-09-21T07:46:57.219594+0000","last_peered":"2024-09-21T07:46:57.219594+0000","last_clean":"2024-09-21T07:46:57.219594+0000","last_became_active":"2024-09-21T07:46:56.236111+0000","last_became_peered":"2024-09-21T07:46:56.236111+0000","last_unstale":"2024-09-21T07:46:57.219594+0000","last_undegraded":"2024-09-21T07:46:57.219594+0000","last_fullsized":"2024-09-21T07:46:57.219594+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:18:25.092510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219132+0000","last_change":"2024-09-21T07:46:55.211160+0000","last_active":"2024-09-21T07:46:57.219132+0000","last_peered":"2024-09-21T07:46:57.219132+0000","last_clean":"2024-09-21T07:46:57.219132+0000","last_became_active":"2024-09-21T07:46:55.210797+0000","last_became_peered":"2024-09-21T07:46:55.210797+0000","last_unstale":"2024-09-21T07:46:57.219132+0000","last_undegraded":"2024-09-21T07:46:57.219132+0000","last_fullsized":"2024-09-21T07:46:57.219132+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:33.882042+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708245+0000","last_change":"2024-09-21T07:46:56.208519+0000","last_active":"2024-09-21T07:46:57.708245+0000","last_peered":"2024-09-21T07:46:57.708245+0000","last_clean":"2024-09-21T07:46:57.708245+0000","last_became_active":"2024-09-21T07:46:56.207695+0000","last_became_peered":"2024-09-21T07:46:56.207695+0000","last_unstale":"2024-09-21T07:46:57.708245+0000","last_undegraded":"2024-09-21T07:46:57.708245+0000","last_fullsized":"2024-09-21T07:46:57.708245+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218445+0000","last_change":"2024-09-21T07:46:55.205503+0000","last_active":"2024-09-21T07:46:57.218445+0000","last_peered":"2024-09-21T07:46:57.218445+0000","last_clean":"2024-09-21T07:46:57.218445+0000","last_became_active":"2024-09-21T07:46:55.204821+0000","last_became_peered":"2024-09-21T07:46:55.204821+0000","last_unstale":"2024-09-21T07:46:57.218445+0000","last_undegraded":"2024-09-21T07:46:57.218445+0000","last_fullsized":"2024-09-21T07:46:57.218445+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220676+0000","last_change":"2024-09-21T07:46:56.212672+0000","last_active":"2024-09-21T07:46:57.220676+0000","last_peered":"2024-09-21T07:46:57.220676+0000","last_clean":"2024-09-21T07:46:57.220676+0000","last_became_active":"2024-09-21T07:46:56.212349+0000","last_became_peered":"2024-09-21T07:46:56.212349+0000","last_unstale":"2024-09-21T07:46:57.220676+0000","last_undegraded":"2024-09-21T07:46:57.220676+0000","last_fullsized":"2024-09-21T07:46:57.220676+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:02.037379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":23,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218410+0000","last_change":"2024-09-21T07:46:55.216736+0000","last_active":"2024-09-21T07:46:57.218410+0000","last_peered":"2024-09-21T07:46:57.218410+0000","last_clean":"2024-09-21T07:46:57.218410+0000","last_became_active":"2024-09-21T07:46:55.216312+0000","last_became_peered":"2024-09-21T07:46:55.216312+0000","last_unstale":"2024-09-21T07:46:57.218410+0000","last_undegraded":"2024-09-21T07:46:57.218410+0000","last_fullsized":"2024-09-21T07:46:57.218410+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:48.195061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221175+0000","last_change":"2024-09-21T07:46:56.209062+0000","last_active":"2024-09-21T07:46:57.221175+0000","last_peered":"2024-09-21T07:46:57.221175+0000","last_clean":"2024-09-21T07:46:57.221175+0000","last_became_active":"2024-09-21T07:46:56.208769+0000","last_became_peered":"2024-09-21T07:46:56.208769+0000","last_unstale":"2024-09-21T07:46:57.221175+0000","last_undegraded":"2024-09-21T07:46:57.221175+0000","last_fullsized":"2024-09-21T07:46:57.221175+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:50.820911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220824+0000","last_change":"2024-09-21T07:46:55.218143+0000","last_active":"2024-09-21T07:46:57.220824+0000","last_peered":"2024-09-21T07:46:57.220824+0000","last_clean":"2024-09-21T07:46:57.220824+0000","last_became_active":"2024-09-21T07:46:55.217800+0000","last_became_peered":"2024-09-21T07:46:55.217800+0000","last_unstale":"2024-09-21T07:46:57.220824+0000","last_undegraded":"2024-09-21T07:46:57.220824+0000","last_fullsized":"2024-09-21T07:46:57.220824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:57.680882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707549+0000","last_change":"2024-09-21T07:46:56.218140+0000","last_active":"2024-09-21T07:46:57.707549+0000","last_peered":"2024-09-21T07:46:57.707549+0000","last_clean":"2024-09-21T07:46:57.707549+0000","last_became_active":"2024-09-21T07:46:56.217824+0000","last_became_peered":"2024-09-21T07:46:56.217824+0000","last_unstale":"2024-09-21T07:46:57.707549+0000","last_undegraded":"2024-09-21T07:46:57.707549+0000","last_fullsized":"2024-09-21T07:46:57.707549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:46.806098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.709120+0000","last_change":"2024-09-21T07:46:55.212733+0000","last_active":"2024-09-21T07:46:57.709120+0000","last_peered":"2024-09-21T07:46:57.709120+0000","last_clean":"2024-09-21T07:46:57.709120+0000","last_became_active":"2024-09-21T07:46:55.212478+0000","last_became_peered":"2024-09-21T07:46:55.212478+0000","last_unstale":"2024-09-21T07:46:57.709120+0000","last_undegraded":"2024-09-21T07:46:57.709120+0000","last_fullsized":"2024-09-21T07:46:57.709120+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228788+0000","last_change":"2024-09-21T07:46:56.223493+0000","last_active":"2024-09-21T07:46:57.228788+0000","last_peered":"2024-09-21T07:46:57.228788+0000","last_clean":"2024-09-21T07:46:57.228788+0000","last_became_active":"2024-09-21T07:46:56.223060+0000","last_became_peered":"2024-09-21T07:46:56.223060+0000","last_unstale":"2024-09-21T07:46:57.228788+0000","last_undegraded":"2024-09-21T07:46:57.228788+0000","last_fullsized":"2024-09-21T07:46:57.228788+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:08.255764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221247+0000","last_change":"2024-09-21T07:46:55.213737+0000","last_active":"2024-09-21T07:46:57.221247+0000","last_peered":"2024-09-21T07:46:57.221247+0000","last_clean":"2024-09-21T07:46:57.221247+0000","last_became_active":"2024-09-21T07:46:55.213452+0000","last_became_peered":"2024-09-21T07:46:55.213452+0000","last_unstale":"2024-09-21T07:46:57.221247+0000","last_undegraded":"2024-09-21T07:46:57.221247+0000","last_fullsized":"2024-09-21T07:46:57.221247+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:38.058509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220927+0000","last_change":"2024-09-21T07:46:56.232119+0000","last_active":"2024-09-21T07:46:57.220927+0000","last_peered":"2024-09-21T07:46:57.220927+0000","last_clean":"2024-09-21T07:46:57.220927+0000","last_became_active":"2024-09-21T07:46:56.231764+0000","last_became_peered":"2024-09-21T07:46:56.231764+0000","last_unstale":"2024-09-21T07:46:57.220927+0000","last_undegraded":"2024-09-21T07:46:57.220927+0000","last_fullsized":"2024-09-21T07:46:57.220927+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:35.746093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707452+0000","last_change":"2024-09-21T07:46:55.206900+0000","last_active":"2024-09-21T07:46:57.707452+0000","last_peered":"2024-09-21T07:46:57.707452+0000","last_clean":"2024-09-21T07:46:57.707452+0000","last_became_active":"2024-09-21T07:46:55.206441+0000","last_became_peered":"2024-09-21T07:46:55.206441+0000","last_unstale":"2024-09-21T07:46:57.707452+0000","last_undegraded":"2024-09-21T07:46:57.707452+0000","last_fullsized":"2024-09-21T07:46:57.707452+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:52.569990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707559+0000","last_change":"2024-09-21T07:46:56.233293+0000","last_active":"2024-09-21T07:46:57.707559+0000","last_peered":"2024-09-21T07:46:57.707559+0000","last_clean":"2024-09-21T07:46:57.707559+0000","last_became_active":"2024-09-21T07:46:56.232951+0000","last_became_peered":"2024-09-21T07:46:56.232951+0000","last_unstale":"2024-09-21T07:46:57.707559+0000","last_undegraded":"2024-09-21T07:46:57.707559+0000","last_fullsized":"2024-09-21T07:46:57.707559+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:03:58.570223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228732+0000","last_change":"2024-09-21T07:46:55.206362+0000","last_active":"2024-09-21T07:46:57.228732+0000","last_peered":"2024-09-21T07:46:57.228732+0000","last_clean":"2024-09-21T07:46:57.228732+0000","last_became_active":"2024-09-21T07:46:55.206044+0000","last_became_peered":"2024-09-21T07:46:55.206044+0000","last_unstale":"2024-09-21T07:46:57.228732+0000","last_undegraded":"2024-09-21T07:46:57.228732+0000","last_fullsized":"2024-09-21T07:46:57.228732+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:20.127719+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228770+0000","last_change":"2024-09-21T07:46:56.221448+0000","last_active":"2024-09-21T07:46:57.228770+0000","last_peered":"2024-09-21T07:46:57.228770+0000","last_clean":"2024-09-21T07:46:57.228770+0000","last_became_active":"2024-09-21T07:46:56.221116+0000","last_became_peered":"2024-09-21T07:46:56.221116+0000","last_unstale":"2024-09-21T07:46:57.228770+0000","last_undegraded":"2024-09-21T07:46:57.228770+0000","last_fullsized":"2024-09-21T07:46:57.228770+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:47:11.694768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708582+0000","last_change":"2024-09-21T07:46:55.213426+0000","last_active":"2024-09-21T07:46:57.708582+0000","last_peered":"2024-09-21T07:46:57.708582+0000","last_clean":"2024-09-21T07:46:57.708582+0000","last_became_active":"2024-09-21T07:46:55.211244+0000","last_became_peered":"2024-09-21T07:46:55.211244+0000","last_unstale":"2024-09-21T07:46:57.708582+0000","last_undegraded":"2024-09-21T07:46:57.708582+0000","last_fullsized":"2024-09-21T07:46:57.708582+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:45:17.627226+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707540+0000","last_change":"2024-09-21T07:46:56.232335+0000","last_active":"2024-09-21T07:46:57.707540+0000","last_peered":"2024-09-21T07:46:57.707540+0000","last_clean":"2024-09-21T07:46:57.707540+0000","last_became_active":"2024-09-21T07:46:56.231953+0000","last_became_peered":"2024-09-21T07:46:56.231953+0000","last_unstale":"2024-09-21T07:46:57.707540+0000","last_undegraded":"2024-09-21T07:46:57.707540+0000","last_fullsized":"2024-09-21T07:46:57.707540+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:07:04.369378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707946+0000","last_change":"2024-09-21T07:46:55.214923+0000","last_active":"2024-09-21T07:46:57.707946+0000","last_peered":"2024-09-21T07:46:57.707946+0000","last_clean":"2024-09-21T07:46:57.707946+0000","last_became_active":"2024-09-21T07:46:55.214607+0000","last_became_peered":"2024-09-21T07:46:55.214607+0000","last_unstale":"2024-09-21T07:46:57.707946+0000","last_undegraded":"2024-09-21T07:46:57.707946+0000","last_fullsized":"2024-09-21T07:46:57.707946+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:04.689624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706862+0000","last_change":"2024-09-21T07:46:56.214731+0000","last_active":"2024-09-21T07:46:57.706862+0000","last_peered":"2024-09-21T07:46:57.706862+0000","last_clean":"2024-09-21T07:46:57.706862+0000","last_became_active":"2024-09-21T07:46:56.214437+0000","last_became_peered":"2024-09-21T07:46:56.214437+0000","last_unstale":"2024-09-21T07:46:57.706862+0000","last_undegraded":"2024-09-21T07:46:57.706862+0000","last_fullsized":"2024-09-21T07:46:57.706862+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:39.838252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220502+0000","last_change":"2024-09-21T07:46:55.208755+0000","last_active":"2024-09-21T07:46:57.220502+0000","last_peered":"2024-09-21T07:46:57.220502+0000","last_clean":"2024-09-21T07:46:57.220502+0000","last_became_active":"2024-09-21T07:46:55.207806+0000","last_became_peered":"2024-09-21T07:46:55.207806+0000","last_unstale":"2024-09-21T07:46:57.220502+0000","last_undegraded":"2024-09-21T07:46:57.220502+0000","last_fullsized":"2024-09-21T07:46:57.220502+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:29.396988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220825+0000","last_change":"2024-09-21T07:46:56.232337+0000","last_active":"2024-09-21T07:46:57.220825+0000","last_peered":"2024-09-21T07:46:57.220825+0000","last_clean":"2024-09-21T07:46:57.220825+0000","last_became_active":"2024-09-21T07:46:56.230029+0000","last_became_peered":"2024-09-21T07:46:56.230029+0000","last_unstale":"2024-09-21T07:46:57.220825+0000","last_undegraded":"2024-09-21T07:46:57.220825+0000","last_fullsized":"2024-09-21T07:46:57.220825+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:08.727543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":17,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708019+0000","last_change":"2024-09-21T07:46:55.211058+0000","last_active":"2024-09-21T07:46:57.708019+0000","last_peered":"2024-09-21T07:46:57.708019+0000","last_clean":"2024-09-21T07:46:57.708019+0000","last_became_active":"2024-09-21T07:46:55.210625+0000","last_became_peered":"2024-09-21T07:46:55.210625+0000","last_unstale":"2024-09-21T07:46:57.708019+0000","last_undegraded":"2024-09-21T07:46:57.708019+0000","last_fullsized":"2024-09-21T07:46:57.708019+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221610+0000","last_change":"2024-09-21T07:46:56.229165+0000","last_active":"2024-09-21T07:46:57.221610+0000","last_peered":"2024-09-21T07:46:57.221610+0000","last_clean":"2024-09-21T07:46:57.221610+0000","last_became_active":"2024-09-21T07:46:56.228862+0000","last_became_peered":"2024-09-21T07:46:56.228862+0000","last_unstale":"2024-09-21T07:46:57.221610+0000","last_undegraded":"2024-09-21T07:46:57.221610+0000","last_fullsized":"2024-09-21T07:46:57.221610+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:22.071500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":16,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.228909+0000","last_change":"2024-09-21T07:46:55.209507+0000","last_active":"2024-09-21T07:46:57.228909+0000","last_peered":"2024-09-21T07:46:57.228909+0000","last_clean":"2024-09-21T07:46:57.228909+0000","last_became_active":"2024-09-21T07:46:55.209003+0000","last_became_peered":"2024-09-21T07:46:55.209003+0000","last_unstale":"2024-09-21T07:46:57.228909+0000","last_undegraded":"2024-09-21T07:46:57.228909+0000","last_fullsized":"2024-09-21T07:46:57.228909+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:25.605174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221110+0000","last_change":"2024-09-21T07:46:56.241826+0000","last_active":"2024-09-21T07:46:57.221110+0000","last_peered":"2024-09-21T07:46:57.221110+0000","last_clean":"2024-09-21T07:46:57.221110+0000","last_became_active":"2024-09-21T07:46:56.241465+0000","last_became_peered":"2024-09-21T07:46:56.241465+0000","last_unstale":"2024-09-21T07:46:57.221110+0000","last_undegraded":"2024-09-21T07:46:57.221110+0000","last_fullsized":"2024-09-21T07:46:57.221110+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:13.411156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708039+0000","last_change":"2024-09-21T07:46:55.209001+0000","last_active":"2024-09-21T07:46:57.708039+0000","last_peered":"2024-09-21T07:46:57.708039+0000","last_clean":"2024-09-21T07:46:57.708039+0000","last_became_active":"2024-09-21T07:46:55.208596+0000","last_became_peered":"2024-09-21T07:46:55.208596+0000","last_unstale":"2024-09-21T07:46:57.708039+0000","last_undegraded":"2024-09-21T07:46:57.708039+0000","last_fullsized":"2024-09-21T07:46:57.708039+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.395008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707408+0000","last_change":"2024-09-21T07:46:56.227045+0000","last_active":"2024-09-21T07:46:57.707408+0000","last_peered":"2024-09-21T07:46:57.707408+0000","last_clean":"2024-09-21T07:46:57.707408+0000","last_became_active":"2024-09-21T07:46:56.220245+0000","last_became_peered":"2024-09-21T07:46:56.220245+0000","last_unstale":"2024-09-21T07:46:57.707408+0000","last_undegraded":"2024-09-21T07:46:57.707408+0000","last_fullsized":"2024-09-21T07:46:57.707408+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:18:51.902373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707914+0000","last_change":"2024-09-21T07:46:55.210484+0000","last_active":"2024-09-21T07:46:57.707914+0000","last_peered":"2024-09-21T07:46:57.707914+0000","last_clean":"2024-09-21T07:46:57.707914+0000","last_became_active":"2024-09-21T07:46:55.210140+0000","last_became_peered":"2024-09-21T07:46:55.210140+0000","last_unstale":"2024-09-21T07:46:57.707914+0000","last_undegraded":"2024-09-21T07:46:57.707914+0000","last_fullsized":"2024-09-21T07:46:57.707914+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:08:42.861822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219755+0000","last_change":"2024-09-21T07:46:56.233200+0000","last_active":"2024-09-21T07:46:57.219755+0000","last_peered":"2024-09-21T07:46:57.219755+0000","last_clean":"2024-09-21T07:46:57.219755+0000","last_became_active":"2024-09-21T07:46:56.232818+0000","last_became_peered":"2024-09-21T07:46:56.232818+0000","last_unstale":"2024-09-21T07:46:57.219755+0000","last_undegraded":"2024-09-21T07:46:57.219755+0000","last_fullsized":"2024-09-21T07:46:57.219755+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:54.180535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707865+0000","last_change":"2024-09-21T07:46:55.215083+0000","last_active":"2024-09-21T07:46:57.707865+0000","last_peered":"2024-09-21T07:46:57.707865+0000","last_clean":"2024-09-21T07:46:57.707865+0000","last_became_active":"2024-09-21T07:46:55.214531+0000","last_became_peered":"2024-09-21T07:46:55.214531+0000","last_unstale":"2024-09-21T07:46:57.707865+0000","last_undegraded":"2024-09-21T07:46:57.707865+0000","last_fullsized":"2024-09-21T07:46:57.707865+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:57:48.181207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.218812+0000","last_change":"2024-09-21T07:46:56.220432+0000","last_active":"2024-09-21T07:46:57.218812+0000","last_peered":"2024-09-21T07:46:57.218812+0000","last_clean":"2024-09-21T07:46:57.218812+0000","last_became_active":"2024-09-21T07:46:56.219527+0000","last_became_peered":"2024-09-21T07:46:56.219527+0000","last_unstale":"2024-09-21T07:46:57.218812+0000","last_undegraded":"2024-09-21T07:46:57.218812+0000","last_fullsized":"2024-09-21T07:46:57.218812+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:30.402592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.708593+0000","last_change":"2024-09-21T07:46:55.212708+0000","last_active":"2024-09-21T07:46:57.708593+0000","last_peered":"2024-09-21T07:46:57.708593+0000","last_clean":"2024-09-21T07:46:57.708593+0000","last_became_active":"2024-09-21T07:46:55.212115+0000","last_became_peered":"2024-09-21T07:46:55.212115+0000","last_unstale":"2024-09-21T07:46:57.708593+0000","last_undegraded":"2024-09-21T07:46:57.708593+0000","last_fullsized":"2024-09-21T07:46:57.708593+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:47.316489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.220717+0000","last_change":"2024-09-21T07:46:56.218638+0000","last_active":"2024-09-21T07:46:57.220717+0000","last_peered":"2024-09-21T07:46:57.220717+0000","last_clean":"2024-09-21T07:46:57.220717+0000","last_became_active":"2024-09-21T07:46:56.218288+0000","last_became_peered":"2024-09-21T07:46:56.218288+0000","last_unstale":"2024-09-21T07:46:57.220717+0000","last_undegraded":"2024-09-21T07:46:57.220717+0000","last_fullsized":"2024-09-21T07:46:57.220717+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:57.104592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.219828+0000","last_change":"2024-09-21T07:46:55.214726+0000","last_active":"2024-09-21T07:46:57.219828+0000","last_peered":"2024-09-21T07:46:57.219828+0000","last_clean":"2024-09-21T07:46:57.219828+0000","last_became_active":"2024-09-21T07:46:55.214331+0000","last_became_peered":"2024-09-21T07:46:55.214331+0000","last_unstale":"2024-09-21T07:46:57.219828+0000","last_undegraded":"2024-09-21T07:46:57.219828+0000","last_fullsized":"2024-09-21T07:46:57.219828+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:52.547289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":13,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.707852+0000","last_change":"2024-09-21T07:46:56.220549+0000","last_active":"2024-09-21T07:46:57.707852+0000","last_peered":"2024-09-21T07:46:57.707852+0000","last_clean":"2024-09-21T07:46:57.707852+0000","last_became_active":"2024-09-21T07:46:56.220238+0000","last_became_peered":"2024-09-21T07:46:56.220238+0000","last_unstale":"2024-09-21T07:46:57.707852+0000","last_undegraded":"2024-09-21T07:46:57.707852+0000","last_fullsized":"2024-09-21T07:46:57.707852+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:55.181730+0000","last_clean_scrub_stamp":"2024-09-21T07:46:55.181730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:59.146665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":15,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.221352+0000","last_change":"2024-09-21T07:46:55.218103+0000","last_active":"2024-09-21T07:46:57.221352+0000","last_peered":"2024-09-21T07:46:57.221352+0000","last_clean":"2024-09-21T07:46:57.221352+0000","last_became_active":"2024-09-21T07:46:55.217796+0000","last_became_peered":"2024-09-21T07:46:55.217796+0000","last_unstale":"2024-09-21T07:46:57.221352+0000","last_undegraded":"2024-09-21T07:46:57.221352+0000","last_fullsized":"2024-09-21T07:46:57.221352+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:46:54.176126+0000","last_clean_scrub_stamp":"2024-09-21T07:46:54.176126+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:44.455935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":133,"reported_epoch":35,"state":"active+clean","last_fresh":"2024-09-21T07:46:57.706779+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:46:57.706779+0000","last_peered":"2024-09-21T07:46:57.706779+0000","last_clean":"2024-09-21T07:46:57.706779+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:46:57.706779+0000","last_undegraded":"2024-09-21T07:46:57.706779+0000","last_fullsized":"2024-09-21T07:46:57.706779+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476756,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27424,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344416,"statfs":{"total":96636764160,"available":96608681984,"internally_reserved":0,"allocated":401408,"data_stored":150002,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476756,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27408,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344432,"statfs":{"total":96636764160,"available":96608698368,"internally_reserved":0,"allocated":389120,"data_stored":149260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476753,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343852,"statfs":{"total":96636764160,"available":96608104448,"internally_reserved":0,"allocated":983040,"data_stored":739664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476753,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27412,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344428,"statfs":{"total":96636764160,"available":96608694272,"internally_reserved":0,"allocated":389120,"data_stored":149316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509456,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":388,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":397312,"data_stored":150838,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509457,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27992,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343848,"statfs":{"total":96636764160,"available":96608100352,"internally_reserved":0,"allocated":983040,"data_stored":740176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27420,"kb_used_data":392,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344420,"statfs":{"total":96636764160,"available":96608686080,"internally_reserved":0,"allocated":401408,"data_stored":150860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542161,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27416,"kb_used_data":380,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344424,"statfs":{"total":96636764160,"available":96608690176,"internally_reserved":0,"allocated":389120,"data_stored":149248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:06.082 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:47:06.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:47:06.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 -- 172.21.15.88:0/2248479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0480cf1c0 msgr2=0x7fb048070630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 --2- 172.21.15.88:0/2248479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0480cf1c0 0x7fb048070630 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb040005fd0 tx=0x7fb04002a820 comp rx=0 tx=0).stop 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 -- 172.21.15.88:0/2248479 shutdown_connections 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 --2- 172.21.15.88:0/2248479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0480cf1c0 0x7fb048070630 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 --2- 172.21.15.88:0/2248479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb04815bca0 0x7fb0480cf9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 -- 172.21.15.88:0/2248479 >> 172.21.15.88:0/2248479 conn(0x7fb0480b1dd0 msgr2=0x7fb0480b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 -- 172.21.15.88:0/2248479 shutdown_connections 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.342+0000 7fb047e00640 1 -- 172.21.15.88:0/2248479 wait complete. 2024-09-21T07:47:06.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.345+0000 7fb047e00640 1 Processor -- start 2024-09-21T07:47:06.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.345+0000 7fb047e00640 1 -- start start 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb048070220 0x7fb048147790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0480cf1c0 0x7fb0481f6b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 0x7fb0481f7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0480cd2f0 con 0x7fb04815bca0 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb04815d510 con 0x7fb048070220 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb0480cd470 con 0x7fb0480cf1c0 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 0x7fb0481f7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 0x7fb0481f7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48354/0 (socket says 172.21.15.88:48354) 2024-09-21T07:47:06.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.346+0000 7fb047400640 1 -- 172.21.15.88:0/4173931786 learned_addr learned my addr 172.21.15.88:0/4173931786 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb03e000640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0480cf1c0 0x7fb0481f6b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb046a00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb048070220 0x7fb048147790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047400640 1 -- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0480cf1c0 msgr2=0x7fb0481f6b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047400640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0480cf1c0 0x7fb0481f6b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047400640 1 -- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb048070220 msgr2=0x7fb048147790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047400640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb048070220 0x7fb048147790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047400640 1 -- 172.21.15.88:0/4173931786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0481f79b0 con 0x7fb04815bca0 2024-09-21T07:47:06.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047400640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 0x7fb0481f7090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb040009cd0 tx=0x7fb040032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040037040 con 0x7fb04815bca0 2024-09-21T07:47:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb040042070 con 0x7fb04815bca0 2024-09-21T07:47:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04809d320 con 0x7fb04815bca0 2024-09-21T07:47:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.347+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb040004260 con 0x7fb04815bca0 2024-09-21T07:47:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.348+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb04809d030 con 0x7fb04815bca0 2024-09-21T07:47:06.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.348+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb048070630 con 0x7fb04815bca0 2024-09-21T07:47:06.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.350+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fb040004400 con 0x7fb04815bca0 2024-09-21T07:47:06.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.351+0000 7fb03f400640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb024078b90 0x7fb02407b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.351+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fb0400bbc20 con 0x7fb04815bca0 2024-09-21T07:47:06.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.351+0000 7fb046a00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb024078b90 0x7fb02407b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.352+0000 7fb046a00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb024078b90 0x7fb02407b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb03800bb00 tx=0x7fb038005e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.355+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb040083fc0 con 0x7fb04815bca0 2024-09-21T07:47:06.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.609+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb0480b2a60 con 0x7fb04815bca0 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.613+0000 7fb03f400640 1 -- 172.21.15.88:0/4173931786 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) ==== 190+0+147 (secure 0 0 0) 0x7fb0400838c0 con 0x7fb04815bca0 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:47:06.615 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:47:06.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb024078b90 msgr2=0x7fb02407b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb024078b90 0x7fb02407b0a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb03800bb00 tx=0x7fb038005e30 comp rx=0 tx=0).stop 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 msgr2=0x7fb0481f7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 0x7fb0481f7090 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb040009cd0 tx=0x7fb040032d10 comp rx=0 tx=0).stop 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 shutdown_connections 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb024078b90 0x7fb02407b0a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb0480cf1c0 0x7fb0481f6b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04815bca0 0x7fb0481f7090 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 --2- 172.21.15.88:0/4173931786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb048070220 0x7fb048147790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 >> 172.21.15.88:0/4173931786 conn(0x7fb0480b1dd0 msgr2=0x7fb0480cf970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 shutdown_connections 2024-09-21T07:47:06.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.616+0000 7fb047e00640 1 -- 172.21.15.88:0/4173931786 wait complete. 2024-09-21T07:47:06.644 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.860+0000 7f5ae4c00640 1 -- 172.21.15.88:0/2335673067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae00ad0e0 msgr2=0x7f5ae00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.860+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/2335673067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae00ad0e0 0x7f5ae00ad4f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005fd0 tx=0x7f5acc02a540 comp rx=0 tx=0).stop 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.860+0000 7f5ae4c00640 1 -- 172.21.15.88:0/2335673067 shutdown_connections 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.860+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/2335673067 >> v2:172.21.15.175:3300/0 conn(0x7f5ae00b3550 0x7f5ae00b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.860+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/2335673067 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae00ad0e0 0x7f5ae00ad4f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.860+0000 7f5ae4c00640 1 -- 172.21.15.88:0/2335673067 >> 172.21.15.88:0/2335673067 conn(0x7f5ae006d730 msgr2=0x7f5ae00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.861+0000 7f5ae4c00640 1 -- 172.21.15.88:0/2335673067 shutdown_connections 2024-09-21T07:47:06.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.861+0000 7f5ae4c00640 1 -- 172.21.15.88:0/2335673067 wait complete. 2024-09-21T07:47:06.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.863+0000 7f5ae4c00640 1 Processor -- start 2024-09-21T07:47:06.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.863+0000 7f5ae4c00640 1 -- start start 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.864+0000 7f5ae4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 0x7f5ae0152080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.864+0000 7f5ae4c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ae00b3550 0x7f5ae00700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.864+0000 7f5ae4c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae015d760 0x7f5ae0070630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.864+0000 7f5ae4c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae00b4da0 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.864+0000 7f5ae4c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ae00b4c20 con 0x7f5ae00b3550 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.864+0000 7f5ae4c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ae00b3ea0 con 0x7f5ae015d760 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 0x7f5ae0152080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 0x7f5ae0152080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48358/0 (socket says 172.21.15.88:48358) 2024-09-21T07:47:06.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 -- 172.21.15.88:0/93542628 learned_addr learned my addr 172.21.15.88:0/93542628 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:06.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5add400640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ae00b3550 0x7f5ae00700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5ade800640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae015d760 0x7f5ae0070630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 -- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae015d760 msgr2=0x7f5ae0070630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae015d760 0x7f5ae0070630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 -- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ae00b3550 msgr2=0x7f5ae00700f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ae00b3550 0x7f5ae00700f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 -- 172.21.15.88:0/93542628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ae0073420 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.865+0000 7f5adde00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 0x7f5ae0152080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000c3c0 tx=0x7f5ad000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.866+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0017070 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.866+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ad00035f0 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.866+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ae0073700 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.866+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad0003a80 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.866+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ae0073cc0 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.867+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ae01562e0 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.868+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f5ad001a070 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.869+0000 7f5ad6000640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ac0078b90 0x7f5ac007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.869+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5ad009a170 con 0x7f5ae00ad0e0 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.870+0000 7f5add400640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ac0078b90 0x7f5ac007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:06.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.871+0000 7f5add400640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ac0078b90 0x7f5ac007b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5acc033000 tx=0x7f5acc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:06.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:06.874+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ae00ada30 con 0x7f5ae00ad0e0 2024-09-21T07:47:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.095+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ae00b4330 con 0x7f5ae00ad0e0 2024-09-21T07:47:07.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.095+0000 7f5ad6000640 1 -- 172.21.15.88:0/93542628 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+5229 (secure 0 0 0) 0x7f5ad0062490 con 0x7f5ae00ad0e0 2024-09-21T07:47:07.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:07.098 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":17,"btime":"2024-09-21T07:46:59:220696+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:55.682651+0000","modified":"2024-09-21T07:46:58.709003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1785638532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1785638532},{"type":"v1","addr":"172.21.15.88:6835","nonce":1785638532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4899,"qdb_cluster":[4899]},"id":2}]} 2024-09-21T07:47:07.098 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 17 2024-09-21T07:47:07.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ac0078b90 msgr2=0x7f5ac007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ac0078b90 0x7f5ac007b0a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f5acc033000 tx=0x7f5acc004730 comp rx=0 tx=0).stop 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 msgr2=0x7f5ae0152080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 0x7f5ae0152080 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f5ad000c3c0 tx=0x7f5ad000c880 comp rx=0 tx=0).stop 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 shutdown_connections 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ac0078b90 0x7f5ac007b0a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ae015d760 0x7f5ae0070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ae00b3550 0x7f5ae00700f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 --2- 172.21.15.88:0/93542628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ae00ad0e0 0x7f5ae0152080 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 >> 172.21.15.88:0/93542628 conn(0x7f5ae006d730 msgr2=0x7f5ae00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 shutdown_connections 2024-09-21T07:47:07.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.098+0000 7f5ae4c00640 1 -- 172.21.15.88:0/93542628 wait complete. 2024-09-21T07:47:07.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:46:55.682651+0000', 'modified': '2024-09-21T07:46:58.709003+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4899}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4899': {'gid': 4899, 'name': 'c', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/1785638532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6833', 'nonce': 1785638532}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 1785638532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 4899, 'qdb_cluster': [4899]} 2024-09-21T07:47:07.526 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:47:07.526 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 -- 172.21.15.88:0/1228842323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 msgr2=0x7fc9e4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 --2- 172.21.15.88:0/1228842323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fc9e4004c50 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec009d20 tx=0x7fc9e4005830 comp rx=0 tx=0).stop 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 -- 172.21.15.88:0/1228842323 shutdown_connections 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 --2- 172.21.15.88:0/1228842323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fc9e4004c50 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 -- 172.21.15.88:0/1228842323 >> 172.21.15.88:0/1228842323 conn(0x7fca0006d3d0 msgr2=0x7fca000b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 -- 172.21.15.88:0/1228842323 shutdown_connections 2024-09-21T07:47:07.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.750+0000 7fca05000640 1 -- 172.21.15.88:0/1228842323 wait complete. 2024-09-21T07:47:07.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.753+0000 7fca05000640 1 Processor -- start 2024-09-21T07:47:07.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.753+0000 7fca05000640 1 -- start start 2024-09-21T07:47:07.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fca05000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fca00156380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fca05000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca001574f0 0x7fca00073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fca05000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca00159fa0 0x7fca000744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fca05000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca0015d240 con 0x7fc9e4004800 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fca05000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca00157320 con 0x7fca00159fa0 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fca05000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca0015d3c0 con 0x7fca001574f0 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fc9fde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fca00156380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fc9fde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fca00156380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48370/0 (socket says 172.21.15.88:48370) 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.754+0000 7fc9fde00640 1 -- 172.21.15.88:0/2427713359 learned_addr learned my addr 172.21.15.88:0/2427713359 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fd400640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca001574f0 0x7fca00073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fe800640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca00159fa0 0x7fca000744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fde00640 1 -- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca001574f0 msgr2=0x7fca00073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fde00640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca001574f0 0x7fca00073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fde00640 1 -- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca00159fa0 msgr2=0x7fca000744b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fde00640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca00159fa0 0x7fca000744b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fde00640 1 -- 172.21.15.88:0/2427713359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca001568c0 con 0x7fc9e4004800 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fd400640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca001574f0 0x7fca00073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fe800640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca00159fa0 0x7fca000744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9fde00640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fca00156380 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0003e10 tx=0x7fc9f0001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9f6000640 1 -- 172.21.15.88:0/2427713359 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0015070 con 0x7fc9e4004800 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9f6000640 1 -- 172.21.15.88:0/2427713359 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f0011920 con 0x7fc9e4004800 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca001588c0 con 0x7fc9e4004800 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.755+0000 7fc9f6000640 1 -- 172.21.15.88:0/2427713359 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0011d70 con 0x7fc9e4004800 2024-09-21T07:47:07.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.756+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001f65e0 con 0x7fc9e4004800 2024-09-21T07:47:07.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.758+0000 7fc9f6000640 1 -- 172.21.15.88:0/2427713359 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7fc9f0018070 con 0x7fc9e4004800 2024-09-21T07:47:07.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.759+0000 7fc9f6000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e0078b90 0x7fc9e007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.759+0000 7fc9f6000640 1 -- 172.21.15.88:0/2427713359 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc9f009bb40 con 0x7fc9e4004800 2024-09-21T07:47:07.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.759+0000 7fc9fd400640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e0078b90 0x7fc9e007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.760+0000 7fca0c060740 1 Processor -- start 2024-09-21T07:47:07.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.760+0000 7fca0c060740 1 -- start start 2024-09-21T07:47:07.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.760+0000 7fc9fd400640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e0078b90 0x7fc9e007b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec034000 tx=0x7fc9ec0046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a56970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fca0c060740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a56eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a26f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fca0c060740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a7829b0430 con 0x55a78265d3b0 2024-09-21T07:47:07.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a7829a05d0 con 0x55a782686270 2024-09-21T07:47:07.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a78298fa00 con 0x55a782665590 2024-09-21T07:47:07.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fc9fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a56eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fc9fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a56eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48386/0 (socket says 172.21.15.88:48386) 2024-09-21T07:47:07.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.761+0000 7fc9fd400640 1 -- 172.21.15.88:0/572380528 learned_addr learned my addr 172.21.15.88:0/572380528 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:07.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fe800640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a26f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fde00640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a56970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fd400640 1 -- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 msgr2=0x55a782a26f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fd400640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a26f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fd400640 1 -- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 msgr2=0x55a782a56970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fd400640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a56970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fd400640 1 -- 172.21.15.88:0/572380528 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a782958520 con 0x55a78265d3b0 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fe800640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a26f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fde00640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a56970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9fd400640 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a56eb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec02b650 tx=0x7fc9ec03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42ede5b395508ba3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9eaa00640 1 -- 172.21.15.88:0/572380528 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec04f070 con 0x55a78265d3b0 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9eaa00640 1 -- 172.21.15.88:0/572380528 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9ec046040 con 0x55a78265d3b0 2024-09-21T07:47:07.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.762+0000 7fc9eaa00640 1 -- 172.21.15.88:0/572380528 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec003850 con 0x55a78265d3b0 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 -- 172.21.15.88:0/572380528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 msgr2=0x55a782a56eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a56eb0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec02b650 tx=0x7fc9ec03c000 comp rx=0 tx=0).stop 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 -- 172.21.15.88:0/572380528 shutdown_connections 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a26f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a56eb0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 --2- 172.21.15.88:0/572380528 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a56970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 -- 172.21.15.88:0/572380528 >> 172.21.15.88:0/572380528 conn(0x55a78267a3c0 msgr2=0x55a782a2c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 -- 172.21.15.88:0/572380528 shutdown_connections 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.763+0000 7fca0c060740 1 -- 172.21.15.88:0/572380528 wait complete. 2024-09-21T07:47:07.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.764+0000 7fca0c060740 1 -- start start 2024-09-21T07:47:07.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.765+0000 7fca0c060740 1 Processor -- start 2024-09-21T07:47:07.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a78265d3b0 0x55a782a84df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a69d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 0x55a782a50510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a782936330 con 0x55a782686270 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a782993810 con 0x55a78265d3b0 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a782700620 con 0x55a782665590 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.766+0000 7fca0c060740 10 client.0 _ll_register_callbacks cb 0x55a78293b420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 0x55a782a50510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 0x55a782a50510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48398/0 (socket says 172.21.15.88:48398) 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 -- 172.21.15.88:0/1645815563 learned_addr learned my addr 172.21.15.88:0/1645815563 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fd400640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a69d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a78265d3b0 0x55a782a84df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 -- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 msgr2=0x55a782a69d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a69d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 -- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a78265d3b0 msgr2=0x55a782a84df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a78265d3b0 0x55a782a84df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 -- 172.21.15.88:0/1645815563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a7826f1850 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 0x55a782a50510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8002040 tx=0x7fc9f8010730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f553c88fc9dd7768 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9df400640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.767+0000 7fc9df400640 1 -- 172.21.15.88:0/1645815563 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8016070 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fc9df400640 1 -- 172.21.15.88:0/1645815563 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc9f8012640 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9ec046040 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fc9df400640 1 -- 172.21.15.88:0/1645815563 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc9f8021050 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 10 client.5091 fetch_fsmap learned FSMap version 17 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a7827686b0 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fc9df400640 1 -- 172.21.15.88:0/1645815563 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9ec046040 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fc9df400640 1 -- 172.21.15.88:0/1645815563 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fc9f8024020 con 0x55a782686270 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 10 client.5091 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 10 client.5091 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 20 client.5091 populate_metadata read hostname 'smithi088' 2024-09-21T07:47:07.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782644e40 0x55a782a4eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 4 client.5091 mds_command: new command op to 4899 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:47:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 --> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a7829a2520 con 0x55a782644e40 2024-09-21T07:47:07.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.768+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782644e40 0x55a782a4eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.769+0000 7fc9df400640 10 client.5091 ms_handle_connect on v2:172.21.15.88:6833/1785638532 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.769+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782644e40 0x55a782a4eb90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87e47c48b3a75d64 server_cookie=ea7cb9c21b1fbabd in_seq=0 out_seq=0 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fc9df400640 1 -- 172.21.15.88:0/1645815563 <== mds.0 v2:172.21.15.88:6833/1785638532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a7829a2520 con 0x55a782644e40 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fc9df400640 10 client.5091 handle_command_reply: tid=0 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 client.5091 shutdown 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 msgr2=0x55a782a50510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 0x55a782a50510 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8002040 tx=0x7fc9f8010730 comp rx=0 tx=0).stop 2024-09-21T07:47:07.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 shutdown_connections 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782644e40 0x55a782a4eb90 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a69d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782686270 0x55a782a50510 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 --2- 172.21.15.88:0/1645815563 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a78265d3b0 0x55a782a84df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fc9df400640 0 client.5091 ms_handle_reset on v2:172.21.15.88:6833/1785638532 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 >> 172.21.15.88:0/1645815563 conn(0x55a78267a3c0 msgr2=0x55a782a738c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 shutdown_connections 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.770+0000 7fca0c060740 1 -- 172.21.15.88:0/1645815563 wait complete. 2024-09-21T07:47:07.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.771+0000 7fca0c060740 20 client.5091 trim_cache size 0 max 16384 2024-09-21T07:47:07.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.789+0000 7fca0c060740 1 Processor -- start 2024-09-21T07:47:07.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.790+0000 7fca0c060740 1 -- start start 2024-09-21T07:47:07.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fca0c060740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a72820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a72d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a4fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fca0c060740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca002012d0 con 0x55a78265d3b0 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fca001f8360 con 0x55a782686270 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fca00200cb0 con 0x55a782665590 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fc9fde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a72820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fc9fde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a72820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48402/0 (socket says 172.21.15.88:48402) 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fc9fde00640 1 -- 172.21.15.88:0/1554626709 learned_addr learned my addr 172.21.15.88:0/1554626709 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fc9fd400640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a72d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.791+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a4fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fde00640 1 -- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 msgr2=0x55a782a72d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a72d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fde00640 1 -- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 msgr2=0x55a782a4fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a4fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fde00640 1 -- 172.21.15.88:0/1554626709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a782a2d650 con 0x55a78265d3b0 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fd400640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a72d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a72820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0063570 tx=0x7fc9f0005040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93e3158773aaf2e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9ea000640 1 -- 172.21.15.88:0/1554626709 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f001f040 con 0x55a78265d3b0 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9ea000640 1 -- 172.21.15.88:0/1554626709 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f0063f70 con 0x55a78265d3b0 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.792+0000 7fc9ea000640 1 -- 172.21.15.88:0/1554626709 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f0063870 con 0x55a78265d3b0 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 -- 172.21.15.88:0/1554626709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 msgr2=0x55a782a72820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a72820 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0063570 tx=0x7fc9f0005040 comp rx=0 tx=0).stop 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 -- 172.21.15.88:0/1554626709 shutdown_connections 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a782665590 0x55a782a72d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782a4fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 --2- 172.21.15.88:0/1554626709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a78265d3b0 0x55a782a72820 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 -- 172.21.15.88:0/1554626709 >> 172.21.15.88:0/1554626709 conn(0x55a78267a3c0 msgr2=0x55a782a28f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 -- 172.21.15.88:0/1554626709 shutdown_connections 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.793+0000 7fca0c060740 1 -- 172.21.15.88:0/1554626709 wait complete. 2024-09-21T07:47:07.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.794+0000 7fca0c060740 1 -- start start 2024-09-21T07:47:07.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.794+0000 7fca0c060740 1 Processor -- start 2024-09-21T07:47:07.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a78265d3b0 0x55a782a4ea90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 0x55a782af0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782af1480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a782a6a2c0 con 0x55a782665590 2024-09-21T07:47:07.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55a782a6b3d0 con 0x55a782686270 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55a782a52af0 con 0x55a78265d3b0 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fca0c060740 10 client.0 _ll_register_callbacks cb 0x7fc9f8004a10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 0x55a782af0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 0x55a782af0f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48416/0 (socket says 172.21.15.88:48416) 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 -- 172.21.15.88:0/1794734806 learned_addr learned my addr 172.21.15.88:0/1794734806 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a78265d3b0 0x55a782a4ea90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782af1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 -- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a78265d3b0 msgr2=0x55a782a4ea90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a78265d3b0 0x55a782a4ea90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 -- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 msgr2=0x55a782af1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782af1480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fd400640 1 -- 172.21.15.88:0/1794734806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a782a69a00 con 0x55a782665590 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a78265d3b0 0x55a782a4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.796+0000 7fc9fe800640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782af1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fc9fd400640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 0x55a782af0f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec04d040 tx=0x7fc9ec0039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e32eefa83037f9bb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fc9df400640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fc9df400640 1 -- 172.21.15.88:0/1794734806 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9ec057070 con 0x55a782665590 2024-09-21T07:47:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fc9df400640 1 -- 172.21.15.88:0/1794734806 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fc9ec003ce0 con 0x55a782665590 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9f0063f70 con 0x55a782665590 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fc9df400640 1 -- 172.21.15.88:0/1794734806 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fc9ec05f050 con 0x55a782665590 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fca0c060740 10 client.5097 fetch_fsmap learned FSMap version 17 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.797+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) -- 0x55a782af1a40 con 0x55a782665590 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fc9df400640 1 -- 172.21.15.88:0/1794734806 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc9f0063f70 con 0x55a782665590 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fc9df400640 1 -- 172.21.15.88:0/1794734806 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 17) ==== 3502+0+0 (secure 0 0 0) 0x7fc9ec062020 con 0x55a782665590 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fca0c060740 10 client.5097 fetch_fsmap finished waiting for FSMap version 17 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fca0c060740 10 client.5097 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fca0c060740 20 client.5097 populate_metadata read hostname 'smithi088' 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fca0c060740 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782a27f80 0x55a782a74d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fca0c060740 4 client.5097 mds_command: new command op to 4899 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 --> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] -- command(tid 0: {"prefix": "status"}) -- 0x55a78299dfd0 con 0x55a782a27f80 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.798+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782a27f80 0x55a782a74d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:07.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fc9df400640 10 client.5097 ms_handle_connect on v2:172.21.15.88:6833/1785638532 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fc9fde00640 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782a27f80 0x55a782a74d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4122d51ba758a23 server_cookie=9f560a641632dd89 in_seq=0 out_seq=0 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fc9df400640 1 -- 172.21.15.88:0/1794734806 <== mds.0 v2:172.21.15.88:6833/1785638532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+319 (crc 0 0 0) 0x55a78299dfd0 con 0x55a782a27f80 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fc9df400640 10 client.5097 handle_command_reply: tid=0 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 client.5097 shutdown 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 msgr2=0x55a782af0f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 0x55a782af0f40 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec04d040 tx=0x7fc9ec0039f0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 shutdown_connections 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:6833/1785638532,v1:172.21.15.88:6835/1785638532] conn(0x55a782a27f80 0x55a782a74d00 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55a782686270 0x55a782af1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a782665590 0x55a782af0f40 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 --2- 172.21.15.88:0/1794734806 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55a78265d3b0 0x55a782a4ea90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fc9df400640 0 client.5097 ms_handle_reset on v2:172.21.15.88:6833/1785638532 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 >> 172.21.15.88:0/1794734806 conn(0x55a78267a3c0 msgr2=0x55a782a72820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 shutdown_connections 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.799+0000 7fca0c060740 1 -- 172.21.15.88:0/1794734806 wait complete. 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca0c060740 20 client.5097 trim_cache size 0 max 16384 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 4899, 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:47:07.805 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 11.589312412, 2024-09-21T07:47:07.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 17, 2024-09-21T07:47:07.806 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 35, 2024-09-21T07:47:07.806 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 35, 2024-09-21T07:47:07.806 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 19.349782412 2024-09-21T07:47:07.806 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:47:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e0078b90 msgr2=0x7fc9e007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e0078b90 0x7fc9e007b0a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec034000 tx=0x7fc9ec0046c0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 msgr2=0x7fca00156380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fca00156380 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0003e10 tx=0x7fc9f0001800 comp rx=0 tx=0).stop 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 shutdown_connections 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc9e0078b90 0x7fc9e007b0a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fca001574f0 0x7fca00073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fca00159fa0 0x7fca000744b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 --2- 172.21.15.88:0/2427713359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9e4004800 0x7fca00156380 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 >> 172.21.15.88:0/2427713359 conn(0x7fca0006d3d0 msgr2=0x7fca0015aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 shutdown_connections 2024-09-21T07:47:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:07.800+0000 7fca05000640 1 -- 172.21.15.88:0/2427713359 wait complete. 2024-09-21T07:47:07.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 -- 172.21.15.88:0/2524450597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 msgr2=0x7f48a80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 --2- 172.21.15.88:0/2524450597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a80b3750 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f48a0005fd0 tx=0x7f48a002a960 comp rx=0 tx=0).stop 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 -- 172.21.15.88:0/2524450597 shutdown_connections 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 --2- 172.21.15.88:0/2524450597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a80b3750 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 --2- 172.21.15.88:0/2524450597 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a8152920 0x7f48a815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 -- 172.21.15.88:0/2524450597 >> 172.21.15.88:0/2524450597 conn(0x7f48a806d730 msgr2=0x7f48a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 -- 172.21.15.88:0/2524450597 shutdown_connections 2024-09-21T07:47:08.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.048+0000 7f48afa00640 1 -- 172.21.15.88:0/2524450597 wait complete. 2024-09-21T07:47:08.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.051+0000 7f48afa00640 1 Processor -- start 2024-09-21T07:47:08.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.051+0000 7f48afa00640 1 -- start start 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.052+0000 7f48afa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a80aebc0 0x7f48a81f5900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.052+0000 7f48afa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a8152920 0x7f48a80ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.052+0000 7f48afa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a8155640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.052+0000 7f48afa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80b41e0 con 0x7f48a815f910 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.052+0000 7f48afa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48a80b4060 con 0x7f48a80aebc0 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.052+0000 7f48afa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48a80b4360 con 0x7f48a8152920 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a8155640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a8155640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48422/0 (socket says 172.21.15.88:48422) 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 -- 172.21.15.88:0/3006413634 learned_addr learned my addr 172.21.15.88:0/3006413634 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48a7e00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a8152920 0x7f48a80ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ad000640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a80aebc0 0x7f48a81f5900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 -- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a8152920 msgr2=0x7f48a80ce730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a8152920 0x7f48a80ce730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 -- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a80aebc0 msgr2=0x7f48a81f5900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a80aebc0 0x7f48a81f5900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 -- 172.21.15.88:0/3006413634 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a81f61a0 con 0x7f48a815f910 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.053+0000 7f48ada00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a8155640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f48a0033040 tx=0x7f48a0031ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.054+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0034040 con 0x7f48a815f910 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.054+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48a0004540 con 0x7f48a815f910 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.054+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a81f6480 con 0x7f48a815f910 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.054+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48a0045480 con 0x7f48a815f910 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.054+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a8153f60 con 0x7f48a815f910 2024-09-21T07:47:08.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.055+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a80b48f0 con 0x7f48a815f910 2024-09-21T07:47:08.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.056+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f48a0037020 con 0x7f48a815f910 2024-09-21T07:47:08.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.057+0000 7f48a4c00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4888078b90 0x7f488807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:08.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.057+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f48a00bc3e0 con 0x7f48a815f910 2024-09-21T07:47:08.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.058+0000 7f48ad000640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4888078b90 0x7f488807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:08.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.058+0000 7f48ad000640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4888078b90 0x7f488807b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f489c006fd0 tx=0x7f489c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:08.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.062+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f48a815fd20 con 0x7f48a815f910 2024-09-21T07:47:08.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.284+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f48a80cf680 con 0x7f48a815f910 2024-09-21T07:47:08.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.284+0000 7f48a4c00640 1 -- 172.21.15.88:0/3006413634 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f48a00841c0 con 0x7f48a815f910 2024-09-21T07:47:08.286 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4888078b90 msgr2=0x7f488807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4888078b90 0x7f488807b0a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f489c006fd0 tx=0x7f489c009060 comp rx=0 tx=0).stop 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 msgr2=0x7f48a8155640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a8155640 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f48a0033040 tx=0x7f48a0031ee0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 shutdown_connections 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4888078b90 0x7f488807b0a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48a8152920 0x7f48a80ce730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48a815f910 0x7f48a8155640 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 --2- 172.21.15.88:0/3006413634 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48a80aebc0 0x7f48a81f5900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.286+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 >> 172.21.15.88:0/3006413634 conn(0x7f48a806d730 msgr2=0x7f48a80d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:08.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.287+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 shutdown_connections 2024-09-21T07:47:08.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08.287+0000 7f48afa00640 1 -- 172.21.15.88:0/3006413634 wait complete. 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:47:08.311 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:08.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:08.312 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:47:08.347 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:47:08.347 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:47:08.347 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:47:08.347 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:47:08.348 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42666sec preferred_lft 42666sec 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:08.349 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:47:08.350 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:47:08.350 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:08.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:08.350 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:47:08.441 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:47:08.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:08.445 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:47:08.481 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:47:08.481 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:47:08.481 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:47:08.481 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:47:08.481 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:47:08.481 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:47:08.482 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:47:08.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:47:08.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:47:08.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:47:08.482 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:47:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:47:08.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:08 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:08.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:08.676 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:47:08.710 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:47:08.710 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:47:08.711 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:08.712 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:47:08.807 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:08.808 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:08.842 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:08.843 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:08.843 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:08.917 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:47:08.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:08.918 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:09.112 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:09.112 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:47:09.112 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:47:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:47:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:47:09.132 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:09.133 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:47:09.163 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:47:09.164 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:47:09.165 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:47:09.165 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:47:09.165 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:47:09.165 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:47:09.165 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:47:09.168 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:47:09.168 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:09.168 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:09.224 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:47:09.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:09.320 INFO:teuthology.orchestra.run.smithi088.stdout:client5106 2024-09-21T07:47:09.320 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:09.320 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:09.356 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:47:09.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 -- 172.21.15.88:0/2715102724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 msgr2=0x7f52e00b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 --2- 172.21.15.88:0/2715102724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e00b31e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f52e00b2cb0 tx=0x7f52d802a560 comp rx=0 tx=0).stop 2024-09-21T07:47:09.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 -- 172.21.15.88:0/2715102724 shutdown_connections 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 --2- 172.21.15.88:0/2715102724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e00b31e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 --2- 172.21.15.88:0/2715102724 >> v2:172.21.15.175:3300/0 conn(0x7f52e0073c10 0x7f52e006f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 -- 172.21.15.88:0/2715102724 >> 172.21.15.88:0/2715102724 conn(0x7f52e006d730 msgr2=0x7f52e00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 -- 172.21.15.88:0/2715102724 shutdown_connections 2024-09-21T07:47:09.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.612+0000 7f52e6800640 1 -- 172.21.15.88:0/2715102724 wait complete. 2024-09-21T07:47:09.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.614+0000 7f52e6800640 1 Processor -- start 2024-09-21T07:47:09.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.614+0000 7f52e6800640 1 -- start start 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52e6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e0156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52e6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e00af630 0x7f52e01e5f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52e6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e0156cb0 0x7f52e01e6470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52e6800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52e00b3b60 con 0x7f52e006f5e0 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52e6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52e00b39e0 con 0x7f52e00af630 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52e6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52e00b3ce0 con 0x7f52e0156cb0 2024-09-21T07:47:09.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e0156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e0156050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48438/0 (socket says 172.21.15.88:48438) 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 -- 172.21.15.88:0/956290327 learned_addr learned my addr 172.21.15.88:0/956290327 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52d6a00640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e00af630 0x7f52e01e5f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52dfe00640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e0156cb0 0x7f52e01e6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 -- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e0156cb0 msgr2=0x7f52e01e6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e0156cb0 0x7f52e01e6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 -- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e00af630 msgr2=0x7f52e01e5f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e00af630 0x7f52e01e5f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.616+0000 7f52df400640 1 -- 172.21.15.88:0/956290327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52e015a280 con 0x7f52e006f5e0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.617+0000 7f52df400640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e0156050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f52d000ba80 tx=0x7f52d000bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.617+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d0019070 con 0x7f52e006f5e0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.617+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f52d000cd80 con 0x7f52e006f5e0 2024-09-21T07:47:09.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.617+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52e015a560 con 0x7f52e006f5e0 2024-09-21T07:47:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.617+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52d00146a0 con 0x7f52e006f5e0 2024-09-21T07:47:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.617+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52e015ab20 con 0x7f52e006f5e0 2024-09-21T07:47:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.618+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52e0159da0 con 0x7f52e006f5e0 2024-09-21T07:47:09.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.619+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f52d001c070 con 0x7f52e006f5e0 2024-09-21T07:47:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.621+0000 7f52d7e00640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f52bc078b90 0x7f52bc07b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.621+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f52d009ab70 con 0x7f52e006f5e0 2024-09-21T07:47:09.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.621+0000 7f52d6a00640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f52bc078b90 0x7f52bc07b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:09.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.622+0000 7f52d6a00640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f52bc078b90 0x7f52bc07b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52e01eaab0 tx=0x7f52c8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:09.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.626+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f52e0159da0 con 0x7f52e006f5e0 2024-09-21T07:47:09.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.848+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f52e00b4210 con 0x7f52e006f5e0 2024-09-21T07:47:09.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.849+0000 7f52d7e00640 1 -- 172.21.15.88:0/956290327 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+73 (secure 0 0 0) 0x7f52d0062fa0 con 0x7f52e006f5e0 2024-09-21T07:47:09.851 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f52bc078b90 msgr2=0x7f52bc07b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f52bc078b90 0x7f52bc07b0a0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f52e01eaab0 tx=0x7f52c8008040 comp rx=0 tx=0).stop 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 msgr2=0x7f52e0156050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e0156050 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f52d000ba80 tx=0x7f52d000bf40 comp rx=0 tx=0).stop 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 shutdown_connections 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f52bc078b90 0x7f52bc07b0a0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52e0156cb0 0x7f52e01e6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52e00af630 0x7f52e01e5f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 --2- 172.21.15.88:0/956290327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52e006f5e0 0x7f52e0156050 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 >> 172.21.15.88:0/956290327 conn(0x7f52e006d730 msgr2=0x7f52e01574f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 shutdown_connections 2024-09-21T07:47:09.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:09.851+0000 7f52e6800640 1 -- 172.21.15.88:0/956290327 wait complete. 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:47:09.875 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:47:09.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:09.876 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:47:09.916 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42615sec preferred_lft 42615sec 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:47:09.917 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:47:09.918 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:09.919 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:09.919 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:47:10.017 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:47:10.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:10.021 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:47:10.057 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:47:10.057 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:47:10.057 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:47:10.057 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:47:10.058 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:47:10.058 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:47:10.058 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:47:10.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:47:10.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:47:10.058 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:47:10.058 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:47:10.175 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:10 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:47:10.246 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:10.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:10.254 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:47:10.289 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:47:10.289 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:47:10.289 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:47:10.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:10.290 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:47:10.397 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:10.397 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:10.432 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:47:10.432 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:10.432 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:10.510 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:47:10.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:10.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:47:10.709 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:47:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:47:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:47:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:10.730 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:47:10.763 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:47:10.765 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:47:10.765 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:47:10.765 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:47:10.765 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:47:10.765 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:47:10.769 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:47:10.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:10.769 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:10.826 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:47:10.905 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:10 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:10.909 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:10 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:10.917 INFO:teuthology.orchestra.run.smithi175.stdout:client4835 2024-09-21T07:47:10.918 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:10.918 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:10.953 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.178+0000 7f0368000640 1 -- 172.21.15.88:0/4110727962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 msgr2=0x7f03600b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.178+0000 7f0368000640 1 --2- 172.21.15.88:0/4110727962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 0x7f03600b4620 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0354009830 tx=0x7f035402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.178+0000 7f0368000640 1 -- 172.21.15.88:0/4110727962 shutdown_connections 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.178+0000 7f0368000640 1 --2- 172.21.15.88:0/4110727962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 0x7f03600b4620 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.178+0000 7f0368000640 1 -- 172.21.15.88:0/4110727962 >> 172.21.15.88:0/4110727962 conn(0x7f03600b2540 msgr2=0x7f03600d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.179+0000 7f0368000640 1 -- 172.21.15.88:0/4110727962 shutdown_connections 2024-09-21T07:47:11.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.179+0000 7f0368000640 1 -- 172.21.15.88:0/4110727962 wait complete. 2024-09-21T07:47:11.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.181+0000 7f0368000640 1 Processor -- start 2024-09-21T07:47:11.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.181+0000 7f0368000640 1 -- start start 2024-09-21T07:47:11.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.182+0000 7f0368000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 0x7f03601f1230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.182+0000 7f0368000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03600af100 0x7f03601f3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.182+0000 7f0368000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 0x7f0360154fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.182+0000 7f0368000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03600b3780 con 0x7f036006cda0 2024-09-21T07:47:11.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.182+0000 7f0368000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03600b3600 con 0x7f03600cc830 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.182+0000 7f0368000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03600b3900 con 0x7f03600af100 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 0x7f03601f1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 0x7f03601f1230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48444/0 (socket says 172.21.15.88:48444) 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 -- 172.21.15.88:0/1871497134 learned_addr learned my addr 172.21.15.88:0/1871497134 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0364c00640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03600af100 0x7f03601f3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0366000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 0x7f0360154fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 -- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03600af100 msgr2=0x7f03601f3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03600af100 0x7f03601f3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 -- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 msgr2=0x7f0360154fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 0x7f0360154fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 -- 172.21.15.88:0/1871497134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03601f1770 con 0x7f036006cda0 2024-09-21T07:47:11.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.183+0000 7f0365600640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 0x7f03601f1230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0354009950 tx=0x7f0354033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:11.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.184+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354031b50 con 0x7f036006cda0 2024-09-21T07:47:11.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.184+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0360146a70 con 0x7f036006cda0 2024-09-21T07:47:11.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.184+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0354031de0 con 0x7f036006cda0 2024-09-21T07:47:11.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.184+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0354005280 con 0x7f036006cda0 2024-09-21T07:47:11.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.184+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0360147030 con 0x7f036006cda0 2024-09-21T07:47:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.185+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03600b3db0 con 0x7f036006cda0 2024-09-21T07:47:11.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.186+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f0354005420 con 0x7f036006cda0 2024-09-21T07:47:11.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.187+0000 7f035d600640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0340078b90 0x7f034007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.187+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f03540bbdd0 con 0x7f036006cda0 2024-09-21T07:47:11.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.188+0000 7f0364c00640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0340078b90 0x7f034007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.188+0000 7f0364c00640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0340078b90 0x7f034007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f036014af80 tx=0x7f034c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:11.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.192+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f035403f030 con 0x7f036006cda0 2024-09-21T07:47:11.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.420+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f03600d1030 con 0x7f036006cda0 2024-09-21T07:47:11.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.472+0000 7f035d600640 1 -- 172.21.15.88:0/1871497134 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v18) ==== 136+0+0 (secure 0 0 0) 0x7f0354031770 con 0x7f036006cda0 2024-09-21T07:47:11.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0340078b90 msgr2=0x7f034007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0340078b90 0x7f034007b0a0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f036014af80 tx=0x7f034c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:11.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 msgr2=0x7f03601f1230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 0x7f03601f1230 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f0354009950 tx=0x7f0354033ba0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 shutdown_connections 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0340078b90 0x7f034007b0a0 secure :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f036014af80 tx=0x7f034c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03600af100 0x7f03601f3f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03600cc830 0x7f0360154fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 --2- 172.21.15.88:0/1871497134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f036006cda0 0x7f03601f1230 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 >> 172.21.15.88:0/1871497134 conn(0x7f03600b2540 msgr2=0x7f0360074740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 shutdown_connections 2024-09-21T07:47:11.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.475+0000 7f0368000640 1 -- 172.21.15.88:0/1871497134 wait complete. 2024-09-21T07:47:11.500 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.718+0000 7f6e6d200640 1 -- 172.21.15.88:0/3727257863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 msgr2=0x7f6e680af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.718+0000 7f6e6d200640 1 --2- 172.21.15.88:0/3727257863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680af170 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6e680af6b0 tx=0x7f6e5802a560 comp rx=0 tx=0).stop 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.719+0000 7f6e6d200640 1 -- 172.21.15.88:0/3727257863 shutdown_connections 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.719+0000 7f6e6d200640 1 --2- 172.21.15.88:0/3727257863 >> v2:172.21.15.175:3300/0 conn(0x7f6e6815a0f0 0x7f6e6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.719+0000 7f6e6d200640 1 --2- 172.21.15.88:0/3727257863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680af170 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.719+0000 7f6e6d200640 1 -- 172.21.15.88:0/3727257863 >> 172.21.15.88:0/3727257863 conn(0x7f6e680b1e40 msgr2=0x7f6e680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.719+0000 7f6e6d200640 1 -- 172.21.15.88:0/3727257863 shutdown_connections 2024-09-21T07:47:11.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.719+0000 7f6e6d200640 1 -- 172.21.15.88:0/3727257863 wait complete. 2024-09-21T07:47:11.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.721+0000 7f6e6d200640 1 Processor -- start 2024-09-21T07:47:11.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.721+0000 7f6e6d200640 1 -- start start 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.722+0000 7f6e6d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.722+0000 7f6e6d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e680b2f60 0x7f6e68147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e6d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e6815a0f0 0x7f6e6809b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e6d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6806f290 con 0x7f6e680ad9c0 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e6d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e6815a5e0 con 0x7f6e6815a0f0 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e6d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e6806f410 con 0x7f6e680b2f60 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48476/0 (socket says 172.21.15.88:48476) 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 -- 172.21.15.88:0/353105312 learned_addr learned my addr 172.21.15.88:0/353105312 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:11.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67e00640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e6815a0f0 0x7f6e6809b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e66a00640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e680b2f60 0x7f6e68147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 -- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e680b2f60 msgr2=0x7f6e68147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e680b2f60 0x7f6e68147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 -- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e6815a0f0 msgr2=0x7f6e6809b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e6815a0f0 0x7f6e6809b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 -- 172.21.15.88:0/353105312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e680cf920 con 0x7f6e680ad9c0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67e00640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e6815a0f0 0x7f6e6809b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.723+0000 7f6e67400640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680ccac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6e580099c0 tx=0x7f6e580039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.724+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e5803e030 con 0x7f6e680ad9c0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.724+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e580047a0 con 0x7f6e680ad9c0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.724+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e58035040 con 0x7f6e680ad9c0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.724+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e6809abf0 con 0x7f6e680ad9c0 2024-09-21T07:47:11.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.724+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e680cf640 con 0x7f6e680ad9c0 2024-09-21T07:47:11.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.725+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e6806f930 con 0x7f6e680ad9c0 2024-09-21T07:47:11.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.726+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f6e58041030 con 0x7f6e680ad9c0 2024-09-21T07:47:11.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.727+0000 7f6e5f400640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6e48078b90 0x7f6e4807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:11.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.727+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f6e580bb640 con 0x7f6e680ad9c0 2024-09-21T07:47:11.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.727+0000 7f6e66a00640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6e48078b90 0x7f6e4807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:11.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.728+0000 7f6e66a00640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6e48078b90 0x7f6e4807b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e6809c4b0 tx=0x7f6e54009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:11.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.732+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e580839e0 con 0x7f6e680ad9c0 2024-09-21T07:47:11.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.936+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6e680b4d80 con 0x7f6e48078b90 2024-09-21T07:47:11.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.939+0000 7f6e5f400640 1 -- 172.21.15.88:0/353105312 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6e680b4d80 con 0x7f6e48078b90 2024-09-21T07:47:11.940 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:47:11.940 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:11.940 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:47:11.940 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:11.940 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6e48078b90 msgr2=0x7f6e4807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6e48078b90 0x7f6e4807b0a0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6e6809c4b0 tx=0x7f6e54009040 comp rx=0 tx=0).stop 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 msgr2=0x7f6e680ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680ccac0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f6e580099c0 tx=0x7f6e580039e0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 shutdown_connections 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6e48078b90 0x7f6e4807b0a0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e680b2f60 0x7f6e68147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e6815a0f0 0x7f6e6809b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 --2- 172.21.15.88:0/353105312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e680ad9c0 0x7f6e680ccac0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 >> 172.21.15.88:0/353105312 conn(0x7f6e680b1e40 msgr2=0x7f6e68070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 shutdown_connections 2024-09-21T07:47:11.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:11.940+0000 7f6e6d200640 1 -- 172.21.15.88:0/353105312 wait complete. 2024-09-21T07:47:11.965 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 -- 172.21.15.88:0/3760303681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0145550 msgr2=0x7f8cf006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 --2- 172.21.15.88:0/3760303681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0145550 0x7f8cf006fb40 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f8cf0160280 tx=0x7f8ce002a6c0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 -- 172.21.15.88:0/3760303681 shutdown_connections 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 --2- 172.21.15.88:0/3760303681 >> v2:172.21.15.175:3300/0 conn(0x7f8cf0145a40 0x7f8cf0073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 --2- 172.21.15.88:0/3760303681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0145550 0x7f8cf006fb40 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 -- 172.21.15.88:0/3760303681 >> 172.21.15.88:0/3760303681 conn(0x7f8cf006d1c0 msgr2=0x7f8cf006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 -- 172.21.15.88:0/3760303681 shutdown_connections 2024-09-21T07:47:12.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.183+0000 7f8cf5400640 1 -- 172.21.15.88:0/3760303681 wait complete. 2024-09-21T07:47:12.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.185+0000 7f8cf5400640 1 Processor -- start 2024-09-21T07:47:12.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.186+0000 7f8cf5400640 1 -- start start 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cf5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 0x7f8cf015ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cf5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cf0145550 0x7f8cf0136c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cf5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cf0145a40 0x7f8cf01f7450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cf5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf0160a30 con 0x7f8cf0073a80 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cf5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8cf00ce870 con 0x7f8cf0145a40 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cf5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8cf0160bb0 con 0x7f8cf0145550 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 0x7f8cf015ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 0x7f8cf015ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48494/0 (socket says 172.21.15.88:48494) 2024-09-21T07:47:12.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cee800640 1 -- 172.21.15.88:0/2355233856 learned_addr learned my addr 172.21.15.88:0/2355233856 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cede00640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cf0145550 0x7f8cf0136c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.187+0000 7f8cef200640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cf0145a40 0x7f8cf01f7450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cee800640 1 -- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cf0145550 msgr2=0x7f8cf0136c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cee800640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cf0145550 0x7f8cf0136c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cee800640 1 -- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cf0145a40 msgr2=0x7f8cf01f7450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cee800640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cf0145a40 0x7f8cf01f7450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cee800640 1 -- 172.21.15.88:0/2355233856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cf015e7d0 con 0x7f8cf0073a80 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cef200640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cf0145a40 0x7f8cf01f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:12.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cee800640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 0x7f8cf015ddf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0034040 tx=0x7f8ce0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce003e030 con 0x7f8cf0073a80 2024-09-21T07:47:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ce0003e00 con 0x7f8cf0073a80 2024-09-21T07:47:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ce0035040 con 0x7f8cf0073a80 2024-09-21T07:47:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.188+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cf015eab0 con 0x7f8cf0073a80 2024-09-21T07:47:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.189+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cf015f070 con 0x7f8cf0073a80 2024-09-21T07:47:12.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.190+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8cf01610d0 con 0x7f8cf0073a80 2024-09-21T07:47:12.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.191+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f8ce0041030 con 0x7f8cf0073a80 2024-09-21T07:47:12.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.192+0000 7f8ce6a00640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cd0078b90 0x7f8cd007b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.192+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f8ce00bb420 con 0x7f8cf0073a80 2024-09-21T07:47:12.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.192+0000 7f8cede00640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cd0078b90 0x7f8cd007b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.193+0000 7f8cede00640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cd0078b90 0x7f8cd007b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cf013b380 tx=0x7f8cdc00b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:12.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.196+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ce0083200 con 0x7f8cf0073a80 2024-09-21T07:47:12.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.388+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8cf00cf050 con 0x7f8cf0073a80 2024-09-21T07:47:12.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.389+0000 7f8ce6a00640 1 -- 172.21.15.88:0/2355233856 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v32)=0 v32) ==== 122+0+0 (secure 0 0 0) 0x7f8ce0087c30 con 0x7f8cf0073a80 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cd0078b90 msgr2=0x7f8cd007b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cd0078b90 0x7f8cd007b0a0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8cf013b380 tx=0x7f8cdc00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 msgr2=0x7f8cf015ddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 0x7f8cf015ddf0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f8ce0034040 tx=0x7f8ce0003910 comp rx=0 tx=0).stop 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 shutdown_connections 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8cd0078b90 0x7f8cd007b0a0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8cf0145550 0x7f8cf0136c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8cf0145a40 0x7f8cf01f7450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 --2- 172.21.15.88:0/2355233856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cf0073a80 0x7f8cf015ddf0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.391+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 >> 172.21.15.88:0/2355233856 conn(0x7f8cf006d1c0 msgr2=0x7f8cf0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:12.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.392+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 shutdown_connections 2024-09-21T07:47:12.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.392+0000 7f8cf5400640 1 -- 172.21.15.88:0/2355233856 wait complete. 2024-09-21T07:47:12.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:12.416 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp12090/subvol94200) 2024-09-21T07:47:12.480 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:12.480 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:47:12.517 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:47:12.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:12.517 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp12090/subvol94200) 2024-09-21T07:47:12.631 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:12.631 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp12090/subvol94200/.meta 2024-09-21T07:47:12.692 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol94200 subvol_grp12090 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 -- 172.21.15.88:0/3926632379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0b3ec0 msgr2=0x7f29fc0b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 --2- 172.21.15.88:0/3926632379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0b3ec0 0x7f29fc0b47c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f29ec005fd0 tx=0x7f29ec02a540 comp rx=0 tx=0).stop 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 -- 172.21.15.88:0/3926632379 shutdown_connections 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 --2- 172.21.15.88:0/3926632379 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29fc0705b0 0x7f29fc0af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 --2- 172.21.15.88:0/3926632379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0b3ec0 0x7f29fc0b47c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 -- 172.21.15.88:0/3926632379 >> 172.21.15.88:0/3926632379 conn(0x7f29fc0b1dd0 msgr2=0x7f29fc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 -- 172.21.15.88:0/3926632379 shutdown_connections 2024-09-21T07:47:12.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.955+0000 7f2a02200640 1 -- 172.21.15.88:0/3926632379 wait complete. 2024-09-21T07:47:12.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.957+0000 7f2a02200640 1 Processor -- start 2024-09-21T07:47:12.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.958+0000 7f2a02200640 1 -- start start 2024-09-21T07:47:12.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a02200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 0x7f29fc0741a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a02200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29fc0b3ec0 0x7f29fc145200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a02200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29fc1532e0 0x7f29fc145d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a02200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc153a80 con 0x7f29fc0705b0 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a02200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29fc153900 con 0x7f29fc1532e0 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a02200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29fc153c00 con 0x7f29fc0b3ec0 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a00e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 0x7f29fc0741a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a00e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 0x7f29fc0741a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48510/0 (socket says 172.21.15.88:48510) 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a00e00640 1 -- 172.21.15.88:0/2780351636 learned_addr learned my addr 172.21.15.88:0/2780351636 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f29f3e00640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29fc0b3ec0 0x7f29fc145200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.959+0000 7f2a01800640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29fc1532e0 0x7f29fc145d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a00e00640 1 -- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29fc0b3ec0 msgr2=0x7f29fc145200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a00e00640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29fc0b3ec0 0x7f29fc145200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a00e00640 1 -- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29fc1532e0 msgr2=0x7f29fc145d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:12.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a00e00640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29fc1532e0 0x7f29fc145d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a00e00640 1 -- 172.21.15.88:0/2780351636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29fc073100 con 0x7f29fc0705b0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a00e00640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 0x7f29fc0741a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f29ec034040 tx=0x7f29ec0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec03e030 con 0x7f29fc0705b0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29ec003d90 con 0x7f29fc0705b0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29fc0733e0 con 0x7f29fc0705b0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.960+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29ec035040 con 0x7f29fc0705b0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.961+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29fc0739a0 con 0x7f29fc0705b0 2024-09-21T07:47:12.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.962+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29fc0af3c0 con 0x7f29fc0705b0 2024-09-21T07:47:12.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.963+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f29ec041030 con 0x7f29fc0705b0 2024-09-21T07:47:12.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.964+0000 7f29f9600640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f29d8078b90 0x7f29d807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:12.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.964+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f29ec0bb8b0 con 0x7f29fc0705b0 2024-09-21T07:47:12.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.964+0000 7f29f3e00640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f29d8078b90 0x7f29d807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:12.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.965+0000 7f29f3e00640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f29d8078b90 0x7f29d807b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0affd0 tx=0x7f29e400b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:12.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:12.968+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f29ec083600 con 0x7f29fc0705b0 2024-09-21T07:47:13.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.184+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol94200", "group_name": "subvol_grp12090", "target": ["mon-mgr", ""]}) -- 0x7f29fc0afd20 con 0x7f29d8078b90 2024-09-21T07:47:13.194 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:13.193+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:13.194 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:13.193+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:13.194 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:13.193+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:13.194 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:13.193+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:13.194 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:13.193+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:13.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.238+0000 7f29f9600640 1 -- 172.21.15.88:0/2780351636 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f29fc0afd20 con 0x7f29d8078b90 2024-09-21T07:47:13.240 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp12090/subvol94200 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.240+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f29d8078b90 msgr2=0x7f29d807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.240+0000 7f2a02200640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f29d8078b90 0x7f29d807b0a0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f29fc0affd0 tx=0x7f29e400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.240+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 msgr2=0x7f29fc0741a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.240+0000 7f2a02200640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 0x7f29fc0741a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f29ec034040 tx=0x7f29ec0038a0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 shutdown_connections 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f29d8078b90 0x7f29d807b0a0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29fc0b3ec0 0x7f29fc145200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29fc1532e0 0x7f29fc145d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 --2- 172.21.15.88:0/2780351636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29fc0705b0 0x7f29fc0741a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 >> 172.21.15.88:0/2780351636 conn(0x7f29fc0b1dd0 msgr2=0x7f29fc068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 shutdown_connections 2024-09-21T07:47:13.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.241+0000 7f2a02200640 1 -- 172.21.15.88:0/2780351636 wait complete. 2024-09-21T07:47:13.265 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol94200 subvol_grp12090 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.485+0000 7f059d200640 1 -- 172.21.15.88:0/1880393430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806d4b0 msgr2=0x7f05980b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.485+0000 7f059d200640 1 --2- 172.21.15.88:0/1880393430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806d4b0 0x7f05980b4620 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f0584009830 tx=0x7f058402a5b0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.486+0000 7f059d200640 1 -- 172.21.15.88:0/1880393430 shutdown_connections 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.486+0000 7f059d200640 1 --2- 172.21.15.88:0/1880393430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f059806d4b0 0x7f05980b4620 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.486+0000 7f059d200640 1 -- 172.21.15.88:0/1880393430 >> 172.21.15.88:0/1880393430 conn(0x7f05980b2540 msgr2=0x7f05980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.486+0000 7f059d200640 1 -- 172.21.15.88:0/1880393430 shutdown_connections 2024-09-21T07:47:13.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.486+0000 7f059d200640 1 -- 172.21.15.88:0/1880393430 wait complete. 2024-09-21T07:47:13.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.488+0000 7f059d200640 1 Processor -- start 2024-09-21T07:47:13.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.488+0000 7f059d200640 1 -- start start 2024-09-21T07:47:13.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.489+0000 7f059d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 0x7f0598074220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.489+0000 7f059d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05980cc830 0x7f0598155790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f059d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0598155cd0 0x7f05980992c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f059d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05980b3780 con 0x7f05980af100 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f059d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05980b3600 con 0x7f0598155cd0 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f059d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f05980b3900 con 0x7f05980cc830 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 0x7f0598074220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 0x7f0598074220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48512/0 (socket says 172.21.15.88:48512) 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 -- 172.21.15.88:0/513971015 learned_addr learned my addr 172.21.15.88:0/513971015 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0596800640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0598155cd0 0x7f05980992c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595400640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05980cc830 0x7f0598155790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 -- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05980cc830 msgr2=0x7f0598155790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05980cc830 0x7f0598155790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 -- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0598155cd0 msgr2=0x7f05980992c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0598155cd0 0x7f05980992c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.490+0000 7f0595e00640 1 -- 172.21.15.88:0/513971015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05981542e0 con 0x7f05980af100 2024-09-21T07:47:13.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.491+0000 7f0595e00640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 0x7f0598074220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f058800c400 tx=0x7f058800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:13.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.491+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0588017070 con 0x7f05980af100 2024-09-21T07:47:13.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.491+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05880053c0 con 0x7f05980af100 2024-09-21T07:47:13.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.491+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05981e6f90 con 0x7f05980af100 2024-09-21T07:47:13.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.491+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05880037f0 con 0x7f05980af100 2024-09-21T07:47:13.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.491+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05981e5480 con 0x7f05980af100 2024-09-21T07:47:13.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.492+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05980b3db0 con 0x7f05980af100 2024-09-21T07:47:13.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.493+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 5) ==== 101441+0+0 (secure 0 0 0) 0x7f058801a070 con 0x7f05980af100 2024-09-21T07:47:13.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.494+0000 7f058e000640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0578078b90 0x7f057807b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:13.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.494+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f0588099ff0 con 0x7f05980af100 2024-09-21T07:47:13.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.495+0000 7f0595400640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0578078b90 0x7f057807b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:13.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.495+0000 7f0595400640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0578078b90 0x7f057807b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0584034000 tx=0x7f0584004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:13.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.499+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0588061d70 con 0x7f05980af100 2024-09-21T07:47:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.708+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol94200", "group_name": "subvol_grp12090", "target": ["mon-mgr", ""]}) -- 0x7f0598074eb0 con 0x7f0578078b90 2024-09-21T07:47:13.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.714+0000 7f058e000640 1 -- 172.21.15.88:0/513971015 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0598074eb0 con 0x7f0578078b90 2024-09-21T07:47:13.716 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp12090/subvol94200 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0578078b90 msgr2=0x7f057807b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0578078b90 0x7f057807b0a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f0584034000 tx=0x7f0584004730 comp rx=0 tx=0).stop 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 msgr2=0x7f0598074220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 0x7f0598074220 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f058800c400 tx=0x7f058800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 shutdown_connections 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0578078b90 0x7f057807b0a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05980cc830 0x7f0598155790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0598155cd0 0x7f05980992c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 --2- 172.21.15.88:0/513971015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05980af100 0x7f0598074220 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:13.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.716+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 >> 172.21.15.88:0/513971015 conn(0x7f05980b2540 msgr2=0x7f05980ca1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:13.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.717+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 shutdown_connections 2024-09-21T07:47:13.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:13.717+0000 7f059d200640 1 -- 172.21.15.88:0/513971015 wait complete. 2024-09-21T07:47:13.743 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:13.743 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/2c1895016f09b4ed73ea6768052b7bb2.meta) 2024-09-21T07:47:18.464 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:47:18.464 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:47:18.464 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:47:18.464 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/subvol_grp12090/subvol94200 2024-09-21T07:47:18.464 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:47:18.464 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:18.468 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol94200 subvol_grp12090 2024-09-21T07:47:18.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351c00640 1 -- 172.21.15.88:0/1250792147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3300041c0 msgr2=0x7fe330004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351c00640 1 --2- 172.21.15.88:0/1250792147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3300041c0 0x7fe330004800 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fe334006f60 tx=0x7fe330005830 comp rx=0 tx=0).stop 2024-09-21T07:47:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351200640 1 -- 172.21.15.88:0/1250792147 reap_dead start 2024-09-21T07:47:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351c00640 1 -- 172.21.15.88:0/1250792147 shutdown_connections 2024-09-21T07:47:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351c00640 1 -- 172.21.15.88:0/1250792147 >> 172.21.15.88:0/1250792147 conn(0x7fe34c0b1dd0 msgr2=0x7fe34c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351c00640 1 -- 172.21.15.88:0/1250792147 shutdown_connections 2024-09-21T07:47:18.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.699+0000 7fe351c00640 1 -- 172.21.15.88:0/1250792147 wait complete. 2024-09-21T07:47:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.701+0000 7fe351c00640 1 Processor -- start 2024-09-21T07:47:18.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.701+0000 7fe351c00640 1 -- start start 2024-09-21T07:47:18.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3300041c0 0x7fe34c154e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 0x7fe34c1f8800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe34c1f8d40 0x7fe34c157220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c160440 con 0x7fe34c14b670 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe34c1602c0 con 0x7fe3300041c0 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe34c1605c0 con 0x7fe34c1f8d40 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 0x7fe34c1f8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 0x7fe34c1f8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59738/0 (socket says 172.21.15.88:59738) 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 -- 172.21.15.88:0/880646913 learned_addr learned my addr 172.21.15.88:0/880646913 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe34be00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3300041c0 0x7fe34c154e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe351200640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe34c1f8d40 0x7fe34c157220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 -- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe34c1f8d40 msgr2=0x7fe34c157220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe34c1f8d40 0x7fe34c157220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 -- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3300041c0 msgr2=0x7fe34c154e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3300041c0 0x7fe34c154e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.703+0000 7fe343400640 1 -- 172.21.15.88:0/880646913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe34c1f96f0 con 0x7fe34c14b670 2024-09-21T07:47:18.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.704+0000 7fe343400640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 0x7fe34c1f8800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c350 tx=0x7fe334006cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.704+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3340037f0 con 0x7fe34c14b670 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.704+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe334003990 con 0x7fe34c14b670 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.704+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe34c1582d0 con 0x7fe34c14b670 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.704+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334003de0 con 0x7fe34c14b670 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.704+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe34c158890 con 0x7fe34c14b670 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.705+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe34c155a30 con 0x7fe34c14b670 2024-09-21T07:47:18.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.706+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 101492+0+0 (secure 0 0 0) 0x7fe334035020 con 0x7fe34c14b670 2024-09-21T07:47:18.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.707+0000 7fe348c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe328078c20 0x7fe32807b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:18.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.708+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fe3340bb940 con 0x7fe34c14b670 2024-09-21T07:47:18.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.708+0000 7fe34be00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe328078c20 0x7fe32807b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:18.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.709+0000 7fe34be00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe328078c20 0x7fe32807b130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009bb0 tx=0x7fe33c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:18.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.712+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe334047060 con 0x7fe34c14b670 2024-09-21T07:47:18.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.928+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol94200", "group_name": "subvol_grp12090", "target": ["mon-mgr", ""]}) -- 0x7fe34c15c3e0 con 0x7fe328078c20 2024-09-21T07:47:18.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.961+0000 7fe348c00640 1 -- 172.21.15.88:0/880646913 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe34c15c3e0 con 0x7fe328078c20 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe328078c20 msgr2=0x7fe32807b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe328078c20 0x7fe32807b130 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fe33c009bb0 tx=0x7fe33c005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 msgr2=0x7fe34c1f8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 0x7fe34c1f8800 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe33400c350 tx=0x7fe334006cc0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 shutdown_connections 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe328078c20 0x7fe32807b130 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe34c1f8d40 0x7fe34c157220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe34c14b670 0x7fe34c1f8800 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 --2- 172.21.15.88:0/880646913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3300041c0 0x7fe34c154e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:18.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 >> 172.21.15.88:0/880646913 conn(0x7fe34c0b1dd0 msgr2=0x7fe34c06f710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 shutdown_connections 2024-09-21T07:47:18.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:18.963+0000 7fe351c00640 1 -- 172.21.15.88:0/880646913 wait complete. 2024-09-21T07:47:18.967 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.965+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.967 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.965+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.967 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.965+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.967 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.965+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.968 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.965+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.988 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:47:18.989 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.986+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.989 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.986+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.989 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.986+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.989 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.986+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:18.989 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:18.986+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:19.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:19.074 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:47:19.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:19.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp12090 2024-09-21T07:47:19.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 -- 172.21.15.88:0/4230752584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900cd1d0 msgr2=0x7f28900af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 --2- 172.21.15.88:0/4230752584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900cd1d0 0x7f28900af0d0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f2890075230 tx=0x7f288802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 -- 172.21.15.88:0/4230752584 shutdown_connections 2024-09-21T07:47:19.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 --2- 172.21.15.88:0/4230752584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28900cd1d0 0x7f28900af0d0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 --2- 172.21.15.88:0/4230752584 >> v2:172.21.15.175:3300/0 conn(0x7f28900b3f30 0x7f2890156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 -- 172.21.15.88:0/4230752584 >> 172.21.15.88:0/4230752584 conn(0x7f289006d620 msgr2=0x7f28900b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:19.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 -- 172.21.15.88:0/4230752584 shutdown_connections 2024-09-21T07:47:19.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.308+0000 7f2896e00640 1 -- 172.21.15.88:0/4230752584 wait complete. 2024-09-21T07:47:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.310+0000 7f2896e00640 1 Processor -- start 2024-09-21T07:47:19.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.310+0000 7f2896e00640 1 -- start start 2024-09-21T07:47:19.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.311+0000 7f2896e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28900b3f30 0x7f2890074a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28900cd1d0 0x7f2890072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 0x7f2890145b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28900af9a0 con 0x7f289015d6e0 2024-09-21T07:47:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28900af820 con 0x7f28900cd1d0 2024-09-21T07:47:19.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28900afb20 con 0x7f28900b3f30 2024-09-21T07:47:19.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 0x7f2890145b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 0x7f2890145b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59744/0 (socket says 172.21.15.88:59744) 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 -- 172.21.15.88:0/2204841973 learned_addr learned my addr 172.21.15.88:0/2204841973 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2895000640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28900cd1d0 0x7f2890072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2895a00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28900b3f30 0x7f2890074a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 -- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28900b3f30 msgr2=0x7f2890074a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28900b3f30 0x7f2890074a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 -- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28900cd1d0 msgr2=0x7f2890072eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28900cd1d0 0x7f2890072eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.312+0000 7f2896400640 1 -- 172.21.15.88:0/2204841973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2890145980 con 0x7f289015d6e0 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.313+0000 7f2896400640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 0x7f2890145b70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2888009ca0 tx=0x7f2888003990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:19.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.313+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2888031990 con 0x7f289015d6e0 2024-09-21T07:47:19.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.313+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2888031c20 con 0x7f289015d6e0 2024-09-21T07:47:19.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.313+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2890144b30 con 0x7f289015d6e0 2024-09-21T07:47:19.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.313+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28880330d0 con 0x7f289015d6e0 2024-09-21T07:47:19.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.313+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2890155410 con 0x7f289015d6e0 2024-09-21T07:47:19.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.314+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2890156ca0 con 0x7f289015d6e0 2024-09-21T07:47:19.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.315+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 6) ==== 101492+0+0 (secure 0 0 0) 0x7f2888033290 con 0x7f289015d6e0 2024-09-21T07:47:19.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.316+0000 7f288d600640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2870078c20 0x7f287007b130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:19.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.316+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f28880bc490 con 0x7f289015d6e0 2024-09-21T07:47:19.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.317+0000 7f2895a00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2870078c20 0x7f287007b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.317+0000 7f2895a00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2870078c20 0x7f287007b130 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f288400bb50 tx=0x7f288400b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.321+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f288803f030 con 0x7f289015d6e0 2024-09-21T07:47:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.533+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp12090", "target": ["mon-mgr", ""]}) -- 0x7f28901544f0 con 0x7f2870078c20 2024-09-21T07:47:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.538+0000 7f288d600640 1 -- 172.21.15.88:0/2204841973 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f28901544f0 con 0x7f2870078c20 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2870078c20 msgr2=0x7f287007b130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2870078c20 0x7f287007b130 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f288400bb50 tx=0x7f288400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 msgr2=0x7f2890145b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 0x7f2890145b70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f2888009ca0 tx=0x7f2888003990 comp rx=0 tx=0).stop 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 shutdown_connections 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2870078c20 0x7f287007b130 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f289015d6e0 0x7f2890145b70 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28900cd1d0 0x7f2890072eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 --2- 172.21.15.88:0/2204841973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28900b3f30 0x7f2890074a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:19.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 >> 172.21.15.88:0/2204841973 conn(0x7f289006d620 msgr2=0x7f28900cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:19.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 shutdown_connections 2024-09-21T07:47:19.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:19.541+0000 7f2896e00640 1 -- 172.21.15.88:0/2204841973 wait complete. 2024-09-21T07:47:19.563 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:47:19.563 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:19.650 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:19.738 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:47:19.739 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:19.739 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:19.739 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:19.775 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:19.775 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:19.850 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.117+0000 7fcb81a00640 1 -- 172.21.15.88:0/488731816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d940 msgr2=0x7fcb7c15edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.117+0000 7fcb81a00640 1 --2- 172.21.15.88:0/488731816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d940 0x7fcb7c15edc0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c072b50 tx=0x7fcb6c02a560 comp rx=0 tx=0).stop 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.118+0000 7fcb81a00640 1 -- 172.21.15.88:0/488731816 shutdown_connections 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.118+0000 7fcb81a00640 1 --2- 172.21.15.88:0/488731816 >> v2:172.21.15.175:3300/0 conn(0x7fcb7c0d06a0 0x7fcb7c06cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.118+0000 7fcb81a00640 1 --2- 172.21.15.88:0/488731816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d940 0x7fcb7c15edc0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.118+0000 7fcb81a00640 1 -- 172.21.15.88:0/488731816 >> 172.21.15.88:0/488731816 conn(0x7fcb7c0b6880 msgr2=0x7fcb7c0b3060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:20.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.118+0000 7fcb81a00640 1 -- 172.21.15.88:0/488731816 shutdown_connections 2024-09-21T07:47:20.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.118+0000 7fcb81a00640 1 -- 172.21.15.88:0/488731816 wait complete. 2024-09-21T07:47:20.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.120+0000 7fcb81a00640 1 Processor -- start 2024-09-21T07:47:20.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.120+0000 7fcb81a00640 1 -- start start 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.121+0000 7fcb81a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 0x7fcb7c0cdad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.121+0000 7fcb81a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb7c06d940 0x7fcb7c0cb730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.121+0000 7fcb81a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb7c0d06a0 0x7fcb7c1f64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.121+0000 7fcb81a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c0ce490 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.121+0000 7fcb81a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb7c0b6190 con 0x7fcb7c06d940 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.121+0000 7fcb81a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb7c0ce610 con 0x7fcb7c0d06a0 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 0x7fcb7c0cdad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 0x7fcb7c0cdad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59766/0 (socket says 172.21.15.88:59766) 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 -- 172.21.15.88:0/3307207236 learned_addr learned my addr 172.21.15.88:0/3307207236 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7b400640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb7c0d06a0 0x7fcb7c1f64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7a000640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb7c06d940 0x7fcb7c0cb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 -- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb7c0d06a0 msgr2=0x7fcb7c1f64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb7c0d06a0 0x7fcb7c1f64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 -- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb7c06d940 msgr2=0x7fcb7c0cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb7c06d940 0x7fcb7c0cb730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 -- 172.21.15.88:0/3307207236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb7c0ccc40 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7a000640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb7c06d940 0x7fcb7c0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.122+0000 7fcb7aa00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 0x7fcb7c0cdad0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c034040 tx=0x7fcb6c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.123+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c03e030 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.123+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcb6c003e00 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.123+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb6c035040 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.123+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb7c0ccf20 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.123+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb7c1f5cf0 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.124+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb7c0ceb30 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.125+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fcb6c041030 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.127+0000 7fcb72a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcb5c078d70 0x7fcb5c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.127+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fcb6c0bb8d0 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.127+0000 7fcb7a000640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcb5c078d70 0x7fcb5c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.128+0000 7fcb7a000640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcb5c078d70 0x7fcb5c07b280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c1f37e0 tx=0x7fcb68005ef0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:20.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.131+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcb6c083290 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.336+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fcb7c15e990 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.337+0000 7fcb72a00640 1 -- 172.21.15.88:0/3307207236 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7fcb6c087cc0 con 0x7fcb7c06d2e0 2024-09-21T07:47:20.338 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:47:20.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcb5c078d70 msgr2=0x7fcb5c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcb5c078d70 0x7fcb5c07b280 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcb7c1f37e0 tx=0x7fcb68005ef0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 msgr2=0x7fcb7c0cdad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 0x7fcb7c0cdad0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fcb6c034040 tx=0x7fcb6c003910 comp rx=0 tx=0).stop 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 shutdown_connections 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcb5c078d70 0x7fcb5c07b280 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb7c0d06a0 0x7fcb7c1f64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb7c06d940 0x7fcb7c0cb730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 --2- 172.21.15.88:0/3307207236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb7c06d2e0 0x7fcb7c0cdad0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 >> 172.21.15.88:0/3307207236 conn(0x7fcb7c0b6880 msgr2=0x7fcb7c071710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 shutdown_connections 2024-09-21T07:47:20.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.339+0000 7fcb81a00640 1 -- 172.21.15.88:0/3307207236 wait complete. 2024-09-21T07:47:20.363 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:47:20.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:20.363 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:20.455 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:47:20.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:20.456 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:47:20.507 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:20.508 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:20.508 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:20.544 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:20.544 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:20.619 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:47:20.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 -- 172.21.15.88:0/3608499037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 msgr2=0x7f97680af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 --2- 172.21.15.88:0/3608499037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f97680af170 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f976806e790 tx=0x7f975802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 -- 172.21.15.88:0/3608499037 shutdown_connections 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 --2- 172.21.15.88:0/3608499037 >> v2:172.21.15.175:3300/0 conn(0x7f97680b2f60 0x7f976815a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 --2- 172.21.15.88:0/3608499037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f97680af170 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 -- 172.21.15.88:0/3608499037 >> 172.21.15.88:0/3608499037 conn(0x7f97680b1e40 msgr2=0x7f97680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 -- 172.21.15.88:0/3608499037 shutdown_connections 2024-09-21T07:47:20.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.838+0000 7f9767200640 1 -- 172.21.15.88:0/3608499037 wait complete. 2024-09-21T07:47:20.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.841+0000 7f9767200640 1 Processor -- start 2024-09-21T07:47:20.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.841+0000 7f9767200640 1 -- start start 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9767200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f9768145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9767200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97680b2f60 0x7f97680cf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9767200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976815d670 0x7f976815bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9767200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f976806f290 con 0x7f97680ad9c0 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9767200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9768070d50 con 0x7f97680b2f60 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9767200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f976806f410 con 0x7f976815d670 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9765e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f9768145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9765e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f9768145a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59780/0 (socket says 172.21.15.88:59780) 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.842+0000 7f9765e00640 1 -- 172.21.15.88:0/3766162677 learned_addr learned my addr 172.21.15.88:0/3766162677 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f975d400640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97680b2f60 0x7f97680cf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9766800640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976815d670 0x7f976815bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9765e00640 1 -- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976815d670 msgr2=0x7f976815bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9765e00640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976815d670 0x7f976815bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9765e00640 1 -- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97680b2f60 msgr2=0x7f97680cf180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9765e00640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97680b2f60 0x7f97680cf180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9765e00640 1 -- 172.21.15.88:0/3766162677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97680cdd60 con 0x7f97680ad9c0 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9766800640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976815d670 0x7f976815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f975d400640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97680b2f60 0x7f97680cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9765e00640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f9768145a40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9758009780 tx=0x7f9758033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758031d60 con 0x7f97680ad9c0 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9758033e10 con 0x7f97680ad9c0 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9768145f80 con 0x7f97680ad9c0 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.843+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9758005260 con 0x7f97680ad9c0 2024-09-21T07:47:20.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.844+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97680ce320 con 0x7f97680ad9c0 2024-09-21T07:47:20.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.845+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976806f8c0 con 0x7f97680ad9c0 2024-09-21T07:47:20.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.846+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f9758005400 con 0x7f97680ad9c0 2024-09-21T07:47:20.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.847+0000 7f975ea00640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9744078d70 0x7f974407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:20.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.847+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f97580bc2a0 con 0x7f97680ad9c0 2024-09-21T07:47:20.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.848+0000 7f975d400640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9744078d70 0x7f974407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:20.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.848+0000 7f975d400640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9744078d70 0x7f974407b280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f97680b40f0 tx=0x7f9750005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:20.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:20.852+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f975803f030 con 0x7f97680ad9c0 2024-09-21T07:47:21.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.049+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9768155780 con 0x7f97680ad9c0 2024-09-21T07:47:21.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.049+0000 7f975ea00640 1 -- 172.21.15.88:0/3766162677 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v35) ==== 80+0+0 (secure 0 0 0) 0x7f97580843d0 con 0x7f97680ad9c0 2024-09-21T07:47:21.051 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:47:21.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9744078d70 msgr2=0x7f974407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9744078d70 0x7f974407b280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f97680b40f0 tx=0x7f9750005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:21.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 msgr2=0x7f9768145a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f9768145a40 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f9758009780 tx=0x7f9758033c20 comp rx=0 tx=0).stop 2024-09-21T07:47:21.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 shutdown_connections 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9744078d70 0x7f974407b280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976815d670 0x7f976815bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97680b2f60 0x7f97680cf180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 --2- 172.21.15.88:0/3766162677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97680ad9c0 0x7f9768145a40 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 >> 172.21.15.88:0/3766162677 conn(0x7f97680b1e40 msgr2=0x7f9768153450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 shutdown_connections 2024-09-21T07:47:21.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.052+0000 7f9767200640 1 -- 172.21.15.88:0/3766162677 wait complete. 2024-09-21T07:47:21.076 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:47:21.076 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:21.076 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:21.141 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:47:21.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:21.141 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:47:21.177 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:21.177 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:21.177 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:21.212 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:21.212 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:47:21.212 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:21.212 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:21.254 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:21.254 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 -- 172.21.15.88:0/2555864414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5280ad0e0 msgr2=0x7fd5280ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 --2- 172.21.15.88:0/2555864414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5280ad0e0 0x7fd5280ad4f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fd510009860 tx=0x7fd51002a510 comp rx=0 tx=0).stop 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 -- 172.21.15.88:0/2555864414 shutdown_connections 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 --2- 172.21.15.88:0/2555864414 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5280ad0e0 0x7fd5280ad4f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 -- 172.21.15.88:0/2555864414 >> 172.21.15.88:0/2555864414 conn(0x7fd52806d730 msgr2=0x7fd5280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 -- 172.21.15.88:0/2555864414 shutdown_connections 2024-09-21T07:47:21.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.473+0000 7fd52fc00640 1 -- 172.21.15.88:0/2555864414 wait complete. 2024-09-21T07:47:21.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.476+0000 7fd52fc00640 1 Processor -- start 2024-09-21T07:47:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.476+0000 7fd52fc00640 1 -- start start 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52fc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 0x7fd528153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52fc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5280b3550 0x7fd5281f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52fc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52815d760 0x7fd528159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52fc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b4da0 con 0x7fd5280ad0e0 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52fc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5280b4c20 con 0x7fd5280b3550 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52fc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5280b4070 con 0x7fd52815d760 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 0x7fd528153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 0x7fd528153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59796/0 (socket says 172.21.15.88:59796) 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.477+0000 7fd52d200640 1 -- 172.21.15.88:0/3111307138 learned_addr learned my addr 172.21.15.88:0/3111307138 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd51fe00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5280b3550 0x7fd5281f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52dc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52815d760 0x7fd528159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52d200640 1 -- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52815d760 msgr2=0x7fd528159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52d200640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52815d760 0x7fd528159b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52d200640 1 -- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5280b3550 msgr2=0x7fd5281f3900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52d200640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5280b3550 0x7fd5281f3900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52d200640 1 -- 172.21.15.88:0/3111307138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd528158af0 con 0x7fd5280ad0e0 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52dc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52815d760 0x7fd528159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52d200640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 0x7fd528153e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c760 tx=0x7fd51800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd518016020 con 0x7fd5280ad0e0 2024-09-21T07:47:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd51800a950 con 0x7fd5280ad0e0 2024-09-21T07:47:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd51800ada0 con 0x7fd5280ad0e0 2024-09-21T07:47:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.478+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd528158d90 con 0x7fd5280ad0e0 2024-09-21T07:47:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.479+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd528159350 con 0x7fd5280ad0e0 2024-09-21T07:47:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.479+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5280ad4f0 con 0x7fd5280ad0e0 2024-09-21T07:47:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.481+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fd518019020 con 0x7fd5280ad0e0 2024-09-21T07:47:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.481+0000 7fd525600640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd504078d70 0x7fd50407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.482+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fd51809b070 con 0x7fd5280ad0e0 2024-09-21T07:47:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.482+0000 7fd51fe00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd504078d70 0x7fd50407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.483+0000 7fd51fe00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd504078d70 0x7fd50407b280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd510034000 tx=0x7fd510004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.486+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd518020020 con 0x7fd5280ad0e0 2024-09-21T07:47:21.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.707+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd5280b4980 con 0x7fd5280ad0e0 2024-09-21T07:47:21.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.707+0000 7fd525600640 1 -- 172.21.15.88:0/3111307138 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fd5280ada30 con 0x7fd5280ad0e0 2024-09-21T07:47:21.709 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:21.709 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"btime":"2024-09-21T07:47:11:472129+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:55.682651+0000","modified":"2024-09-21T07:46:58.709003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1785638532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1785638532},{"type":"v1","addr":"172.21.15.88:6835","nonce":1785638532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4899,"qdb_cluster":[4899]},"id":2}]} 2024-09-21T07:47:21.709 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2024-09-21T07:47:21.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd504078d70 msgr2=0x7fd50407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd504078d70 0x7fd50407b280 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd510034000 tx=0x7fd510004730 comp rx=0 tx=0).stop 2024-09-21T07:47:21.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 msgr2=0x7fd528153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 0x7fd528153e40 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd51800c760 tx=0x7fd51800cc20 comp rx=0 tx=0).stop 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 shutdown_connections 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd504078d70 0x7fd50407b280 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd52815d760 0x7fd528159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5280b3550 0x7fd5281f3900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 --2- 172.21.15.88:0/3111307138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5280ad0e0 0x7fd528153e40 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 >> 172.21.15.88:0/3111307138 conn(0x7fd52806d730 msgr2=0x7fd5281535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 shutdown_connections 2024-09-21T07:47:21.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.710+0000 7fd52fc00640 1 -- 172.21.15.88:0/3111307138 wait complete. 2024-09-21T07:47:21.733 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 -- 172.21.15.88:0/50819126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa988005250 msgr2=0x7fa98800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/50819126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa988005250 0x7fa98800b9c0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fa99c001dc0 tx=0x7fa988005830 comp rx=0 tx=0).stop 2024-09-21T07:47:21.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 -- 172.21.15.88:0/50819126 shutdown_connections 2024-09-21T07:47:21.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/50819126 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa988005250 0x7fa98800b9c0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 -- 172.21.15.88:0/50819126 >> 172.21.15.88:0/50819126 conn(0x7fa9a40b1e40 msgr2=0x7fa9a40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:21.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 -- 172.21.15.88:0/50819126 shutdown_connections 2024-09-21T07:47:21.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.953+0000 7fa9a9e00640 1 -- 172.21.15.88:0/50819126 wait complete. 2024-09-21T07:47:21.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.955+0000 7fa9a9e00640 1 Processor -- start 2024-09-21T07:47:21.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.955+0000 7fa9a9e00640 1 -- start start 2024-09-21T07:47:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 0x7fa9a409c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a414ed00 0x7fa9a41ee2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a41f4420 0x7fa9a41f4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a4072e60 con 0x7fa988005250 2024-09-21T07:47:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9a40702c0 con 0x7fa9a41f4420 2024-09-21T07:47:21.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9a4072fe0 con 0x7fa9a414ed00 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 0x7fa9a409c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 0x7fa9a409c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59810/0 (socket says 172.21.15.88:59810) 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a8a00640 1 -- 172.21.15.88:0/2736865026 learned_addr learned my addr 172.21.15.88:0/2736865026 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa9a9400640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a41f4420 0x7fa9a41f4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.957+0000 7fa99be00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a414ed00 0x7fa9a41ee2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a8a00640 1 -- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a414ed00 msgr2=0x7fa9a41ee2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a8a00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a414ed00 0x7fa9a41ee2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a8a00640 1 -- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a41f4420 msgr2=0x7fa9a41f4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a8a00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a41f4420 0x7fa9a41f4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a8a00640 1 -- 172.21.15.88:0/2736865026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9a41f4d70 con 0x7fa988005250 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a8a00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 0x7fa9a409c100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9940066b0 tx=0x7fa9940039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994011700 con 0x7fa988005250 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa994016020 con 0x7fa988005250 2024-09-21T07:47:21.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9a413b520 con 0x7fa988005250 2024-09-21T07:47:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa994011b60 con 0x7fa988005250 2024-09-21T07:47:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.958+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9a4149e50 con 0x7fa988005250 2024-09-21T07:47:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.959+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9a4149960 con 0x7fa988005250 2024-09-21T07:47:21.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.960+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa994011d00 con 0x7fa988005250 2024-09-21T07:47:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.961+0000 7fa9a1600640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa980078d40 0x7fa98007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:21.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.961+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa99409bd30 con 0x7fa988005250 2024-09-21T07:47:21.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.962+0000 7fa99be00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa980078d40 0x7fa98007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:21.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.962+0000 7fa99be00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa980078d40 0x7fa98007b250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4074ad0 tx=0x7fa98c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:21.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:21.966+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa994020020 con 0x7fa988005250 2024-09-21T07:47:22.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.194+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa9a4073540 con 0x7fa988005250 2024-09-21T07:47:22.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.195+0000 7fa9a1600640 1 -- 172.21.15.88:0/2736865026 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7fa9a41f4d70 con 0x7fa988005250 2024-09-21T07:47:22.196 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:22.196 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"btime":"2024-09-21T07:47:11:472129+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:55.682651+0000","modified":"2024-09-21T07:46:58.709003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1785638532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1785638532},{"type":"v1","addr":"172.21.15.88:6835","nonce":1785638532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4899,"qdb_cluster":[4899]},"id":2}]} 2024-09-21T07:47:22.197 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.197+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa980078d40 msgr2=0x7fa98007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.197+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa980078d40 0x7fa98007b250 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4074ad0 tx=0x7fa98c005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.197+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 msgr2=0x7fa9a409c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.197+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 0x7fa9a409c100 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fa9940066b0 tx=0x7fa9940039c0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 shutdown_connections 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa980078d40 0x7fa98007b250 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9a414ed00 0x7fa9a41ee2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9a41f4420 0x7fa9a41f4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 --2- 172.21.15.88:0/2736865026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa988005250 0x7fa9a409c100 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 >> 172.21.15.88:0/2736865026 conn(0x7fa9a40b1e40 msgr2=0x7fa9a415e4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:22.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 shutdown_connections 2024-09-21T07:47:22.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.198+0000 7fa9a9e00640 1 -- 172.21.15.88:0/2736865026 wait complete. 2024-09-21T07:47:22.224 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.445+0000 7fdc58c00640 1 -- 172.21.15.88:0/1608289113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc38003c10 msgr2=0x7fdc38004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.445+0000 7fdc58c00640 1 --2- 172.21.15.88:0/1608289113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc38003c10 0x7fdc38004060 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fdc44009b50 tx=0x7fdc38005830 comp rx=0 tx=0).stop 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.445+0000 7fdc52800640 1 -- 172.21.15.88:0/1608289113 reap_dead start 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.446+0000 7fdc58c00640 1 -- 172.21.15.88:0/1608289113 shutdown_connections 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.446+0000 7fdc58c00640 1 -- 172.21.15.88:0/1608289113 >> 172.21.15.88:0/1608289113 conn(0x7fdc5406d730 msgr2=0x7fdc540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.446+0000 7fdc58c00640 1 -- 172.21.15.88:0/1608289113 shutdown_connections 2024-09-21T07:47:22.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.446+0000 7fdc58c00640 1 -- 172.21.15.88:0/1608289113 wait complete. 2024-09-21T07:47:22.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.448+0000 7fdc58c00640 1 Processor -- start 2024-09-21T07:47:22.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.448+0000 7fdc58c00640 1 -- start start 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.449+0000 7fdc58c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc38003c10 0x7fdc54151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.449+0000 7fdc58c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc54074510 0x7fdc54074920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc58c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 0x7fdc540b10d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc58c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540b3fe0 con 0x7fdc540b0cc0 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc58c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc540b3e60 con 0x7fdc54074510 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc58c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc540b4160 con 0x7fdc38003c10 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 0x7fdc540b10d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 0x7fdc540b10d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59816/0 (socket says 172.21.15.88:59816) 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 -- 172.21.15.88:0/165462261 learned_addr learned my addr 172.21.15.88:0/165462261 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc51e00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc38003c10 0x7fdc54151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc51400640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc54074510 0x7fdc54074920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 -- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc38003c10 msgr2=0x7fdc54151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc38003c10 0x7fdc54151b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 -- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc54074510 msgr2=0x7fdc54074920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc54074510 0x7fdc54074920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.450+0000 7fdc52800640 1 -- 172.21.15.88:0/165462261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc540b1610 con 0x7fdc540b0cc0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.451+0000 7fdc52800640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 0x7fdc540b10d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc54072e10 tx=0x7fdc4c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.451+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c011900 con 0x7fdc540b0cc0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.451+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc4c017070 con 0x7fdc540b0cc0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.451+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c011d40 con 0x7fdc540b0cc0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.451+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc54073320 con 0x7fdc540b0cc0 2024-09-21T07:47:22.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.451+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc540738e0 con 0x7fdc540b0cc0 2024-09-21T07:47:22.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.452+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc540b46f0 con 0x7fdc540b0cc0 2024-09-21T07:47:22.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.453+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fdc4c01c460 con 0x7fdc540b0cc0 2024-09-21T07:47:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.454+0000 7fdc4a000640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdc34078d40 0x7fdc3407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.454+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fdc4c09bac0 con 0x7fdc540b0cc0 2024-09-21T07:47:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.454+0000 7fdc51e00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdc34078d40 0x7fdc3407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.455+0000 7fdc51e00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdc34078d40 0x7fdc3407b250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc44035000 tx=0x7fdc4402d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.459+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc540b1610 con 0x7fdc540b0cc0 2024-09-21T07:47:22.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.647+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdc54073090 con 0x7fdc540b0cc0 2024-09-21T07:47:22.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.648+0000 7fdc4a000640 1 -- 172.21.15.88:0/165462261 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7fdc4c063480 con 0x7fdc540b0cc0 2024-09-21T07:47:22.650 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:22.650 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":35,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:57.207741+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:53.697978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:54.649546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:22.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdc34078d40 msgr2=0x7fdc3407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdc34078d40 0x7fdc3407b250 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fdc44035000 tx=0x7fdc4402d040 comp rx=0 tx=0).stop 2024-09-21T07:47:22.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 msgr2=0x7fdc540b10d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 0x7fdc540b10d0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fdc54072e10 tx=0x7fdc4c006430 comp rx=0 tx=0).stop 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 shutdown_connections 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdc34078d40 0x7fdc3407b250 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc540b0cc0 0x7fdc540b10d0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc54074510 0x7fdc54074920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 --2- 172.21.15.88:0/165462261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc38003c10 0x7fdc54151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 >> 172.21.15.88:0/165462261 conn(0x7fdc5406d730 msgr2=0x7fdc5415b500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 shutdown_connections 2024-09-21T07:47:22.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.651+0000 7fdc58c00640 1 -- 172.21.15.88:0/165462261 wait complete. 2024-09-21T07:47:22.674 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:47:22.674 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 -- 172.21.15.88:0/342672857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9834152920 msgr2=0x7f983415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 --2- 172.21.15.88:0/342672857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9834152920 0x7f983415f3d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f9820009830 tx=0x7f982002a480 comp rx=0 tx=0).stop 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 -- 172.21.15.88:0/342672857 shutdown_connections 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 --2- 172.21.15.88:0/342672857 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f983415f910 0x7f98340b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 --2- 172.21.15.88:0/342672857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9834152920 0x7f983415f3d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 -- 172.21.15.88:0/342672857 >> 172.21.15.88:0/342672857 conn(0x7f983406d730 msgr2=0x7f98340b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 -- 172.21.15.88:0/342672857 shutdown_connections 2024-09-21T07:47:22.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.895+0000 7f9839600640 1 -- 172.21.15.88:0/342672857 wait complete. 2024-09-21T07:47:22.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.898+0000 7f9839600640 1 Processor -- start 2024-09-21T07:47:22.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.898+0000 7f9839600640 1 -- start start 2024-09-21T07:47:22.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.899+0000 7f9839600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98340aebc0 0x7f98341f6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.899+0000 7f9839600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9834152920 0x7f98341402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.899+0000 7f9839600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 0x7f98341e9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.899+0000 7f9839600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98340b4190 con 0x7f983415f910 2024-09-21T07:47:22.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.899+0000 7f9839600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98340b4010 con 0x7f9834152920 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.899+0000 7f9839600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98340b4310 con 0x7f98340aebc0 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 0x7f98341e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9832a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98340aebc0 0x7f98341f6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9832000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9834152920 0x7f98341402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 0x7f98341e9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59820/0 (socket says 172.21.15.88:59820) 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 -- 172.21.15.88:0/2072159215 learned_addr learned my addr 172.21.15.88:0/2072159215 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9832a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98340aebc0 0x7f98341f6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:51804/0 (socket says 172.21.15.88:51804) 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 -- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98340aebc0 msgr2=0x7f98341f6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98340aebc0 0x7f98341f6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 -- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9834152920 msgr2=0x7f98341402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9834152920 0x7f98341402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:22.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 -- 172.21.15.88:0/2072159215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9834142f00 con 0x7f983415f910 2024-09-21T07:47:22.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.900+0000 7f9833400640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 0x7f98341e9620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f982c00ea80 tx=0x7f982c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:22.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.901+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c004b20 con 0x7f983415f910 2024-09-21T07:47:22.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.901+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f982c004ce0 con 0x7f983415f910 2024-09-21T07:47:22.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.901+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98341431e0 con 0x7f983415f910 2024-09-21T07:47:22.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.901+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f982c005280 con 0x7f983415f910 2024-09-21T07:47:22.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.901+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98341437a0 con 0x7f983415f910 2024-09-21T07:47:22.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.902+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98340b48a0 con 0x7f983415f910 2024-09-21T07:47:22.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.903+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f982c005420 con 0x7f983415f910 2024-09-21T07:47:22.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.905+0000 7f982aa00640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9814078d40 0x7f981407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:22.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.905+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f982c09bc40 con 0x7f983415f910 2024-09-21T07:47:22.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.905+0000 7f9832a00640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9814078d40 0x7f981407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:22.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.906+0000 7f9832a00640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9814078d40 0x7f981407b250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9824001720 tx=0x7f9824007e90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:22.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:22.910+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f982c063730 con 0x7f983415f910 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.132+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9834142c90 con 0x7f983415f910 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.132+0000 7f982aa00640 1 -- 172.21.15.88:0/2072159215 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7f982c068160 con 0x7f983415f910 2024-09-21T07:47:23.133 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"btime":"2024-09-21T07:47:11:472129+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:55.682651+0000","modified":"2024-09-21T07:46:58.709003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1785638532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1785638532},{"type":"v1","addr":"172.21.15.88:6835","nonce":1785638532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4899,"qdb_cluster":[4899]},"id":2}]} 2024-09-21T07:47:23.134 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9814078d40 msgr2=0x7f981407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9814078d40 0x7f981407b250 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9824001720 tx=0x7f9824007e90 comp rx=0 tx=0).stop 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 msgr2=0x7f98341e9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 0x7f98341e9620 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f982c00ea80 tx=0x7f982c00ef40 comp rx=0 tx=0).stop 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 shutdown_connections 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9814078d40 0x7f981407b250 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f983415f910 0x7f98341e9620 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9834152920 0x7f98341402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 --2- 172.21.15.88:0/2072159215 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98340aebc0 0x7f98341f6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 >> 172.21.15.88:0/2072159215 conn(0x7f983406d730 msgr2=0x7f983406f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 shutdown_connections 2024-09-21T07:47:23.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.135+0000 7f9839600640 1 -- 172.21.15.88:0/2072159215 wait complete. 2024-09-21T07:47:23.160 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 -- 172.21.15.88:0/3252520848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c815f910 msgr2=0x7f96c815bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3252520848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c815f910 0x7f96c815bd90 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f96c8071320 tx=0x7f96c002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 -- 172.21.15.88:0/3252520848 shutdown_connections 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3252520848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c815f910 0x7f96c815bd90 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3252520848 >> v2:172.21.15.175:3300/0 conn(0x7f96c80b2750 0x7f96c815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3252520848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96c80aebc0 0x7f96c8152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 -- 172.21.15.88:0/3252520848 >> 172.21.15.88:0/3252520848 conn(0x7f96c806d730 msgr2=0x7f96c80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 -- 172.21.15.88:0/3252520848 shutdown_connections 2024-09-21T07:47:23.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.379+0000 7f96cfa00640 1 -- 172.21.15.88:0/3252520848 wait complete. 2024-09-21T07:47:23.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.382+0000 7f96cfa00640 1 Processor -- start 2024-09-21T07:47:23.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.382+0000 7f96cfa00640 1 -- start start 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96cfa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96c80aebc0 0x7f96c80cb1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96cfa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 0x7f96c81f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96cfa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c815f910 0x7f96c80b12f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96cfa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f96c815c660 con 0x7f96c80b2750 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96cfa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f96c815c4e0 con 0x7f96c815f910 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96cfa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f96c815c7e0 con 0x7f96c80aebc0 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.383+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 0x7f96c81f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 0x7f96c81f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59832/0 (socket says 172.21.15.88:59832) 2024-09-21T07:47:23.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 -- 172.21.15.88:0/3615024607 learned_addr learned my addr 172.21.15.88:0/3615024607 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96cd000640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96c80aebc0 0x7f96c80cb1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96cda00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c815f910 0x7f96c80b12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 -- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96c80aebc0 msgr2=0x7f96c80cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96c80aebc0 0x7f96c80cb1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 -- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c815f910 msgr2=0x7f96c80b12f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c815f910 0x7f96c80b12f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 -- 172.21.15.88:0/3615024607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96c80c94c0 con 0x7f96c80b2750 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.384+0000 7f96bfe00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 0x7f96c81f6670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f96b000d770 tx=0x7f96b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.385+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b0017070 con 0x7f96c80b2750 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.385+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f96b0009d20 con 0x7f96c80b2750 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.385+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96b00106c0 con 0x7f96c80b2750 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.385+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96c80c97a0 con 0x7f96c80b2750 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.385+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96c80c9d60 con 0x7f96c80b2750 2024-09-21T07:47:23.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.386+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96c80cb830 con 0x7f96c80b2750 2024-09-21T07:47:23.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.387+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f96b001a070 con 0x7f96c80b2750 2024-09-21T07:47:23.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.388+0000 7f96c5600640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f96a4078d40 0x7f96a407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.388+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f96b009b1e0 con 0x7f96c80b2750 2024-09-21T07:47:23.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.388+0000 7f96cd000640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f96a4078d40 0x7f96a407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.389+0000 7f96cd000640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f96a4078d40 0x7f96a407b250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005fd0 tx=0x7f96b800b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.392+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f96b0062ce0 con 0x7f96c80b2750 2024-09-21T07:47:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.615+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f96c815f3d0 con 0x7f96c80b2750 2024-09-21T07:47:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.615+0000 7f96c5600640 1 -- 172.21.15.88:0/3615024607 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) ==== 94+0+5229 (secure 0 0 0) 0x7f96b0067710 con 0x7f96c80b2750 2024-09-21T07:47:23.617 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:23.617 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":18,"btime":"2024-09-21T07:47:11:472129+0000","default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:46:55.682651+0000","modified":"2024-09-21T07:46:58.709003+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":4899},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4899":{"gid":4899,"name":"c","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1785638532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1785638532},{"type":"v1","addr":"172.21.15.88:6835","nonce":1785638532}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":4899,"qdb_cluster":[4899]},"id":2}]} 2024-09-21T07:47:23.617 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 18 2024-09-21T07:47:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f96a4078d40 msgr2=0x7f96a407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f96a4078d40 0x7f96a407b250 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f96b8005fd0 tx=0x7f96b800b260 comp rx=0 tx=0).stop 2024-09-21T07:47:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 msgr2=0x7f96c81f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 0x7f96c81f6670 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f96b000d770 tx=0x7f96b000dc30 comp rx=0 tx=0).stop 2024-09-21T07:47:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 shutdown_connections 2024-09-21T07:47:23.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f96a4078d40 0x7f96a407b250 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f96c815f910 0x7f96c80b12f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f96c80b2750 0x7f96c81f6670 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 --2- 172.21.15.88:0/3615024607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f96c80aebc0 0x7f96c80cb1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 >> 172.21.15.88:0/3615024607 conn(0x7f96c806d730 msgr2=0x7f96c80b4550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 shutdown_connections 2024-09-21T07:47:23.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.618+0000 7f96cfa00640 1 -- 172.21.15.88:0/3615024607 wait complete. 2024-09-21T07:47:23.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:23.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 -- 172.21.15.88:0/1757897258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec80ad0e0 msgr2=0x7f5ec80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1757897258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec80ad0e0 0x7f5ec80ad4f0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f5ec80b3100 tx=0x7f5eb402a560 comp rx=0 tx=0).stop 2024-09-21T07:47:23.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 -- 172.21.15.88:0/1757897258 shutdown_connections 2024-09-21T07:47:23.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1757897258 >> v2:172.21.15.175:3300/0 conn(0x7f5ec80b3550 0x7f5ec80b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1757897258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec80ad0e0 0x7f5ec80ad4f0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 -- 172.21.15.88:0/1757897258 >> 172.21.15.88:0/1757897258 conn(0x7f5ec806d730 msgr2=0x7f5ec80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:23.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 -- 172.21.15.88:0/1757897258 shutdown_connections 2024-09-21T07:47:23.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.861+0000 7f5ecd800640 1 -- 172.21.15.88:0/1757897258 wait complete. 2024-09-21T07:47:23.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.863+0000 7f5ecd800640 1 Processor -- start 2024-09-21T07:47:23.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.863+0000 7f5ecd800640 1 -- start start 2024-09-21T07:47:23.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ecd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ec80ad0e0 0x7f5ec8073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ecd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ec80b3550 0x7f5ec81f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ecd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 0x7f5ec81500b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ecd800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80b4da0 con 0x7f5ec815d760 2024-09-21T07:47:23.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ecd800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ec80b4c20 con 0x7f5ec80ad0e0 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ecd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ec80b3ea0 con 0x7f5ec80b3550 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 0x7f5ec81500b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 0x7f5ec81500b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59850/0 (socket says 172.21.15.88:59850) 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec7400640 1 -- 172.21.15.88:0/1085863965 learned_addr learned my addr 172.21.15.88:0/1085863965 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec6a00640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ec80ad0e0 0x7f5ec8073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec6000640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ec80b3550 0x7f5ec81f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec7400640 1 -- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ec80b3550 msgr2=0x7f5ec81f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.865+0000 7f5ec7400640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ec80b3550 0x7f5ec81f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ec7400640 1 -- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ec80ad0e0 msgr2=0x7f5ec8073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ec7400640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ec80ad0e0 0x7f5ec8073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ec7400640 1 -- 172.21.15.88:0/1085863965 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ec81f9b10 con 0x7f5ec815d760 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ec7400640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 0x7f5ec81500b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000e9c0 tx=0x7f5ec000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0004d20 con 0x7f5ec815d760 2024-09-21T07:47:23.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ec000cb20 con 0x7f5ec815d760 2024-09-21T07:47:23.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec0005020 con 0x7f5ec815d760 2024-09-21T07:47:23.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ec814d8a0 con 0x7f5ec815d760 2024-09-21T07:47:23.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.866+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ec81f9db0 con 0x7f5ec815d760 2024-09-21T07:47:23.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.867+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ec81f9ff0 con 0x7f5ec815d760 2024-09-21T07:47:23.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.868+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f5ec00051c0 con 0x7f5ec815d760 2024-09-21T07:47:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.869+0000 7f5ebea00640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ea8078d70 0x7f5ea807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:23.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.869+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7f5ec009ba90 con 0x7f5ec815d760 2024-09-21T07:47:23.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.870+0000 7f5ec6a00640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ea8078d70 0x7f5ea807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:23.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.871+0000 7f5ec6a00640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ea8078d70 0x7f5ea807b280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80015e0 tx=0x7f5eb8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:23.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:23.874+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ec81f9ff0 con 0x7f5ec815d760 2024-09-21T07:47:24.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.068+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5ec80b4360 con 0x7f5ec815d760 2024-09-21T07:47:24.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.069+0000 7f5ebea00640 1 -- 172.21.15.88:0/1085863965 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) ==== 74+0+16681 (secure 0 0 0) 0x7f5ec0016030 con 0x7f5ec815d760 2024-09-21T07:47:24.071 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:24.071 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":35,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:46:57.207741+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:46:53.697978+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2024-09-21T07:46:54.649546+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.071+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ea8078d70 msgr2=0x7f5ea807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.071+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ea8078d70 0x7f5ea807b280 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5eb80015e0 tx=0x7f5eb8009040 comp rx=0 tx=0).stop 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.071+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 msgr2=0x7f5ec81500b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.071+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 0x7f5ec81500b0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f5ec000e9c0 tx=0x7f5ec000ee80 comp rx=0 tx=0).stop 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 shutdown_connections 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ea8078d70 0x7f5ea807b280 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ec80b3550 0x7f5ec81f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec815d760 0x7f5ec81500b0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 --2- 172.21.15.88:0/1085863965 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ec80ad0e0 0x7f5ec8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 >> 172.21.15.88:0/1085863965 conn(0x7f5ec806d730 msgr2=0x7f5ec81535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 shutdown_connections 2024-09-21T07:47:24.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.072+0000 7f5ecd800640 1 -- 172.21.15.88:0/1085863965 wait complete. 2024-09-21T07:47:24.097 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:47:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 -- 172.21.15.88:0/2738943691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9b4003a50 msgr2=0x7fa9b4003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/2738943691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9b4003a50 0x7fa9b4003ec0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa9d00cf0d0 tx=0x7fa9b4005830 comp rx=0 tx=0).stop 2024-09-21T07:47:24.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 -- 172.21.15.88:0/2738943691 shutdown_connections 2024-09-21T07:47:24.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/2738943691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9b4003a50 0x7fa9b4003ec0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 -- 172.21.15.88:0/2738943691 >> 172.21.15.88:0/2738943691 conn(0x7fa9d006d3d0 msgr2=0x7fa9d00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 -- 172.21.15.88:0/2738943691 shutdown_connections 2024-09-21T07:47:24.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.318+0000 7fa9d4e00640 1 -- 172.21.15.88:0/2738943691 wait complete. 2024-09-21T07:47:24.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.320+0000 7fa9d4e00640 1 Processor -- start 2024-09-21T07:47:24.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.320+0000 7fa9d4e00640 1 -- start start 2024-09-21T07:47:24.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9d4e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b4003a50 0x7fa9d0074060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9d4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 0x7fa9d01f3e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9d4e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01551e0 0x7fa9d01e78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9d4e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d015ce30 con 0x7fa9d0074e40 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9d4e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9d0157210 con 0x7fa9d01551e0 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9d4e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9d015cfb0 con 0x7fa9b4003a50 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 0x7fa9d01f3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 0x7fa9d01f3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59866/0 (socket says 172.21.15.88:59866) 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 -- 172.21.15.88:0/3962735613 learned_addr learned my addr 172.21.15.88:0/3962735613 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9cde00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b4003a50 0x7fa9d0074060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9ce800640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01551e0 0x7fa9d01e78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 -- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b4003a50 msgr2=0x7fa9d0074060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b4003a50 0x7fa9d0074060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 -- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01551e0 msgr2=0x7fa9d01e78e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01551e0 0x7fa9d01e78e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9c5400640 1 -- 172.21.15.88:0/3962735613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9d0072fc0 con 0x7fa9d0074e40 2024-09-21T07:47:24.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.322+0000 7fa9ce800640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01551e0 0x7fa9d01e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:24.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.323+0000 7fa9c5400640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 0x7fa9d01f3e70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8004d90 tx=0x7fa9b8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.323+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8001d40 con 0x7fa9d0074e40 2024-09-21T07:47:24.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.323+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa9b8016070 con 0x7fa9d0074e40 2024-09-21T07:47:24.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.323+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9b8010b60 con 0x7fa9d0074e40 2024-09-21T07:47:24.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.323+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9d00732a0 con 0x7fa9d0074e40 2024-09-21T07:47:24.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.323+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9d0073860 con 0x7fa9d0074e40 2024-09-21T07:47:24.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.324+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9d00afa70 con 0x7fa9d0074e40 2024-09-21T07:47:24.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.325+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa9b8010d00 con 0x7fa9d0074e40 2024-09-21T07:47:24.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.326+0000 7fa9c6a00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa9ac078d40 0x7fa9ac07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.326+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(35..35 src has 1..35) ==== 5709+0+0 (secure 0 0 0) 0x7fa9b809b870 con 0x7fa9d0074e40 2024-09-21T07:47:24.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.327+0000 7fa9cde00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa9ac078d40 0x7fa9ac07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.327+0000 7fa9cde00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa9ac078d40 0x7fa9ac07b250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0033000 tx=0x7fa9c002d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.331+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa9b8063910 con 0x7fa9d0074e40 2024-09-21T07:47:24.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.555+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fa9d015aeb0 con 0x7fa9d0074e40 2024-09-21T07:47:24.557 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:47:24.556+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:47:24.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.558+0000 7fa9c6a00640 1 -- 172.21.15.88:0/3962735613 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v19) ==== 161+0+0 (secure 0 0 0) 0x7fa9b8063210 con 0x7fa9d0074e40 2024-09-21T07:47:24.559 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa9ac078d40 msgr2=0x7fa9ac07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa9ac078d40 0x7fa9ac07b250 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa9c0033000 tx=0x7fa9c002d000 comp rx=0 tx=0).stop 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 msgr2=0x7fa9d01f3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 0x7fa9d01f3e70 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa9b8004d90 tx=0x7fa9b8001630 comp rx=0 tx=0).stop 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 shutdown_connections 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa9ac078d40 0x7fa9ac07b250 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9d01551e0 0x7fa9d01e78e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9d0074e40 0x7fa9d01f3e70 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 --2- 172.21.15.88:0/3962735613 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9b4003a50 0x7fa9d0074060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 >> 172.21.15.88:0/3962735613 conn(0x7fa9d006d3d0 msgr2=0x7fa9d015f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.561+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 shutdown_connections 2024-09-21T07:47:24.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.562+0000 7fa9d4e00640 1 -- 172.21.15.88:0/3962735613 wait complete. 2024-09-21T07:47:24.590 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:47:24.702 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 -- 172.21.15.88:0/630805890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c156ca0 msgr2=0x7f9a5c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 --2- 172.21.15.88:0/630805890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c156ca0 0x7f9a5c0ccac0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009830 tx=0x7f9a4802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 -- 172.21.15.88:0/630805890 shutdown_connections 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 --2- 172.21.15.88:0/630805890 >> v2:172.21.15.175:3300/0 conn(0x7f9a5c0cd000 0x7f9a5c0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 --2- 172.21.15.88:0/630805890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c156ca0 0x7f9a5c0ccac0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 -- 172.21.15.88:0/630805890 >> 172.21.15.88:0/630805890 conn(0x7f9a5c06d620 msgr2=0x7f9a5c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 -- 172.21.15.88:0/630805890 shutdown_connections 2024-09-21T07:47:24.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.818+0000 7f9a5b200640 1 -- 172.21.15.88:0/630805890 wait complete. 2024-09-21T07:47:24.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.820+0000 7f9a5b200640 1 Processor -- start 2024-09-21T07:47:24.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.821+0000 7f9a5b200640 1 -- start start 2024-09-21T07:47:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c0b3f30 0x7f9a5c15bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 0x7f9a5c1f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a5b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a5c156ca0 0x7f9a5c1f60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a5b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c0af9a0 con 0x7f9a5c0cd000 2024-09-21T07:47:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a5b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c0af820 con 0x7f9a5c156ca0 2024-09-21T07:47:24.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a5b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5c0afb20 con 0x7f9a5c0b3f30 2024-09-21T07:47:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 0x7f9a5c1f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 0x7f9a5c1f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59902/0 (socket says 172.21.15.88:59902) 2024-09-21T07:47:24.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.822+0000 7f9a59400640 1 -- 172.21.15.88:0/2528662382 learned_addr learned my addr 172.21.15.88:0/2528662382 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a5a800640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a5c156ca0 0x7f9a5c1f60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59e00640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c0b3f30 0x7f9a5c15bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59400640 1 -- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c0b3f30 msgr2=0x7f9a5c15bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59400640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c0b3f30 0x7f9a5c15bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59400640 1 -- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a5c156ca0 msgr2=0x7f9a5c1f60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59400640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a5c156ca0 0x7f9a5c1f60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59400640 1 -- 172.21.15.88:0/2528662382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c15a2f0 con 0x7f9a5c0cd000 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a5a800640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a5c156ca0 0x7f9a5c1f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:24.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59e00640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c0b3f30 0x7f9a5c15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a59400640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 0x7f9a5c1f6970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009950 tx=0x7f9a48003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.823+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48005c90 con 0x7f9a5c0cd000 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.824+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a48005020 con 0x7f9a5c0cd000 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.824+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c15a5d0 con 0x7f9a5c0cd000 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.824+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a48005450 con 0x7f9a5c0cd000 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.824+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c1f5ba0 con 0x7f9a5c0cd000 2024-09-21T07:47:24.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.825+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5c0af0d0 con 0x7f9a5c0cd000 2024-09-21T07:47:24.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.826+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f9a480317c0 con 0x7f9a5c0cd000 2024-09-21T07:47:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.827+0000 7f9a52000640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a3c078d70 0x7f9a3c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:24.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.827+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f9a480bc350 con 0x7f9a5c0cd000 2024-09-21T07:47:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.828+0000 7f9a59e00640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a3c078d70 0x7f9a3c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:24.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.828+0000 7f9a59e00640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a3c078d70 0x7f9a3c07b280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c001220 tx=0x7f9a4c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:24.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:24.831+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9a4803f030 con 0x7f9a5c0cd000 2024-09-21T07:47:25.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.056+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f9a5c0b1670 con 0x7f9a5c0cd000 2024-09-21T07:47:25.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.560+0000 7f9a52000640 1 -- 172.21.15.88:0/2528662382 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v20) ==== 104+0+0 (secure 0 0 0) 0x7f9a48084300 con 0x7f9a5c0cd000 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a3c078d70 msgr2=0x7f9a3c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a3c078d70 0x7f9a3c07b280 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c001220 tx=0x7f9a4c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 msgr2=0x7f9a5c1f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 0x7f9a5c1f6970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f9a48009950 tx=0x7f9a48003c10 comp rx=0 tx=0).stop 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 shutdown_connections 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a3c078d70 0x7f9a3c07b280 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a5c156ca0 0x7f9a5c1f60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c0cd000 0x7f9a5c1f6970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 --2- 172.21.15.88:0/2528662382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5c0b3f30 0x7f9a5c15bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.562+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 >> 172.21.15.88:0/2528662382 conn(0x7f9a5c06d620 msgr2=0x7f9a5c0adbe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:25.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.563+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 shutdown_connections 2024-09-21T07:47:25.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.563+0000 7f9a5b200640 1 -- 172.21.15.88:0/2528662382 wait complete. 2024-09-21T07:47:25.587 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 -- 172.21.15.88:0/2727069421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0af3c0 msgr2=0x7f745c1602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 --2- 172.21.15.88:0/2727069421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0af3c0 0x7f745c1602b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f745c06ddf0 tx=0x7f745402a820 comp rx=0 tx=0).stop 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 -- 172.21.15.88:0/2727069421 shutdown_connections 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 --2- 172.21.15.88:0/2727069421 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0af3c0 0x7f745c1602b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 --2- 172.21.15.88:0/2727069421 >> v2:172.21.15.175:3300/0 conn(0x7f745c0b3ec0 0x7f745c0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 -- 172.21.15.88:0/2727069421 >> 172.21.15.88:0/2727069421 conn(0x7f745c0b1dd0 msgr2=0x7f745c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:25.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 -- 172.21.15.88:0/2727069421 shutdown_connections 2024-09-21T07:47:25.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.812+0000 7f7461e00640 1 -- 172.21.15.88:0/2727069421 wait complete. 2024-09-21T07:47:25.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.815+0000 7f7461e00640 1 Processor -- start 2024-09-21T07:47:25.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.815+0000 7f7461e00640 1 -- start start 2024-09-21T07:47:25.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c0705b0 0x7f745c141a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:25.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c0af3c0 0x7f745c0744e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:25.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 0x7f745c074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:25.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f745c160b80 con 0x7f745c0b3ec0 2024-09-21T07:47:25.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f745c160a00 con 0x7f745c0af3c0 2024-09-21T07:47:25.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f745c153c40 con 0x7f745c0705b0 2024-09-21T07:47:25.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 0x7f745c074a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:25.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 0x7f745c074a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49252/0 (socket says 172.21.15.88:49252) 2024-09-21T07:47:25.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.816+0000 7f7461400640 1 -- 172.21.15.88:0/2606380422 learned_addr learned my addr 172.21.15.88:0/2606380422 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:25.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f745be00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c0af3c0 0x7f745c0744e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:25.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7460a00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c0705b0 0x7f745c141a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:25.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461400640 1 -- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c0705b0 msgr2=0x7f745c141a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:25.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461400640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c0705b0 0x7f745c141a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461400640 1 -- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c0af3c0 msgr2=0x7f745c0744e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461400640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c0af3c0 0x7f745c0744e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461400640 1 -- 172.21.15.88:0/2606380422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f745c141fd0 con 0x7f745c0b3ec0 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7460a00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c0705b0 0x7f745c141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461400640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 0x7f745c074a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f745402ae10 tx=0x7f7454032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7454037040 con 0x7f745c0b3ec0 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7454042070 con 0x7f745c0b3ec0 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74540042a0 con 0x7f745c0b3ec0 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.817+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f745c09bcf0 con 0x7f745c0b3ec0 2024-09-21T07:47:25.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.818+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f745c09c2b0 con 0x7f745c0b3ec0 2024-09-21T07:47:25.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.819+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f745c0b4a20 con 0x7f745c0b3ec0 2024-09-21T07:47:25.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.820+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f7454004440 con 0x7f745c0b3ec0 2024-09-21T07:47:25.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.821+0000 7f7458c00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f743c078d40 0x7f743c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:25.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.821+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 5795+0+0 (secure 0 0 0) 0x7f74540bc1d0 con 0x7f745c0b3ec0 2024-09-21T07:47:25.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.821+0000 7f7460a00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f743c078d40 0x7f743c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:25.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.822+0000 7f7460a00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f743c078d40 0x7f743c07b250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7450009aa0 tx=0x7f7450005db0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:25.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:25.825+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f745c0b4a20 con 0x7f745c0b3ec0 2024-09-21T07:47:26.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:26.012+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f745c1602b0 con 0x7f745c0b3ec0 2024-09-21T07:47:26.026 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:26.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:27.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.006+0000 7f7458c00640 1 -- 172.21.15.88:0/2606380422 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v37) ==== 181+0+0 (secure 0 0 0) 0x7f7454084180 con 0x7f745c0b3ec0 2024-09-21T07:47:27.008 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.010+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f743c078d40 msgr2=0x7f743c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.010+0000 7f7461e00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f743c078d40 0x7f743c07b250 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f7450009aa0 tx=0x7f7450005db0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.010+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 msgr2=0x7f745c074a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.010+0000 7f7461e00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 0x7f745c074a20 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f745402ae10 tx=0x7f7454032d10 comp rx=0 tx=0).stop 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 shutdown_connections 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f743c078d40 0x7f743c07b250 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f745c0b3ec0 0x7f745c074a20 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f745c0af3c0 0x7f745c0744e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 --2- 172.21.15.88:0/2606380422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f745c0705b0 0x7f745c141a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 >> 172.21.15.88:0/2606380422 conn(0x7f745c0b1dd0 msgr2=0x7f745c06ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 shutdown_connections 2024-09-21T07:47:27.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.011+0000 7f7461e00640 1 -- 172.21.15.88:0/2606380422 wait complete. 2024-09-21T07:47:27.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:47:27.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.263+0000 7f461a200640 1 -- 172.21.15.88:0/3090261296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 msgr2=0x7f45f800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:27.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.263+0000 7f461a200640 1 --2- 172.21.15.88:0/3090261296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f45f800b9c0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f461406dd10 tx=0x7f45f8005830 comp rx=0 tx=0).stop 2024-09-21T07:47:27.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.263+0000 7f4613e00640 1 -- 172.21.15.88:0/3090261296 reap_dead start 2024-09-21T07:47:27.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.264+0000 7f461a200640 1 -- 172.21.15.88:0/3090261296 shutdown_connections 2024-09-21T07:47:27.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.264+0000 7f461a200640 1 -- 172.21.15.88:0/3090261296 >> 172.21.15.88:0/3090261296 conn(0x7f461409aa50 msgr2=0x7f46140b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:27.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.264+0000 7f461a200640 1 -- 172.21.15.88:0/3090261296 shutdown_connections 2024-09-21T07:47:27.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.264+0000 7f461a200640 1 -- 172.21.15.88:0/3090261296 wait complete. 2024-09-21T07:47:27.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.266+0000 7f461a200640 1 Processor -- start 2024-09-21T07:47:27.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.266+0000 7f461a200640 1 -- start start 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.267+0000 7f461a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f46141f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.267+0000 7f461a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4614160020 0x7f4614160430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.267+0000 7f461a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46141f8840 0x7f46141f8c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f461a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f461406d930 con 0x7f45f8005150 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f461a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f461406d7b0 con 0x7f46141f8840 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f461a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f461406dab0 con 0x7f4614160020 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f46141f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f46141f7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49270/0 (socket says 172.21.15.88:49270) 2024-09-21T07:47:27.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 -- 172.21.15.88:0/1925951638 learned_addr learned my addr 172.21.15.88:0/1925951638 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:27.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 -- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4614160020 msgr2=0x7f4614160430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:27.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4612a00640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4614160020 0x7f4614160430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4614160020 0x7f4614160430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 -- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46141f8840 msgr2=0x7f46141f8c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613e00640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46141f8840 0x7f46141f8c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46141f8840 0x7f46141f8c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 -- 172.21.15.88:0/1925951638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46141f9190 con 0x7f45f8005150 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613e00640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46141f8840 0x7f46141f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.268+0000 7f4613400640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f46141f7ed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4604006720 tx=0x7f4604003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.269+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460400f860 con 0x7f45f8005150 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.269+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4604016020 con 0x7f45f8005150 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.269+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4614160d70 con 0x7f45f8005150 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.269+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460400fcc0 con 0x7f45f8005150 2024-09-21T07:47:27.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.269+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4614161330 con 0x7f45f8005150 2024-09-21T07:47:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.270+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46141f6420 con 0x7f45f8005150 2024-09-21T07:47:27.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.271+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f4604011430 con 0x7f45f8005150 2024-09-21T07:47:27.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.272+0000 7f460b400640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45f4078d40 0x7f45f407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:27.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.272+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(37..37 src has 1..37) ==== 5357+0+0 (secure 0 0 0) 0x7f460409bc20 con 0x7f45f8005150 2024-09-21T07:47:27.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.273+0000 7f4612a00640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45f4078d40 0x7f45f407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:27.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.274+0000 7f4612a00640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45f4078d40 0x7f45f407b250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f461415c850 tx=0x7f460000b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:27.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.277+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f46141f6420 con 0x7f45f8005150 2024-09-21T07:47:27.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:27.462+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46140b05a0 con 0x7f45f8005150 2024-09-21T07:47:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.012+0000 7f460b400640 1 -- 172.21.15.88:0/1925951638 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v38) ==== 169+0+0 (secure 0 0 0) 0x7f4604020020 con 0x7f45f8005150 2024-09-21T07:47:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.017+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45f4078d40 msgr2=0x7f45f407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.017+0000 7f461a200640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45f4078d40 0x7f45f407b250 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f461415c850 tx=0x7f460000b260 comp rx=0 tx=0).stop 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.017+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 msgr2=0x7f46141f7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.017+0000 7f461a200640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f46141f7ed0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f4604006720 tx=0x7f4604003a80 comp rx=0 tx=0).stop 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 shutdown_connections 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f45f4078d40 0x7f45f407b250 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4614160020 0x7f4614160430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f46141f8840 0x7f46141f8c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 --2- 172.21.15.88:0/1925951638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45f8005150 0x7f46141f7ed0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 >> 172.21.15.88:0/1925951638 conn(0x7f461409aa50 msgr2=0x7f461415df00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 shutdown_connections 2024-09-21T07:47:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.018+0000 7f461a200640 1 -- 172.21.15.88:0/1925951638 wait complete. 2024-09-21T07:47:28.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:47:28.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 -- 172.21.15.88:0/3956657718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e006f280 msgr2=0x7fd5e006f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 --2- 172.21.15.88:0/3956657718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e006f280 0x7fd5e006f690 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fd5e006f220 tx=0x7fd5d002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 -- 172.21.15.88:0/3956657718 shutdown_connections 2024-09-21T07:47:28.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 --2- 172.21.15.88:0/3956657718 >> v2:172.21.15.175:3300/0 conn(0x7fd5e01623a0 0x7fd5e01627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 --2- 172.21.15.88:0/3956657718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e006f280 0x7fd5e006f690 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 -- 172.21.15.88:0/3956657718 >> 172.21.15.88:0/3956657718 conn(0x7fd5e00b1e40 msgr2=0x7fd5e00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:28.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 -- 172.21.15.88:0/3956657718 shutdown_connections 2024-09-21T07:47:28.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.269+0000 7fd5e5000640 1 -- 172.21.15.88:0/3956657718 wait complete. 2024-09-21T07:47:28.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.271+0000 7fd5e5000640 1 Processor -- start 2024-09-21T07:47:28.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.271+0000 7fd5e5000640 1 -- start start 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5e5000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e006f280 0x7fd5e01f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5e5000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5e0155370 0x7fd5e0156d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5e5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 0x7fd5e01572b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5e5000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e00b3890 con 0x7fd5e01623a0 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5e5000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5e0073e30 con 0x7fd5e006f280 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5e5000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5e00b3a10 con 0x7fd5e0155370 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5dfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 0x7fd5e01572b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5dfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 0x7fd5e01572b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49286/0 (socket says 172.21.15.88:49286) 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.273+0000 7fd5dfe00640 1 -- 172.21.15.88:0/2772880250 learned_addr learned my addr 172.21.15.88:0/2772880250 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5dfe00640 1 -- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5e0155370 msgr2=0x7fd5e0156d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5dfe00640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5e0155370 0x7fd5e0156d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5dfe00640 1 -- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e006f280 msgr2=0x7fd5e01f4d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5df400640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e006f280 0x7fd5e01f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5dfe00640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e006f280 0x7fd5e01f4d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5dfe00640 1 -- 172.21.15.88:0/2772880250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5e009cd40 con 0x7fd5e01623a0 2024-09-21T07:47:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5df400640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e006f280 0x7fd5e01f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5dfe00640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 0x7fd5e01572b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800b9e0 tx=0x7fd5d800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:28.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8019070 con 0x7fd5e01623a0 2024-09-21T07:47:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5d800cc10 con 0x7fd5e01623a0 2024-09-21T07:47:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5e009d020 con 0x7fd5e01623a0 2024-09-21T07:47:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5d8014440 con 0x7fd5e01623a0 2024-09-21T07:47:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.274+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5e009d5e0 con 0x7fd5e01623a0 2024-09-21T07:47:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.275+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5e006f690 con 0x7fd5e01623a0 2024-09-21T07:47:28.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.277+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fd5d801c070 con 0x7fd5e01623a0 2024-09-21T07:47:28.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.278+0000 7fd5d7e00640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd5bc078d40 0x7fd5bc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.278+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fd5d809ac60 con 0x7fd5e01623a0 2024-09-21T07:47:28.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.279+0000 7fd5df400640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd5bc078d40 0x7fd5bc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.279+0000 7fd5df400640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd5bc078d40 0x7fd5bc07b250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0033000 tx=0x7fd5d0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:28.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.283+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5d8062f70 con 0x7fd5e01623a0 2024-09-21T07:47:28.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.474+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fd5e01f5260 con 0x7fd5e01623a0 2024-09-21T07:47:28.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.478+0000 7fd5d7e00640 1 -- 172.21.15.88:0/2772880250 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v33) ==== 71+0+0 (secure 0 0 0) 0x7fd5d8062870 con 0x7fd5e01623a0 2024-09-21T07:47:28.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd5bc078d40 msgr2=0x7fd5bc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd5bc078d40 0x7fd5bc07b250 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0033000 tx=0x7fd5d0004730 comp rx=0 tx=0).stop 2024-09-21T07:47:28.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 msgr2=0x7fd5e01572b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 0x7fd5e01572b0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd5d800b9e0 tx=0x7fd5d800bea0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 shutdown_connections 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd5bc078d40 0x7fd5bc07b250 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5e0155370 0x7fd5e0156d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5e01623a0 0x7fd5e01572b0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 --2- 172.21.15.88:0/2772880250 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5e006f280 0x7fd5e01f4d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 >> 172.21.15.88:0/2772880250 conn(0x7fd5e00b1e40 msgr2=0x7fd5e01618e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 shutdown_connections 2024-09-21T07:47:28.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.480+0000 7fd5e5000640 1 -- 172.21.15.88:0/2772880250 wait complete. 2024-09-21T07:47:28.503 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade' 2024-09-21T07:47:28.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 -- 172.21.15.88:0/730940689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46f8003a50 msgr2=0x7f46f8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 --2- 172.21.15.88:0/730940689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46f8003a50 0x7f46f8003ec0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f47040016c0 tx=0x7f46f8005830 comp rx=0 tx=0).stop 2024-09-21T07:47:28.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 -- 172.21.15.88:0/730940689 shutdown_connections 2024-09-21T07:47:28.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 --2- 172.21.15.88:0/730940689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46f8003a50 0x7f46f8003ec0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 -- 172.21.15.88:0/730940689 >> 172.21.15.88:0/730940689 conn(0x7f471406d730 msgr2=0x7f47140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:28.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 -- 172.21.15.88:0/730940689 shutdown_connections 2024-09-21T07:47:28.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.726+0000 7f471b200640 1 -- 172.21.15.88:0/730940689 wait complete. 2024-09-21T07:47:28.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.728+0000 7f471b200640 1 Processor -- start 2024-09-21T07:47:28.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.728+0000 7f471b200640 1 -- start start 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f471b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f8003a50 0x7f4714160940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f471b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47140999e0 0x7f47140afd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f471b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 0x7f47140b1840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f471b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47140ad4f0 con 0x7f4714073970 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f471b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47140b4e30 con 0x7f47140999e0 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f471b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47140ad670 con 0x7f46f8003a50 2024-09-21T07:47:28.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 0x7f47140b1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 0x7f47140b1840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49312/0 (socket says 172.21.15.88:49312) 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 -- 172.21.15.88:0/2279981800 learned_addr learned my addr 172.21.15.88:0/2279981800 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4713e00640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f8003a50 0x7f4714160940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f470b400640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47140999e0 0x7f47140afd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 -- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f8003a50 msgr2=0x7f4714160940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f8003a50 0x7f4714160940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 -- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47140999e0 msgr2=0x7f47140afd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47140999e0 0x7f47140afd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f4719200640 1 -- 172.21.15.88:0/2279981800 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4714160e80 con 0x7f4714073970 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.730+0000 7f470b400640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47140999e0 0x7f47140afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.731+0000 7f4719200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 0x7f47140b1840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f47140ae780 tx=0x7f470c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.731+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c004b20 con 0x7f4714073970 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.731+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f470c015070 con 0x7f4714073970 2024-09-21T07:47:28.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.731+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4714074aa0 con 0x7f4714073970 2024-09-21T07:47:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.731+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c003bd0 con 0x7f4714073970 2024-09-21T07:47:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.731+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4714073170 con 0x7f4714073970 2024-09-21T07:47:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.732+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47141535c0 con 0x7f4714073970 2024-09-21T07:47:28.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.733+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f470c004030 con 0x7f4714073970 2024-09-21T07:47:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.735+0000 7f4710c00640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46f0078d40 0x7f46f007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.735+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f470c09b5c0 con 0x7f4714073970 2024-09-21T07:47:28.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.735+0000 7f4713e00640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46f0078d40 0x7f46f007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:28.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.736+0000 7f4713e00640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46f0078d40 0x7f46f007b250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4704032000 tx=0x7f4704001190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:28.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.739+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f470c063960 con 0x7f4714073970 2024-09-21T07:47:28.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:28.996+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7f471415e340 con 0x7f4714073970 2024-09-21T07:47:29.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.014+0000 7f4710c00640 1 -- 172.21.15.88:0/2279981800 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_binary_metafile_on_legacy_to_v1_upgrade"]}]=0 v102) ==== 158+0+0 (secure 0 0 0) 0x7f470c063260 con 0x7f4714073970 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.016+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46f0078d40 msgr2=0x7f46f007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.016+0000 7f471b200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46f0078d40 0x7f46f007b250 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4704032000 tx=0x7f4704001190 comp rx=0 tx=0).stop 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.016+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 msgr2=0x7f47140b1840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.016+0000 7f471b200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 0x7f47140b1840 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f47140ae780 tx=0x7f470c010f40 comp rx=0 tx=0).stop 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 shutdown_connections 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46f0078d40 0x7f46f007b250 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4714073970 0x7f47140b1840 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47140999e0 0x7f47140afd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 --2- 172.21.15.88:0/2279981800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46f8003a50 0x7f4714160940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 >> 172.21.15.88:0/2279981800 conn(0x7f471406d730 msgr2=0x7f47140d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:29.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 shutdown_connections 2024-09-21T07:47:29.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.017+0000 7f471b200640 1 -- 172.21.15.88:0/2279981800 wait complete. 2024-09-21T07:47:29.040 INFO:tasks.cephfs_test_runner:Validate binary .meta file on legacy subvol root doesn't break the system ... ok 2024-09-21T07:47:29.040 INFO:tasks.cephfs_test_runner:Starting test: test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:47:29.040 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.257+0000 7ff260a00640 1 -- 172.21.15.88:0/3613679394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 msgr2=0x7ff25c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.257+0000 7ff260a00640 1 --2- 172.21.15.88:0/3613679394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b3960 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7ff25c072d60 tx=0x7ff25402a820 comp rx=0 tx=0).stop 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.257+0000 7ff260a00640 1 -- 172.21.15.88:0/3613679394 shutdown_connections 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.257+0000 7ff260a00640 1 --2- 172.21.15.88:0/3613679394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b3960 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.257+0000 7ff260a00640 1 --2- 172.21.15.88:0/3613679394 >> v2:172.21.15.175:3300/0 conn(0x7ff25c0ad0e0 0x7ff25c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.257+0000 7ff260a00640 1 -- 172.21.15.88:0/3613679394 >> 172.21.15.88:0/3613679394 conn(0x7ff25c06d730 msgr2=0x7ff25c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:29.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.258+0000 7ff260a00640 1 -- 172.21.15.88:0/3613679394 shutdown_connections 2024-09-21T07:47:29.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.258+0000 7ff260a00640 1 -- 172.21.15.88:0/3613679394 wait complete. 2024-09-21T07:47:29.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.260+0000 7ff260a00640 1 Processor -- start 2024-09-21T07:47:29.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.260+0000 7ff260a00640 1 -- start start 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.261+0000 7ff260a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff25c0ad0e0 0x7ff25c1f5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.261+0000 7ff260a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b11d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.261+0000 7ff260a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c15d760 0x7ff25c14cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.261+0000 7ff260a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0b4da0 con 0x7ff25c0b3550 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.261+0000 7ff260a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff25c0b4c20 con 0x7ff25c15d760 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.261+0000 7ff260a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff25c0b3ea0 con 0x7ff25c0ad0e0 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49324/0 (socket says 172.21.15.88:49324) 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 -- 172.21.15.88:0/1649438916 learned_addr learned my addr 172.21.15.88:0/1649438916 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff25a800640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c15d760 0x7ff25c14cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259e00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff25c0ad0e0 0x7ff25c1f5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 -- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff25c0ad0e0 msgr2=0x7ff25c1f5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff25c0ad0e0 0x7ff25c1f5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 -- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c15d760 msgr2=0x7ff25c14cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c15d760 0x7ff25c14cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 -- 172.21.15.88:0/1649438916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff25c1f58b0 con 0x7ff25c0b3550 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff25a800640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c15d760 0x7ff25c14cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.262+0000 7ff259400640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b11d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff24800e790 tx=0x7ff24800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.263+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff24800eea0 con 0x7ff25c0b3550 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.263+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff24800ccc0 con 0x7ff25c0b3550 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.263+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff248004280 con 0x7ff25c0b3550 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.263+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff25c0b1710 con 0x7ff25c0b3550 2024-09-21T07:47:29.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.263+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff25c0b0410 con 0x7ff25c0b3550 2024-09-21T07:47:29.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.264+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff25c0b43c0 con 0x7ff25c0b3550 2024-09-21T07:47:29.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.265+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7ff2480035f0 con 0x7ff25c0b3550 2024-09-21T07:47:29.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.266+0000 7ff252000640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff23c078d40 0x7ff23c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.267+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7ff24809b920 con 0x7ff25c0b3550 2024-09-21T07:47:29.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.267+0000 7ff259e00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff23c078d40 0x7ff23c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.268+0000 7ff259e00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff23c078d40 0x7ff23c07b250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0013f0 tx=0x7ff24c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:29.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.271+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff25c1f58b0 con 0x7ff25c0b3550 2024-09-21T07:47:29.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.456+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff25c153900 con 0x7ff25c0b3550 2024-09-21T07:47:29.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.457+0000 7ff252000640 1 -- 172.21.15.88:0/1649438916 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v38) ==== 74+0+13044 (secure 0 0 0) 0x7ff2480635e0 con 0x7ff25c0b3550 2024-09-21T07:47:29.458 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:29.458 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":38,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:47:28.007485+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/1785638532":"2024-09-22T07:47:24.556399+0000","172.21.15.88:6833/1785638532":"2024-09-22T07:47:24.556399+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff23c078d40 msgr2=0x7ff23c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff23c078d40 0x7ff23c07b250 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0013f0 tx=0x7ff24c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 msgr2=0x7ff25c0b11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b11d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7ff24800e790 tx=0x7ff24800ec50 comp rx=0 tx=0).stop 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 shutdown_connections 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff23c078d40 0x7ff23c07b250 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff25c15d760 0x7ff25c14cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff25c0b3550 0x7ff25c0b11d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 --2- 172.21.15.88:0/1649438916 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff25c0ad0e0 0x7ff25c1f5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 >> 172.21.15.88:0/1649438916 conn(0x7ff25c06d730 msgr2=0x7ff25c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 shutdown_connections 2024-09-21T07:47:29.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.459+0000 7ff260a00640 1 -- 172.21.15.88:0/1649438916 wait complete. 2024-09-21T07:47:29.482 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:47:29.482 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:47:29.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 -- 172.21.15.88:0/1296369018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 msgr2=0x7f04280733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 --2- 172.21.15.88:0/1296369018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 0x7f04280733d0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f0414009830 tx=0x7f041402a510 comp rx=0 tx=0).stop 2024-09-21T07:47:29.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 -- 172.21.15.88:0/1296369018 shutdown_connections 2024-09-21T07:47:29.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 --2- 172.21.15.88:0/1296369018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0428152060 0x7f0428152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 --2- 172.21.15.88:0/1296369018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 0x7f04280733d0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 -- 172.21.15.88:0/1296369018 >> 172.21.15.88:0/1296369018 conn(0x7f042806d710 msgr2=0x7f04280b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:29.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 -- 172.21.15.88:0/1296369018 shutdown_connections 2024-09-21T07:47:29.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.704+0000 7f042f200640 1 -- 172.21.15.88:0/1296369018 wait complete. 2024-09-21T07:47:29.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.706+0000 7f042f200640 1 Processor -- start 2024-09-21T07:47:29.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.706+0000 7f042f200640 1 -- start start 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 0x7f042814b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 0x7f04281f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042815f250 0x7f04281f6750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04280aec60 con 0x7f0428152060 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04280aeae0 con 0x7f0428072fc0 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04280aede0 con 0x7f042815f250 2024-09-21T07:47:29.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f0427400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 0x7f04281f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f0427400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 0x7f04281f6f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49338/0 (socket says 172.21.15.88:49338) 2024-09-21T07:47:29.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f0427400640 1 -- 172.21.15.88:0/419766482 learned_addr learned my addr 172.21.15.88:0/419766482 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:29.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f042d200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042815f250 0x7f04281f6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f0427e00640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 0x7f042814b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f0427400640 1 -- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042815f250 msgr2=0x7f04281f6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.708+0000 7f0427400640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042815f250 0x7f04281f6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f0427400640 1 -- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 msgr2=0x7f042814b1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f0427400640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 0x7f042814b1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f0427400640 1 -- 172.21.15.88:0/419766482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0428153930 con 0x7f0428152060 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f0427400640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 0x7f04281f6f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0414009950 tx=0x7f0414033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414031d60 con 0x7f0428152060 2024-09-21T07:47:29.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0414033e10 con 0x7f0428152060 2024-09-21T07:47:29.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414033310 con 0x7f0428152060 2024-09-21T07:47:29.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04281f5f50 con 0x7f0428152060 2024-09-21T07:47:29.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.709+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04281f77c0 con 0x7f0428152060 2024-09-21T07:47:29.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.711+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04280733d0 con 0x7f0428152060 2024-09-21T07:47:29.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.711+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f04140334b0 con 0x7f0428152060 2024-09-21T07:47:29.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.712+0000 7f041fe00640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0408078d40 0x7f040807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:29.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.713+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f04140bc090 con 0x7f0428152060 2024-09-21T07:47:29.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.713+0000 7f0427e00640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0408078d40 0x7f040807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:29.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.713+0000 7f0427e00640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0408078d40 0x7f040807b250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0418001720 tx=0x7f0418007e90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:29.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.717+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f041403f030 con 0x7f0428152060 2024-09-21T07:47:29.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.904+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f0428150530 con 0x7f0428152060 2024-09-21T07:47:29.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.905+0000 7f041fe00640 1 -- 172.21.15.88:0/419766482 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v38) ==== 93+0+10 (secure 0 0 0) 0x7f0414083e90 con 0x7f0428152060 2024-09-21T07:47:29.906 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:47:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0408078d40 msgr2=0x7f040807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0408078d40 0x7f040807b250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f0418001720 tx=0x7f0418007e90 comp rx=0 tx=0).stop 2024-09-21T07:47:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 msgr2=0x7f04281f6f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 0x7f04281f6f90 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f0414009950 tx=0x7f0414033c20 comp rx=0 tx=0).stop 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 shutdown_connections 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0408078d40 0x7f040807b250 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f042815f250 0x7f04281f6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0428152060 0x7f04281f6f90 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 --2- 172.21.15.88:0/419766482 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0428072fc0 0x7f042814b1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 >> 172.21.15.88:0/419766482 conn(0x7f042806d710 msgr2=0x7f042815e580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 shutdown_connections 2024-09-21T07:47:29.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:29.907+0000 7f042f200640 1 -- 172.21.15.88:0/419766482 wait complete. 2024-09-21T07:47:29.929 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-21T07:47:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 -- 172.21.15.88:0/3703728774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc0ad0e0 msgr2=0x7f04bc0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:30.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 --2- 172.21.15.88:0/3703728774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc0ad0e0 0x7f04bc0ad4f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f04a8009830 tx=0x7f04a802a480 comp rx=0 tx=0).stop 2024-09-21T07:47:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 -- 172.21.15.88:0/3703728774 shutdown_connections 2024-09-21T07:47:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 --2- 172.21.15.88:0/3703728774 >> v2:172.21.15.175:3300/0 conn(0x7f04bc0b3550 0x7f04bc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 --2- 172.21.15.88:0/3703728774 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc0ad0e0 0x7f04bc0ad4f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 -- 172.21.15.88:0/3703728774 >> 172.21.15.88:0/3703728774 conn(0x7f04bc06d730 msgr2=0x7f04bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 -- 172.21.15.88:0/3703728774 shutdown_connections 2024-09-21T07:47:30.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.149+0000 7f04c3e00640 1 -- 172.21.15.88:0/3703728774 wait complete. 2024-09-21T07:47:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.151+0000 7f04c3e00640 1 Processor -- start 2024-09-21T07:47:30.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.152+0000 7f04c3e00640 1 -- start start 2024-09-21T07:47:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 0x7f04bc15b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b3550 0x7f04bc14b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:30.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc15d760 0x7f04bc14b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0b4da0 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04bc0b4c20 con 0x7f04bc0b3550 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04bc0b3ea0 con 0x7f04bc15d760 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 0x7f04bc15b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 0x7f04bc15b2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49350/0 (socket says 172.21.15.88:49350) 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.153+0000 7f04c1400640 1 -- 172.21.15.88:0/2268670190 learned_addr learned my addr 172.21.15.88:0/2268670190 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc15d760 0x7f04bc14b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:30.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c0a00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b3550 0x7f04bc14b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1400640 1 -- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc15d760 msgr2=0x7f04bc14b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1400640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc15d760 0x7f04bc14b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1400640 1 -- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b3550 msgr2=0x7f04bc14b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1400640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b3550 0x7f04bc14b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:30.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1400640 1 -- 172.21.15.88:0/2268670190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04bc156010 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c1400640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 0x7f04bc15b2c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c760 tx=0x7f04b000cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0016020 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04bc1562f0 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b000a950 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.154+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b000ada0 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.155+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04bc1568b0 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.156+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04bc0b4330 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.157+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f04b0019020 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.158+0000 7f04b9600640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f049c078d70 0x7f049c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:30.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.158+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f04b009ace0 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.159+0000 7f04c0a00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f049c078d70 0x7f049c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:30.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.160+0000 7f04c0a00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f049c078d70 0x7f049c07b280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:30.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.163+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04b0020020 con 0x7f04bc0ad0e0 2024-09-21T07:47:30.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:30.419+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f04bc09e450 con 0x7f04bc0ad0e0 2024-09-21T07:47:31.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.024+0000 7f04b9600640 1 -- 172.21.15.88:0/2268670190 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v104) ==== 143+0+0 (secure 0 0 0) 0x7f04bc0ada30 con 0x7f04bc0ad0e0 2024-09-21T07:47:31.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.027+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f049c078d70 msgr2=0x7f049c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.027+0000 7f04c3e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f049c078d70 0x7f049c07b280 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f04a8034000 tx=0x7f04a8004730 comp rx=0 tx=0).stop 2024-09-21T07:47:31.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.027+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 msgr2=0x7f04bc15b2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 0x7f04bc15b2c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f04b000c760 tx=0x7f04b000cb60 comp rx=0 tx=0).stop 2024-09-21T07:47:31.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 shutdown_connections 2024-09-21T07:47:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f049c078d70 0x7f049c07b280 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04bc15d760 0x7f04bc14b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04bc0b3550 0x7f04bc14b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 --2- 172.21.15.88:0/2268670190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04bc0ad0e0 0x7f04bc15b2c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 >> 172.21.15.88:0/2268670190 conn(0x7f04bc06d730 msgr2=0x7f04bc1535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:31.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 shutdown_connections 2024-09-21T07:47:31.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.028+0000 7f04c3e00640 1 -- 172.21.15.88:0/2268670190 wait complete. 2024-09-21T07:47:31.054 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 -- 172.21.15.88:0/1912358861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd015d760 msgr2=0x7f0bd01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 --2- 172.21.15.88:0/1912358861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd015d760 0x7f0bd01535c0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f0bc0009830 tx=0x7f0bc002a530 comp rx=0 tx=0).stop 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 -- 172.21.15.88:0/1912358861 shutdown_connections 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 --2- 172.21.15.88:0/1912358861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd015d760 0x7f0bd01535c0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 -- 172.21.15.88:0/1912358861 >> 172.21.15.88:0/1912358861 conn(0x7f0bd006d730 msgr2=0x7f0bd00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 -- 172.21.15.88:0/1912358861 shutdown_connections 2024-09-21T07:47:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.271+0000 7f0bd5200640 1 -- 172.21.15.88:0/1912358861 wait complete. 2024-09-21T07:47:31.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.273+0000 7f0bd5200640 1 Processor -- start 2024-09-21T07:47:31.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.274+0000 7f0bd5200640 1 -- start start 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bd5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 0x7f0bd014f5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bd5200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bd00b3550 0x7f0bd01f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bd5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd015d760 0x7f0bd01f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bd5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd006f240 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bd5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bd00b4e30 con 0x7f0bd015d760 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bd5200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0bd00b4070 con 0x7f0bd00b3550 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bcde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 0x7f0bd014f5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bcde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 0x7f0bd014f5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49366/0 (socket says 172.21.15.88:49366) 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.275+0000 7f0bcde00640 1 -- 172.21.15.88:0/2162029029 learned_addr learned my addr 172.21.15.88:0/2162029029 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bce800640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd015d760 0x7f0bd01f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcd400640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bd00b3550 0x7f0bd01f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcde00640 1 -- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bd00b3550 msgr2=0x7f0bd01f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcde00640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bd00b3550 0x7f0bd01f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcde00640 1 -- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd015d760 msgr2=0x7f0bd01f7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcde00640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd015d760 0x7f0bd01f7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcde00640 1 -- 172.21.15.88:0/2162029029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bd01568b0 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bce800640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd015d760 0x7f0bd01f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bcde00640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 0x7f0bd014f5f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0bc002ab20 tx=0x7f0bc0033c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.276+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0031b60 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.277+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0bc0031df0 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.277+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0bc0005280 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.277+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bd0146c50 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.277+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bd0147210 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.278+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bd01535c0 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.279+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f0bc0005420 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.280+0000 7f0bc6000640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0bb0078d40 0x7f0bb007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.281+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f0bc00bc170 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.281+0000 7f0bcd400640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0bb0078d40 0x7f0bb007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.282+0000 7f0bcd400640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0bb0078d40 0x7f0bb007b250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bd014f320 tx=0x7f0bbc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.285+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0bc003f030 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.470+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0bd00b0790 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.471+0000 7f0bc6000640 1 -- 172.21.15.88:0/2162029029 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v33) ==== 59+0+2294 (secure 0 0 0) 0x7f0bc0031770 con 0x7f0bd00ad0e0 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:47:31.472 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:47:31.473 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:47:31.474 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:47:31.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0bb0078d40 msgr2=0x7f0bb007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0bb0078d40 0x7f0bb007b250 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f0bd014f320 tx=0x7f0bbc009040 comp rx=0 tx=0).stop 2024-09-21T07:47:31.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 msgr2=0x7f0bd014f5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 0x7f0bd014f5f0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f0bc002ab20 tx=0x7f0bc0033c10 comp rx=0 tx=0).stop 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 shutdown_connections 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0bb0078d40 0x7f0bb007b250 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bd00b3550 0x7f0bd01f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bd015d760 0x7f0bd01f7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 --2- 172.21.15.88:0/2162029029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bd00ad0e0 0x7f0bd014f5f0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 >> 172.21.15.88:0/2162029029 conn(0x7f0bd006d730 msgr2=0x7f0bd0070280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 shutdown_connections 2024-09-21T07:47:31.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.473+0000 7f0bd5200640 1 -- 172.21.15.88:0/2162029029 wait complete. 2024-09-21T07:47:31.496 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:47:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a50e00640 1 -- 172.21.15.88:0/4055462112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 msgr2=0x7f6a30003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a50e00640 1 --2- 172.21.15.88:0/4055462112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a30003e70 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c4a0 tx=0x7f6a30005830 comp rx=0 tx=0).stop 2024-09-21T07:47:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a4a800640 1 -- 172.21.15.88:0/4055462112 reap_dead start 2024-09-21T07:47:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a50e00640 1 -- 172.21.15.88:0/4055462112 shutdown_connections 2024-09-21T07:47:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a50e00640 1 -- 172.21.15.88:0/4055462112 >> 172.21.15.88:0/4055462112 conn(0x7f6a4c06d1c0 msgr2=0x7f6a4c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:31.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a50e00640 1 -- 172.21.15.88:0/4055462112 shutdown_connections 2024-09-21T07:47:31.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.717+0000 7f6a50e00640 1 -- 172.21.15.88:0/4055462112 wait complete. 2024-09-21T07:47:31.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.720+0000 7f6a50e00640 1 Processor -- start 2024-09-21T07:47:31.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.720+0000 7f6a50e00640 1 -- start start 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a50e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a4c152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a50e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4c15bf60 0x7f6a4c1f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a50e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4c1f72b0 0x7f6a4c1f76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a50e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0cdcd0 con 0x7f6a30003a00 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a50e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6a4c0cdb50 con 0x7f6a4c1f72b0 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a50e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6a4c0cde50 con 0x7f6a4c15bf60 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a49e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a4c152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a49e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a4c152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49386/0 (socket says 172.21.15.88:49386) 2024-09-21T07:47:31.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.721+0000 7f6a49e00640 1 -- 172.21.15.88:0/545198848 learned_addr learned my addr 172.21.15.88:0/545198848 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49400640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4c15bf60 0x7f6a4c1f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a4a800640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4c1f72b0 0x7f6a4c1f76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49e00640 1 -- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4c15bf60 msgr2=0x7f6a4c1f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4c15bf60 0x7f6a4c1f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49e00640 1 -- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4c1f72b0 msgr2=0x7f6a4c1f76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4c1f72b0 0x7f6a4c1f76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49e00640 1 -- 172.21.15.88:0/545198848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a4c15e640 con 0x7f6a30003a00 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a4a800640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4c1f72b0 0x7f6a4c1f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:31.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a49e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a4c152d80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c5c0 tx=0x7f6a3c006f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:31.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c0065f0 con 0x7f6a30003a00 2024-09-21T07:47:31.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6a3c02b6f0 con 0x7f6a30003a00 2024-09-21T07:47:31.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a4c15cf20 con 0x7f6a30003a00 2024-09-21T07:47:31.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.722+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a3c02bb20 con 0x7f6a30003a00 2024-09-21T07:47:31.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.723+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a4c1532c0 con 0x7f6a30003a00 2024-09-21T07:47:31.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.723+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a4c0cd610 con 0x7f6a30003a00 2024-09-21T07:47:31.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.725+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f6a3c02bcc0 con 0x7f6a30003a00 2024-09-21T07:47:31.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.725+0000 7f6a42000640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6a2c078d40 0x7f6a2c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:31.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.726+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7f6a3c0ba7d0 con 0x7f6a30003a00 2024-09-21T07:47:31.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.726+0000 7f6a49400640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6a2c078d40 0x7f6a2c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:31.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.727+0000 7f6a49400640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6a2c078d40 0x7f6a2c07b250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c13a7a0 tx=0x7f6a38005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:31.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.730+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6a4c073e90 con 0x7f6a30003a00 2024-09-21T07:47:31.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.918+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6a4c0cd610 con 0x7f6a30003a00 2024-09-21T07:47:31.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.922+0000 7f6a42000640 1 -- 172.21.15.88:0/545198848 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f6a3c082ae0 con 0x7f6a30003a00 2024-09-21T07:47:31.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6a2c078d40 msgr2=0x7f6a2c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6a2c078d40 0x7f6a2c07b250 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f6a4c13a7a0 tx=0x7f6a38005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:31.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 msgr2=0x7f6a4c152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a4c152d80 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f6a3c00c5c0 tx=0x7f6a3c006f10 comp rx=0 tx=0).stop 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 shutdown_connections 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6a2c078d40 0x7f6a2c07b250 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6a4c15bf60 0x7f6a4c1f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6a4c1f72b0 0x7f6a4c1f76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 --2- 172.21.15.88:0/545198848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a30003a00 0x7f6a4c152d80 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 >> 172.21.15.88:0/545198848 conn(0x7f6a4c06d1c0 msgr2=0x7f6a4c142bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 shutdown_connections 2024-09-21T07:47:31.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:31.925+0000 7f6a50e00640 1 -- 172.21.15.88:0/545198848 wait complete. 2024-09-21T07:47:31.947 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:47:31.947 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:32.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:47:32.129 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:47:32.129 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:32.129 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:32.164 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:32.165 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:32.165 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:32.201 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:32.201 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:32.202 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:32.239 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:47:32.239 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:32.239 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:32.278 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:47:32.278 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:47:32.278 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 -- 172.21.15.88:0/2343725752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 msgr2=0x7fe4400b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 --2- 172.21.15.88:0/2343725752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 0x7fe4400b3960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe438005fd0 tx=0x7fe43802aa10 comp rx=0 tx=0).stop 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 -- 172.21.15.88:0/2343725752 shutdown_connections 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 --2- 172.21.15.88:0/2343725752 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 0x7fe4400b3960 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 --2- 172.21.15.88:0/2343725752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe44015d760 0x7fe4401535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 -- 172.21.15.88:0/2343725752 >> 172.21.15.88:0/2343725752 conn(0x7fe44006d730 msgr2=0x7fe4400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 -- 172.21.15.88:0/2343725752 shutdown_connections 2024-09-21T07:47:32.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.498+0000 7fe447c00640 1 -- 172.21.15.88:0/2343725752 wait complete. 2024-09-21T07:47:32.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.501+0000 7fe447c00640 1 Processor -- start 2024-09-21T07:47:32.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.501+0000 7fe447c00640 1 -- start start 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.502+0000 7fe447c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 0x7fe44014a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.502+0000 7fe447c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 0x7fe44014b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.502+0000 7fe447c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe44015d760 0x7fe4401f5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.502+0000 7fe447c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe4400b4da0 con 0x7fe4400ad0e0 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.502+0000 7fe447c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4400b4c20 con 0x7fe4400b3550 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.502+0000 7fe447c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4400b3ea0 con 0x7fe44015d760 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 0x7fe44014a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 0x7fe44014a590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49398/0 (socket says 172.21.15.88:49398) 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 -- 172.21.15.88:0/3480046744 learned_addr learned my addr 172.21.15.88:0/3480046744 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe44015d760 0x7fe4401f5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:32.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe43fe00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 0x7fe44014b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 -- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe44015d760 msgr2=0x7fe4401f5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe44015d760 0x7fe4401f5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 -- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 msgr2=0x7fe44014b910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 0x7fe44014b910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 -- 172.21.15.88:0/3480046744 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4401f9480 con 0x7fe4400ad0e0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.503+0000 7fe445200640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 0x7fe44014a590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe43000ea50 tx=0x7fe43000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.504+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe43000cdc0 con 0x7fe4400ad0e0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.504+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4401f9760 con 0x7fe4400ad0e0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.504+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe430004020 con 0x7fe4400ad0e0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.504+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe430004470 con 0x7fe4400ad0e0 2024-09-21T07:47:32.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.504+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe44014abd0 con 0x7fe4400ad0e0 2024-09-21T07:47:32.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.505+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4401f5790 con 0x7fe4400ad0e0 2024-09-21T07:47:32.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.506+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fe4300035f0 con 0x7fe4400ad0e0 2024-09-21T07:47:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.507+0000 7fe43cc00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe420078d70 0x7fe42007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.507+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4943+0+0 (secure 0 0 0) 0x7fe43009b940 con 0x7fe4400ad0e0 2024-09-21T07:47:32.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.507+0000 7fe43fe00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe420078d70 0x7fe42007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:32.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.508+0000 7fe43fe00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe420078d70 0x7fe42007b280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4400699f0 tx=0x7fe42c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:32.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.511+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe430063580 con 0x7fe4400ad0e0 2024-09-21T07:47:32.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.732+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe4400b4330 con 0x7fe4400ad0e0 2024-09-21T07:47:32.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.732+0000 7fe43cc00640 1 -- 172.21.15.88:0/3480046744 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+3914 (secure 0 0 0) 0x7fe430067fb0 con 0x7fe4400ad0e0 2024-09-21T07:47:32.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:32.734 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":20,"btime":"2024-09-21T07:47:25:558569+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4694,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3019127515","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3019127515},{"type":"v1","addr":"172.21.15.175:6836","nonce":3019127515}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4891,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3121099863","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3121099863},{"type":"v1","addr":"172.21.15.175:6837","nonce":3121099863}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":4896,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3120344095","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3120344095},{"type":"v1","addr":"172.21.15.88:6834","nonce":3120344095}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":5247,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/997641555","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":997641555},{"type":"v1","addr":"172.21.15.88:6835","nonce":997641555}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":20}],"filesystems":[]} 2024-09-21T07:47:32.734 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 20 2024-09-21T07:47:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe420078d70 msgr2=0x7fe42007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe420078d70 0x7fe42007b280 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fe4400699f0 tx=0x7fe42c009040 comp rx=0 tx=0).stop 2024-09-21T07:47:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 msgr2=0x7fe44014a590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:32.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 0x7fe44014a590 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fe43000ea50 tx=0x7fe43000ef10 comp rx=0 tx=0).stop 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 shutdown_connections 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe420078d70 0x7fe42007b280 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe44015d760 0x7fe4401f5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4400b3550 0x7fe44014b910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 --2- 172.21.15.88:0/3480046744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe4400ad0e0 0x7fe44014a590 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 >> 172.21.15.88:0/3480046744 conn(0x7fe44006d730 msgr2=0x7fe44015ea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 shutdown_connections 2024-09-21T07:47:32.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:32.735+0000 7fe447c00640 1 -- 172.21.15.88:0/3480046744 wait complete. 2024-09-21T07:47:32.757 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:47:32.757 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:47:32.757 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:47:32.757 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:32.758 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:47:32.758 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:47:32.758 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:47:32.758 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:32.758 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:47:32.758 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:47:32.758 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:47:32.758 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:32.759 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:47:32.759 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:47:32.759 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:47:32.759 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:47:32.779 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:47:32.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:47:32.782 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:47:32.782 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:47:32.783 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:47:32.783 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:47:32.785 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:47:32.785 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:47:32.826 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:47:32.826 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:32.827 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:47:32.828 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:32.828 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:47:32.829 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:32.830 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:47:32.830 DEBUG:teuthology.parallel:result is None 2024-09-21T07:47:32.831 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:47:33.023 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:47:33.023 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:47:33.024 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:47:33.024 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.063+0000 7fea00c00640 1 -- 172.21.15.88:0/4246837388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc15a0f0 msgr2=0x7fe9fc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.063+0000 7fea00c00640 1 --2- 172.21.15.88:0/4246837388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc15a0f0 0x7fe9fc15d670 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4009c10 tx=0x7fe9f402a870 comp rx=0 tx=0).stop 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.063+0000 7fea00c00640 1 -- 172.21.15.88:0/4246837388 shutdown_connections 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.063+0000 7fea00c00640 1 --2- 172.21.15.88:0/4246837388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc15a0f0 0x7fe9fc15d670 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.063+0000 7fea00c00640 1 --2- 172.21.15.88:0/4246837388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc0af170 0x7fe9fc0b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.063+0000 7fea00c00640 1 -- 172.21.15.88:0/4246837388 >> 172.21.15.88:0/4246837388 conn(0x7fe9fc0b1e40 msgr2=0x7fe9fc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.064+0000 7fea00c00640 1 -- 172.21.15.88:0/4246837388 shutdown_connections 2024-09-21T07:47:33.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.064+0000 7fea00c00640 1 -- 172.21.15.88:0/4246837388 wait complete. 2024-09-21T07:47:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.066+0000 7fea00c00640 1 Processor -- start 2024-09-21T07:47:33.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.066+0000 7fea00c00640 1 -- start start 2024-09-21T07:47:33.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fea00c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ad9c0 0x7fe9fc146700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:33.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fea00c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 0x7fe9fc1f30e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:33.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fea00c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc15a0f0 0x7fe9fc143250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fea00c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc06f410 con 0x7fe9fc0af170 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fea00c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc06f290 con 0x7fe9fc0ad9c0 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fea00c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc06f590 con 0x7fe9fc15a0f0 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fe9faa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 0x7fe9fc1f30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fe9faa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 0x7fe9fc1f30e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49456/0 (socket says 172.21.15.88:49456) 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fe9faa00640 1 -- 172.21.15.88:0/1198284702 learned_addr learned my addr 172.21.15.88:0/1198284702 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fe9fbe00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc15a0f0 0x7fe9fc143250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.068+0000 7fe9fb400640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ad9c0 0x7fe9fc146700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9faa00640 1 -- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc15a0f0 msgr2=0x7fe9fc143250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9faa00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc15a0f0 0x7fe9fc143250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9faa00640 1 -- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ad9c0 msgr2=0x7fe9fc146700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9faa00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ad9c0 0x7fe9fc146700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:33.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9faa00640 1 -- 172.21.15.88:0/1198284702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc1f3620 con 0x7fe9fc0af170 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9faa00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 0x7fe9fc1f30e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e9f0 tx=0x7fe9e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e800cd80 con 0x7fe9fc0af170 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9e8004020 con 0x7fe9fc0af170 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9e8004450 con 0x7fe9fc0af170 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc157a90 con 0x7fe9fc0af170 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.069+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1f3de0 con 0x7fe9fc0af170 2024-09-21T07:47:33.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.071+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc1f3870 con 0x7fe9fc0af170 2024-09-21T07:47:33.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.072+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fe9e80035f0 con 0x7fe9fc0af170 2024-09-21T07:47:33.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.073+0000 7fe9f3400640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe9dc078da0 0x7fe9dc07b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:33.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.073+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4943+0+0 (secure 0 0 0) 0x7fe9e809b780 con 0x7fe9fc0af170 2024-09-21T07:47:33.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.073+0000 7fe9fb400640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe9dc078da0 0x7fe9dc07b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:33.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.074+0000 7fe9fb400640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe9dc078da0 0x7fe9dc07b2b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec001720 tx=0x7fe9ec007e90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:33.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.077+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9fc1f3620 con 0x7fe9fc0af170 2024-09-21T07:47:33.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:33.275+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fe9fc143790 con 0x7fe9fc0af170 2024-09-21T07:47:34.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.070+0000 7fe9f3400640 1 -- 172.21.15.88:0/1198284702 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v43) ==== 97+0+0 (secure 0 0 0) 0x7fe9e8063440 con 0x7fe9fc0af170 2024-09-21T07:47:34.071 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:47:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe9dc078da0 msgr2=0x7fe9dc07b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe9dc078da0 0x7fe9dc07b2b0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec001720 tx=0x7fe9ec007e90 comp rx=0 tx=0).stop 2024-09-21T07:47:34.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 msgr2=0x7fe9fc1f30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 0x7fe9fc1f30e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe9e800e9f0 tx=0x7fe9e800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 shutdown_connections 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe9dc078da0 0x7fe9dc07b2b0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9fc15a0f0 0x7fe9fc143250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc0af170 0x7fe9fc1f30e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 --2- 172.21.15.88:0/1198284702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc0ad9c0 0x7fe9fc146700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 >> 172.21.15.88:0/1198284702 conn(0x7fe9fc0b1e40 msgr2=0x7fe9fc15bf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 shutdown_connections 2024-09-21T07:47:34.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.073+0000 7fea00c00640 1 -- 172.21.15.88:0/1198284702 wait complete. 2024-09-21T07:47:34.100 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:47:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 -- 172.21.15.88:0/3009439549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30200ad0e0 msgr2=0x7f30200ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 --2- 172.21.15.88:0/3009439549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30200ad0e0 0x7f30200ad4f0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f300c005fd0 tx=0x7f300c02a540 comp rx=0 tx=0).stop 2024-09-21T07:47:34.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 -- 172.21.15.88:0/3009439549 shutdown_connections 2024-09-21T07:47:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 --2- 172.21.15.88:0/3009439549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30200ad0e0 0x7f30200ad4f0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 --2- 172.21.15.88:0/3009439549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f302015d760 0x7f30201535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 -- 172.21.15.88:0/3009439549 >> 172.21.15.88:0/3009439549 conn(0x7f302006d730 msgr2=0x7f30200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 -- 172.21.15.88:0/3009439549 shutdown_connections 2024-09-21T07:47:34.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.323+0000 7f3025800640 1 -- 172.21.15.88:0/3009439549 wait complete. 2024-09-21T07:47:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.325+0000 7f3025800640 1 Processor -- start 2024-09-21T07:47:34.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.325+0000 7f3025800640 1 -- start start 2024-09-21T07:47:34.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.326+0000 7f3025800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 0x7f3020153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.326+0000 7f3025800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f302015d760 0x7f3020158940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f3025800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3020160310 0x7f3020158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f3025800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f302006f220 con 0x7f30200ad0e0 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f3025800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f302006f0a0 con 0x7f3020160310 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f3025800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30200b4c20 con 0x7f302015d760 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 0x7f3020153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 0x7f3020153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49470/0 (socket says 172.21.15.88:49470) 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 -- 172.21.15.88:0/1630763079 learned_addr learned my addr 172.21.15.88:0/1630763079 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301f400640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3020160310 0x7f3020158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301e000640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f302015d760 0x7f3020158940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 -- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f302015d760 msgr2=0x7f3020158940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f302015d760 0x7f3020158940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 -- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3020160310 msgr2=0x7f3020158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3020160310 0x7f3020158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301ea00640 1 -- 172.21.15.88:0/1630763079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30201f5690 con 0x7f30200ad0e0 2024-09-21T07:47:34.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.327+0000 7f301f400640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3020160310 0x7f3020158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:34.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.328+0000 7f301ea00640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 0x7f3020153e40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f301000e9f0 tx=0x7f301000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:34.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.328+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f301000cd80 con 0x7f30200ad0e0 2024-09-21T07:47:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.328+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3010004020 con 0x7f30200ad0e0 2024-09-21T07:47:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.328+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3010004470 con 0x7f30200ad0e0 2024-09-21T07:47:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.328+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3020154790 con 0x7f30200ad0e0 2024-09-21T07:47:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.328+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3020154d50 con 0x7f30200ad0e0 2024-09-21T07:47:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.329+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3020160fa0 con 0x7f30200ad0e0 2024-09-21T07:47:34.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.330+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f30100035f0 con 0x7f30200ad0e0 2024-09-21T07:47:34.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.331+0000 7f3016a00640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3000078d40 0x7f300007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.331+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f301009b7b0 con 0x7f30200ad0e0 2024-09-21T07:47:34.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.332+0000 7f301e000640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3000078d40 0x7f300007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.332+0000 7f301e000640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3000078d40 0x7f300007b250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f300c033000 tx=0x7f300c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:34.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.336+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3010063b10 con 0x7f30200ad0e0 2024-09-21T07:47:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.589+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f3020160850 con 0x7f30200ad0e0 2024-09-21T07:47:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.589+0000 7f3016a00640 1 -- 172.21.15.88:0/1630763079 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v9) ==== 80+0+5304 (secure 0 0 0) 0x7f3010063410 con 0x7f30200ad0e0 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:47:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:47:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:47:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:47:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:47:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:47:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:47:34.606 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.591+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3000078d40 msgr2=0x7f300007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.591+0000 7f3025800640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3000078d40 0x7f300007b250 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f300c033000 tx=0x7f300c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.591+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 msgr2=0x7f3020153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.591+0000 7f3025800640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 0x7f3020153e40 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f301000e9f0 tx=0x7f301000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 shutdown_connections 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3000078d40 0x7f300007b250 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f302015d760 0x7f3020158940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3020160310 0x7f3020158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 --2- 172.21.15.88:0/1630763079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30200ad0e0 0x7f3020153e40 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 >> 172.21.15.88:0/1630763079 conn(0x7f302006d730 msgr2=0x7f302006f3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:34.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 shutdown_connections 2024-09-21T07:47:34.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.592+0000 7f3025800640 1 -- 172.21.15.88:0/1630763079 wait complete. 2024-09-21T07:47:34.619 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 -- 172.21.15.88:0/172521154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 msgr2=0x7fec600ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 --2- 172.21.15.88:0/172521154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec600ad4f0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fec60075250 tx=0x7fec4c02a560 comp rx=0 tx=0).stop 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 -- 172.21.15.88:0/172521154 shutdown_connections 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 --2- 172.21.15.88:0/172521154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec600ad4f0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 --2- 172.21.15.88:0/172521154 >> v2:172.21.15.175:3300/0 conn(0x7fec6015d760 0x7fec601535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 -- 172.21.15.88:0/172521154 >> 172.21.15.88:0/172521154 conn(0x7fec6006d730 msgr2=0x7fec600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 -- 172.21.15.88:0/172521154 shutdown_connections 2024-09-21T07:47:34.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.836+0000 7fec67a00640 1 -- 172.21.15.88:0/172521154 wait complete. 2024-09-21T07:47:34.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.839+0000 7fec67a00640 1 Processor -- start 2024-09-21T07:47:34.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.839+0000 7fec67a00640 1 -- start start 2024-09-21T07:47:34.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec67a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec60159cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec67a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec6015d760 0x7fec601f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec67a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60160310 0x7fec601f5700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec67a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec600b4c20 con 0x7fec600ad0e0 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec67a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fec60153c70 con 0x7fec60160310 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec67a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fec600b4da0 con 0x7fec6015d760 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec65000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec60159cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec65000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec60159cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49480/0 (socket says 172.21.15.88:49480) 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.840+0000 7fec65000640 1 -- 172.21.15.88:0/1235134757 learned_addr learned my addr 172.21.15.88:0/1235134757 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec5fe00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec6015d760 0x7fec601f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65000640 1 -- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec6015d760 msgr2=0x7fec601f51c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60160310 0x7fec601f5700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65000640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec6015d760 0x7fec601f51c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65000640 1 -- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60160310 msgr2=0x7fec601f5700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65000640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60160310 0x7fec601f5700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65000640 1 -- 172.21.15.88:0/1235134757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec6015ba60 con 0x7fec600ad0e0 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec5fe00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec6015d760 0x7fec601f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60160310 0x7fec601f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec65000640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec60159cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fec5000ea50 tx=0x7fec5000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec5000cd60 con 0x7fec600ad0e0 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fec50011470 con 0x7fec600ad0e0 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec500118a0 con 0x7fec600ad0e0 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.841+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec60156680 con 0x7fec600ad0e0 2024-09-21T07:47:34.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.842+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec60156c40 con 0x7fec600ad0e0 2024-09-21T07:47:34.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.843+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec60160ff0 con 0x7fec600ad0e0 2024-09-21T07:47:34.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.844+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fec50011a40 con 0x7fec600ad0e0 2024-09-21T07:47:34.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.845+0000 7fec5cc00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078d70 0x7fec4007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:34.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.845+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7fec5009ba80 con 0x7fec600ad0e0 2024-09-21T07:47:34.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.845+0000 7fec5fe00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078d70 0x7fec4007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:34.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.846+0000 7fec5fe00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078d70 0x7fec4007b280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec4c033000 tx=0x7fec4c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:34.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:34.849+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fec50016030 con 0x7fec600ad0e0 2024-09-21T07:47:35.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.035+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fec60152db0 con 0x7fec600ad0e0 2024-09-21T07:47:35.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.035+0000 7fec5cc00640 1 -- 172.21.15.88:0/1235134757 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v43) ==== 84+0+65 (secure 0 0 0) 0x7fec50063e60 con 0x7fec600ad0e0 2024-09-21T07:47:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:35.038 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:47:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:47:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:47:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:35.039 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078d70 msgr2=0x7fec4007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078d70 0x7fec4007b280 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fec4c033000 tx=0x7fec4c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 msgr2=0x7fec60159cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec60159cf0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fec5000ea50 tx=0x7fec5000ef10 comp rx=0 tx=0).stop 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 shutdown_connections 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fec40078d70 0x7fec4007b280 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fec6015d760 0x7fec601f51c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fec60160310 0x7fec601f5700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 --2- 172.21.15.88:0/1235134757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec600ad0e0 0x7fec60159cf0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 >> 172.21.15.88:0/1235134757 conn(0x7fec6006d730 msgr2=0x7fec600cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:35.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 shutdown_connections 2024-09-21T07:47:35.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.038+0000 7fec67a00640 1 -- 172.21.15.88:0/1235134757 wait complete. 2024-09-21T07:47:35.067 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:47:35.067 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:47:35.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 -- 172.21.15.88:0/828442815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd80af100 msgr2=0x7f0cd80af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:35.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 --2- 172.21.15.88:0/828442815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd80af100 0x7f0cd80af510 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0005fd0 tx=0x7f0cd002aa90 comp rx=0 tx=0).stop 2024-09-21T07:47:35.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 -- 172.21.15.88:0/828442815 shutdown_connections 2024-09-21T07:47:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 --2- 172.21.15.88:0/828442815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd80af100 0x7f0cd80af510 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 --2- 172.21.15.88:0/828442815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd806d4b0 0x7f0cd80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 -- 172.21.15.88:0/828442815 >> 172.21.15.88:0/828442815 conn(0x7f0cd80b2540 msgr2=0x7f0cd80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 -- 172.21.15.88:0/828442815 shutdown_connections 2024-09-21T07:47:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.287+0000 7f0cdd800640 1 -- 172.21.15.88:0/828442815 wait complete. 2024-09-21T07:47:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.290+0000 7f0cdd800640 1 Processor -- start 2024-09-21T07:47:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.290+0000 7f0cdd800640 1 -- start start 2024-09-21T07:47:35.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cdd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd80af100 0x7f0cd81f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:35.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cdd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd80cc830 0x7f0cd81f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:35.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cdd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 0x7f0cd815b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cdd800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd80b3670 con 0x7f0cd815ee00 2024-09-21T07:47:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cdd800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cd80b34f0 con 0x7f0cd80cc830 2024-09-21T07:47:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cdd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cd80b37f0 con 0x7f0cd80af100 2024-09-21T07:47:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cd7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 0x7f0cd815b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cd6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd80af100 0x7f0cd81f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cd7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 0x7f0cd815b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49510/0 (socket says 172.21.15.88:49510) 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.291+0000 7f0cd7400640 1 -- 172.21.15.88:0/1223551995 learned_addr learned my addr 172.21.15.88:0/1223551995 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd6000640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd80cc830 0x7f0cd81f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd7400640 1 -- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd80af100 msgr2=0x7f0cd81f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd7400640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd80af100 0x7f0cd81f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd7400640 1 -- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd80cc830 msgr2=0x7f0cd81f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd7400640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd80cc830 0x7f0cd81f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:35.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd7400640 1 -- 172.21.15.88:0/1223551995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cd815d140 con 0x7f0cd815ee00 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd6a00640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd80af100 0x7f0cd81f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.292+0000 7f0cd7400640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 0x7f0cd815b8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009d90 tx=0x7f0cd0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.293+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0044070 con 0x7f0cd815ee00 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.293+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cd0004040 con 0x7f0cd815ee00 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.293+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cd81f5400 con 0x7f0cd815ee00 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.293+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cd0004430 con 0x7f0cd815ee00 2024-09-21T07:47:35.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.293+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cd815d690 con 0x7f0cd815ee00 2024-09-21T07:47:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.294+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cd80b1c20 con 0x7f0cd815ee00 2024-09-21T07:47:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.295+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f0cd0047070 con 0x7f0cd815ee00 2024-09-21T07:47:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.296+0000 7f0ccea00640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0cb8078d40 0x7f0cb807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.296+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4857+0+0 (secure 0 0 0) 0x7f0cd00bbc60 con 0x7f0cd815ee00 2024-09-21T07:47:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.297+0000 7f0cd6a00640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0cb8078d40 0x7f0cb807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.298+0000 7f0cd6a00640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0cb8078d40 0x7f0cb807b250 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8001b10 tx=0x7f0cc8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:35.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.300+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0cd80b1c20 con 0x7f0cd815ee00 2024-09-21T07:47:35.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:35.485+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0cd80b3d10 con 0x7f0cd815ee00 2024-09-21T07:47:36.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.049+0000 7f0ccea00640 1 -- 172.21.15.88:0/1223551995 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v44) ==== 138+0+0 (secure 0 0 0) 0x7f0cd81f5400 con 0x7f0cd815ee00 2024-09-21T07:47:36.051 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:47:36.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0cb8078d40 msgr2=0x7f0cb807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0cb8078d40 0x7f0cb807b250 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f0cc8001b10 tx=0x7f0cc8009040 comp rx=0 tx=0).stop 2024-09-21T07:47:36.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 msgr2=0x7f0cd815b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 0x7f0cd815b8b0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f0cd0009d90 tx=0x7f0cd0003ce0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 shutdown_connections 2024-09-21T07:47:36.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0cb8078d40 0x7f0cb807b250 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cd815ee00 0x7f0cd815b8b0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cd80cc830 0x7f0cd81f94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 --2- 172.21.15.88:0/1223551995 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cd80af100 0x7f0cd81f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 >> 172.21.15.88:0/1223551995 conn(0x7f0cd80b2540 msgr2=0x7f0cd80b1510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:36.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 shutdown_connections 2024-09-21T07:47:36.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.052+0000 7f0cdd800640 1 -- 172.21.15.88:0/1223551995 wait complete. 2024-09-21T07:47:36.085 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:47:36.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 -- 172.21.15.88:0/1265669864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4145550 msgr2=0x7f27f406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 --2- 172.21.15.88:0/1265669864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4145550 0x7f27f406fb40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f27e4005fd0 tx=0x7f27e402a670 comp rx=0 tx=0).stop 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 -- 172.21.15.88:0/1265669864 shutdown_connections 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 --2- 172.21.15.88:0/1265669864 >> v2:172.21.15.175:3300/0 conn(0x7f27f4145a40 0x7f27f4073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 --2- 172.21.15.88:0/1265669864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4145550 0x7f27f406fb40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 -- 172.21.15.88:0/1265669864 >> 172.21.15.88:0/1265669864 conn(0x7f27f406d1c0 msgr2=0x7f27f406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 -- 172.21.15.88:0/1265669864 shutdown_connections 2024-09-21T07:47:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.311+0000 7f27fb000640 1 -- 172.21.15.88:0/1265669864 wait complete. 2024-09-21T07:47:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.313+0000 7f27fb000640 1 Processor -- start 2024-09-21T07:47:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.314+0000 7f27fb000640 1 -- start start 2024-09-21T07:47:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27fb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4073a80 0x7f27f4137570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27fb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4145550 0x7f27f413d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27fb000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 0x7f27f413b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27fb000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27f4160a30 con 0x7f27f4145a40 2024-09-21T07:47:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27fb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27f40ce940 con 0x7f27f4145550 2024-09-21T07:47:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27fb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27f4160bb0 con 0x7f27f4073a80 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27f9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 0x7f27f413b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27f9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 0x7f27f413b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39014/0 (socket says 172.21.15.88:39014) 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.315+0000 7f27f9000640 1 -- 172.21.15.88:0/266740659 learned_addr learned my addr 172.21.15.88:0/266740659 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f3e00640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4073a80 0x7f27f4137570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27eb400640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4145550 0x7f27f413d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f9000640 1 -- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4073a80 msgr2=0x7f27f4137570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f9000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4073a80 0x7f27f4137570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f9000640 1 -- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4145550 msgr2=0x7f27f413d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f9000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4145550 0x7f27f413d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f9000640 1 -- 172.21.15.88:0/266740659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27f413e0c0 con 0x7f27f4145a40 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.316+0000 7f27f9000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 0x7f27f413b320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27ec002230 tx=0x7f27ec00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.317+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec003b70 con 0x7f27f4145a40 2024-09-21T07:47:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.317+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27ec003d10 con 0x7f27f4145a40 2024-09-21T07:47:36.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.317+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27f413e3a0 con 0x7f27f4145a40 2024-09-21T07:47:36.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.317+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27ec005330 con 0x7f27f4145a40 2024-09-21T07:47:36.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.317+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27f4137ab0 con 0x7f27f4145a40 2024-09-21T07:47:36.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.318+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27f414d970 con 0x7f27f4145a40 2024-09-21T07:47:36.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.319+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f27ec0056e0 con 0x7f27f4145a40 2024-09-21T07:47:36.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.320+0000 7f27f0c00640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f27d0078d40 0x7f27d007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.320+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(44..44 src has 1..44) ==== 5227+0+0 (secure 0 0 0) 0x7f27ec09ad70 con 0x7f27f4145a40 2024-09-21T07:47:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.321+0000 7f27f3e00640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f27d0078d40 0x7f27d007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:36.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.321+0000 7f27f3e00640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f27d0078d40 0x7f27d007b250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f27e4033000 tx=0x7f27e4004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:36.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.325+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27f4073e90 con 0x7f27f4145a40 2024-09-21T07:47:36.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:36.512+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f27f413de50 con 0x7f27f4145a40 2024-09-21T07:47:37.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.061+0000 7f27f0c00640 1 -- 172.21.15.88:0/266740659 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v45) ==== 170+0+0 (secure 0 0 0) 0x7f27ec062a30 con 0x7f27f4145a40 2024-09-21T07:47:37.064 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:47:37.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f27d0078d40 msgr2=0x7f27d007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f27d0078d40 0x7f27d007b250 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f27e4033000 tx=0x7f27e4004730 comp rx=0 tx=0).stop 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 msgr2=0x7f27f413b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 0x7f27f413b320 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f27ec002230 tx=0x7f27ec00cb90 comp rx=0 tx=0).stop 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 shutdown_connections 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f27d0078d40 0x7f27d007b250 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4145a40 0x7f27f413b320 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4145550 0x7f27f413d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 --2- 172.21.15.88:0/266740659 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27f4073a80 0x7f27f4137570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 >> 172.21.15.88:0/266740659 conn(0x7f27f406d1c0 msgr2=0x7f27f4068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:37.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.065+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 shutdown_connections 2024-09-21T07:47:37.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.066+0000 7f27fb000640 1 -- 172.21.15.88:0/266740659 wait complete. 2024-09-21T07:47:37.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 -- 172.21.15.88:0/1445166620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 msgr2=0x7f7224156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 --2- 172.21.15.88:0/1445166620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f7224156ca0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f7214005fd0 tx=0x7f721402a8d0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 -- 172.21.15.88:0/1445166620 shutdown_connections 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 --2- 172.21.15.88:0/1445166620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f7224156ca0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 -- 172.21.15.88:0/1445166620 >> 172.21.15.88:0/1445166620 conn(0x7f722406d620 msgr2=0x7f72240b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 -- 172.21.15.88:0/1445166620 shutdown_connections 2024-09-21T07:47:37.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.336+0000 7f7229400640 1 -- 172.21.15.88:0/1445166620 wait complete. 2024-09-21T07:47:37.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.339+0000 7f7229400640 1 Processor -- start 2024-09-21T07:47:37.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.339+0000 7f7229400640 1 -- start start 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f7229400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f72241f4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f7229400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 0x7f722409d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f7229400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f722415d6e0 0x7f722409dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f7229400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f72240af7e0 con 0x7f72240cd1d0 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f7229400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7224070370 con 0x7f72240b3f30 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f7229400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72240af960 con 0x7f722415d6e0 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.340+0000 7f721b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 0x7f722409d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 0x7f722409d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39018/0 (socket says 172.21.15.88:39018) 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 -- 172.21.15.88:0/422039209 learned_addr learned my addr 172.21.15.88:0/422039209 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f7228a00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f722415d6e0 0x7f722409dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f7223e00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f72241f4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 -- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f722415d6e0 msgr2=0x7f722409dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f722415d6e0 0x7f722409dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 -- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 msgr2=0x7f72241f4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f72241f4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 -- 172.21.15.88:0/422039209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f722409b160 con 0x7f72240cd1d0 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f7228a00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f722415d6e0 0x7f722409dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f7223e00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f72241f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f721b400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 0x7f722409d850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f720c001eb0 tx=0x7f720c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.341+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c017070 con 0x7f72240cd1d0 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.342+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f720c0053c0 con 0x7f72240cd1d0 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.342+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f720c003830 con 0x7f72240cd1d0 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.342+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f722409a8c0 con 0x7f72240cd1d0 2024-09-21T07:47:37.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.342+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f722409ae80 con 0x7f72240cd1d0 2024-09-21T07:47:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.343+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f722409a630 con 0x7f72240cd1d0 2024-09-21T07:47:37.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.344+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f720c01a070 con 0x7f72240cd1d0 2024-09-21T07:47:37.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.345+0000 7f7220c00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7200078d40 0x7f720007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:37.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.345+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(45..45 src has 1..45) ==== 5609+0+0 (secure 0 0 0) 0x7f720c09a5f0 con 0x7f72240cd1d0 2024-09-21T07:47:37.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.346+0000 7f7223e00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7200078d40 0x7f720007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:37.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.346+0000 7f7223e00640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7200078d40 0x7f720007b250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7214034000 tx=0x7f7214004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:37.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.350+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f720c0625e0 con 0x7f72240cd1d0 2024-09-21T07:47:37.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:37.570+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f7224156ca0 con 0x7f72240cd1d0 2024-09-21T07:47:38.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.074+0000 7f7220c00640 1 -- 172.21.15.88:0/422039209 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v23) ==== 172+0+0 (secure 0 0 0) 0x7f720c061ee0 con 0x7f72240cd1d0 2024-09-21T07:47:38.079 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 6 and data pool 7 2024-09-21T07:47:38.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.079+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7200078d40 msgr2=0x7f720007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.079+0000 7f7229400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7200078d40 0x7f720007b250 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f7214034000 tx=0x7f7214004ab0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.079+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 msgr2=0x7f722409d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.079+0000 7f7229400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 0x7f722409d850 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f720c001eb0 tx=0x7f720c00c890 comp rx=0 tx=0).stop 2024-09-21T07:47:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.082+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 shutdown_connections 2024-09-21T07:47:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.082+0000 7f7229400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7200078d40 0x7f720007b250 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.082+0000 7f7229400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f722415d6e0 0x7f722409dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.082+0000 7f7229400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72240cd1d0 0x7f722409d850 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.082+0000 7f7229400640 1 --2- 172.21.15.88:0/422039209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72240b3f30 0x7f72241f4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.082+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 >> 172.21.15.88:0/422039209 conn(0x7f722406d620 msgr2=0x7f72240af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.085+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 shutdown_connections 2024-09-21T07:47:38.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.086+0000 7f7229400640 1 -- 172.21.15.88:0/422039209 wait complete. 2024-09-21T07:47:38.126 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e8a00640 1 -- 172.21.15.88:0/3176355134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c8004d40 msgr2=0x7f28c8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e8a00640 1 --2- 172.21.15.88:0/3176355134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28c8004d40 0x7f28c8005150 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f28dc009b20 tx=0x7f28c8005830 comp rx=0 tx=0).stop 2024-09-21T07:47:38.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e3200640 1 -- 172.21.15.88:0/3176355134 reap_dead start 2024-09-21T07:47:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e8a00640 1 -- 172.21.15.88:0/3176355134 shutdown_connections 2024-09-21T07:47:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e8a00640 1 -- 172.21.15.88:0/3176355134 >> 172.21.15.88:0/3176355134 conn(0x7f28e40b1e40 msgr2=0x7f28e40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e8a00640 1 -- 172.21.15.88:0/3176355134 shutdown_connections 2024-09-21T07:47:38.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.343+0000 7f28e8a00640 1 -- 172.21.15.88:0/3176355134 wait complete. 2024-09-21T07:47:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.345+0000 7f28e8a00640 1 Processor -- start 2024-09-21T07:47:38.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.346+0000 7f28e8a00640 1 -- start start 2024-09-21T07:47:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28e8a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c8004d40 0x7f28e40ca410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28e8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 0x7f28e409cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28e8a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28e40ceee0 0x7f28e409d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28e8a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28e415d7f0 con 0x7f28e40cead0 2024-09-21T07:47:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28e8a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28e415d670 con 0x7f28c8004d40 2024-09-21T07:47:38.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28e8a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28e415d970 con 0x7f28e40ceee0 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28d9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 0x7f28e409cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28d9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 0x7f28e409cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39032/0 (socket says 172.21.15.88:39032) 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.347+0000 7f28d9e00640 1 -- 172.21.15.88:0/2405738736 learned_addr learned my addr 172.21.15.88:0/2405738736 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28e2800640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c8004d40 0x7f28e40ca410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28e3200640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28e40ceee0 0x7f28e409d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28d9e00640 1 -- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28e40ceee0 msgr2=0x7f28e409d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28d9e00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28e40ceee0 0x7f28e409d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28d9e00640 1 -- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c8004d40 msgr2=0x7f28e40ca410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28d9e00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c8004d40 0x7f28e40ca410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28d9e00640 1 -- 172.21.15.88:0/2405738736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28e41f5390 con 0x7f28e40cead0 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28d9e00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 0x7f28e409cfc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0065c0 tx=0x7f28cc0066b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc015070 con 0x7f28e40cead0 2024-09-21T07:47:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28cc011920 con 0x7f28e40cead0 2024-09-21T07:47:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28e41f5670 con 0x7f28e40cead0 2024-09-21T07:47:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28cc011d70 con 0x7f28e40cead0 2024-09-21T07:47:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.348+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28e41f5c30 con 0x7f28e40cead0 2024-09-21T07:47:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.349+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28e40705d0 con 0x7f28e40cead0 2024-09-21T07:47:38.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.351+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f28cc018070 con 0x7f28e40cead0 2024-09-21T07:47:38.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.352+0000 7f28db400640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f28c0078d70 0x7f28c007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.352+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f28cc09bfe0 con 0x7f28e40cead0 2024-09-21T07:47:38.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.352+0000 7f28e2800640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f28c0078d70 0x7f28c007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.353+0000 7f28e2800640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f28c0078d70 0x7f28c007b280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28d4005fd0 tx=0x7f28d4009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.357+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28cc064050 con 0x7f28e40cead0 2024-09-21T07:47:38.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.540+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28e40cf300 con 0x7f28e40cead0 2024-09-21T07:47:38.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.542+0000 7f28db400640 1 -- 172.21.15.88:0/2405738736 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16694 (secure 0 0 0) 0x7f28cc063950 con 0x7f28e40cead0 2024-09-21T07:47:38.544 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:38.544 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:47:37.572691+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:35.498681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:36.524995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f28c0078d70 msgr2=0x7f28c007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f28c0078d70 0x7f28c007b280 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f28d4005fd0 tx=0x7f28d4009500 comp rx=0 tx=0).stop 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 msgr2=0x7f28e409cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 0x7f28e409cfc0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f28cc0065c0 tx=0x7f28cc0066b0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 shutdown_connections 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f28c0078d70 0x7f28c007b280 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28e40ceee0 0x7f28e409d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28e40cead0 0x7f28e409cfc0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 --2- 172.21.15.88:0/2405738736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28c8004d40 0x7f28e40ca410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 >> 172.21.15.88:0/2405738736 conn(0x7f28e40b1e40 msgr2=0x7f28e4155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 shutdown_connections 2024-09-21T07:47:38.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.544+0000 7f28e8a00640 1 -- 172.21.15.88:0/2405738736 wait complete. 2024-09-21T07:47:38.569 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 -- 172.21.15.88:0/2429968609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 msgr2=0x7f39a40d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 --2- 172.21.15.88:0/2429968609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a40d1090 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f39a408e630 tx=0x7f399002a4f0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 -- 172.21.15.88:0/2429968609 shutdown_connections 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 --2- 172.21.15.88:0/2429968609 >> v2:172.21.15.175:3300/0 conn(0x7f39a40d15d0 0x7f39a4162a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 --2- 172.21.15.88:0/2429968609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a40d1090 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 --2- 172.21.15.88:0/2429968609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 0x7f39a40b3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 -- 172.21.15.88:0/2429968609 >> 172.21.15.88:0/2429968609 conn(0x7f39a40d4680 msgr2=0x7f39a40d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.786+0000 7f39aa400640 1 -- 172.21.15.88:0/2429968609 shutdown_connections 2024-09-21T07:47:38.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.787+0000 7f39aa400640 1 -- 172.21.15.88:0/2429968609 wait complete. 2024-09-21T07:47:38.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.789+0000 7f39aa400640 1 Processor -- start 2024-09-21T07:47:38.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.789+0000 7f39aa400640 1 -- start start 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39aa400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 0x7f39a4161bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39aa400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a40d15d0 0x7f39a4151b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39aa400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a406edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39aa400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39a4163330 con 0x7f39a415ee50 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39aa400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39a41631b0 con 0x7f39a40d15d0 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39aa400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f39a41634b0 con 0x7f39a40b9780 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a406edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.790+0000 7f39a3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a406edb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39052/0 (socket says 172.21.15.88:39052) 2024-09-21T07:47:38.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 -- 172.21.15.88:0/3916393940 learned_addr learned my addr 172.21.15.88:0/3916393940 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 0x7f39a4161bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a2a00640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a40d15d0 0x7f39a4151b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 -- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 msgr2=0x7f39a4161bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 0x7f39a4161bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 -- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a40d15d0 msgr2=0x7f39a4151b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a40d15d0 0x7f39a4151b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 -- 172.21.15.88:0/3916393940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39a4162290 con 0x7f39a415ee50 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a2a00640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a40d15d0 0x7f39a4151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:38.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 0x7f39a4161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:38.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f39a3e00640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f399c00ba10 tx=0x7f399c00bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.791+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c019070 con 0x7f39a415ee50 2024-09-21T07:47:38.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.792+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f399c00cbb0 con 0x7f39a415ee50 2024-09-21T07:47:38.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.792+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39a4162570 con 0x7f39a415ee50 2024-09-21T07:47:38.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.792+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f399c014440 con 0x7f39a415ee50 2024-09-21T07:47:38.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.792+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39a4160dc0 con 0x7f39a415ee50 2024-09-21T07:47:38.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.793+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39a408d100 con 0x7f39a415ee50 2024-09-21T07:47:38.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.794+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f399c01c070 con 0x7f39a415ee50 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.795+0000 7f399b400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3984078d40 0x7f398407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:38.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.796+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(46..46 src has 1..46) ==== 5709+0+0 (secure 0 0 0) 0x7f399c09ae40 con 0x7f39a415ee50 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.796+0000 7f39a3400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3984078d40 0x7f398407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:38.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.797+0000 7f39a3400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3984078d40 0x7f398407b250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3994005860 tx=0x7f3994005da0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:38.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.800+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39a408d100 con 0x7f39a415ee50 2024-09-21T07:47:38.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.985+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f39a415f370 con 0x7f39a415ee50 2024-09-21T07:47:38.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.987+0000 7f399b400640 1 -- 172.21.15.88:0/3916393940 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) ==== 74+0+16694 (secure 0 0 0) 0x7f399c062ee0 con 0x7f39a415ee50 2024-09-21T07:47:38.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:38.989 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:47:37.572691+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:35.498681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:36.524995+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:38.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3984078d40 msgr2=0x7f398407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3984078d40 0x7f398407b250 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f3994005860 tx=0x7f3994005da0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 msgr2=0x7f39a406edb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:38.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a406edb0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f399c00ba10 tx=0x7f399c00bed0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 shutdown_connections 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3984078d40 0x7f398407b250 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39a415ee50 0x7f39a406edb0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f39a40d15d0 0x7f39a4151b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 --2- 172.21.15.88:0/3916393940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39a40b9780 0x7f39a4161bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 >> 172.21.15.88:0/3916393940 conn(0x7f39a40d4680 msgr2=0x7f39a40d0de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 shutdown_connections 2024-09-21T07:47:38.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:38.989+0000 7f39aa400640 1 -- 172.21.15.88:0/3916393940 wait complete. 2024-09-21T07:47:39.013 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:47:39.050 INFO:tasks.ceph.mds.c.smithi088.stderr:2024-09-21T07:47:39.048+0000 7f2efd600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.237+0000 7f6019400640 1 -- 172.21.15.88:0/2941975327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014073a80 msgr2=0x7f60140cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.237+0000 7f6019400640 1 --2- 172.21.15.88:0/2941975327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014073a80 0x7f60140cd610 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f6014160280 tx=0x7f600c02ab40 comp rx=0 tx=0).stop 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.237+0000 7f6019400640 1 -- 172.21.15.88:0/2941975327 shutdown_connections 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.237+0000 7f6019400640 1 --2- 172.21.15.88:0/2941975327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014073a80 0x7f60140cd610 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.237+0000 7f6019400640 1 --2- 172.21.15.88:0/2941975327 >> v2:172.21.15.175:3300/0 conn(0x7f6014145a40 0x7f6014073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.238+0000 7f6019400640 1 -- 172.21.15.88:0/2941975327 >> 172.21.15.88:0/2941975327 conn(0x7f601406d1c0 msgr2=0x7f601406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:39.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.238+0000 7f6019400640 1 -- 172.21.15.88:0/2941975327 shutdown_connections 2024-09-21T07:47:39.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.238+0000 7f6019400640 1 -- 172.21.15.88:0/2941975327 wait complete. 2024-09-21T07:47:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.240+0000 7f6019400640 1 Processor -- start 2024-09-21T07:47:39.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.240+0000 7f6019400640 1 -- start start 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.241+0000 7f6019400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6014073a80 0x7f60140b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.241+0000 7f6019400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6014145550 0x7f6014137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.241+0000 7f6019400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 0x7f6014139340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.241+0000 7f6019400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60140ce090 con 0x7f6014145a40 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.241+0000 7f6019400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60140cdf10 con 0x7f6014073a80 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.241+0000 7f6019400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60140ce210 con 0x7f6014145550 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 0x7f6014139340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 0x7f6014139340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39066/0 (socket says 172.21.15.88:39066) 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 -- 172.21.15.88:0/1627767011 learned_addr learned my addr 172.21.15.88:0/1627767011 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:39.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6011e00640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6014145550 0x7f6014137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6012800640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6014073a80 0x7f60140b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 -- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6014145550 msgr2=0x7f6014137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6014145550 0x7f6014137ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 -- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6014073a80 msgr2=0x7f60140b00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6014073a80 0x7f60140b00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 -- 172.21.15.88:0/1627767011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60141381e0 con 0x7f6014145a40 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6012800640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6014073a80 0x7f60140b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6011e00640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6014145550 0x7f6014137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.242+0000 7f6013200640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 0x7f6014139340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f600c009d60 tx=0x7f600c02c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.243+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c044070 con 0x7f6014145a40 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.243+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f600c004250 con 0x7f6014145a40 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.243+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f600c004680 con 0x7f6014145a40 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.243+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6014138580 con 0x7f6014145a40 2024-09-21T07:47:39.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.243+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6014138b40 con 0x7f6014145a40 2024-09-21T07:47:39.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.244+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60140b2e20 con 0x7f6014145a40 2024-09-21T07:47:39.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.245+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f600c036400 con 0x7f6014145a40 2024-09-21T07:47:39.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.246+0000 7f600aa00640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ff4078d70 0x7f5ff407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:39.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.246+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f600c0bba60 con 0x7f6014145a40 2024-09-21T07:47:39.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.247+0000 7f6012800640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ff4078d70 0x7f5ff407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:39.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.247+0000 7f6012800640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ff4078d70 0x7f5ff407b280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6004007fd0 tx=0x7f6004009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:39.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.251+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60141381e0 con 0x7f6014145a40 2024-09-21T07:47:39.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:39.472+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f60140ce730 con 0x7f6014145a40 2024-09-21T07:47:40.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.093+0000 7f600aa00640 1 -- 172.21.15.88:0/1627767011 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v25) ==== 118+0+0 (secure 0 0 0) 0x7f600c032030 con 0x7f6014145a40 2024-09-21T07:47:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.096+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ff4078d70 msgr2=0x7f5ff407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.096+0000 7f6019400640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ff4078d70 0x7f5ff407b280 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f6004007fd0 tx=0x7f6004009040 comp rx=0 tx=0).stop 2024-09-21T07:47:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.096+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 msgr2=0x7f6014139340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.096+0000 7f6019400640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 0x7f6014139340 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f600c009d60 tx=0x7f600c02c8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 shutdown_connections 2024-09-21T07:47:40.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5ff4078d70 0x7f5ff407b280 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6014145550 0x7f6014137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6014145a40 0x7f6014139340 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 --2- 172.21.15.88:0/1627767011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6014073a80 0x7f60140b00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 >> 172.21.15.88:0/1627767011 conn(0x7f601406d1c0 msgr2=0x7f601406fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:40.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 shutdown_connections 2024-09-21T07:47:40.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.097+0000 7f6019400640 1 -- 172.21.15.88:0/1627767011 wait complete. 2024-09-21T07:47:40.125 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:47:40.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.390+0000 7f2266800640 1 -- 172.21.15.88:0/3838157924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260145550 msgr2=0x7f226006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.390+0000 7f2266800640 1 --2- 172.21.15.88:0/3838157924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260145550 0x7f226006fb40 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f2250005fd0 tx=0x7f225002a670 comp rx=0 tx=0).stop 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.390+0000 7f2266800640 1 -- 172.21.15.88:0/3838157924 shutdown_connections 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.390+0000 7f2266800640 1 --2- 172.21.15.88:0/3838157924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2260145a40 0x7f2260073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.390+0000 7f2266800640 1 --2- 172.21.15.88:0/3838157924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260145550 0x7f226006fb40 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.390+0000 7f2266800640 1 -- 172.21.15.88:0/3838157924 >> 172.21.15.88:0/3838157924 conn(0x7f226006d1c0 msgr2=0x7f226006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.391+0000 7f2266800640 1 -- 172.21.15.88:0/3838157924 shutdown_connections 2024-09-21T07:47:40.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.391+0000 7f2266800640 1 -- 172.21.15.88:0/3838157924 wait complete. 2024-09-21T07:47:40.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.393+0000 7f2266800640 1 Processor -- start 2024-09-21T07:47:40.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.393+0000 7f2266800640 1 -- start start 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f2266800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 0x7f22601e8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f2266800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2260145550 0x7f226014cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f2266800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2260145a40 0x7f226014d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f2266800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2260160a30 con 0x7f2260073a80 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f2266800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22600ce940 con 0x7f2260145a40 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f2266800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2260160bb0 con 0x7f2260145550 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.394+0000 7f225f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 0x7f22601e8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 0x7f22601e8ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39078/0 (socket says 172.21.15.88:39078) 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 -- 172.21.15.88:0/1415124399 learned_addr learned my addr 172.21.15.88:0/1415124399 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:40.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225ea00640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2260145550 0x7f226014cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225fe00640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2260145a40 0x7f226014d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 -- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2260145550 msgr2=0x7f226014cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2260145550 0x7f226014cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 -- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2260145a40 msgr2=0x7f226014d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2260145a40 0x7f226014d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 -- 172.21.15.88:0/1415124399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22600adbf0 con 0x7f2260073a80 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.395+0000 7f225f400640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 0x7f22601e8ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2250000c00 tx=0x7f22500038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.396+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f225003e030 con 0x7f2260073a80 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.396+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2250003db0 con 0x7f2260073a80 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.396+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22601e8ff0 con 0x7f2260073a80 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.396+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2250035040 con 0x7f2260073a80 2024-09-21T07:47:40.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.396+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22600b4a10 con 0x7f2260073a80 2024-09-21T07:47:40.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.397+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22600addc0 con 0x7f2260073a80 2024-09-21T07:47:40.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.398+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f2250041030 con 0x7f2260073a80 2024-09-21T07:47:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.399+0000 7f2257400640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2240078d40 0x7f224007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.399+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f22500bb910 con 0x7f2260073a80 2024-09-21T07:47:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.400+0000 7f225ea00640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2240078d40 0x7f224007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:40.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.400+0000 7f225ea00640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2240078d40 0x7f224007b250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f22600b3bd0 tx=0x7f224c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.404+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f22500832f0 con 0x7f2260073a80 2024-09-21T07:47:40.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:40.628+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f22601610f0 con 0x7f2260073a80 2024-09-21T07:47:41.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.095+0000 7f2257400640 1 -- 172.21.15.88:0/1415124399 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v26) ==== 122+0+0 (secure 0 0 0) 0x7f2250087d20 con 0x7f2260073a80 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2240078d40 msgr2=0x7f224007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2240078d40 0x7f224007b250 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f22600b3bd0 tx=0x7f224c005f90 comp rx=0 tx=0).stop 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 msgr2=0x7f22601e8ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 0x7f22601e8ab0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2250000c00 tx=0x7f22500038a0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 shutdown_connections 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2240078d40 0x7f224007b250 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2260145550 0x7f226014cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2260145a40 0x7f226014d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 --2- 172.21.15.88:0/1415124399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2260073a80 0x7f22601e8ab0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.097+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 >> 172.21.15.88:0/1415124399 conn(0x7f226006d1c0 msgr2=0x7f226013ea90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:41.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.098+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 shutdown_connections 2024-09-21T07:47:41.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.098+0000 7f2266800640 1 -- 172.21.15.88:0/1415124399 wait complete. 2024-09-21T07:47:41.126 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:47:41.127 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 -- 172.21.15.88:0/3150391032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280aebc0 msgr2=0x7fae28152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 --2- 172.21.15.88:0/3150391032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280aebc0 0x7fae28152920 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fae18005fd0 tx=0x7fae1802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 -- 172.21.15.88:0/3150391032 shutdown_connections 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 --2- 172.21.15.88:0/3150391032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280aebc0 0x7fae28152920 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 -- 172.21.15.88:0/3150391032 >> 172.21.15.88:0/3150391032 conn(0x7fae2806d730 msgr2=0x7fae280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 -- 172.21.15.88:0/3150391032 shutdown_connections 2024-09-21T07:47:41.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.344+0000 7fae2f000640 1 -- 172.21.15.88:0/3150391032 wait complete. 2024-09-21T07:47:41.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.346+0000 7fae2f000640 1 Processor -- start 2024-09-21T07:47:41.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.347+0000 7fae2f000640 1 -- start start 2024-09-21T07:47:41.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae2f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae280aebc0 0x7fae28151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae2f000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 0x7fae280734c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae2f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae280b20b0 0x7fae28073a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae2f000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae280b4360 con 0x7fae280b2750 2024-09-21T07:47:41.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae2f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae280b41e0 con 0x7fae280aebc0 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae2f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae280b44e0 con 0x7fae280b20b0 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae27400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 0x7fae280734c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae27400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 0x7fae280734c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39098/0 (socket says 172.21.15.88:39098) 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.348+0000 7fae27400640 1 -- 172.21.15.88:0/2713329676 learned_addr learned my addr 172.21.15.88:0/2713329676 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27e00640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae280aebc0 0x7fae28151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae2d000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae280b20b0 0x7fae28073a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27400640 1 -- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae280b20b0 msgr2=0x7fae28073a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27400640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae280b20b0 0x7fae28073a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27400640 1 -- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae280aebc0 msgr2=0x7fae28151190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27400640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae280aebc0 0x7fae28151190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27400640 1 -- 172.21.15.88:0/2713329676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae28154470 con 0x7fae280b2750 2024-09-21T07:47:41.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae27400640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 0x7fae280734c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c400 tx=0x7fae1400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:41.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14017070 con 0x7fae280b2750 2024-09-21T07:47:41.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae140053c0 con 0x7fae280b2750 2024-09-21T07:47:41.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae14003850 con 0x7fae280b2750 2024-09-21T07:47:41.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae280b15d0 con 0x7fae280b2750 2024-09-21T07:47:41.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.349+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae28074120 con 0x7fae280b2750 2024-09-21T07:47:41.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.351+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae280b4990 con 0x7fae280b2750 2024-09-21T07:47:41.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.352+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fae1401a070 con 0x7fae280b2750 2024-09-21T07:47:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.353+0000 7fae1fe00640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fae08078d70 0x7fae0807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.353+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fae1409a5e0 con 0x7fae280b2750 2024-09-21T07:47:41.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.353+0000 7fae27e00640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fae08078d70 0x7fae0807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.354+0000 7fae27e00640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fae08078d70 0x7fae0807b280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae18033000 tx=0x7fae18004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:41.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.358+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae140625f0 con 0x7fae280b2750 2024-09-21T07:47:41.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.579+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae2815f460 con 0x7fae280b2750 2024-09-21T07:47:41.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.580+0000 7fae1fe00640 1 -- 172.21.15.88:0/2713329676 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7fae14061ef0 con 0x7fae280b2750 2024-09-21T07:47:41.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:41.582 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":26,"btime":"2024-09-21T07:47:41:094343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:47:41.583 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 26 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.582+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fae08078d70 msgr2=0x7fae0807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.582+0000 7fae2f000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fae08078d70 0x7fae0807b280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fae18033000 tx=0x7fae18004730 comp rx=0 tx=0).stop 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.582+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 msgr2=0x7fae280734c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.582+0000 7fae2f000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 0x7fae280734c0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fae1400c400 tx=0x7fae1400c8c0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 shutdown_connections 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fae08078d70 0x7fae0807b280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae280b20b0 0x7fae28073a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae280b2750 0x7fae280734c0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 --2- 172.21.15.88:0/2713329676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae280aebc0 0x7fae28151190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 >> 172.21.15.88:0/2713329676 conn(0x7fae2806d730 msgr2=0x7fae2815e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 shutdown_connections 2024-09-21T07:47:41.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.583+0000 7fae2f000640 1 -- 172.21.15.88:0/2713329676 wait complete. 2024-09-21T07:47:41.606 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:47:41.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 -- 172.21.15.88:0/603540137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f180aebc0 msgr2=0x7f3f18152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 --2- 172.21.15.88:0/603540137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f180aebc0 0x7f3f18152920 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f3f08009830 tx=0x7f3f0802a480 comp rx=0 tx=0).stop 2024-09-21T07:47:41.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 -- 172.21.15.88:0/603540137 shutdown_connections 2024-09-21T07:47:41.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 --2- 172.21.15.88:0/603540137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1815fae0 0x7f3f180b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 --2- 172.21.15.88:0/603540137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f180aebc0 0x7f3f18152920 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 -- 172.21.15.88:0/603540137 >> 172.21.15.88:0/603540137 conn(0x7f3f1806d730 msgr2=0x7f3f180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:41.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 -- 172.21.15.88:0/603540137 shutdown_connections 2024-09-21T07:47:41.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.824+0000 7f3f1e000640 1 -- 172.21.15.88:0/603540137 wait complete. 2024-09-21T07:47:41.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.826+0000 7f3f1e000640 1 Processor -- start 2024-09-21T07:47:41.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.826+0000 7f3f1e000640 1 -- start start 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.827+0000 7f3f1e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f180aebc0 0x7f3f18155d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.827+0000 7f3f1e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 0x7f3f18155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.827+0000 7f3f1e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f181f6800 0x7f3f181f6c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.827+0000 7f3f1e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180b4190 con 0x7f3f180b2750 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.827+0000 7f3f1e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f180b4010 con 0x7f3f181f6800 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.827+0000 7f3f1e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f180b4310 con 0x7f3f180aebc0 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 0x7f3f18155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 0x7f3f18155560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39102/0 (socket says 172.21.15.88:39102) 2024-09-21T07:47:41.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 -- 172.21.15.88:0/208402526 learned_addr learned my addr 172.21.15.88:0/208402526 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:41.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f17e00640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f181f6800 0x7f3f181f6c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f17400640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f180aebc0 0x7f3f18155d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 -- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f180aebc0 msgr2=0x7f3f18155d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f180aebc0 0x7f3f18155d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 -- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f181f6800 msgr2=0x7f3f181f6c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f181f6800 0x7f3f181f6c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.828+0000 7f3f16a00640 1 -- 172.21.15.88:0/208402526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f181f7150 con 0x7f3f180b2750 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.829+0000 7f3f16a00640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 0x7f3f18155560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c460 tx=0x7f3f0400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.829+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04016020 con 0x7f3f180b2750 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.829+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f04012940 con 0x7f3f180b2750 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.829+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04012d90 con 0x7f3f180b2750 2024-09-21T07:47:41.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.829+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f1809ddf0 con 0x7f3f180b2750 2024-09-21T07:47:41.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.829+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f181f7630 con 0x7f3f180b2750 2024-09-21T07:47:41.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.830+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f180b47c0 con 0x7f3f180b2750 2024-09-21T07:47:41.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.831+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f3f04019020 con 0x7f3f180b2750 2024-09-21T07:47:41.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.832+0000 7f3f0f400640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef8078d40 0x7f3ef807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:41.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.832+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f3f0409abf0 con 0x7f3f180b2750 2024-09-21T07:47:41.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.833+0000 7f3f17400640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef8078d40 0x7f3ef807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:41.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.833+0000 7f3f17400640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef8078d40 0x7f3ef807b250 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f08034000 tx=0x7f3f08004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:41.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:41.837+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f04020030 con 0x7f3f180b2750 2024-09-21T07:47:42.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.029+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f1809e080 con 0x7f3f180b2750 2024-09-21T07:47:42.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.030+0000 7f3f0f400640 1 -- 172.21.15.88:0/208402526 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) ==== 74+0+16681 (secure 0 0 0) 0x7f3f181f7150 con 0x7f3f180b2750 2024-09-21T07:47:42.033 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:42.033 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:47:39.073308+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:35.498681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:36.524995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:47:42.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef8078d40 msgr2=0x7f3ef807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef8078d40 0x7f3ef807b250 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f3f08034000 tx=0x7f3f08004730 comp rx=0 tx=0).stop 2024-09-21T07:47:42.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 msgr2=0x7f3f18155560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 0x7f3f18155560 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c460 tx=0x7f3f0400c920 comp rx=0 tx=0).stop 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 shutdown_connections 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef8078d40 0x7f3ef807b250 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f181f6800 0x7f3f181f6c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f180b2750 0x7f3f18155560 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 --2- 172.21.15.88:0/208402526 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f180aebc0 0x7f3f18155d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 >> 172.21.15.88:0/208402526 conn(0x7f3f1806d730 msgr2=0x7f3f18074730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 shutdown_connections 2024-09-21T07:47:42.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.033+0000 7f3f1e000640 1 -- 172.21.15.88:0/208402526 wait complete. 2024-09-21T07:47:42.056 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:47:42.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.276+0000 7f5806e00640 1 -- 172.21.15.88:0/3628304803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 msgr2=0x7f580015d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.276+0000 7f5806e00640 1 --2- 172.21.15.88:0/3628304803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f580015d670 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f5800051590 tx=0x7f57f802aa30 comp rx=0 tx=0).stop 2024-09-21T07:47:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.277+0000 7f5806e00640 1 -- 172.21.15.88:0/3628304803 shutdown_connections 2024-09-21T07:47:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.277+0000 7f5806e00640 1 --2- 172.21.15.88:0/3628304803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f580015d670 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.277+0000 7f5806e00640 1 --2- 172.21.15.88:0/3628304803 >> v2:172.21.15.175:3300/0 conn(0x7f58000ad9c0 0x7f58000af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.277+0000 7f5806e00640 1 -- 172.21.15.88:0/3628304803 >> 172.21.15.88:0/3628304803 conn(0x7f58000b1e40 msgr2=0x7f58000b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.277+0000 7f5806e00640 1 -- 172.21.15.88:0/3628304803 shutdown_connections 2024-09-21T07:47:42.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.277+0000 7f5806e00640 1 -- 172.21.15.88:0/3628304803 wait complete. 2024-09-21T07:47:42.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.279+0000 7f5806e00640 1 Processor -- start 2024-09-21T07:47:42.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.279+0000 7f5806e00640 1 -- start start 2024-09-21T07:47:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.280+0000 7f5806e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58000ad9c0 0x7f5800148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.280+0000 7f5806e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58000b2f60 0x7f5800158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.280+0000 7f5806e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f58001589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.280+0000 7f5806e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f580006f290 con 0x7f580015a0f0 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.280+0000 7f5806e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f580015a5e0 con 0x7f58000ad9c0 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.280+0000 7f5806e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f580006f410 con 0x7f58000b2f60 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f58001589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f58001589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39120/0 (socket says 172.21.15.88:39120) 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 -- 172.21.15.88:0/2537017677 learned_addr learned my addr 172.21.15.88:0/2537017677 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5805a00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58000ad9c0 0x7f5800148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5805000640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58000b2f60 0x7f5800158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 -- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58000b2f60 msgr2=0x7f5800158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58000b2f60 0x7f5800158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 -- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58000ad9c0 msgr2=0x7f5800148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58000ad9c0 0x7f5800148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5806400640 1 -- 172.21.15.88:0/2537017677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5800157710 con 0x7f580015a0f0 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.281+0000 7f5805000640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58000b2f60 0x7f5800158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:42.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.282+0000 7f5806400640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f58001589a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f57f802b020 tx=0x7f57f8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.282+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8033050 con 0x7f580015a0f0 2024-09-21T07:47:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.282+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57f8004550 con 0x7f580015a0f0 2024-09-21T07:47:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.282+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57f8038450 con 0x7f580015a0f0 2024-09-21T07:47:42.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.282+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58001489c0 con 0x7f580015a0f0 2024-09-21T07:47:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.282+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5800157c60 con 0x7f580015a0f0 2024-09-21T07:47:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.283+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f580006f930 con 0x7f580015a0f0 2024-09-21T07:47:42.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.284+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f57f8036070 con 0x7f580015a0f0 2024-09-21T07:47:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.285+0000 7f57fd600640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f57e0078d40 0x7f57e007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.285+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f57f80bc770 con 0x7f580015a0f0 2024-09-21T07:47:42.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.287+0000 7f5805a00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f57e0078d40 0x7f57e007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.289+0000 7f5805a00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f57e0078d40 0x7f57e007b250 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f400a250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:42.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.290+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57f8084820 con 0x7f580015a0f0 2024-09-21T07:47:42.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.471+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f58000b24a0 con 0x7f57e0078d40 2024-09-21T07:47:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.477+0000 7f57fd600640 1 -- 172.21.15.88:0/2537017677 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7f58000b24a0 con 0x7f57e0078d40 2024-09-21T07:47:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:42.483 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f57e0078d40 msgr2=0x7f57e007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f57e0078d40 0x7f57e007b250 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f57f4005fd0 tx=0x7f57f400a250 comp rx=0 tx=0).stop 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 msgr2=0x7f58001589a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f58001589a0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f57f802b020 tx=0x7f57f8003f10 comp rx=0 tx=0).stop 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 shutdown_connections 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f57e0078d40 0x7f57e007b250 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58000b2f60 0x7f5800158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f580015a0f0 0x7f58001589a0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 --2- 172.21.15.88:0/2537017677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f58000ad9c0 0x7f5800148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 >> 172.21.15.88:0/2537017677 conn(0x7f58000b1e40 msgr2=0x7f58000cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 shutdown_connections 2024-09-21T07:47:42.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.481+0000 7f5806e00640 1 -- 172.21.15.88:0/2537017677 wait complete. 2024-09-21T07:47:42.504 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-21T07:47:41.878231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230000,"kb_used_data":5436,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744720,"statfs":{"total":773094113280,"available":772858593280,"internally_reserved":0,"allocated":5566464,"data_stored":2921184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.825033"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144362+0000","last_change":"2024-09-21T07:47:38.088987+0000","last_active":"2024-09-21T07:47:39.144362+0000","last_peered":"2024-09-21T07:47:39.144362+0000","last_clean":"2024-09-21T07:47:39.144362+0000","last_became_active":"2024-09-21T07:47:38.088613+0000","last_became_peered":"2024-09-21T07:47:38.088613+0000","last_unstale":"2024-09-21T07:47:39.144362+0000","last_undegraded":"2024-09-21T07:47:39.144362+0000","last_fullsized":"2024-09-21T07:47:39.144362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:16:10.683521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110178+0000","last_change":"2024-09-21T07:47:38.078774+0000","last_active":"2024-09-21T07:47:39.110178+0000","last_peered":"2024-09-21T07:47:39.110178+0000","last_clean":"2024-09-21T07:47:39.110178+0000","last_became_active":"2024-09-21T07:47:38.078523+0000","last_became_peered":"2024-09-21T07:47:38.078523+0000","last_unstale":"2024-09-21T07:47:39.110178+0000","last_undegraded":"2024-09-21T07:47:39.110178+0000","last_fullsized":"2024-09-21T07:47:39.110178+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.629109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081655+0000","last_change":"2024-09-21T07:47:38.081946+0000","last_active":"2024-09-21T07:47:39.081655+0000","last_peered":"2024-09-21T07:47:39.081655+0000","last_clean":"2024-09-21T07:47:39.081655+0000","last_became_active":"2024-09-21T07:47:38.081604+0000","last_became_peered":"2024-09-21T07:47:38.081604+0000","last_unstale":"2024-09-21T07:47:39.081655+0000","last_undegraded":"2024-09-21T07:47:39.081655+0000","last_fullsized":"2024-09-21T07:47:39.081655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.938971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083462+0000","last_change":"2024-09-21T07:47:38.101683+0000","last_active":"2024-09-21T07:47:39.083462+0000","last_peered":"2024-09-21T07:47:39.083462+0000","last_clean":"2024-09-21T07:47:39.083462+0000","last_became_active":"2024-09-21T07:47:38.101344+0000","last_became_peered":"2024-09-21T07:47:38.101344+0000","last_unstale":"2024-09-21T07:47:39.083462+0000","last_undegraded":"2024-09-21T07:47:39.083462+0000","last_fullsized":"2024-09-21T07:47:39.083462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:07.985028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144074+0000","last_change":"2024-09-21T07:47:38.090516+0000","last_active":"2024-09-21T07:47:39.144074+0000","last_peered":"2024-09-21T07:47:39.144074+0000","last_clean":"2024-09-21T07:47:39.144074+0000","last_became_active":"2024-09-21T07:47:38.090197+0000","last_became_peered":"2024-09-21T07:47:38.090197+0000","last_unstale":"2024-09-21T07:47:39.144074+0000","last_undegraded":"2024-09-21T07:47:39.144074+0000","last_fullsized":"2024-09-21T07:47:39.144074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:23.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144079+0000","last_change":"2024-09-21T07:47:38.099125+0000","last_active":"2024-09-21T07:47:39.144079+0000","last_peered":"2024-09-21T07:47:39.144079+0000","last_clean":"2024-09-21T07:47:39.144079+0000","last_became_active":"2024-09-21T07:47:38.098820+0000","last_became_peered":"2024-09-21T07:47:38.098820+0000","last_unstale":"2024-09-21T07:47:39.144079+0000","last_undegraded":"2024-09-21T07:47:39.144079+0000","last_fullsized":"2024-09-21T07:47:39.144079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:45.573811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092440+0000","last_change":"2024-09-21T07:47:38.082352+0000","last_active":"2024-09-21T07:47:39.092440+0000","last_peered":"2024-09-21T07:47:39.092440+0000","last_clean":"2024-09-21T07:47:39.092440+0000","last_became_active":"2024-09-21T07:47:38.081939+0000","last_became_peered":"2024-09-21T07:47:38.081939+0000","last_unstale":"2024-09-21T07:47:39.092440+0000","last_undegraded":"2024-09-21T07:47:39.092440+0000","last_fullsized":"2024-09-21T07:47:39.092440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:06.007142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084513+0000","last_change":"2024-09-21T07:47:38.096353+0000","last_active":"2024-09-21T07:47:39.084513+0000","last_peered":"2024-09-21T07:47:39.084513+0000","last_clean":"2024-09-21T07:47:39.084513+0000","last_became_active":"2024-09-21T07:47:38.096001+0000","last_became_peered":"2024-09-21T07:47:38.096001+0000","last_unstale":"2024-09-21T07:47:39.084513+0000","last_undegraded":"2024-09-21T07:47:39.084513+0000","last_fullsized":"2024-09-21T07:47:39.084513+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:53.467974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144583+0000","last_change":"2024-09-21T07:47:38.096562+0000","last_active":"2024-09-21T07:47:39.144583+0000","last_peered":"2024-09-21T07:47:39.144583+0000","last_clean":"2024-09-21T07:47:39.144583+0000","last_became_active":"2024-09-21T07:47:38.096173+0000","last_became_peered":"2024-09-21T07:47:38.096173+0000","last_unstale":"2024-09-21T07:47:39.144583+0000","last_undegraded":"2024-09-21T07:47:39.144583+0000","last_fullsized":"2024-09-21T07:47:39.144583+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:23.048496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085556+0000","last_change":"2024-09-21T07:47:38.089144+0000","last_active":"2024-09-21T07:47:39.085556+0000","last_peered":"2024-09-21T07:47:39.085556+0000","last_clean":"2024-09-21T07:47:39.085556+0000","last_became_active":"2024-09-21T07:47:38.088813+0000","last_became_peered":"2024-09-21T07:47:38.088813+0000","last_unstale":"2024-09-21T07:47:39.085556+0000","last_undegraded":"2024-09-21T07:47:39.085556+0000","last_fullsized":"2024-09-21T07:47:39.085556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:46.126791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083993+0000","last_change":"2024-09-21T07:47:38.093514+0000","last_active":"2024-09-21T07:47:39.083993+0000","last_peered":"2024-09-21T07:47:39.083993+0000","last_clean":"2024-09-21T07:47:39.083993+0000","last_became_active":"2024-09-21T07:47:38.093159+0000","last_became_peered":"2024-09-21T07:47:38.093159+0000","last_unstale":"2024-09-21T07:47:39.083993+0000","last_undegraded":"2024-09-21T07:47:39.083993+0000","last_fullsized":"2024-09-21T07:47:39.083993+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:40.465534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110260+0000","last_change":"2024-09-21T07:47:38.101535+0000","last_active":"2024-09-21T07:47:39.110260+0000","last_peered":"2024-09-21T07:47:39.110260+0000","last_clean":"2024-09-21T07:47:39.110260+0000","last_became_active":"2024-09-21T07:47:38.101231+0000","last_became_peered":"2024-09-21T07:47:38.101231+0000","last_unstale":"2024-09-21T07:47:39.110260+0000","last_undegraded":"2024-09-21T07:47:39.110260+0000","last_fullsized":"2024-09-21T07:47:39.110260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:12.606382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144421+0000","last_change":"2024-09-21T07:47:38.098466+0000","last_active":"2024-09-21T07:47:39.144421+0000","last_peered":"2024-09-21T07:47:39.144421+0000","last_clean":"2024-09-21T07:47:39.144421+0000","last_became_active":"2024-09-21T07:47:38.098126+0000","last_became_peered":"2024-09-21T07:47:38.098126+0000","last_unstale":"2024-09-21T07:47:39.144421+0000","last_undegraded":"2024-09-21T07:47:39.144421+0000","last_fullsized":"2024-09-21T07:47:39.144421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:56.285797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083127+0000","last_change":"2024-09-21T07:47:38.090993+0000","last_active":"2024-09-21T07:47:39.083127+0000","last_peered":"2024-09-21T07:47:39.083127+0000","last_clean":"2024-09-21T07:47:39.083127+0000","last_became_active":"2024-09-21T07:47:38.090643+0000","last_became_peered":"2024-09-21T07:47:38.090643+0000","last_unstale":"2024-09-21T07:47:39.083127+0000","last_undegraded":"2024-09-21T07:47:39.083127+0000","last_fullsized":"2024-09-21T07:47:39.083127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.948221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083635+0000","last_change":"2024-09-21T07:47:38.091569+0000","last_active":"2024-09-21T07:47:39.083635+0000","last_peered":"2024-09-21T07:47:39.083635+0000","last_clean":"2024-09-21T07:47:39.083635+0000","last_became_active":"2024-09-21T07:47:38.090147+0000","last_became_peered":"2024-09-21T07:47:38.090147+0000","last_unstale":"2024-09-21T07:47:39.083635+0000","last_undegraded":"2024-09-21T07:47:39.083635+0000","last_fullsized":"2024-09-21T07:47:39.083635+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:04.448926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144051+0000","last_change":"2024-09-21T07:47:38.094826+0000","last_active":"2024-09-21T07:47:39.144051+0000","last_peered":"2024-09-21T07:47:39.144051+0000","last_clean":"2024-09-21T07:47:39.144051+0000","last_became_active":"2024-09-21T07:47:38.094472+0000","last_became_peered":"2024-09-21T07:47:38.094472+0000","last_unstale":"2024-09-21T07:47:39.144051+0000","last_undegraded":"2024-09-21T07:47:39.144051+0000","last_fullsized":"2024-09-21T07:47:39.144051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:18.360590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110385+0000","last_change":"2024-09-21T07:47:38.096358+0000","last_active":"2024-09-21T07:47:39.110385+0000","last_peered":"2024-09-21T07:47:39.110385+0000","last_clean":"2024-09-21T07:47:39.110385+0000","last_became_active":"2024-09-21T07:47:38.096089+0000","last_became_peered":"2024-09-21T07:47:38.096089+0000","last_unstale":"2024-09-21T07:47:39.110385+0000","last_undegraded":"2024-09-21T07:47:39.110385+0000","last_fullsized":"2024-09-21T07:47:39.110385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:42.013947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081368+0000","last_change":"2024-09-21T07:47:38.086044+0000","last_active":"2024-09-21T07:47:39.081368+0000","last_peered":"2024-09-21T07:47:39.081368+0000","last_clean":"2024-09-21T07:47:39.081368+0000","last_became_active":"2024-09-21T07:47:38.085701+0000","last_became_peered":"2024-09-21T07:47:38.085701+0000","last_unstale":"2024-09-21T07:47:39.081368+0000","last_undegraded":"2024-09-21T07:47:39.081368+0000","last_fullsized":"2024-09-21T07:47:39.081368+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:42.556673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144219+0000","last_change":"2024-09-21T07:47:38.092858+0000","last_active":"2024-09-21T07:47:39.144219+0000","last_peered":"2024-09-21T07:47:39.144219+0000","last_clean":"2024-09-21T07:47:39.144219+0000","last_became_active":"2024-09-21T07:47:38.092507+0000","last_became_peered":"2024-09-21T07:47:38.092507+0000","last_unstale":"2024-09-21T07:47:39.144219+0000","last_undegraded":"2024-09-21T07:47:39.144219+0000","last_fullsized":"2024-09-21T07:47:39.144219+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:33.773460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085175+0000","last_change":"2024-09-21T07:47:38.097972+0000","last_active":"2024-09-21T07:47:39.085175+0000","last_peered":"2024-09-21T07:47:39.085175+0000","last_clean":"2024-09-21T07:47:39.085175+0000","last_became_active":"2024-09-21T07:47:38.097671+0000","last_became_peered":"2024-09-21T07:47:38.097671+0000","last_unstale":"2024-09-21T07:47:39.085175+0000","last_undegraded":"2024-09-21T07:47:39.085175+0000","last_fullsized":"2024-09-21T07:47:39.085175+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:39.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083647+0000","last_change":"2024-09-21T07:47:38.081145+0000","last_active":"2024-09-21T07:47:39.083647+0000","last_peered":"2024-09-21T07:47:39.083647+0000","last_clean":"2024-09-21T07:47:39.083647+0000","last_became_active":"2024-09-21T07:47:38.080382+0000","last_became_peered":"2024-09-21T07:47:38.080382+0000","last_unstale":"2024-09-21T07:47:39.083647+0000","last_undegraded":"2024-09-21T07:47:39.083647+0000","last_fullsized":"2024-09-21T07:47:39.083647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.629109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.099641+0000","last_change":"2024-09-21T07:47:38.097772+0000","last_active":"2024-09-21T07:47:38.099641+0000","last_peered":"2024-09-21T07:47:38.099641+0000","last_clean":"2024-09-21T07:47:38.099641+0000","last_became_active":"2024-09-21T07:47:38.097429+0000","last_became_peered":"2024-09-21T07:47:38.097429+0000","last_unstale":"2024-09-21T07:47:38.099641+0000","last_undegraded":"2024-09-21T07:47:38.099641+0000","last_fullsized":"2024-09-21T07:47:38.099641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:44.178854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092166+0000","last_change":"2024-09-21T07:47:38.086850+0000","last_active":"2024-09-21T07:47:39.092166+0000","last_peered":"2024-09-21T07:47:39.092166+0000","last_clean":"2024-09-21T07:47:39.092166+0000","last_became_active":"2024-09-21T07:47:38.086504+0000","last_became_peered":"2024-09-21T07:47:38.086504+0000","last_unstale":"2024-09-21T07:47:39.092166+0000","last_undegraded":"2024-09-21T07:47:39.092166+0000","last_fullsized":"2024-09-21T07:47:39.092166+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:31.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110227+0000","last_change":"2024-09-21T07:47:38.088451+0000","last_active":"2024-09-21T07:47:39.110227+0000","last_peered":"2024-09-21T07:47:39.110227+0000","last_clean":"2024-09-21T07:47:39.110227+0000","last_became_active":"2024-09-21T07:47:38.088129+0000","last_became_peered":"2024-09-21T07:47:38.088129+0000","last_unstale":"2024-09-21T07:47:39.110227+0000","last_undegraded":"2024-09-21T07:47:39.110227+0000","last_fullsized":"2024-09-21T07:47:39.110227+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:18.360590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144410+0000","last_change":"2024-09-21T07:47:38.096517+0000","last_active":"2024-09-21T07:47:39.144410+0000","last_peered":"2024-09-21T07:47:39.144410+0000","last_clean":"2024-09-21T07:47:39.144410+0000","last_became_active":"2024-09-21T07:47:38.096186+0000","last_became_peered":"2024-09-21T07:47:38.096186+0000","last_unstale":"2024-09-21T07:47:39.144410+0000","last_undegraded":"2024-09-21T07:47:39.144410+0000","last_fullsized":"2024-09-21T07:47:39.144410+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:20.474763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084185+0000","last_change":"2024-09-21T07:47:38.093599+0000","last_active":"2024-09-21T07:47:39.084185+0000","last_peered":"2024-09-21T07:47:39.084185+0000","last_clean":"2024-09-21T07:47:39.084185+0000","last_became_active":"2024-09-21T07:47:38.092937+0000","last_became_peered":"2024-09-21T07:47:38.092937+0000","last_unstale":"2024-09-21T07:47:39.084185+0000","last_undegraded":"2024-09-21T07:47:39.084185+0000","last_fullsized":"2024-09-21T07:47:39.084185+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:16:54.050789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092655+0000","last_change":"2024-09-21T07:47:38.095340+0000","last_active":"2024-09-21T07:47:39.092655+0000","last_peered":"2024-09-21T07:47:39.092655+0000","last_clean":"2024-09-21T07:47:39.092655+0000","last_became_active":"2024-09-21T07:47:38.095040+0000","last_became_peered":"2024-09-21T07:47:38.095040+0000","last_unstale":"2024-09-21T07:47:39.092655+0000","last_undegraded":"2024-09-21T07:47:39.092655+0000","last_fullsized":"2024-09-21T07:47:39.092655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:33.045018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092303+0000","last_change":"2024-09-21T07:47:38.087689+0000","last_active":"2024-09-21T07:47:39.092303+0000","last_peered":"2024-09-21T07:47:39.092303+0000","last_clean":"2024-09-21T07:47:39.092303+0000","last_became_active":"2024-09-21T07:47:38.087316+0000","last_became_peered":"2024-09-21T07:47:38.087316+0000","last_unstale":"2024-09-21T07:47:39.092303+0000","last_undegraded":"2024-09-21T07:47:39.092303+0000","last_fullsized":"2024-09-21T07:47:39.092303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:46.250888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144040+0000","last_change":"2024-09-21T07:47:38.097421+0000","last_active":"2024-09-21T07:47:39.144040+0000","last_peered":"2024-09-21T07:47:39.144040+0000","last_clean":"2024-09-21T07:47:39.144040+0000","last_became_active":"2024-09-21T07:47:38.095733+0000","last_became_peered":"2024-09-21T07:47:38.095733+0000","last_unstale":"2024-09-21T07:47:39.144040+0000","last_undegraded":"2024-09-21T07:47:39.144040+0000","last_fullsized":"2024-09-21T07:47:39.144040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:30.016507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.109928+0000","last_change":"2024-09-21T07:47:38.082968+0000","last_active":"2024-09-21T07:47:39.109928+0000","last_peered":"2024-09-21T07:47:39.109928+0000","last_clean":"2024-09-21T07:47:39.109928+0000","last_became_active":"2024-09-21T07:47:38.081334+0000","last_became_peered":"2024-09-21T07:47:38.081334+0000","last_unstale":"2024-09-21T07:47:39.109928+0000","last_undegraded":"2024-09-21T07:47:39.109928+0000","last_fullsized":"2024-09-21T07:47:39.109928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:04.448926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143995+0000","last_change":"2024-09-21T07:47:38.106778+0000","last_active":"2024-09-21T07:47:39.143995+0000","last_peered":"2024-09-21T07:47:39.143995+0000","last_clean":"2024-09-21T07:47:39.143995+0000","last_became_active":"2024-09-21T07:47:38.105636+0000","last_became_peered":"2024-09-21T07:47:38.105636+0000","last_unstale":"2024-09-21T07:47:39.143995+0000","last_undegraded":"2024-09-21T07:47:39.143995+0000","last_fullsized":"2024-09-21T07:47:39.143995+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:03.940811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084605+0000","last_change":"2024-09-21T07:47:38.100391+0000","last_active":"2024-09-21T07:47:39.084605+0000","last_peered":"2024-09-21T07:47:39.084605+0000","last_clean":"2024-09-21T07:47:39.084605+0000","last_became_active":"2024-09-21T07:47:38.099940+0000","last_became_peered":"2024-09-21T07:47:38.099940+0000","last_unstale":"2024-09-21T07:47:39.084605+0000","last_undegraded":"2024-09-21T07:47:39.084605+0000","last_fullsized":"2024-09-21T07:47:39.084605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:49.088537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.106963+0000","last_change":"2024-09-21T07:47:38.101129+0000","last_active":"2024-09-21T07:47:38.106963+0000","last_peered":"2024-09-21T07:47:38.106963+0000","last_clean":"2024-09-21T07:47:38.106963+0000","last_became_active":"2024-09-21T07:47:38.099102+0000","last_became_peered":"2024-09-21T07:47:38.099102+0000","last_unstale":"2024-09-21T07:47:38.106963+0000","last_undegraded":"2024-09-21T07:47:38.106963+0000","last_fullsized":"2024-09-21T07:47:38.106963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:26.118846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144646+0000","last_change":"2024-09-21T07:47:37.094676+0000","last_active":"2024-09-21T07:47:39.144646+0000","last_peered":"2024-09-21T07:47:39.144646+0000","last_clean":"2024-09-21T07:47:39.144646+0000","last_became_active":"2024-09-21T07:47:37.093016+0000","last_became_peered":"2024-09-21T07:47:37.093016+0000","last_unstale":"2024-09-21T07:47:39.144646+0000","last_undegraded":"2024-09-21T07:47:39.144646+0000","last_fullsized":"2024-09-21T07:47:39.144646+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:38.743173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144509+0000","last_change":"2024-09-21T07:47:38.092907+0000","last_active":"2024-09-21T07:47:39.144509+0000","last_peered":"2024-09-21T07:47:39.144509+0000","last_clean":"2024-09-21T07:47:39.144509+0000","last_became_active":"2024-09-21T07:47:38.092573+0000","last_became_peered":"2024-09-21T07:47:38.092573+0000","last_unstale":"2024-09-21T07:47:39.144509+0000","last_undegraded":"2024-09-21T07:47:39.144509+0000","last_fullsized":"2024-09-21T07:47:39.144509+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:23.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092585+0000","last_change":"2024-09-21T07:47:37.078705+0000","last_active":"2024-09-21T07:47:39.092585+0000","last_peered":"2024-09-21T07:47:39.092585+0000","last_clean":"2024-09-21T07:47:39.092585+0000","last_became_active":"2024-09-21T07:47:37.078358+0000","last_became_peered":"2024-09-21T07:47:37.078358+0000","last_unstale":"2024-09-21T07:47:39.092585+0000","last_undegraded":"2024-09-21T07:47:39.092585+0000","last_fullsized":"2024-09-21T07:47:39.092585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:18.594370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110303+0000","last_change":"2024-09-21T07:47:38.080169+0000","last_active":"2024-09-21T07:47:39.110303+0000","last_peered":"2024-09-21T07:47:39.110303+0000","last_clean":"2024-09-21T07:47:39.110303+0000","last_became_active":"2024-09-21T07:47:38.079881+0000","last_became_peered":"2024-09-21T07:47:38.079881+0000","last_unstale":"2024-09-21T07:47:39.110303+0000","last_undegraded":"2024-09-21T07:47:39.110303+0000","last_fullsized":"2024-09-21T07:47:39.110303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:40.465534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144231+0000","last_change":"2024-09-21T07:47:37.088481+0000","last_active":"2024-09-21T07:47:39.144231+0000","last_peered":"2024-09-21T07:47:39.144231+0000","last_clean":"2024-09-21T07:47:39.144231+0000","last_became_active":"2024-09-21T07:47:37.088163+0000","last_became_peered":"2024-09-21T07:47:37.088163+0000","last_unstale":"2024-09-21T07:47:39.144231+0000","last_undegraded":"2024-09-21T07:47:39.144231+0000","last_fullsized":"2024-09-21T07:47:39.144231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:52.070517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084185+0000","last_change":"2024-09-21T07:47:38.107122+0000","last_active":"2024-09-21T07:47:39.084185+0000","last_peered":"2024-09-21T07:47:39.084185+0000","last_clean":"2024-09-21T07:47:39.084185+0000","last_became_active":"2024-09-21T07:47:38.106606+0000","last_became_peered":"2024-09-21T07:47:38.106606+0000","last_unstale":"2024-09-21T07:47:39.084185+0000","last_undegraded":"2024-09-21T07:47:39.084185+0000","last_fullsized":"2024-09-21T07:47:39.084185+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.622169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092314+0000","last_change":"2024-09-21T07:47:37.080482+0000","last_active":"2024-09-21T07:47:39.092314+0000","last_peered":"2024-09-21T07:47:39.092314+0000","last_clean":"2024-09-21T07:47:39.092314+0000","last_became_active":"2024-09-21T07:47:37.080170+0000","last_became_peered":"2024-09-21T07:47:37.080170+0000","last_unstale":"2024-09-21T07:47:39.092314+0000","last_undegraded":"2024-09-21T07:47:39.092314+0000","last_fullsized":"2024-09-21T07:47:39.092314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:11.372734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091939+0000","last_change":"2024-09-21T07:47:38.083936+0000","last_active":"2024-09-21T07:47:39.091939+0000","last_peered":"2024-09-21T07:47:39.091939+0000","last_clean":"2024-09-21T07:47:39.091939+0000","last_became_active":"2024-09-21T07:47:38.083583+0000","last_became_peered":"2024-09-21T07:47:38.083583+0000","last_unstale":"2024-09-21T07:47:39.091939+0000","last_undegraded":"2024-09-21T07:47:39.091939+0000","last_fullsized":"2024-09-21T07:47:39.091939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:58.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144572+0000","last_change":"2024-09-21T07:47:37.081597+0000","last_active":"2024-09-21T07:47:39.144572+0000","last_peered":"2024-09-21T07:47:39.144572+0000","last_clean":"2024-09-21T07:47:39.144572+0000","last_became_active":"2024-09-21T07:47:37.081231+0000","last_became_peered":"2024-09-21T07:47:37.081231+0000","last_unstale":"2024-09-21T07:47:39.144572+0000","last_undegraded":"2024-09-21T07:47:39.144572+0000","last_fullsized":"2024-09-21T07:47:39.144572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:46.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084780+0000","last_change":"2024-09-21T07:47:38.090129+0000","last_active":"2024-09-21T07:47:39.084780+0000","last_peered":"2024-09-21T07:47:39.084780+0000","last_clean":"2024-09-21T07:47:39.084780+0000","last_became_active":"2024-09-21T07:47:38.089807+0000","last_became_peered":"2024-09-21T07:47:38.089807+0000","last_unstale":"2024-09-21T07:47:39.084780+0000","last_undegraded":"2024-09-21T07:47:39.084780+0000","last_fullsized":"2024-09-21T07:47:39.084780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:06.573911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091878+0000","last_change":"2024-09-21T07:47:37.089253+0000","last_active":"2024-09-21T07:47:39.091878+0000","last_peered":"2024-09-21T07:47:39.091878+0000","last_clean":"2024-09-21T07:47:39.091878+0000","last_became_active":"2024-09-21T07:47:37.088785+0000","last_became_peered":"2024-09-21T07:47:37.088785+0000","last_unstale":"2024-09-21T07:47:39.091878+0000","last_undegraded":"2024-09-21T07:47:39.091878+0000","last_fullsized":"2024-09-21T07:47:39.091878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.559407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083024+0000","last_change":"2024-09-21T07:47:38.106650+0000","last_active":"2024-09-21T07:47:39.083024+0000","last_peered":"2024-09-21T07:47:39.083024+0000","last_clean":"2024-09-21T07:47:39.083024+0000","last_became_active":"2024-09-21T07:47:38.106299+0000","last_became_peered":"2024-09-21T07:47:38.106299+0000","last_unstale":"2024-09-21T07:47:39.083024+0000","last_undegraded":"2024-09-21T07:47:39.083024+0000","last_fullsized":"2024-09-21T07:47:39.083024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:40.565767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143956+0000","last_change":"2024-09-21T07:47:37.094320+0000","last_active":"2024-09-21T07:47:39.143956+0000","last_peered":"2024-09-21T07:47:39.143956+0000","last_clean":"2024-09-21T07:47:39.143956+0000","last_became_active":"2024-09-21T07:47:37.094041+0000","last_became_peered":"2024-09-21T07:47:37.094041+0000","last_unstale":"2024-09-21T07:47:39.143956+0000","last_undegraded":"2024-09-21T07:47:39.143956+0000","last_fullsized":"2024-09-21T07:47:39.143956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:16.849675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091876+0000","last_change":"2024-09-21T07:47:38.090178+0000","last_active":"2024-09-21T07:47:39.091876+0000","last_peered":"2024-09-21T07:47:39.091876+0000","last_clean":"2024-09-21T07:47:39.091876+0000","last_became_active":"2024-09-21T07:47:38.089884+0000","last_became_peered":"2024-09-21T07:47:38.089884+0000","last_unstale":"2024-09-21T07:47:39.091876+0000","last_undegraded":"2024-09-21T07:47:39.091876+0000","last_fullsized":"2024-09-21T07:47:39.091876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:39.584964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083043+0000","last_change":"2024-09-21T07:47:37.085490+0000","last_active":"2024-09-21T07:47:39.083043+0000","last_peered":"2024-09-21T07:47:39.083043+0000","last_clean":"2024-09-21T07:47:39.083043+0000","last_became_active":"2024-09-21T07:47:37.085143+0000","last_became_peered":"2024-09-21T07:47:37.085143+0000","last_unstale":"2024-09-21T07:47:39.083043+0000","last_undegraded":"2024-09-21T07:47:39.083043+0000","last_fullsized":"2024-09-21T07:47:39.083043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:22.042136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.082027+0000","last_change":"2024-09-21T07:47:38.105161+0000","last_active":"2024-09-21T07:47:39.082027+0000","last_peered":"2024-09-21T07:47:39.082027+0000","last_clean":"2024-09-21T07:47:39.082027+0000","last_became_active":"2024-09-21T07:47:38.103450+0000","last_became_peered":"2024-09-21T07:47:38.103450+0000","last_unstale":"2024-09-21T07:47:39.082027+0000","last_undegraded":"2024-09-21T07:47:39.082027+0000","last_fullsized":"2024-09-21T07:47:39.082027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:46.013625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.082006+0000","last_change":"2024-09-21T07:47:37.089929+0000","last_active":"2024-09-21T07:47:39.082006+0000","last_peered":"2024-09-21T07:47:39.082006+0000","last_clean":"2024-09-21T07:47:39.082006+0000","last_became_active":"2024-09-21T07:47:37.089621+0000","last_became_peered":"2024-09-21T07:47:37.089621+0000","last_unstale":"2024-09-21T07:47:39.082006+0000","last_undegraded":"2024-09-21T07:47:39.082006+0000","last_fullsized":"2024-09-21T07:47:39.082006+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:24.665789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081830+0000","last_change":"2024-09-21T07:47:38.090141+0000","last_active":"2024-09-21T07:47:39.081830+0000","last_peered":"2024-09-21T07:47:39.081830+0000","last_clean":"2024-09-21T07:47:39.081830+0000","last_became_active":"2024-09-21T07:47:38.089824+0000","last_became_peered":"2024-09-21T07:47:38.089824+0000","last_unstale":"2024-09-21T07:47:39.081830+0000","last_undegraded":"2024-09-21T07:47:39.081830+0000","last_fullsized":"2024-09-21T07:47:39.081830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:04.677097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085495+0000","last_change":"2024-09-21T07:47:37.089606+0000","last_active":"2024-09-21T07:47:39.085495+0000","last_peered":"2024-09-21T07:47:39.085495+0000","last_clean":"2024-09-21T07:47:39.085495+0000","last_became_active":"2024-09-21T07:47:37.087153+0000","last_became_peered":"2024-09-21T07:47:37.087153+0000","last_unstale":"2024-09-21T07:47:39.085495+0000","last_undegraded":"2024-09-21T07:47:39.085495+0000","last_fullsized":"2024-09-21T07:47:39.085495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:32.767100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083724+0000","last_change":"2024-09-21T07:47:38.094149+0000","last_active":"2024-09-21T07:47:39.083724+0000","last_peered":"2024-09-21T07:47:39.083724+0000","last_clean":"2024-09-21T07:47:39.083724+0000","last_became_active":"2024-09-21T07:47:38.093827+0000","last_became_peered":"2024-09-21T07:47:38.093827+0000","last_unstale":"2024-09-21T07:47:39.083724+0000","last_undegraded":"2024-09-21T07:47:39.083724+0000","last_fullsized":"2024-09-21T07:47:39.083724+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:12.606382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110056+0000","last_change":"2024-09-21T07:47:37.086480+0000","last_active":"2024-09-21T07:47:39.110056+0000","last_peered":"2024-09-21T07:47:39.110056+0000","last_clean":"2024-09-21T07:47:39.110056+0000","last_became_active":"2024-09-21T07:47:37.086159+0000","last_became_peered":"2024-09-21T07:47:37.086159+0000","last_unstale":"2024-09-21T07:47:39.110056+0000","last_undegraded":"2024-09-21T07:47:39.110056+0000","last_fullsized":"2024-09-21T07:47:39.110056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:37.967542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092142+0000","last_change":"2024-09-21T07:47:38.086940+0000","last_active":"2024-09-21T07:47:39.092142+0000","last_peered":"2024-09-21T07:47:39.092142+0000","last_clean":"2024-09-21T07:47:39.092142+0000","last_became_active":"2024-09-21T07:47:38.086557+0000","last_became_peered":"2024-09-21T07:47:38.086557+0000","last_unstale":"2024-09-21T07:47:39.092142+0000","last_undegraded":"2024-09-21T07:47:39.092142+0000","last_fullsized":"2024-09-21T07:47:39.092142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:23.238523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085352+0000","last_change":"2024-09-21T07:47:37.090277+0000","last_active":"2024-09-21T07:47:39.085352+0000","last_peered":"2024-09-21T07:47:39.085352+0000","last_clean":"2024-09-21T07:47:39.085352+0000","last_became_active":"2024-09-21T07:47:37.089931+0000","last_became_peered":"2024-09-21T07:47:37.089931+0000","last_unstale":"2024-09-21T07:47:39.085352+0000","last_undegraded":"2024-09-21T07:47:39.085352+0000","last_fullsized":"2024-09-21T07:47:39.085352+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:22.637572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081628+0000","last_change":"2024-09-21T07:47:38.087067+0000","last_active":"2024-09-21T07:47:39.081628+0000","last_peered":"2024-09-21T07:47:39.081628+0000","last_clean":"2024-09-21T07:47:39.081628+0000","last_became_active":"2024-09-21T07:47:38.086557+0000","last_became_peered":"2024-09-21T07:47:38.086557+0000","last_unstale":"2024-09-21T07:47:39.081628+0000","last_undegraded":"2024-09-21T07:47:39.081628+0000","last_fullsized":"2024-09-21T07:47:39.081628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:06.961821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.113392+0000","last_change":"2024-09-21T07:47:37.094651+0000","last_active":"2024-09-21T07:47:38.113392+0000","last_peered":"2024-09-21T07:47:38.113392+0000","last_clean":"2024-09-21T07:47:38.113392+0000","last_became_active":"2024-09-21T07:47:37.094327+0000","last_became_peered":"2024-09-21T07:47:37.094327+0000","last_unstale":"2024-09-21T07:47:38.113392+0000","last_undegraded":"2024-09-21T07:47:38.113392+0000","last_fullsized":"2024-09-21T07:47:38.113392+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:46.016163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.111627+0000","last_change":"2024-09-21T07:47:38.109411+0000","last_active":"2024-09-21T07:47:38.111627+0000","last_peered":"2024-09-21T07:47:38.111627+0000","last_clean":"2024-09-21T07:47:38.111627+0000","last_became_active":"2024-09-21T07:47:38.109021+0000","last_became_peered":"2024-09-21T07:47:38.109021+0000","last_unstale":"2024-09-21T07:47:38.111627+0000","last_undegraded":"2024-09-21T07:47:38.111627+0000","last_fullsized":"2024-09-21T07:47:38.111627+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:40.301303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144255+0000","last_change":"2024-09-21T07:47:37.091616+0000","last_active":"2024-09-21T07:47:39.144255+0000","last_peered":"2024-09-21T07:47:39.144255+0000","last_clean":"2024-09-21T07:47:39.144255+0000","last_became_active":"2024-09-21T07:47:37.091140+0000","last_became_peered":"2024-09-21T07:47:37.091140+0000","last_unstale":"2024-09-21T07:47:39.144255+0000","last_undegraded":"2024-09-21T07:47:39.144255+0000","last_fullsized":"2024-09-21T07:47:39.144255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:42.900329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.111499+0000","last_change":"2024-09-21T07:47:38.107834+0000","last_active":"2024-09-21T07:47:38.111499+0000","last_peered":"2024-09-21T07:47:38.111499+0000","last_clean":"2024-09-21T07:47:38.111499+0000","last_became_active":"2024-09-21T07:47:38.107552+0000","last_became_peered":"2024-09-21T07:47:38.107552+0000","last_unstale":"2024-09-21T07:47:38.111499+0000","last_undegraded":"2024-09-21T07:47:38.111499+0000","last_fullsized":"2024-09-21T07:47:38.111499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:25.439718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143707+0000","last_change":"2024-09-21T07:47:37.085379+0000","last_active":"2024-09-21T07:47:39.143707+0000","last_peered":"2024-09-21T07:47:39.143707+0000","last_clean":"2024-09-21T07:47:39.143707+0000","last_became_active":"2024-09-21T07:47:37.084111+0000","last_became_peered":"2024-09-21T07:47:37.084111+0000","last_unstale":"2024-09-21T07:47:39.143707+0000","last_undegraded":"2024-09-21T07:47:39.143707+0000","last_fullsized":"2024-09-21T07:47:39.143707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:11.524792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.109898+0000","last_change":"2024-09-21T07:47:38.094796+0000","last_active":"2024-09-21T07:47:39.109898+0000","last_peered":"2024-09-21T07:47:39.109898+0000","last_clean":"2024-09-21T07:47:39.109898+0000","last_became_active":"2024-09-21T07:47:38.094510+0000","last_became_peered":"2024-09-21T07:47:38.094510+0000","last_unstale":"2024-09-21T07:47:39.109898+0000","last_undegraded":"2024-09-21T07:47:39.109898+0000","last_fullsized":"2024-09-21T07:47:39.109898+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:23.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083190+0000","last_change":"2024-09-21T07:47:37.085298+0000","last_active":"2024-09-21T07:47:39.083190+0000","last_peered":"2024-09-21T07:47:39.083190+0000","last_clean":"2024-09-21T07:47:39.083190+0000","last_became_active":"2024-09-21T07:47:37.084949+0000","last_became_peered":"2024-09-21T07:47:37.084949+0000","last_unstale":"2024-09-21T07:47:39.083190+0000","last_undegraded":"2024-09-21T07:47:39.083190+0000","last_fullsized":"2024-09-21T07:47:39.083190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:29.010147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.111403+0000","last_change":"2024-09-21T07:47:38.106652+0000","last_active":"2024-09-21T07:47:38.111403+0000","last_peered":"2024-09-21T07:47:38.111403+0000","last_clean":"2024-09-21T07:47:38.111403+0000","last_became_active":"2024-09-21T07:47:38.099507+0000","last_became_peered":"2024-09-21T07:47:38.099507+0000","last_unstale":"2024-09-21T07:47:38.111403+0000","last_undegraded":"2024-09-21T07:47:38.111403+0000","last_fullsized":"2024-09-21T07:47:38.111403+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:04.554725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084100+0000","last_change":"2024-09-21T07:47:37.092638+0000","last_active":"2024-09-21T07:47:39.084100+0000","last_peered":"2024-09-21T07:47:39.084100+0000","last_clean":"2024-09-21T07:47:39.084100+0000","last_became_active":"2024-09-21T07:47:37.091373+0000","last_became_peered":"2024-09-21T07:47:37.091373+0000","last_unstale":"2024-09-21T07:47:39.084100+0000","last_undegraded":"2024-09-21T07:47:39.084100+0000","last_fullsized":"2024-09-21T07:47:39.084100+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:28.428685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.100110+0000","last_change":"2024-09-21T07:47:38.095639+0000","last_active":"2024-09-21T07:47:38.100110+0000","last_peered":"2024-09-21T07:47:38.100110+0000","last_clean":"2024-09-21T07:47:38.100110+0000","last_became_active":"2024-09-21T07:47:38.095240+0000","last_became_peered":"2024-09-21T07:47:38.095240+0000","last_unstale":"2024-09-21T07:47:38.100110+0000","last_undegraded":"2024-09-21T07:47:38.100110+0000","last_fullsized":"2024-09-21T07:47:38.100110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:03.411340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081477+0000","last_change":"2024-09-21T07:47:37.082957+0000","last_active":"2024-09-21T07:47:39.081477+0000","last_peered":"2024-09-21T07:47:39.081477+0000","last_clean":"2024-09-21T07:47:39.081477+0000","last_became_active":"2024-09-21T07:47:37.082549+0000","last_became_peered":"2024-09-21T07:47:37.082549+0000","last_unstale":"2024-09-21T07:47:39.081477+0000","last_undegraded":"2024-09-21T07:47:39.081477+0000","last_fullsized":"2024-09-21T07:47:39.081477+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.343306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110414+0000","last_change":"2024-09-21T07:47:38.091299+0000","last_active":"2024-09-21T07:47:39.110414+0000","last_peered":"2024-09-21T07:47:39.110414+0000","last_clean":"2024-09-21T07:47:39.110414+0000","last_became_active":"2024-09-21T07:47:38.090898+0000","last_became_peered":"2024-09-21T07:47:38.090898+0000","last_unstale":"2024-09-21T07:47:39.110414+0000","last_undegraded":"2024-09-21T07:47:39.110414+0000","last_fullsized":"2024-09-21T07:47:39.110414+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:16:54.050789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081964+0000","last_change":"2024-09-21T07:47:37.090141+0000","last_active":"2024-09-21T07:47:39.081964+0000","last_peered":"2024-09-21T07:47:39.081964+0000","last_clean":"2024-09-21T07:47:39.081964+0000","last_became_active":"2024-09-21T07:47:37.089859+0000","last_became_peered":"2024-09-21T07:47:37.089859+0000","last_unstale":"2024-09-21T07:47:39.081964+0000","last_undegraded":"2024-09-21T07:47:39.081964+0000","last_fullsized":"2024-09-21T07:47:39.081964+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:33.703657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081852+0000","last_change":"2024-09-21T07:47:38.082314+0000","last_active":"2024-09-21T07:47:39.081852+0000","last_peered":"2024-09-21T07:47:39.081852+0000","last_clean":"2024-09-21T07:47:39.081852+0000","last_became_active":"2024-09-21T07:47:38.081268+0000","last_became_peered":"2024-09-21T07:47:38.081268+0000","last_unstale":"2024-09-21T07:47:39.081852+0000","last_undegraded":"2024-09-21T07:47:39.081852+0000","last_fullsized":"2024-09-21T07:47:39.081852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:36.049845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144193+0000","last_change":"2024-09-21T07:47:37.082753+0000","last_active":"2024-09-21T07:47:39.144193+0000","last_peered":"2024-09-21T07:47:39.144193+0000","last_clean":"2024-09-21T07:47:39.144193+0000","last_became_active":"2024-09-21T07:47:37.082403+0000","last_became_peered":"2024-09-21T07:47:37.082403+0000","last_unstale":"2024-09-21T07:47:39.144193+0000","last_undegraded":"2024-09-21T07:47:39.144193+0000","last_fullsized":"2024-09-21T07:47:39.144193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:27.740391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083607+0000","last_change":"2024-09-21T07:47:38.091431+0000","last_active":"2024-09-21T07:47:39.083607+0000","last_peered":"2024-09-21T07:47:39.083607+0000","last_clean":"2024-09-21T07:47:39.083607+0000","last_became_active":"2024-09-21T07:47:38.090171+0000","last_became_peered":"2024-09-21T07:47:38.090171+0000","last_unstale":"2024-09-21T07:47:39.083607+0000","last_undegraded":"2024-09-21T07:47:39.083607+0000","last_fullsized":"2024-09-21T07:47:39.083607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:08.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084787+0000","last_change":"2024-09-21T07:47:37.088028+0000","last_active":"2024-09-21T07:47:39.084787+0000","last_peered":"2024-09-21T07:47:39.084787+0000","last_clean":"2024-09-21T07:47:39.084787+0000","last_became_active":"2024-09-21T07:47:37.087782+0000","last_became_peered":"2024-09-21T07:47:37.087782+0000","last_unstale":"2024-09-21T07:47:39.084787+0000","last_undegraded":"2024-09-21T07:47:39.084787+0000","last_fullsized":"2024-09-21T07:47:39.084787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:17.354229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144888+0000","last_change":"2024-09-21T07:47:38.096562+0000","last_active":"2024-09-21T07:47:39.144888+0000","last_peered":"2024-09-21T07:47:39.144888+0000","last_clean":"2024-09-21T07:47:39.144888+0000","last_became_active":"2024-09-21T07:47:38.096116+0000","last_became_peered":"2024-09-21T07:47:38.096116+0000","last_unstale":"2024-09-21T07:47:39.144888+0000","last_undegraded":"2024-09-21T07:47:39.144888+0000","last_fullsized":"2024-09-21T07:47:39.144888+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:30.016507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143955+0000","last_change":"2024-09-21T07:47:37.092974+0000","last_active":"2024-09-21T07:47:39.143955+0000","last_peered":"2024-09-21T07:47:39.143955+0000","last_clean":"2024-09-21T07:47:39.143955+0000","last_became_active":"2024-09-21T07:47:37.092661+0000","last_became_peered":"2024-09-21T07:47:37.092661+0000","last_unstale":"2024-09-21T07:47:39.143955+0000","last_undegraded":"2024-09-21T07:47:39.143955+0000","last_fullsized":"2024-09-21T07:47:39.143955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:37.967542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084871+0000","last_change":"2024-09-21T07:47:38.090655+0000","last_active":"2024-09-21T07:47:39.084871+0000","last_peered":"2024-09-21T07:47:39.084871+0000","last_clean":"2024-09-21T07:47:39.084871+0000","last_became_active":"2024-09-21T07:47:38.090302+0000","last_became_peered":"2024-09-21T07:47:38.090302+0000","last_unstale":"2024-09-21T07:47:39.084871+0000","last_undegraded":"2024-09-21T07:47:39.084871+0000","last_fullsized":"2024-09-21T07:47:39.084871+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:28.174591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.082955+0000","last_change":"2024-09-21T07:47:37.083425+0000","last_active":"2024-09-21T07:47:39.082955+0000","last_peered":"2024-09-21T07:47:39.082955+0000","last_clean":"2024-09-21T07:47:39.082955+0000","last_became_active":"2024-09-21T07:47:37.082065+0000","last_became_peered":"2024-09-21T07:47:37.082065+0000","last_unstale":"2024-09-21T07:47:39.082955+0000","last_undegraded":"2024-09-21T07:47:39.082955+0000","last_fullsized":"2024-09-21T07:47:39.082955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:55.279437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084051+0000","last_change":"2024-09-21T07:47:38.089132+0000","last_active":"2024-09-21T07:47:39.084051+0000","last_peered":"2024-09-21T07:47:39.084051+0000","last_clean":"2024-09-21T07:47:39.084051+0000","last_became_active":"2024-09-21T07:47:38.088893+0000","last_became_peered":"2024-09-21T07:47:38.088893+0000","last_unstale":"2024-09-21T07:47:39.084051+0000","last_undegraded":"2024-09-21T07:47:39.084051+0000","last_fullsized":"2024-09-21T07:47:39.084051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:42.013947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091749+0000","last_change":"2024-09-21T07:47:37.079797+0000","last_active":"2024-09-21T07:47:39.091749+0000","last_peered":"2024-09-21T07:47:39.091749+0000","last_clean":"2024-09-21T07:47:39.091749+0000","last_became_active":"2024-09-21T07:47:37.079462+0000","last_became_peered":"2024-09-21T07:47:37.079462+0000","last_unstale":"2024-09-21T07:47:39.091749+0000","last_undegraded":"2024-09-21T07:47:39.091749+0000","last_fullsized":"2024-09-21T07:47:39.091749+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:04.615809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092348+0000","last_change":"2024-09-21T07:47:38.098242+0000","last_active":"2024-09-21T07:47:39.092348+0000","last_peered":"2024-09-21T07:47:39.092348+0000","last_clean":"2024-09-21T07:47:39.092348+0000","last_became_active":"2024-09-21T07:47:38.097907+0000","last_became_peered":"2024-09-21T07:47:38.097907+0000","last_unstale":"2024-09-21T07:47:39.092348+0000","last_undegraded":"2024-09-21T07:47:39.092348+0000","last_fullsized":"2024-09-21T07:47:39.092348+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:28.227677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091043+0000","last_change":"2024-09-21T07:47:37.089407+0000","last_active":"2024-09-21T07:47:39.091043+0000","last_peered":"2024-09-21T07:47:39.091043+0000","last_clean":"2024-09-21T07:47:39.091043+0000","last_became_active":"2024-09-21T07:47:37.089094+0000","last_became_peered":"2024-09-21T07:47:37.089094+0000","last_unstale":"2024-09-21T07:47:39.091043+0000","last_undegraded":"2024-09-21T07:47:39.091043+0000","last_fullsized":"2024-09-21T07:47:39.091043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:44.944425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085289+0000","last_change":"2024-09-21T07:47:38.102803+0000","last_active":"2024-09-21T07:47:39.085289+0000","last_peered":"2024-09-21T07:47:39.085289+0000","last_clean":"2024-09-21T07:47:39.085289+0000","last_became_active":"2024-09-21T07:47:38.102365+0000","last_became_peered":"2024-09-21T07:47:38.102365+0000","last_unstale":"2024-09-21T07:47:39.085289+0000","last_undegraded":"2024-09-21T07:47:39.085289+0000","last_fullsized":"2024-09-21T07:47:39.085289+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:53.273731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144136+0000","last_change":"2024-09-21T07:47:37.087874+0000","last_active":"2024-09-21T07:47:39.144136+0000","last_peered":"2024-09-21T07:47:39.144136+0000","last_clean":"2024-09-21T07:47:39.144136+0000","last_became_active":"2024-09-21T07:47:37.087477+0000","last_became_peered":"2024-09-21T07:47:37.087477+0000","last_unstale":"2024-09-21T07:47:39.144136+0000","last_undegraded":"2024-09-21T07:47:39.144136+0000","last_fullsized":"2024-09-21T07:47:39.144136+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:56.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092271+0000","last_change":"2024-09-21T07:47:38.098268+0000","last_active":"2024-09-21T07:47:39.092271+0000","last_peered":"2024-09-21T07:47:39.092271+0000","last_clean":"2024-09-21T07:47:39.092271+0000","last_became_active":"2024-09-21T07:47:38.097979+0000","last_became_peered":"2024-09-21T07:47:38.097979+0000","last_unstale":"2024-09-21T07:47:39.092271+0000","last_undegraded":"2024-09-21T07:47:39.092271+0000","last_fullsized":"2024-09-21T07:47:39.092271+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:13.995955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.082329+0000","last_change":"2024-09-21T07:47:37.094946+0000","last_active":"2024-09-21T07:47:38.082329+0000","last_peered":"2024-09-21T07:47:38.082329+0000","last_clean":"2024-09-21T07:47:38.082329+0000","last_became_active":"2024-09-21T07:47:37.094169+0000","last_became_peered":"2024-09-21T07:47:37.094169+0000","last_unstale":"2024-09-21T07:47:38.082329+0000","last_undegraded":"2024-09-21T07:47:38.082329+0000","last_fullsized":"2024-09-21T07:47:38.082329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:40.234680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083861+0000","last_change":"2024-09-21T07:47:38.096655+0000","last_active":"2024-09-21T07:47:39.083861+0000","last_peered":"2024-09-21T07:47:39.083861+0000","last_clean":"2024-09-21T07:47:39.083861+0000","last_became_active":"2024-09-21T07:47:38.096041+0000","last_became_peered":"2024-09-21T07:47:38.096041+0000","last_unstale":"2024-09-21T07:47:39.083861+0000","last_undegraded":"2024-09-21T07:47:39.083861+0000","last_fullsized":"2024-09-21T07:47:39.083861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:12.379095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144156+0000","last_change":"2024-09-21T07:47:37.084447+0000","last_active":"2024-09-21T07:47:39.144156+0000","last_peered":"2024-09-21T07:47:39.144156+0000","last_clean":"2024-09-21T07:47:39.144156+0000","last_became_active":"2024-09-21T07:47:37.084079+0000","last_became_peered":"2024-09-21T07:47:37.084079+0000","last_unstale":"2024-09-21T07:47:39.144156+0000","last_undegraded":"2024-09-21T07:47:39.144156+0000","last_fullsized":"2024-09-21T07:47:39.144156+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:31.683862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083374+0000","last_change":"2024-09-21T07:47:38.081554+0000","last_active":"2024-09-21T07:47:39.083374+0000","last_peered":"2024-09-21T07:47:39.083374+0000","last_clean":"2024-09-21T07:47:39.083374+0000","last_became_active":"2024-09-21T07:47:38.080806+0000","last_became_peered":"2024-09-21T07:47:38.080806+0000","last_unstale":"2024-09-21T07:47:39.083374+0000","last_undegraded":"2024-09-21T07:47:39.083374+0000","last_fullsized":"2024-09-21T07:47:39.083374+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:23.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091999+0000","last_change":"2024-09-21T07:47:37.084257+0000","last_active":"2024-09-21T07:47:39.091999+0000","last_peered":"2024-09-21T07:47:39.091999+0000","last_clean":"2024-09-21T07:47:39.091999+0000","last_became_active":"2024-09-21T07:47:37.083905+0000","last_became_peered":"2024-09-21T07:47:37.083905+0000","last_unstale":"2024-09-21T07:47:39.091999+0000","last_undegraded":"2024-09-21T07:47:39.091999+0000","last_fullsized":"2024-09-21T07:47:39.091999+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:07.933495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081088+0000","last_change":"2024-09-21T07:47:38.087700+0000","last_active":"2024-09-21T07:47:39.081088+0000","last_peered":"2024-09-21T07:47:39.081088+0000","last_clean":"2024-09-21T07:47:39.081088+0000","last_became_active":"2024-09-21T07:47:38.085190+0000","last_became_peered":"2024-09-21T07:47:38.085190+0000","last_unstale":"2024-09-21T07:47:39.081088+0000","last_undegraded":"2024-09-21T07:47:39.081088+0000","last_fullsized":"2024-09-21T07:47:39.081088+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:27.330849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143821+0000","last_change":"2024-09-21T07:47:37.088866+0000","last_active":"2024-09-21T07:47:39.143821+0000","last_peered":"2024-09-21T07:47:39.143821+0000","last_clean":"2024-09-21T07:47:39.143821+0000","last_became_active":"2024-09-21T07:47:37.088596+0000","last_became_peered":"2024-09-21T07:47:37.088596+0000","last_unstale":"2024-09-21T07:47:39.143821+0000","last_undegraded":"2024-09-21T07:47:39.143821+0000","last_fullsized":"2024-09-21T07:47:39.143821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:05.735469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.109885+0000","last_change":"2024-09-21T07:47:38.083232+0000","last_active":"2024-09-21T07:47:39.109885+0000","last_peered":"2024-09-21T07:47:39.109885+0000","last_clean":"2024-09-21T07:47:39.109885+0000","last_became_active":"2024-09-21T07:47:38.082933+0000","last_became_peered":"2024-09-21T07:47:38.082933+0000","last_unstale":"2024-09-21T07:47:39.109885+0000","last_undegraded":"2024-09-21T07:47:39.109885+0000","last_fullsized":"2024-09-21T07:47:39.109885+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.948221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.088383+0000","last_change":"2024-09-21T07:47:37.090134+0000","last_active":"2024-09-21T07:47:38.088383+0000","last_peered":"2024-09-21T07:47:38.088383+0000","last_clean":"2024-09-21T07:47:38.088383+0000","last_became_active":"2024-09-21T07:47:37.089772+0000","last_became_peered":"2024-09-21T07:47:37.089772+0000","last_unstale":"2024-09-21T07:47:38.088383+0000","last_undegraded":"2024-09-21T07:47:38.088383+0000","last_fullsized":"2024-09-21T07:47:38.088383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:09.964846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084556+0000","last_change":"2024-09-21T07:47:38.084284+0000","last_active":"2024-09-21T07:47:39.084556+0000","last_peered":"2024-09-21T07:47:39.084556+0000","last_clean":"2024-09-21T07:47:39.084556+0000","last_became_active":"2024-09-21T07:47:38.083923+0000","last_became_peered":"2024-09-21T07:47:38.083923+0000","last_unstale":"2024-09-21T07:47:39.084556+0000","last_undegraded":"2024-09-21T07:47:39.084556+0000","last_fullsized":"2024-09-21T07:47:39.084556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:46.490520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092466+0000","last_change":"2024-09-21T07:47:37.085411+0000","last_active":"2024-09-21T07:47:39.092466+0000","last_peered":"2024-09-21T07:47:39.092466+0000","last_clean":"2024-09-21T07:47:39.092466+0000","last_became_active":"2024-09-21T07:47:37.085136+0000","last_became_peered":"2024-09-21T07:47:37.085136+0000","last_unstale":"2024-09-21T07:47:39.092466+0000","last_undegraded":"2024-09-21T07:47:39.092466+0000","last_fullsized":"2024-09-21T07:47:39.092466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:05.567551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.080989+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:47:39.080989+0000","last_peered":"2024-09-21T07:47:39.080989+0000","last_clean":"2024-09-21T07:47:39.080989+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:47:39.080989+0000","last_undegraded":"2024-09-21T07:47:39.080989+0000","last_fullsized":"2024-09-21T07:47:39.080989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476763,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":219353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":217831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476760,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":808179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476760,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":548864,"data_stored":218413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":548864,"data_stored":218357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":808131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":219445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":532480,"data_stored":211475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:42.511 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 -- 172.21.15.88:0/553492664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 msgr2=0x7fe9141535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 --2- 172.21.15.88:0/553492664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141535c0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fe914072d60 tx=0x7fe90802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 -- 172.21.15.88:0/553492664 shutdown_connections 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 --2- 172.21.15.88:0/553492664 >> v2:172.21.15.175:3300/0 conn(0x7fe9140ad0e0 0x7fe9140ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 --2- 172.21.15.88:0/553492664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141535c0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 -- 172.21.15.88:0/553492664 >> 172.21.15.88:0/553492664 conn(0x7fe91406d730 msgr2=0x7fe9140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.763+0000 7fe91bc00640 1 -- 172.21.15.88:0/553492664 shutdown_connections 2024-09-21T07:47:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.764+0000 7fe91bc00640 1 -- 172.21.15.88:0/553492664 wait complete. 2024-09-21T07:47:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.766+0000 7fe91bc00640 1 Processor -- start 2024-09-21T07:47:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.766+0000 7fe91bc00640 1 -- start start 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe91bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140ad0e0 0x7fe9141f8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe91bc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141f64a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe91bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe914160310 0x7fe9141f5300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe91bc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9140b4c20 con 0x7fe91415d760 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe91bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe914153c70 con 0x7fe914160310 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe91bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9140b4da0 con 0x7fe9140ad0e0 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.767+0000 7fe913e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141f64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141f64a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39132/0 (socket says 172.21.15.88:39132) 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 -- 172.21.15.88:0/1962498061 learned_addr learned my addr 172.21.15.88:0/1962498061 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe919200640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140ad0e0 0x7fe9141f8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe919c00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe914160310 0x7fe9141f5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 -- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140ad0e0 msgr2=0x7fe9141f8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140ad0e0 0x7fe9141f8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 -- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe914160310 msgr2=0x7fe9141f5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe914160310 0x7fe9141f5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 -- 172.21.15.88:0/1962498061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9141f9740 con 0x7fe91415d760 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe919200640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140ad0e0 0x7fe9141f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe913e00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141f64a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c9a0 tx=0x7fe90000ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.768+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe900007bb0 con 0x7fe91415d760 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.769+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe900007e40 con 0x7fe91415d760 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.769+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9141f9a20 con 0x7fe91415d760 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.769+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90000f6b0 con 0x7fe91415d760 2024-09-21T07:47:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.769+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe914073e90 con 0x7fe91415d760 2024-09-21T07:47:42.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.770+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe914161010 con 0x7fe91415d760 2024-09-21T07:47:42.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.771+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fe90000f850 con 0x7fe91415d760 2024-09-21T07:47:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.772+0000 7fe910c00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe8f4078d40 0x7fe8f407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.772+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fe90009b980 con 0x7fe91415d760 2024-09-21T07:47:42.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.772+0000 7fe919200640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe8f4078d40 0x7fe8f407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:42.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.773+0000 7fe919200640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe8f4078d40 0x7fe8f407b250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe908033000 tx=0x7fe908004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:42.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.776+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe900014030 con 0x7fe91415d760 2024-09-21T07:47:42.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.960+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe91415e260 con 0x7fe8f4078d40 2024-09-21T07:47:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.966+0000 7fe910c00640 1 -- 172.21.15.88:0/1962498061 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251946 (secure 0 0 0) 0x7fe91415e260 con 0x7fe8f4078d40 2024-09-21T07:47:42.971 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:42.972 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:47:42.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe8f4078d40 msgr2=0x7fe8f407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe8f4078d40 0x7fe8f407b250 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fe908033000 tx=0x7fe908004730 comp rx=0 tx=0).stop 2024-09-21T07:47:42.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 msgr2=0x7fe9141f64a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:42.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141f64a0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fe90000c9a0 tx=0x7fe90000ce60 comp rx=0 tx=0).stop 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 shutdown_connections 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe8f4078d40 0x7fe8f407b250 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe914160310 0x7fe9141f5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91415d760 0x7fe9141f64a0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 --2- 172.21.15.88:0/1962498061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140ad0e0 0x7fe9141f8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 >> 172.21.15.88:0/1962498061 conn(0x7fe91406d730 msgr2=0x7fe91415e5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 shutdown_connections 2024-09-21T07:47:42.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:42.970+0000 7fe91bc00640 1 -- 172.21.15.88:0/1962498061 wait complete. 2024-09-21T07:47:42.993 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":97,"stamp":"2024-09-21T07:47:41.878231+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":230000,"kb_used_data":5436,"kb_used_omap":12,"kb_used_meta":224371,"kb_avail":754744720,"statfs":{"total":773094113280,"available":772858593280,"internally_reserved":0,"allocated":5566464,"data_stored":2921184,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":229756465},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.825033"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144362+0000","last_change":"2024-09-21T07:47:38.088987+0000","last_active":"2024-09-21T07:47:39.144362+0000","last_peered":"2024-09-21T07:47:39.144362+0000","last_clean":"2024-09-21T07:47:39.144362+0000","last_became_active":"2024-09-21T07:47:38.088613+0000","last_became_peered":"2024-09-21T07:47:38.088613+0000","last_unstale":"2024-09-21T07:47:39.144362+0000","last_undegraded":"2024-09-21T07:47:39.144362+0000","last_fullsized":"2024-09-21T07:47:39.144362+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:16:10.683521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110178+0000","last_change":"2024-09-21T07:47:38.078774+0000","last_active":"2024-09-21T07:47:39.110178+0000","last_peered":"2024-09-21T07:47:39.110178+0000","last_clean":"2024-09-21T07:47:39.110178+0000","last_became_active":"2024-09-21T07:47:38.078523+0000","last_became_peered":"2024-09-21T07:47:38.078523+0000","last_unstale":"2024-09-21T07:47:39.110178+0000","last_undegraded":"2024-09-21T07:47:39.110178+0000","last_fullsized":"2024-09-21T07:47:39.110178+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.629109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081655+0000","last_change":"2024-09-21T07:47:38.081946+0000","last_active":"2024-09-21T07:47:39.081655+0000","last_peered":"2024-09-21T07:47:39.081655+0000","last_clean":"2024-09-21T07:47:39.081655+0000","last_became_active":"2024-09-21T07:47:38.081604+0000","last_became_peered":"2024-09-21T07:47:38.081604+0000","last_unstale":"2024-09-21T07:47:39.081655+0000","last_undegraded":"2024-09-21T07:47:39.081655+0000","last_fullsized":"2024-09-21T07:47:39.081655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.938971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083462+0000","last_change":"2024-09-21T07:47:38.101683+0000","last_active":"2024-09-21T07:47:39.083462+0000","last_peered":"2024-09-21T07:47:39.083462+0000","last_clean":"2024-09-21T07:47:39.083462+0000","last_became_active":"2024-09-21T07:47:38.101344+0000","last_became_peered":"2024-09-21T07:47:38.101344+0000","last_unstale":"2024-09-21T07:47:39.083462+0000","last_undegraded":"2024-09-21T07:47:39.083462+0000","last_fullsized":"2024-09-21T07:47:39.083462+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:53:07.985028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144074+0000","last_change":"2024-09-21T07:47:38.090516+0000","last_active":"2024-09-21T07:47:39.144074+0000","last_peered":"2024-09-21T07:47:39.144074+0000","last_clean":"2024-09-21T07:47:39.144074+0000","last_became_active":"2024-09-21T07:47:38.090197+0000","last_became_peered":"2024-09-21T07:47:38.090197+0000","last_unstale":"2024-09-21T07:47:39.144074+0000","last_undegraded":"2024-09-21T07:47:39.144074+0000","last_fullsized":"2024-09-21T07:47:39.144074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:23.643932+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144079+0000","last_change":"2024-09-21T07:47:38.099125+0000","last_active":"2024-09-21T07:47:39.144079+0000","last_peered":"2024-09-21T07:47:39.144079+0000","last_clean":"2024-09-21T07:47:39.144079+0000","last_became_active":"2024-09-21T07:47:38.098820+0000","last_became_peered":"2024-09-21T07:47:38.098820+0000","last_unstale":"2024-09-21T07:47:39.144079+0000","last_undegraded":"2024-09-21T07:47:39.144079+0000","last_fullsized":"2024-09-21T07:47:39.144079+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:45.573811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092440+0000","last_change":"2024-09-21T07:47:38.082352+0000","last_active":"2024-09-21T07:47:39.092440+0000","last_peered":"2024-09-21T07:47:39.092440+0000","last_clean":"2024-09-21T07:47:39.092440+0000","last_became_active":"2024-09-21T07:47:38.081939+0000","last_became_peered":"2024-09-21T07:47:38.081939+0000","last_unstale":"2024-09-21T07:47:39.092440+0000","last_undegraded":"2024-09-21T07:47:39.092440+0000","last_fullsized":"2024-09-21T07:47:39.092440+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:45:06.007142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084513+0000","last_change":"2024-09-21T07:47:38.096353+0000","last_active":"2024-09-21T07:47:39.084513+0000","last_peered":"2024-09-21T07:47:39.084513+0000","last_clean":"2024-09-21T07:47:39.084513+0000","last_became_active":"2024-09-21T07:47:38.096001+0000","last_became_peered":"2024-09-21T07:47:38.096001+0000","last_unstale":"2024-09-21T07:47:39.084513+0000","last_undegraded":"2024-09-21T07:47:39.084513+0000","last_fullsized":"2024-09-21T07:47:39.084513+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:53.467974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144583+0000","last_change":"2024-09-21T07:47:38.096562+0000","last_active":"2024-09-21T07:47:39.144583+0000","last_peered":"2024-09-21T07:47:39.144583+0000","last_clean":"2024-09-21T07:47:39.144583+0000","last_became_active":"2024-09-21T07:47:38.096173+0000","last_became_peered":"2024-09-21T07:47:38.096173+0000","last_unstale":"2024-09-21T07:47:39.144583+0000","last_undegraded":"2024-09-21T07:47:39.144583+0000","last_fullsized":"2024-09-21T07:47:39.144583+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:23.048496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085556+0000","last_change":"2024-09-21T07:47:38.089144+0000","last_active":"2024-09-21T07:47:39.085556+0000","last_peered":"2024-09-21T07:47:39.085556+0000","last_clean":"2024-09-21T07:47:39.085556+0000","last_became_active":"2024-09-21T07:47:38.088813+0000","last_became_peered":"2024-09-21T07:47:38.088813+0000","last_unstale":"2024-09-21T07:47:39.085556+0000","last_undegraded":"2024-09-21T07:47:39.085556+0000","last_fullsized":"2024-09-21T07:47:39.085556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:46.126791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083993+0000","last_change":"2024-09-21T07:47:38.093514+0000","last_active":"2024-09-21T07:47:39.083993+0000","last_peered":"2024-09-21T07:47:39.083993+0000","last_clean":"2024-09-21T07:47:39.083993+0000","last_became_active":"2024-09-21T07:47:38.093159+0000","last_became_peered":"2024-09-21T07:47:38.093159+0000","last_unstale":"2024-09-21T07:47:39.083993+0000","last_undegraded":"2024-09-21T07:47:39.083993+0000","last_fullsized":"2024-09-21T07:47:39.083993+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:40.465534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110260+0000","last_change":"2024-09-21T07:47:38.101535+0000","last_active":"2024-09-21T07:47:39.110260+0000","last_peered":"2024-09-21T07:47:39.110260+0000","last_clean":"2024-09-21T07:47:39.110260+0000","last_became_active":"2024-09-21T07:47:38.101231+0000","last_became_peered":"2024-09-21T07:47:38.101231+0000","last_unstale":"2024-09-21T07:47:39.110260+0000","last_undegraded":"2024-09-21T07:47:39.110260+0000","last_fullsized":"2024-09-21T07:47:39.110260+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:12.606382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144421+0000","last_change":"2024-09-21T07:47:38.098466+0000","last_active":"2024-09-21T07:47:39.144421+0000","last_peered":"2024-09-21T07:47:39.144421+0000","last_clean":"2024-09-21T07:47:39.144421+0000","last_became_active":"2024-09-21T07:47:38.098126+0000","last_became_peered":"2024-09-21T07:47:38.098126+0000","last_unstale":"2024-09-21T07:47:39.144421+0000","last_undegraded":"2024-09-21T07:47:39.144421+0000","last_fullsized":"2024-09-21T07:47:39.144421+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:56.285797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083127+0000","last_change":"2024-09-21T07:47:38.090993+0000","last_active":"2024-09-21T07:47:39.083127+0000","last_peered":"2024-09-21T07:47:39.083127+0000","last_clean":"2024-09-21T07:47:39.083127+0000","last_became_active":"2024-09-21T07:47:38.090643+0000","last_became_peered":"2024-09-21T07:47:38.090643+0000","last_unstale":"2024-09-21T07:47:39.083127+0000","last_undegraded":"2024-09-21T07:47:39.083127+0000","last_fullsized":"2024-09-21T07:47:39.083127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.948221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083635+0000","last_change":"2024-09-21T07:47:38.091569+0000","last_active":"2024-09-21T07:47:39.083635+0000","last_peered":"2024-09-21T07:47:39.083635+0000","last_clean":"2024-09-21T07:47:39.083635+0000","last_became_active":"2024-09-21T07:47:38.090147+0000","last_became_peered":"2024-09-21T07:47:38.090147+0000","last_unstale":"2024-09-21T07:47:39.083635+0000","last_undegraded":"2024-09-21T07:47:39.083635+0000","last_fullsized":"2024-09-21T07:47:39.083635+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:04.448926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144051+0000","last_change":"2024-09-21T07:47:38.094826+0000","last_active":"2024-09-21T07:47:39.144051+0000","last_peered":"2024-09-21T07:47:39.144051+0000","last_clean":"2024-09-21T07:47:39.144051+0000","last_became_active":"2024-09-21T07:47:38.094472+0000","last_became_peered":"2024-09-21T07:47:38.094472+0000","last_unstale":"2024-09-21T07:47:39.144051+0000","last_undegraded":"2024-09-21T07:47:39.144051+0000","last_fullsized":"2024-09-21T07:47:39.144051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:18.360590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110385+0000","last_change":"2024-09-21T07:47:38.096358+0000","last_active":"2024-09-21T07:47:39.110385+0000","last_peered":"2024-09-21T07:47:39.110385+0000","last_clean":"2024-09-21T07:47:39.110385+0000","last_became_active":"2024-09-21T07:47:38.096089+0000","last_became_peered":"2024-09-21T07:47:38.096089+0000","last_unstale":"2024-09-21T07:47:39.110385+0000","last_undegraded":"2024-09-21T07:47:39.110385+0000","last_fullsized":"2024-09-21T07:47:39.110385+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:42.013947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081368+0000","last_change":"2024-09-21T07:47:38.086044+0000","last_active":"2024-09-21T07:47:39.081368+0000","last_peered":"2024-09-21T07:47:39.081368+0000","last_clean":"2024-09-21T07:47:39.081368+0000","last_became_active":"2024-09-21T07:47:38.085701+0000","last_became_peered":"2024-09-21T07:47:38.085701+0000","last_unstale":"2024-09-21T07:47:39.081368+0000","last_undegraded":"2024-09-21T07:47:39.081368+0000","last_fullsized":"2024-09-21T07:47:39.081368+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:42.556673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144219+0000","last_change":"2024-09-21T07:47:38.092858+0000","last_active":"2024-09-21T07:47:39.144219+0000","last_peered":"2024-09-21T07:47:39.144219+0000","last_clean":"2024-09-21T07:47:39.144219+0000","last_became_active":"2024-09-21T07:47:38.092507+0000","last_became_peered":"2024-09-21T07:47:38.092507+0000","last_unstale":"2024-09-21T07:47:39.144219+0000","last_undegraded":"2024-09-21T07:47:39.144219+0000","last_fullsized":"2024-09-21T07:47:39.144219+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:33.773460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085175+0000","last_change":"2024-09-21T07:47:38.097972+0000","last_active":"2024-09-21T07:47:39.085175+0000","last_peered":"2024-09-21T07:47:39.085175+0000","last_clean":"2024-09-21T07:47:39.085175+0000","last_became_active":"2024-09-21T07:47:38.097671+0000","last_became_peered":"2024-09-21T07:47:38.097671+0000","last_unstale":"2024-09-21T07:47:39.085175+0000","last_undegraded":"2024-09-21T07:47:39.085175+0000","last_fullsized":"2024-09-21T07:47:39.085175+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:39.793352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083647+0000","last_change":"2024-09-21T07:47:38.081145+0000","last_active":"2024-09-21T07:47:39.083647+0000","last_peered":"2024-09-21T07:47:39.083647+0000","last_clean":"2024-09-21T07:47:39.083647+0000","last_became_active":"2024-09-21T07:47:38.080382+0000","last_became_peered":"2024-09-21T07:47:38.080382+0000","last_unstale":"2024-09-21T07:47:39.083647+0000","last_undegraded":"2024-09-21T07:47:39.083647+0000","last_fullsized":"2024-09-21T07:47:39.083647+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:40:40.629109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.099641+0000","last_change":"2024-09-21T07:47:38.097772+0000","last_active":"2024-09-21T07:47:38.099641+0000","last_peered":"2024-09-21T07:47:38.099641+0000","last_clean":"2024-09-21T07:47:38.099641+0000","last_became_active":"2024-09-21T07:47:38.097429+0000","last_became_peered":"2024-09-21T07:47:38.097429+0000","last_unstale":"2024-09-21T07:47:38.099641+0000","last_undegraded":"2024-09-21T07:47:38.099641+0000","last_fullsized":"2024-09-21T07:47:38.099641+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:44.178854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092166+0000","last_change":"2024-09-21T07:47:38.086850+0000","last_active":"2024-09-21T07:47:39.092166+0000","last_peered":"2024-09-21T07:47:39.092166+0000","last_clean":"2024-09-21T07:47:39.092166+0000","last_became_active":"2024-09-21T07:47:38.086504+0000","last_became_peered":"2024-09-21T07:47:38.086504+0000","last_unstale":"2024-09-21T07:47:39.092166+0000","last_undegraded":"2024-09-21T07:47:39.092166+0000","last_fullsized":"2024-09-21T07:47:39.092166+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:31.574133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110227+0000","last_change":"2024-09-21T07:47:38.088451+0000","last_active":"2024-09-21T07:47:39.110227+0000","last_peered":"2024-09-21T07:47:39.110227+0000","last_clean":"2024-09-21T07:47:39.110227+0000","last_became_active":"2024-09-21T07:47:38.088129+0000","last_became_peered":"2024-09-21T07:47:38.088129+0000","last_unstale":"2024-09-21T07:47:39.110227+0000","last_undegraded":"2024-09-21T07:47:39.110227+0000","last_fullsized":"2024-09-21T07:47:39.110227+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:18.360590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144410+0000","last_change":"2024-09-21T07:47:38.096517+0000","last_active":"2024-09-21T07:47:39.144410+0000","last_peered":"2024-09-21T07:47:39.144410+0000","last_clean":"2024-09-21T07:47:39.144410+0000","last_became_active":"2024-09-21T07:47:38.096186+0000","last_became_peered":"2024-09-21T07:47:38.096186+0000","last_unstale":"2024-09-21T07:47:39.144410+0000","last_undegraded":"2024-09-21T07:47:39.144410+0000","last_fullsized":"2024-09-21T07:47:39.144410+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:11:20.474763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084185+0000","last_change":"2024-09-21T07:47:38.093599+0000","last_active":"2024-09-21T07:47:39.084185+0000","last_peered":"2024-09-21T07:47:39.084185+0000","last_clean":"2024-09-21T07:47:39.084185+0000","last_became_active":"2024-09-21T07:47:38.092937+0000","last_became_peered":"2024-09-21T07:47:38.092937+0000","last_unstale":"2024-09-21T07:47:39.084185+0000","last_undegraded":"2024-09-21T07:47:39.084185+0000","last_fullsized":"2024-09-21T07:47:39.084185+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:16:54.050789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092655+0000","last_change":"2024-09-21T07:47:38.095340+0000","last_active":"2024-09-21T07:47:39.092655+0000","last_peered":"2024-09-21T07:47:39.092655+0000","last_clean":"2024-09-21T07:47:39.092655+0000","last_became_active":"2024-09-21T07:47:38.095040+0000","last_became_peered":"2024-09-21T07:47:38.095040+0000","last_unstale":"2024-09-21T07:47:39.092655+0000","last_undegraded":"2024-09-21T07:47:39.092655+0000","last_fullsized":"2024-09-21T07:47:39.092655+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:33.045018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092303+0000","last_change":"2024-09-21T07:47:38.087689+0000","last_active":"2024-09-21T07:47:39.092303+0000","last_peered":"2024-09-21T07:47:39.092303+0000","last_clean":"2024-09-21T07:47:39.092303+0000","last_became_active":"2024-09-21T07:47:38.087316+0000","last_became_peered":"2024-09-21T07:47:38.087316+0000","last_unstale":"2024-09-21T07:47:39.092303+0000","last_undegraded":"2024-09-21T07:47:39.092303+0000","last_fullsized":"2024-09-21T07:47:39.092303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:09:46.250888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144040+0000","last_change":"2024-09-21T07:47:38.097421+0000","last_active":"2024-09-21T07:47:39.144040+0000","last_peered":"2024-09-21T07:47:39.144040+0000","last_clean":"2024-09-21T07:47:39.144040+0000","last_became_active":"2024-09-21T07:47:38.095733+0000","last_became_peered":"2024-09-21T07:47:38.095733+0000","last_unstale":"2024-09-21T07:47:39.144040+0000","last_undegraded":"2024-09-21T07:47:39.144040+0000","last_fullsized":"2024-09-21T07:47:39.144040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:30.016507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.109928+0000","last_change":"2024-09-21T07:47:38.082968+0000","last_active":"2024-09-21T07:47:39.109928+0000","last_peered":"2024-09-21T07:47:39.109928+0000","last_clean":"2024-09-21T07:47:39.109928+0000","last_became_active":"2024-09-21T07:47:38.081334+0000","last_became_peered":"2024-09-21T07:47:38.081334+0000","last_unstale":"2024-09-21T07:47:39.109928+0000","last_undegraded":"2024-09-21T07:47:39.109928+0000","last_fullsized":"2024-09-21T07:47:39.109928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:04.448926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143995+0000","last_change":"2024-09-21T07:47:38.106778+0000","last_active":"2024-09-21T07:47:39.143995+0000","last_peered":"2024-09-21T07:47:39.143995+0000","last_clean":"2024-09-21T07:47:39.143995+0000","last_became_active":"2024-09-21T07:47:38.105636+0000","last_became_peered":"2024-09-21T07:47:38.105636+0000","last_unstale":"2024-09-21T07:47:39.143995+0000","last_undegraded":"2024-09-21T07:47:39.143995+0000","last_fullsized":"2024-09-21T07:47:39.143995+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:03.940811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084605+0000","last_change":"2024-09-21T07:47:38.100391+0000","last_active":"2024-09-21T07:47:39.084605+0000","last_peered":"2024-09-21T07:47:39.084605+0000","last_clean":"2024-09-21T07:47:39.084605+0000","last_became_active":"2024-09-21T07:47:38.099940+0000","last_became_peered":"2024-09-21T07:47:38.099940+0000","last_unstale":"2024-09-21T07:47:39.084605+0000","last_undegraded":"2024-09-21T07:47:39.084605+0000","last_fullsized":"2024-09-21T07:47:39.084605+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:53:49.088537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.106963+0000","last_change":"2024-09-21T07:47:38.101129+0000","last_active":"2024-09-21T07:47:38.106963+0000","last_peered":"2024-09-21T07:47:38.106963+0000","last_clean":"2024-09-21T07:47:38.106963+0000","last_became_active":"2024-09-21T07:47:38.099102+0000","last_became_peered":"2024-09-21T07:47:38.099102+0000","last_unstale":"2024-09-21T07:47:38.106963+0000","last_undegraded":"2024-09-21T07:47:38.106963+0000","last_fullsized":"2024-09-21T07:47:38.106963+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:26.118846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1f","version":"46'4","reported_seq":19,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144646+0000","last_change":"2024-09-21T07:47:37.094676+0000","last_active":"2024-09-21T07:47:39.144646+0000","last_peered":"2024-09-21T07:47:39.144646+0000","last_clean":"2024-09-21T07:47:39.144646+0000","last_became_active":"2024-09-21T07:47:37.093016+0000","last_became_peered":"2024-09-21T07:47:37.093016+0000","last_unstale":"2024-09-21T07:47:39.144646+0000","last_undegraded":"2024-09-21T07:47:39.144646+0000","last_fullsized":"2024-09-21T07:47:39.144646+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:38.743173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144509+0000","last_change":"2024-09-21T07:47:38.092907+0000","last_active":"2024-09-21T07:47:39.144509+0000","last_peered":"2024-09-21T07:47:39.144509+0000","last_clean":"2024-09-21T07:47:39.144509+0000","last_became_active":"2024-09-21T07:47:38.092573+0000","last_became_peered":"2024-09-21T07:47:38.092573+0000","last_unstale":"2024-09-21T07:47:39.144509+0000","last_undegraded":"2024-09-21T07:47:39.144509+0000","last_fullsized":"2024-09-21T07:47:39.144509+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:54:23.911296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092585+0000","last_change":"2024-09-21T07:47:37.078705+0000","last_active":"2024-09-21T07:47:39.092585+0000","last_peered":"2024-09-21T07:47:39.092585+0000","last_clean":"2024-09-21T07:47:39.092585+0000","last_became_active":"2024-09-21T07:47:37.078358+0000","last_became_peered":"2024-09-21T07:47:37.078358+0000","last_unstale":"2024-09-21T07:47:39.092585+0000","last_undegraded":"2024-09-21T07:47:39.092585+0000","last_fullsized":"2024-09-21T07:47:39.092585+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:18.594370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110303+0000","last_change":"2024-09-21T07:47:38.080169+0000","last_active":"2024-09-21T07:47:39.110303+0000","last_peered":"2024-09-21T07:47:39.110303+0000","last_clean":"2024-09-21T07:47:39.110303+0000","last_became_active":"2024-09-21T07:47:38.079881+0000","last_became_peered":"2024-09-21T07:47:38.079881+0000","last_unstale":"2024-09-21T07:47:39.110303+0000","last_undegraded":"2024-09-21T07:47:39.110303+0000","last_fullsized":"2024-09-21T07:47:39.110303+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:40.465534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1d","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144231+0000","last_change":"2024-09-21T07:47:37.088481+0000","last_active":"2024-09-21T07:47:39.144231+0000","last_peered":"2024-09-21T07:47:39.144231+0000","last_clean":"2024-09-21T07:47:39.144231+0000","last_became_active":"2024-09-21T07:47:37.088163+0000","last_became_peered":"2024-09-21T07:47:37.088163+0000","last_unstale":"2024-09-21T07:47:39.144231+0000","last_undegraded":"2024-09-21T07:47:39.144231+0000","last_fullsized":"2024-09-21T07:47:39.144231+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:52.070517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084185+0000","last_change":"2024-09-21T07:47:38.107122+0000","last_active":"2024-09-21T07:47:39.084185+0000","last_peered":"2024-09-21T07:47:39.084185+0000","last_clean":"2024-09-21T07:47:39.084185+0000","last_became_active":"2024-09-21T07:47:38.106606+0000","last_became_peered":"2024-09-21T07:47:38.106606+0000","last_unstale":"2024-09-21T07:47:39.084185+0000","last_undegraded":"2024-09-21T07:47:39.084185+0000","last_fullsized":"2024-09-21T07:47:39.084185+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:05.622169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1c","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092314+0000","last_change":"2024-09-21T07:47:37.080482+0000","last_active":"2024-09-21T07:47:39.092314+0000","last_peered":"2024-09-21T07:47:39.092314+0000","last_clean":"2024-09-21T07:47:39.092314+0000","last_became_active":"2024-09-21T07:47:37.080170+0000","last_became_peered":"2024-09-21T07:47:37.080170+0000","last_unstale":"2024-09-21T07:47:39.092314+0000","last_undegraded":"2024-09-21T07:47:39.092314+0000","last_fullsized":"2024-09-21T07:47:39.092314+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:11.372734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091939+0000","last_change":"2024-09-21T07:47:38.083936+0000","last_active":"2024-09-21T07:47:39.091939+0000","last_peered":"2024-09-21T07:47:39.091939+0000","last_clean":"2024-09-21T07:47:39.091939+0000","last_became_active":"2024-09-21T07:47:38.083583+0000","last_became_peered":"2024-09-21T07:47:38.083583+0000","last_unstale":"2024-09-21T07:47:39.091939+0000","last_undegraded":"2024-09-21T07:47:39.091939+0000","last_fullsized":"2024-09-21T07:47:39.091939+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:58.443712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1b","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144572+0000","last_change":"2024-09-21T07:47:37.081597+0000","last_active":"2024-09-21T07:47:39.144572+0000","last_peered":"2024-09-21T07:47:39.144572+0000","last_clean":"2024-09-21T07:47:39.144572+0000","last_became_active":"2024-09-21T07:47:37.081231+0000","last_became_peered":"2024-09-21T07:47:37.081231+0000","last_unstale":"2024-09-21T07:47:39.144572+0000","last_undegraded":"2024-09-21T07:47:39.144572+0000","last_fullsized":"2024-09-21T07:47:39.144572+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:46.179686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084780+0000","last_change":"2024-09-21T07:47:38.090129+0000","last_active":"2024-09-21T07:47:39.084780+0000","last_peered":"2024-09-21T07:47:39.084780+0000","last_clean":"2024-09-21T07:47:39.084780+0000","last_became_active":"2024-09-21T07:47:38.089807+0000","last_became_peered":"2024-09-21T07:47:38.089807+0000","last_unstale":"2024-09-21T07:47:39.084780+0000","last_undegraded":"2024-09-21T07:47:39.084780+0000","last_fullsized":"2024-09-21T07:47:39.084780+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:06.573911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1a","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091878+0000","last_change":"2024-09-21T07:47:37.089253+0000","last_active":"2024-09-21T07:47:39.091878+0000","last_peered":"2024-09-21T07:47:39.091878+0000","last_clean":"2024-09-21T07:47:39.091878+0000","last_became_active":"2024-09-21T07:47:37.088785+0000","last_became_peered":"2024-09-21T07:47:37.088785+0000","last_unstale":"2024-09-21T07:47:39.091878+0000","last_undegraded":"2024-09-21T07:47:39.091878+0000","last_fullsized":"2024-09-21T07:47:39.091878+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:39.559407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083024+0000","last_change":"2024-09-21T07:47:38.106650+0000","last_active":"2024-09-21T07:47:39.083024+0000","last_peered":"2024-09-21T07:47:39.083024+0000","last_clean":"2024-09-21T07:47:39.083024+0000","last_became_active":"2024-09-21T07:47:38.106299+0000","last_became_peered":"2024-09-21T07:47:38.106299+0000","last_unstale":"2024-09-21T07:47:39.083024+0000","last_undegraded":"2024-09-21T07:47:39.083024+0000","last_fullsized":"2024-09-21T07:47:39.083024+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:47:40.565767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143956+0000","last_change":"2024-09-21T07:47:37.094320+0000","last_active":"2024-09-21T07:47:39.143956+0000","last_peered":"2024-09-21T07:47:39.143956+0000","last_clean":"2024-09-21T07:47:39.143956+0000","last_became_active":"2024-09-21T07:47:37.094041+0000","last_became_peered":"2024-09-21T07:47:37.094041+0000","last_unstale":"2024-09-21T07:47:39.143956+0000","last_undegraded":"2024-09-21T07:47:39.143956+0000","last_fullsized":"2024-09-21T07:47:39.143956+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:16.849675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091876+0000","last_change":"2024-09-21T07:47:38.090178+0000","last_active":"2024-09-21T07:47:39.091876+0000","last_peered":"2024-09-21T07:47:39.091876+0000","last_clean":"2024-09-21T07:47:39.091876+0000","last_became_active":"2024-09-21T07:47:38.089884+0000","last_became_peered":"2024-09-21T07:47:38.089884+0000","last_unstale":"2024-09-21T07:47:39.091876+0000","last_undegraded":"2024-09-21T07:47:39.091876+0000","last_fullsized":"2024-09-21T07:47:39.091876+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:21:39.584964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083043+0000","last_change":"2024-09-21T07:47:37.085490+0000","last_active":"2024-09-21T07:47:39.083043+0000","last_peered":"2024-09-21T07:47:39.083043+0000","last_clean":"2024-09-21T07:47:39.083043+0000","last_became_active":"2024-09-21T07:47:37.085143+0000","last_became_peered":"2024-09-21T07:47:37.085143+0000","last_unstale":"2024-09-21T07:47:39.083043+0000","last_undegraded":"2024-09-21T07:47:39.083043+0000","last_fullsized":"2024-09-21T07:47:39.083043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:22.042136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.082027+0000","last_change":"2024-09-21T07:47:38.105161+0000","last_active":"2024-09-21T07:47:39.082027+0000","last_peered":"2024-09-21T07:47:39.082027+0000","last_clean":"2024-09-21T07:47:39.082027+0000","last_became_active":"2024-09-21T07:47:38.103450+0000","last_became_peered":"2024-09-21T07:47:38.103450+0000","last_unstale":"2024-09-21T07:47:39.082027+0000","last_undegraded":"2024-09-21T07:47:39.082027+0000","last_fullsized":"2024-09-21T07:47:39.082027+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:46.013625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.082006+0000","last_change":"2024-09-21T07:47:37.089929+0000","last_active":"2024-09-21T07:47:39.082006+0000","last_peered":"2024-09-21T07:47:39.082006+0000","last_clean":"2024-09-21T07:47:39.082006+0000","last_became_active":"2024-09-21T07:47:37.089621+0000","last_became_peered":"2024-09-21T07:47:37.089621+0000","last_unstale":"2024-09-21T07:47:39.082006+0000","last_undegraded":"2024-09-21T07:47:39.082006+0000","last_fullsized":"2024-09-21T07:47:39.082006+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:03:24.665789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081830+0000","last_change":"2024-09-21T07:47:38.090141+0000","last_active":"2024-09-21T07:47:39.081830+0000","last_peered":"2024-09-21T07:47:39.081830+0000","last_clean":"2024-09-21T07:47:39.081830+0000","last_became_active":"2024-09-21T07:47:38.089824+0000","last_became_peered":"2024-09-21T07:47:38.089824+0000","last_unstale":"2024-09-21T07:47:39.081830+0000","last_undegraded":"2024-09-21T07:47:39.081830+0000","last_fullsized":"2024-09-21T07:47:39.081830+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:04.677097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085495+0000","last_change":"2024-09-21T07:47:37.089606+0000","last_active":"2024-09-21T07:47:39.085495+0000","last_peered":"2024-09-21T07:47:39.085495+0000","last_clean":"2024-09-21T07:47:39.085495+0000","last_became_active":"2024-09-21T07:47:37.087153+0000","last_became_peered":"2024-09-21T07:47:37.087153+0000","last_unstale":"2024-09-21T07:47:39.085495+0000","last_undegraded":"2024-09-21T07:47:39.085495+0000","last_fullsized":"2024-09-21T07:47:39.085495+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:12:32.767100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083724+0000","last_change":"2024-09-21T07:47:38.094149+0000","last_active":"2024-09-21T07:47:39.083724+0000","last_peered":"2024-09-21T07:47:39.083724+0000","last_clean":"2024-09-21T07:47:39.083724+0000","last_became_active":"2024-09-21T07:47:38.093827+0000","last_became_peered":"2024-09-21T07:47:38.093827+0000","last_unstale":"2024-09-21T07:47:39.083724+0000","last_undegraded":"2024-09-21T07:47:39.083724+0000","last_fullsized":"2024-09-21T07:47:39.083724+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:12.606382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"46'3","reported_seq":18,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110056+0000","last_change":"2024-09-21T07:47:37.086480+0000","last_active":"2024-09-21T07:47:39.110056+0000","last_peered":"2024-09-21T07:47:39.110056+0000","last_clean":"2024-09-21T07:47:39.110056+0000","last_became_active":"2024-09-21T07:47:37.086159+0000","last_became_peered":"2024-09-21T07:47:37.086159+0000","last_unstale":"2024-09-21T07:47:39.110056+0000","last_undegraded":"2024-09-21T07:47:39.110056+0000","last_fullsized":"2024-09-21T07:47:39.110056+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:37.967542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092142+0000","last_change":"2024-09-21T07:47:38.086940+0000","last_active":"2024-09-21T07:47:39.092142+0000","last_peered":"2024-09-21T07:47:39.092142+0000","last_clean":"2024-09-21T07:47:39.092142+0000","last_became_active":"2024-09-21T07:47:38.086557+0000","last_became_peered":"2024-09-21T07:47:38.086557+0000","last_unstale":"2024-09-21T07:47:39.092142+0000","last_undegraded":"2024-09-21T07:47:39.092142+0000","last_fullsized":"2024-09-21T07:47:39.092142+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:23.238523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"46'6","reported_seq":21,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085352+0000","last_change":"2024-09-21T07:47:37.090277+0000","last_active":"2024-09-21T07:47:39.085352+0000","last_peered":"2024-09-21T07:47:39.085352+0000","last_clean":"2024-09-21T07:47:39.085352+0000","last_became_active":"2024-09-21T07:47:37.089931+0000","last_became_peered":"2024-09-21T07:47:37.089931+0000","last_unstale":"2024-09-21T07:47:39.085352+0000","last_undegraded":"2024-09-21T07:47:39.085352+0000","last_fullsized":"2024-09-21T07:47:39.085352+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:22.637572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081628+0000","last_change":"2024-09-21T07:47:38.087067+0000","last_active":"2024-09-21T07:47:39.081628+0000","last_peered":"2024-09-21T07:47:39.081628+0000","last_clean":"2024-09-21T07:47:39.081628+0000","last_became_active":"2024-09-21T07:47:38.086557+0000","last_became_peered":"2024-09-21T07:47:38.086557+0000","last_unstale":"2024-09-21T07:47:39.081628+0000","last_undegraded":"2024-09-21T07:47:39.081628+0000","last_fullsized":"2024-09-21T07:47:39.081628+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:19:06.961821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"46'2","reported_seq":15,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.113392+0000","last_change":"2024-09-21T07:47:37.094651+0000","last_active":"2024-09-21T07:47:38.113392+0000","last_peered":"2024-09-21T07:47:38.113392+0000","last_clean":"2024-09-21T07:47:38.113392+0000","last_became_active":"2024-09-21T07:47:37.094327+0000","last_became_peered":"2024-09-21T07:47:37.094327+0000","last_unstale":"2024-09-21T07:47:38.113392+0000","last_undegraded":"2024-09-21T07:47:38.113392+0000","last_fullsized":"2024-09-21T07:47:38.113392+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:45:46.016163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.111627+0000","last_change":"2024-09-21T07:47:38.109411+0000","last_active":"2024-09-21T07:47:38.111627+0000","last_peered":"2024-09-21T07:47:38.111627+0000","last_clean":"2024-09-21T07:47:38.111627+0000","last_became_active":"2024-09-21T07:47:38.109021+0000","last_became_peered":"2024-09-21T07:47:38.109021+0000","last_unstale":"2024-09-21T07:47:38.111627+0000","last_undegraded":"2024-09-21T07:47:38.111627+0000","last_fullsized":"2024-09-21T07:47:38.111627+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:40.301303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144255+0000","last_change":"2024-09-21T07:47:37.091616+0000","last_active":"2024-09-21T07:47:39.144255+0000","last_peered":"2024-09-21T07:47:39.144255+0000","last_clean":"2024-09-21T07:47:39.144255+0000","last_became_active":"2024-09-21T07:47:37.091140+0000","last_became_peered":"2024-09-21T07:47:37.091140+0000","last_unstale":"2024-09-21T07:47:39.144255+0000","last_undegraded":"2024-09-21T07:47:39.144255+0000","last_fullsized":"2024-09-21T07:47:39.144255+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:15:42.900329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.111499+0000","last_change":"2024-09-21T07:47:38.107834+0000","last_active":"2024-09-21T07:47:38.111499+0000","last_peered":"2024-09-21T07:47:38.111499+0000","last_clean":"2024-09-21T07:47:38.111499+0000","last_became_active":"2024-09-21T07:47:38.107552+0000","last_became_peered":"2024-09-21T07:47:38.107552+0000","last_unstale":"2024-09-21T07:47:38.111499+0000","last_undegraded":"2024-09-21T07:47:38.111499+0000","last_fullsized":"2024-09-21T07:47:38.111499+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:25.439718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143707+0000","last_change":"2024-09-21T07:47:37.085379+0000","last_active":"2024-09-21T07:47:39.143707+0000","last_peered":"2024-09-21T07:47:39.143707+0000","last_clean":"2024-09-21T07:47:39.143707+0000","last_became_active":"2024-09-21T07:47:37.084111+0000","last_became_peered":"2024-09-21T07:47:37.084111+0000","last_unstale":"2024-09-21T07:47:39.143707+0000","last_undegraded":"2024-09-21T07:47:39.143707+0000","last_fullsized":"2024-09-21T07:47:39.143707+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:20:11.524792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.109898+0000","last_change":"2024-09-21T07:47:38.094796+0000","last_active":"2024-09-21T07:47:39.109898+0000","last_peered":"2024-09-21T07:47:39.109898+0000","last_clean":"2024-09-21T07:47:39.109898+0000","last_became_active":"2024-09-21T07:47:38.094510+0000","last_became_peered":"2024-09-21T07:47:38.094510+0000","last_unstale":"2024-09-21T07:47:39.109898+0000","last_undegraded":"2024-09-21T07:47:39.109898+0000","last_fullsized":"2024-09-21T07:47:39.109898+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:23.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.10","version":"46'8","reported_seq":23,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083190+0000","last_change":"2024-09-21T07:47:37.085298+0000","last_active":"2024-09-21T07:47:39.083190+0000","last_peered":"2024-09-21T07:47:39.083190+0000","last_clean":"2024-09-21T07:47:39.083190+0000","last_became_active":"2024-09-21T07:47:37.084949+0000","last_became_peered":"2024-09-21T07:47:37.084949+0000","last_unstale":"2024-09-21T07:47:39.083190+0000","last_undegraded":"2024-09-21T07:47:39.083190+0000","last_fullsized":"2024-09-21T07:47:39.083190+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:29.010147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.111403+0000","last_change":"2024-09-21T07:47:38.106652+0000","last_active":"2024-09-21T07:47:38.111403+0000","last_peered":"2024-09-21T07:47:38.111403+0000","last_clean":"2024-09-21T07:47:38.111403+0000","last_became_active":"2024-09-21T07:47:38.099507+0000","last_became_peered":"2024-09-21T07:47:38.099507+0000","last_unstale":"2024-09-21T07:47:38.111403+0000","last_undegraded":"2024-09-21T07:47:38.111403+0000","last_fullsized":"2024-09-21T07:47:38.111403+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:04.554725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.f","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084100+0000","last_change":"2024-09-21T07:47:37.092638+0000","last_active":"2024-09-21T07:47:39.084100+0000","last_peered":"2024-09-21T07:47:39.084100+0000","last_clean":"2024-09-21T07:47:39.084100+0000","last_became_active":"2024-09-21T07:47:37.091373+0000","last_became_peered":"2024-09-21T07:47:37.091373+0000","last_unstale":"2024-09-21T07:47:39.084100+0000","last_undegraded":"2024-09-21T07:47:39.084100+0000","last_fullsized":"2024-09-21T07:47:39.084100+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:28.428685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":10,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.100110+0000","last_change":"2024-09-21T07:47:38.095639+0000","last_active":"2024-09-21T07:47:38.100110+0000","last_peered":"2024-09-21T07:47:38.100110+0000","last_clean":"2024-09-21T07:47:38.100110+0000","last_became_active":"2024-09-21T07:47:38.095240+0000","last_became_peered":"2024-09-21T07:47:38.095240+0000","last_unstale":"2024-09-21T07:47:38.100110+0000","last_undegraded":"2024-09-21T07:47:38.100110+0000","last_fullsized":"2024-09-21T07:47:38.100110+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:03.411340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.e","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081477+0000","last_change":"2024-09-21T07:47:37.082957+0000","last_active":"2024-09-21T07:47:39.081477+0000","last_peered":"2024-09-21T07:47:39.081477+0000","last_clean":"2024-09-21T07:47:39.081477+0000","last_became_active":"2024-09-21T07:47:37.082549+0000","last_became_peered":"2024-09-21T07:47:37.082549+0000","last_unstale":"2024-09-21T07:47:39.081477+0000","last_undegraded":"2024-09-21T07:47:39.081477+0000","last_fullsized":"2024-09-21T07:47:39.081477+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.343306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.110414+0000","last_change":"2024-09-21T07:47:38.091299+0000","last_active":"2024-09-21T07:47:39.110414+0000","last_peered":"2024-09-21T07:47:39.110414+0000","last_clean":"2024-09-21T07:47:39.110414+0000","last_became_active":"2024-09-21T07:47:38.090898+0000","last_became_peered":"2024-09-21T07:47:38.090898+0000","last_unstale":"2024-09-21T07:47:39.110414+0000","last_undegraded":"2024-09-21T07:47:39.110414+0000","last_fullsized":"2024-09-21T07:47:39.110414+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:16:54.050789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.d","version":"46'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081964+0000","last_change":"2024-09-21T07:47:37.090141+0000","last_active":"2024-09-21T07:47:39.081964+0000","last_peered":"2024-09-21T07:47:39.081964+0000","last_clean":"2024-09-21T07:47:39.081964+0000","last_became_active":"2024-09-21T07:47:37.089859+0000","last_became_peered":"2024-09-21T07:47:37.089859+0000","last_unstale":"2024-09-21T07:47:39.081964+0000","last_undegraded":"2024-09-21T07:47:39.081964+0000","last_fullsized":"2024-09-21T07:47:39.081964+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:33.703657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081852+0000","last_change":"2024-09-21T07:47:38.082314+0000","last_active":"2024-09-21T07:47:39.081852+0000","last_peered":"2024-09-21T07:47:39.081852+0000","last_clean":"2024-09-21T07:47:39.081852+0000","last_became_active":"2024-09-21T07:47:38.081268+0000","last_became_peered":"2024-09-21T07:47:38.081268+0000","last_unstale":"2024-09-21T07:47:39.081852+0000","last_undegraded":"2024-09-21T07:47:39.081852+0000","last_fullsized":"2024-09-21T07:47:39.081852+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:36.049845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144193+0000","last_change":"2024-09-21T07:47:37.082753+0000","last_active":"2024-09-21T07:47:39.144193+0000","last_peered":"2024-09-21T07:47:39.144193+0000","last_clean":"2024-09-21T07:47:39.144193+0000","last_became_active":"2024-09-21T07:47:37.082403+0000","last_became_peered":"2024-09-21T07:47:37.082403+0000","last_unstale":"2024-09-21T07:47:39.144193+0000","last_undegraded":"2024-09-21T07:47:39.144193+0000","last_fullsized":"2024-09-21T07:47:39.144193+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:10:27.740391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083607+0000","last_change":"2024-09-21T07:47:38.091431+0000","last_active":"2024-09-21T07:47:39.083607+0000","last_peered":"2024-09-21T07:47:39.083607+0000","last_clean":"2024-09-21T07:47:39.083607+0000","last_became_active":"2024-09-21T07:47:38.090171+0000","last_became_peered":"2024-09-21T07:47:38.090171+0000","last_unstale":"2024-09-21T07:47:39.083607+0000","last_undegraded":"2024-09-21T07:47:39.083607+0000","last_fullsized":"2024-09-21T07:47:39.083607+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:08.939855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.b","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084787+0000","last_change":"2024-09-21T07:47:37.088028+0000","last_active":"2024-09-21T07:47:39.084787+0000","last_peered":"2024-09-21T07:47:39.084787+0000","last_clean":"2024-09-21T07:47:39.084787+0000","last_became_active":"2024-09-21T07:47:37.087782+0000","last_became_peered":"2024-09-21T07:47:37.087782+0000","last_unstale":"2024-09-21T07:47:39.084787+0000","last_undegraded":"2024-09-21T07:47:39.084787+0000","last_fullsized":"2024-09-21T07:47:39.084787+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:58:17.354229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144888+0000","last_change":"2024-09-21T07:47:38.096562+0000","last_active":"2024-09-21T07:47:39.144888+0000","last_peered":"2024-09-21T07:47:39.144888+0000","last_clean":"2024-09-21T07:47:39.144888+0000","last_became_active":"2024-09-21T07:47:38.096116+0000","last_became_peered":"2024-09-21T07:47:38.096116+0000","last_unstale":"2024-09-21T07:47:39.144888+0000","last_undegraded":"2024-09-21T07:47:39.144888+0000","last_fullsized":"2024-09-21T07:47:39.144888+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:30.016507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143955+0000","last_change":"2024-09-21T07:47:37.092974+0000","last_active":"2024-09-21T07:47:39.143955+0000","last_peered":"2024-09-21T07:47:39.143955+0000","last_clean":"2024-09-21T07:47:39.143955+0000","last_became_active":"2024-09-21T07:47:37.092661+0000","last_became_peered":"2024-09-21T07:47:37.092661+0000","last_unstale":"2024-09-21T07:47:39.143955+0000","last_undegraded":"2024-09-21T07:47:39.143955+0000","last_fullsized":"2024-09-21T07:47:39.143955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:50:37.967542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084871+0000","last_change":"2024-09-21T07:47:38.090655+0000","last_active":"2024-09-21T07:47:39.084871+0000","last_peered":"2024-09-21T07:47:39.084871+0000","last_clean":"2024-09-21T07:47:39.084871+0000","last_became_active":"2024-09-21T07:47:38.090302+0000","last_became_peered":"2024-09-21T07:47:38.090302+0000","last_unstale":"2024-09-21T07:47:39.084871+0000","last_undegraded":"2024-09-21T07:47:39.084871+0000","last_fullsized":"2024-09-21T07:47:39.084871+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:28.174591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.082955+0000","last_change":"2024-09-21T07:47:37.083425+0000","last_active":"2024-09-21T07:47:39.082955+0000","last_peered":"2024-09-21T07:47:39.082955+0000","last_clean":"2024-09-21T07:47:39.082955+0000","last_became_active":"2024-09-21T07:47:37.082065+0000","last_became_peered":"2024-09-21T07:47:37.082065+0000","last_unstale":"2024-09-21T07:47:39.082955+0000","last_undegraded":"2024-09-21T07:47:39.082955+0000","last_fullsized":"2024-09-21T07:47:39.082955+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:55.279437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084051+0000","last_change":"2024-09-21T07:47:38.089132+0000","last_active":"2024-09-21T07:47:39.084051+0000","last_peered":"2024-09-21T07:47:39.084051+0000","last_clean":"2024-09-21T07:47:39.084051+0000","last_became_active":"2024-09-21T07:47:38.088893+0000","last_became_peered":"2024-09-21T07:47:38.088893+0000","last_unstale":"2024-09-21T07:47:39.084051+0000","last_undegraded":"2024-09-21T07:47:39.084051+0000","last_fullsized":"2024-09-21T07:47:39.084051+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:05:42.013947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091749+0000","last_change":"2024-09-21T07:47:37.079797+0000","last_active":"2024-09-21T07:47:39.091749+0000","last_peered":"2024-09-21T07:47:39.091749+0000","last_clean":"2024-09-21T07:47:39.091749+0000","last_became_active":"2024-09-21T07:47:37.079462+0000","last_became_peered":"2024-09-21T07:47:37.079462+0000","last_unstale":"2024-09-21T07:47:39.091749+0000","last_undegraded":"2024-09-21T07:47:39.091749+0000","last_fullsized":"2024-09-21T07:47:39.091749+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:04.615809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092348+0000","last_change":"2024-09-21T07:47:38.098242+0000","last_active":"2024-09-21T07:47:39.092348+0000","last_peered":"2024-09-21T07:47:39.092348+0000","last_clean":"2024-09-21T07:47:39.092348+0000","last_became_active":"2024-09-21T07:47:38.097907+0000","last_became_peered":"2024-09-21T07:47:38.097907+0000","last_unstale":"2024-09-21T07:47:39.092348+0000","last_undegraded":"2024-09-21T07:47:39.092348+0000","last_fullsized":"2024-09-21T07:47:39.092348+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:36:28.227677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.6","version":"47'1","reported_seq":16,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091043+0000","last_change":"2024-09-21T07:47:37.089407+0000","last_active":"2024-09-21T07:47:39.091043+0000","last_peered":"2024-09-21T07:47:39.091043+0000","last_clean":"2024-09-21T07:47:39.091043+0000","last_became_active":"2024-09-21T07:47:37.089094+0000","last_became_peered":"2024-09-21T07:47:37.089094+0000","last_unstale":"2024-09-21T07:47:39.091043+0000","last_undegraded":"2024-09-21T07:47:39.091043+0000","last_fullsized":"2024-09-21T07:47:39.091043+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:44.944425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.085289+0000","last_change":"2024-09-21T07:47:38.102803+0000","last_active":"2024-09-21T07:47:39.085289+0000","last_peered":"2024-09-21T07:47:39.085289+0000","last_clean":"2024-09-21T07:47:39.085289+0000","last_became_active":"2024-09-21T07:47:38.102365+0000","last_became_peered":"2024-09-21T07:47:38.102365+0000","last_unstale":"2024-09-21T07:47:39.085289+0000","last_undegraded":"2024-09-21T07:47:39.085289+0000","last_fullsized":"2024-09-21T07:47:39.085289+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:53.273731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144136+0000","last_change":"2024-09-21T07:47:37.087874+0000","last_active":"2024-09-21T07:47:39.144136+0000","last_peered":"2024-09-21T07:47:39.144136+0000","last_clean":"2024-09-21T07:47:39.144136+0000","last_became_active":"2024-09-21T07:47:37.087477+0000","last_became_peered":"2024-09-21T07:47:37.087477+0000","last_unstale":"2024-09-21T07:47:39.144136+0000","last_undegraded":"2024-09-21T07:47:39.144136+0000","last_fullsized":"2024-09-21T07:47:39.144136+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:52:56.035451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092271+0000","last_change":"2024-09-21T07:47:38.098268+0000","last_active":"2024-09-21T07:47:39.092271+0000","last_peered":"2024-09-21T07:47:39.092271+0000","last_clean":"2024-09-21T07:47:39.092271+0000","last_became_active":"2024-09-21T07:47:38.097979+0000","last_became_peered":"2024-09-21T07:47:38.097979+0000","last_unstale":"2024-09-21T07:47:39.092271+0000","last_undegraded":"2024-09-21T07:47:39.092271+0000","last_fullsized":"2024-09-21T07:47:39.092271+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:13.995955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.082329+0000","last_change":"2024-09-21T07:47:37.094946+0000","last_active":"2024-09-21T07:47:38.082329+0000","last_peered":"2024-09-21T07:47:38.082329+0000","last_clean":"2024-09-21T07:47:38.082329+0000","last_became_active":"2024-09-21T07:47:37.094169+0000","last_became_peered":"2024-09-21T07:47:37.094169+0000","last_unstale":"2024-09-21T07:47:38.082329+0000","last_undegraded":"2024-09-21T07:47:38.082329+0000","last_fullsized":"2024-09-21T07:47:38.082329+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:31:40.234680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083861+0000","last_change":"2024-09-21T07:47:38.096655+0000","last_active":"2024-09-21T07:47:39.083861+0000","last_peered":"2024-09-21T07:47:39.083861+0000","last_clean":"2024-09-21T07:47:39.083861+0000","last_became_active":"2024-09-21T07:47:38.096041+0000","last_became_peered":"2024-09-21T07:47:38.096041+0000","last_unstale":"2024-09-21T07:47:39.083861+0000","last_undegraded":"2024-09-21T07:47:39.083861+0000","last_fullsized":"2024-09-21T07:47:39.083861+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:47:12.379095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.144156+0000","last_change":"2024-09-21T07:47:37.084447+0000","last_active":"2024-09-21T07:47:39.144156+0000","last_peered":"2024-09-21T07:47:39.144156+0000","last_clean":"2024-09-21T07:47:39.144156+0000","last_became_active":"2024-09-21T07:47:37.084079+0000","last_became_peered":"2024-09-21T07:47:37.084079+0000","last_unstale":"2024-09-21T07:47:39.144156+0000","last_undegraded":"2024-09-21T07:47:39.144156+0000","last_fullsized":"2024-09-21T07:47:39.144156+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:18:31.683862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.083374+0000","last_change":"2024-09-21T07:47:38.081554+0000","last_active":"2024-09-21T07:47:39.083374+0000","last_peered":"2024-09-21T07:47:39.083374+0000","last_clean":"2024-09-21T07:47:39.083374+0000","last_became_active":"2024-09-21T07:47:38.080806+0000","last_became_peered":"2024-09-21T07:47:38.080806+0000","last_unstale":"2024-09-21T07:47:39.083374+0000","last_undegraded":"2024-09-21T07:47:39.083374+0000","last_fullsized":"2024-09-21T07:47:39.083374+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:16:23.638474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.091999+0000","last_change":"2024-09-21T07:47:37.084257+0000","last_active":"2024-09-21T07:47:39.091999+0000","last_peered":"2024-09-21T07:47:39.091999+0000","last_clean":"2024-09-21T07:47:39.091999+0000","last_became_active":"2024-09-21T07:47:37.083905+0000","last_became_peered":"2024-09-21T07:47:37.083905+0000","last_unstale":"2024-09-21T07:47:39.091999+0000","last_undegraded":"2024-09-21T07:47:39.091999+0000","last_fullsized":"2024-09-21T07:47:39.091999+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:07.933495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.081088+0000","last_change":"2024-09-21T07:47:38.087700+0000","last_active":"2024-09-21T07:47:39.081088+0000","last_peered":"2024-09-21T07:47:39.081088+0000","last_clean":"2024-09-21T07:47:39.081088+0000","last_became_active":"2024-09-21T07:47:38.085190+0000","last_became_peered":"2024-09-21T07:47:38.085190+0000","last_unstale":"2024-09-21T07:47:39.081088+0000","last_undegraded":"2024-09-21T07:47:39.081088+0000","last_fullsized":"2024-09-21T07:47:39.081088+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:46:27.330849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":15,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.143821+0000","last_change":"2024-09-21T07:47:37.088866+0000","last_active":"2024-09-21T07:47:39.143821+0000","last_peered":"2024-09-21T07:47:39.143821+0000","last_clean":"2024-09-21T07:47:39.143821+0000","last_became_active":"2024-09-21T07:47:37.088596+0000","last_became_peered":"2024-09-21T07:47:37.088596+0000","last_unstale":"2024-09-21T07:47:39.143821+0000","last_undegraded":"2024-09-21T07:47:39.143821+0000","last_fullsized":"2024-09-21T07:47:39.143821+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:20:05.735469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.109885+0000","last_change":"2024-09-21T07:47:38.083232+0000","last_active":"2024-09-21T07:47:39.109885+0000","last_peered":"2024-09-21T07:47:39.109885+0000","last_clean":"2024-09-21T07:47:39.109885+0000","last_became_active":"2024-09-21T07:47:38.082933+0000","last_became_peered":"2024-09-21T07:47:38.082933+0000","last_unstale":"2024-09-21T07:47:39.109885+0000","last_undegraded":"2024-09-21T07:47:39.109885+0000","last_fullsized":"2024-09-21T07:47:39.109885+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:49:31.948221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":13,"reported_epoch":46,"state":"active+clean","last_fresh":"2024-09-21T07:47:38.088383+0000","last_change":"2024-09-21T07:47:37.090134+0000","last_active":"2024-09-21T07:47:38.088383+0000","last_peered":"2024-09-21T07:47:38.088383+0000","last_clean":"2024-09-21T07:47:38.088383+0000","last_became_active":"2024-09-21T07:47:37.089772+0000","last_became_peered":"2024-09-21T07:47:37.089772+0000","last_unstale":"2024-09-21T07:47:38.088383+0000","last_undegraded":"2024-09-21T07:47:38.088383+0000","last_fullsized":"2024-09-21T07:47:38.088383+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:09.964846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":13,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.084556+0000","last_change":"2024-09-21T07:47:38.084284+0000","last_active":"2024-09-21T07:47:39.084556+0000","last_peered":"2024-09-21T07:47:39.084556+0000","last_clean":"2024-09-21T07:47:39.084556+0000","last_became_active":"2024-09-21T07:47:38.083923+0000","last_became_peered":"2024-09-21T07:47:38.083923+0000","last_unstale":"2024-09-21T07:47:39.084556+0000","last_undegraded":"2024-09-21T07:47:39.084556+0000","last_fullsized":"2024-09-21T07:47:39.084556+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":45,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:37.051041+0000","last_clean_scrub_stamp":"2024-09-21T07:47:37.051041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:57:46.490520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.7","version":"46'2","reported_seq":17,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.092466+0000","last_change":"2024-09-21T07:47:37.085411+0000","last_active":"2024-09-21T07:47:39.092466+0000","last_peered":"2024-09-21T07:47:39.092466+0000","last_clean":"2024-09-21T07:47:39.092466+0000","last_became_active":"2024-09-21T07:47:37.085136+0000","last_became_peered":"2024-09-21T07:47:37.085136+0000","last_unstale":"2024-09-21T07:47:39.092466+0000","last_undegraded":"2024-09-21T07:47:39.092466+0000","last_fullsized":"2024-09-21T07:47:39.092466+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":44,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:47:36.044680+0000","last_clean_scrub_stamp":"2024-09-21T07:47:36.044680+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:05.567551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":155,"reported_epoch":47,"state":"active+clean","last_fresh":"2024-09-21T07:47:39.080989+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:47:39.080989+0000","last_peered":"2024-09-21T07:47:39.080989+0000","last_clean":"2024-09-21T07:47:39.080989+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:47:39.080989+0000","last_undegraded":"2024-09-21T07:47:39.080989+0000","last_fullsized":"2024-09-21T07:47:39.080989+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476763,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27572,"kb_used_data":540,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344268,"statfs":{"total":96636764160,"available":96608530432,"internally_reserved":0,"allocated":552960,"data_stored":219353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476763,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27560,"kb_used_data":532,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344280,"statfs":{"total":96636764160,"available":96608542720,"internally_reserved":0,"allocated":544768,"data_stored":217831,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476760,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32300,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339540,"statfs":{"total":96636764160,"available":96603688960,"internally_reserved":0,"allocated":1138688,"data_stored":808179,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476760,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":548864,"data_stored":218413,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509463,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27568,"kb_used_data":536,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344272,"statfs":{"total":96636764160,"available":96608534528,"internally_reserved":0,"allocated":548864,"data_stored":218357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509464,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28144,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343696,"statfs":{"total":96636764160,"available":96607944704,"internally_reserved":0,"allocated":1138688,"data_stored":808131,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509464,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27576,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344264,"statfs":{"total":96636764160,"available":96608526336,"internally_reserved":0,"allocated":561152,"data_stored":219445,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542168,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31712,"kb_used_data":520,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94340128,"statfs":{"total":96636764160,"available":96604291072,"internally_reserved":0,"allocated":532480,"data_stored":211475,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:47:42.996 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:47:42.997 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:47:43.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 -- 172.21.15.88:0/925003004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c145550 msgr2=0x7fa46c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 --2- 172.21.15.88:0/925003004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c145550 0x7fa46c06fb40 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fa45c005fd0 tx=0x7fa45c02a660 comp rx=0 tx=0).stop 2024-09-21T07:47:43.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 -- 172.21.15.88:0/925003004 shutdown_connections 2024-09-21T07:47:43.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 --2- 172.21.15.88:0/925003004 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa46c073c50 0x7fa46c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 --2- 172.21.15.88:0/925003004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c145550 0x7fa46c06fb40 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 -- 172.21.15.88:0/925003004 >> 172.21.15.88:0/925003004 conn(0x7fa46c06d1c0 msgr2=0x7fa46c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 -- 172.21.15.88:0/925003004 shutdown_connections 2024-09-21T07:47:43.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.259+0000 7fa472400640 1 -- 172.21.15.88:0/925003004 wait complete. 2024-09-21T07:47:43.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.261+0000 7fa472400640 1 Processor -- start 2024-09-21T07:47:43.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.261+0000 7fa472400640 1 -- start start 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa472400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa46c145550 0x7fa46c13dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa472400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa46c145a40 0x7fa46c15ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa472400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 0x7fa46c0adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa472400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c0ce150 con 0x7fa46c0b3f20 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa472400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa46c0cdfd0 con 0x7fa46c145a40 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa472400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa46c0ce2d0 con 0x7fa46c145550 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa46be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 0x7fa46c0adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa46be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 0x7fa46c0adbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39146/0 (socket says 172.21.15.88:39146) 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa46be00640 1 -- 172.21.15.88:0/2603405892 learned_addr learned my addr 172.21.15.88:0/2603405892 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa46b400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa46c145550 0x7fa46c13dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa462a00640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa46c145a40 0x7fa46c15ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa46be00640 1 -- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa46c145550 msgr2=0x7fa46c13dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.263+0000 7fa46be00640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa46c145550 0x7fa46c13dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa46be00640 1 -- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa46c145a40 msgr2=0x7fa46c15ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa46be00640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa46c145a40 0x7fa46c15ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa46be00640 1 -- 172.21.15.88:0/2603405892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa46c0b3620 con 0x7fa46c0b3f20 2024-09-21T07:47:43.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa46be00640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 0x7fa46c0adbf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa46400e960 tx=0x7fa46400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464004d20 con 0x7fa46c0b3f20 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa46400cac0 con 0x7fa46c0b3f20 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa464005020 con 0x7fa46c0b3f20 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa46c0b3900 con 0x7fa46c0b3f20 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.264+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa46c0b3be0 con 0x7fa46c0b3f20 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.265+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa46c0b4400 con 0x7fa46c0b3f20 2024-09-21T07:47:43.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.267+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fa4640051c0 con 0x7fa46c0b3f20 2024-09-21T07:47:43.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.267+0000 7fa463e00640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa448078d40 0x7fa44807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.268+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7fa46409b910 con 0x7fa46c0b3f20 2024-09-21T07:47:43.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.268+0000 7fa46b400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa448078d40 0x7fa44807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.269+0000 7fa46b400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa448078d40 0x7fa44807b250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa45c033000 tx=0x7fa45c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.272+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa464016030 con 0x7fa46c0b3f20 2024-09-21T07:47:43.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.526+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fa46c0ce7f0 con 0x7fa46c0b3f20 2024-09-21T07:47:43.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.530+0000 7fa463e00640 1 -- 172.21.15.88:0/2603405892 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v10) ==== 190+0+147 (secure 0 0 0) 0x7fa464063410 con 0x7fa46c0b3f20 2024-09-21T07:47:43.532 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:47:43.533 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:47:43.533 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:47:43.533 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:47:43.533 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:47:43.533 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa448078d40 msgr2=0x7fa44807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa448078d40 0x7fa44807b250 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fa45c033000 tx=0x7fa45c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 msgr2=0x7fa46c0adbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 0x7fa46c0adbf0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fa46400e960 tx=0x7fa46400ee20 comp rx=0 tx=0).stop 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 shutdown_connections 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa448078d40 0x7fa44807b250 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa46c0b3f20 0x7fa46c0adbf0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa46c145a40 0x7fa46c15ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 --2- 172.21.15.88:0/2603405892 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa46c145550 0x7fa46c13dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 >> 172.21.15.88:0/2603405892 conn(0x7fa46c06d1c0 msgr2=0x7fa46c0b2100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 shutdown_connections 2024-09-21T07:47:43.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.533+0000 7fa472400640 1 -- 172.21.15.88:0/2603405892 wait complete. 2024-09-21T07:47:43.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:43.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 -- 172.21.15.88:0/1689135717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 msgr2=0x7f3fc0003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 --2- 172.21.15.88:0/1689135717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fc0003ec0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009b50 tx=0x7f3fc0005830 comp rx=0 tx=0).stop 2024-09-21T07:47:43.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 -- 172.21.15.88:0/1689135717 shutdown_connections 2024-09-21T07:47:43.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 --2- 172.21.15.88:0/1689135717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fc0003ec0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 -- 172.21.15.88:0/1689135717 >> 172.21.15.88:0/1689135717 conn(0x7f3fdc06d730 msgr2=0x7f3fdc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:43.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 -- 172.21.15.88:0/1689135717 shutdown_connections 2024-09-21T07:47:43.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.776+0000 7f3fe2000640 1 -- 172.21.15.88:0/1689135717 wait complete. 2024-09-21T07:47:43.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.778+0000 7f3fe2000640 1 Processor -- start 2024-09-21T07:47:43.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.778+0000 7f3fe2000640 1 -- start start 2024-09-21T07:47:43.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fe2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fdc1f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fe2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fdc148550 0x7f3fdc073d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fe2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc074270 0x7f3fdc14b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fe2000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc0ad4f0 con 0x7f3fc0003a50 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fe2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3fdc0ad670 con 0x7f3fdc074270 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fe2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3fdc0ad7f0 con 0x7f3fdc148550 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fdb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fdc1f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fdb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fdc1f9d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39170/0 (socket says 172.21.15.88:39170) 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fdb400640 1 -- 172.21.15.88:0/2879439773 learned_addr learned my addr 172.21.15.88:0/2879439773 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fd2a00640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fdc148550 0x7f3fdc073d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.780+0000 7f3fdbe00640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc074270 0x7f3fdc14b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdb400640 1 -- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fdc148550 msgr2=0x7f3fdc073d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdb400640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fdc148550 0x7f3fdc073d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdb400640 1 -- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc074270 msgr2=0x7f3fdc14b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:43.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdb400640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc074270 0x7f3fdc14b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdb400640 1 -- 172.21.15.88:0/2879439773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fdc1fa240 con 0x7f3fc0003a50 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdbe00640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc074270 0x7f3fdc14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fdb400640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fdc1f9d00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009a70 tx=0x7f3fcc0336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc038040 con 0x7f3fc0003a50 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3fcc004230 con 0x7f3fc0003a50 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fdc074680 con 0x7f3fc0003a50 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fcc0338e0 con 0x7f3fc0003a50 2024-09-21T07:47:43.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.781+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fdc14cd00 con 0x7f3fc0003a50 2024-09-21T07:47:43.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.783+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fdc152a40 con 0x7f3fc0003a50 2024-09-21T07:47:43.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.784+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f3fcc02b8a0 con 0x7f3fc0003a50 2024-09-21T07:47:43.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.785+0000 7f3fd3e00640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3fb8078d70 0x7f3fb807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:43.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.785+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f3fcc0bb990 con 0x7f3fc0003a50 2024-09-21T07:47:43.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.785+0000 7f3fd2a00640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3fb8078d70 0x7f3fb807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:43.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.786+0000 7f3fd2a00640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3fb8078d70 0x7f3fb807b280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40018a0 tx=0x7f3fc4000ed0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:43.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:43.790+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3fdc152a40 con 0x7f3fc0003a50 2024-09-21T07:47:44.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.014+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3fdc153730 con 0x7f3fc0003a50 2024-09-21T07:47:44.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.014+0000 7f3fd3e00640 1 -- 172.21.15.88:0/2879439773 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) ==== 94+0+5226 (secure 0 0 0) 0x7f3fdc074680 con 0x7f3fc0003a50 2024-09-21T07:47:44.139 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:44.139 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":26,"btime":"2024-09-21T07:47:41:094343+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:47:44.140 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 26 2024-09-21T07:47:44.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3fb8078d70 msgr2=0x7f3fb807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3fb8078d70 0x7f3fb807b280 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f3fc40018a0 tx=0x7f3fc4000ed0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 msgr2=0x7f3fdc1f9d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fdc1f9d00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f3fcc009a70 tx=0x7f3fcc0336f0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 shutdown_connections 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3fb8078d70 0x7f3fb807b280 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3fdc148550 0x7f3fdc073d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3fdc074270 0x7f3fdc14b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.016+0000 7f3fe2000640 1 --2- 172.21.15.88:0/2879439773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fc0003a50 0x7f3fdc1f9d00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.017+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 >> 172.21.15.88:0/2879439773 conn(0x7f3fdc06d730 msgr2=0x7f3fdc1602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.017+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 shutdown_connections 2024-09-21T07:47:44.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.017+0000 7f3fe2000640 1 -- 172.21.15.88:0/2879439773 wait complete. 2024-09-21T07:47:44.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:47:37.572652+0000', 'modified': '2024-09-21T07:47:40.630013+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5337}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5337': {'gid': 5337, 'name': 'c', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/909378044', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6833', 'nonce': 909378044}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 909378044}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5337, 'qdb_cluster': [5337]} 2024-09-21T07:47:44.545 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:47:44.545 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 -- 172.21.15.88:0/1521673933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d780cfc80 msgr2=0x7f3d78156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/1521673933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d780cfc80 0x7f3d78156e40 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f3d68005fd0 tx=0x7f3d6802a4f0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 -- 172.21.15.88:0/1521673933 shutdown_connections 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/1521673933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d7815ce30 0x7f3d7815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/1521673933 >> v2:172.21.15.175:3300/0 conn(0x7f3d780705b0 0x7f3d780709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/1521673933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d780cfc80 0x7f3d78156e40 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 -- 172.21.15.88:0/1521673933 >> 172.21.15.88:0/1521673933 conn(0x7f3d7806d3d0 msgr2=0x7f3d780b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 -- 172.21.15.88:0/1521673933 shutdown_connections 2024-09-21T07:47:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.766+0000 7f3d7fa00640 1 -- 172.21.15.88:0/1521673933 wait complete. 2024-09-21T07:47:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.768+0000 7f3d7fa00640 1 Processor -- start 2024-09-21T07:47:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.768+0000 7f3d7fa00640 1 -- start start 2024-09-21T07:47:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d780705b0 0x7f3d780c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d780cfc80 0x7f3d780c95f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7fa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 0x7f3d78073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7fa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d780cce40 con 0x7f3d7815ce30 2024-09-21T07:47:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d780cccc0 con 0x7f3d780705b0 2024-09-21T07:47:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d780ccfc0 con 0x7f3d780cfc80 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 0x7f3d78073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 0x7f3d78073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39190/0 (socket says 172.21.15.88:39190) 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.770+0000 7f3d7da00640 1 -- 172.21.15.88:0/3430345758 learned_addr learned my addr 172.21.15.88:0/3430345758 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d780cfc80 0x7f3d780c95f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7da00640 1 -- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d780cfc80 msgr2=0x7f3d780c95f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7d000640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d780705b0 0x7f3d780c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7da00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d780cfc80 0x7f3d780c95f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7da00640 1 -- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d780705b0 msgr2=0x7f3d780c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7da00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d780705b0 0x7f3d780c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7da00640 1 -- 172.21.15.88:0/3430345758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d780c7d80 con 0x7f3d7815ce30 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d780cfc80 0x7f3d780c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:44.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7d000640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d780705b0 0x7f3d780c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7da00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 0x7f3d78073f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e990 tx=0x7f3d7000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d75600640 1 -- 172.21.15.88:0/3430345758 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70004d20 con 0x7f3d7815ce30 2024-09-21T07:47:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d75600640 1 -- 172.21.15.88:0/3430345758 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d7000cb00 con 0x7f3d7815ce30 2024-09-21T07:47:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d75600640 1 -- 172.21.15.88:0/3430345758 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70005020 con 0x7f3d7815ce30 2024-09-21T07:47:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d780c8060 con 0x7f3d7815ce30 2024-09-21T07:47:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.771+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d780c8c00 con 0x7f3d7815ce30 2024-09-21T07:47:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.774+0000 7f3d75600640 1 -- 172.21.15.88:0/3430345758 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f3d700051c0 con 0x7f3d7815ce30 2024-09-21T07:47:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.775+0000 7f3d75600640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3d54078d70 0x7f3d5407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.775+0000 7f3d75600640 1 -- 172.21.15.88:0/3430345758 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f3d7009ba90 con 0x7f3d7815ce30 2024-09-21T07:47:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.775+0000 7f3d86c51740 1 Processor -- start 2024-09-21T07:47:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.775+0000 7f3d7d000640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3d54078d70 0x7f3d5407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.775+0000 7f3d86c51740 1 -- start start 2024-09-21T07:47:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.776+0000 7f3d7d000640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3d54078d70 0x7f3d5407b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d68033000 tx=0x7f3d68004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e07970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e07eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9dd7f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d86c51740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55acf9d61430 con 0x55acf9a16590 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55acf9d515d0 con 0x55acf9a0e3b0 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55acf9d40a00 con 0x55acf9a37270 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d7da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9dd7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d7da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9dd7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39198/0 (socket says 172.21.15.88:39198) 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d7da00640 1 -- 172.21.15.88:0/46541700 learned_addr learned my addr 172.21.15.88:0/46541700 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d7d000640 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e07970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.777+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e07eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d7da00640 1 -- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 msgr2=0x55acf9e07970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d7da00640 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e07970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d7da00640 1 -- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 msgr2=0x55acf9e07eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d7da00640 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e07eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d7da00640 1 -- 172.21.15.88:0/46541700 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf9d09520 con 0x55acf9a16590 2024-09-21T07:47:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d7da00640 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9dd7f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000c640 tx=0x7f3d70062c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25322773eab02b56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d6d600640 1 -- 172.21.15.88:0/46541700 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d70089020 con 0x55acf9a16590 2024-09-21T07:47:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d6d600640 1 -- 172.21.15.88:0/46541700 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d70067f10 con 0x55acf9a16590 2024-09-21T07:47:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.778+0000 7f3d6d600640 1 -- 172.21.15.88:0/46541700 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7001f040 con 0x55acf9a16590 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 -- 172.21.15.88:0/46541700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 msgr2=0x55acf9dd7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9dd7f60 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000c640 tx=0x7f3d70062c90 comp rx=0 tx=0).stop 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 -- 172.21.15.88:0/46541700 shutdown_connections 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9dd7f60 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e07eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 --2- 172.21.15.88:0/46541700 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e07970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 -- 172.21.15.88:0/46541700 >> 172.21.15.88:0/46541700 conn(0x55acf9a2b3c0 msgr2=0x55acf9ddd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 -- 172.21.15.88:0/46541700 shutdown_connections 2024-09-21T07:47:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.779+0000 7f3d86c51740 1 -- 172.21.15.88:0/46541700 wait complete. 2024-09-21T07:47:44.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.780+0000 7f3d86c51740 1 -- start start 2024-09-21T07:47:44.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.780+0000 7f3d86c51740 1 Processor -- start 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e1d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9e04120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55acf9ce7330 con 0x55acf9a37270 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55acf9d44810 con 0x55acf9a0e3b0 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55acf9ab1620 con 0x55acf9a16590 2024-09-21T07:47:44.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d7d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e1d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d7d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e1d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:53282/0 (socket says 172.21.15.88:53282) 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d7d000640 1 -- 172.21.15.88:0/957730406 learned_addr learned my addr 172.21.15.88:0/957730406 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.782+0000 7f3d86c51740 10 client.0 _ll_register_callbacks cb 0x55acf9cec420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9e04120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 -- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 msgr2=0x55acf9dfe410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 -- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 msgr2=0x55acf9e1d750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e1d750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 -- 172.21.15.88:0/957730406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf9aa2850 con 0x55acf9a37270 2024-09-21T07:47:44.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7d000640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e1d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d7da00640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9e04120 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e960 tx=0x7f3d700628c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77a7fe8f8dd4c6f6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.783+0000 7f3d66000640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d66000640 1 -- 172.21.15.88:0/957730406 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7007d020 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d66000640 1 -- 172.21.15.88:0/957730406 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f3d7006a330 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d70067f10 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d66000640 1 -- 172.21.15.88:0/957730406 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d70089050 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 10 client.5445 fetch_fsmap learned FSMap version 26 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55acf9b196b0 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d66000640 1 -- 172.21.15.88:0/957730406 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d70067f10 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d66000640 1 -- 172.21.15.88:0/957730406 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f3d7000f040 con 0x55acf9a37270 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 10 client.5445 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 10 client.5445 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:44.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 20 client.5445 populate_metadata read hostname 'smithi088' 2024-09-21T07:47:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf99f5e40 0x55acf9e31c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 4 client.5445 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:47:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.784+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 --> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55acf9d53520 con 0x55acf99f5e40 2024-09-21T07:47:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.785+0000 7f3d7d000640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf99f5e40 0x55acf9e31c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.785+0000 7f3d66000640 10 client.5445 ms_handle_connect on v2:172.21.15.88:6833/909378044 2024-09-21T07:47:44.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.785+0000 7f3d7d000640 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf99f5e40 0x55acf9e31c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8652a42e1de9549f server_cookie=fffa1a24f9b05666 in_seq=0 out_seq=0 2024-09-21T07:47:44.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d66000640 1 -- 172.21.15.88:0/957730406 <== mds.0 v2:172.21.15.88:6833/909378044 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55acf9d53520 con 0x55acf99f5e40 2024-09-21T07:47:44.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d66000640 10 client.5445 handle_command_reply: tid=0 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 client.5445 shutdown 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 msgr2=0x55acf9e04120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9e04120 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e960 tx=0x7f3d700628c0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 shutdown_connections 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf99f5e40 0x55acf9e31c70 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9e04120 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 --2- 172.21.15.88:0/957730406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e1d750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d66000640 0 client.5445 ms_handle_reset on v2:172.21.15.88:6833/909378044 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 >> 172.21.15.88:0/957730406 conn(0x55acf9a2b3c0 msgr2=0x55acf9e25d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 shutdown_connections 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.786+0000 7f3d86c51740 1 -- 172.21.15.88:0/957730406 wait complete. 2024-09-21T07:47:44.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.787+0000 7f3d86c51740 20 client.5445 trim_cache size 0 max 16384 2024-09-21T07:47:44.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.805+0000 7f3d86c51740 1 Processor -- start 2024-09-21T07:47:44.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.805+0000 7f3d86c51740 1 -- start start 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e45cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9dfe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d86c51740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d781ff140 con 0x55acf9a37270 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3d781e7590 con 0x55acf9a0e3b0 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3d781f4260 con 0x55acf9a16590 2024-09-21T07:47:44.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9dfe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9dfe730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39210/0 (socket says 172.21.15.88:39210) 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 -- 172.21.15.88:0/1433510085 learned_addr learned my addr 172.21.15.88:0/1433510085 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7d000640 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e45cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 -- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 msgr2=0x55acf9dfe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 -- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 msgr2=0x55acf9e45cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.807+0000 7f3d7da00640 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e45cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.808+0000 7f3d7da00640 1 -- 172.21.15.88:0/1433510085 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55acf9e352b0 con 0x55acf9a37270 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.808+0000 7f3d7da00640 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9dfe730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d70063130 tx=0x7f3d7000cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a65e20504439e4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.808+0000 7f3d6cc00640 1 -- 172.21.15.88:0/1433510085 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7007d030 con 0x55acf9a37270 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.808+0000 7f3d6cc00640 1 -- 172.21.15.88:0/1433510085 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d70067f10 con 0x55acf9a37270 2024-09-21T07:47:44.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.808+0000 7f3d6cc00640 1 -- 172.21.15.88:0/1433510085 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d7001f070 con 0x55acf9a37270 2024-09-21T07:47:44.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 -- 172.21.15.88:0/1433510085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 msgr2=0x55acf9dfe730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9dfe730 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f3d70063130 tx=0x7f3d7000cf50 comp rx=0 tx=0).stop 2024-09-21T07:47:44.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 -- 172.21.15.88:0/1433510085 shutdown_connections 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a16590 0x55acf9dfe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a37270 0x55acf9dfe730 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 --2- 172.21.15.88:0/1433510085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e45cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 -- 172.21.15.88:0/1433510085 >> 172.21.15.88:0/1433510085 conn(0x55acf9a2b3c0 msgr2=0x55acf9dd3560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 -- 172.21.15.88:0/1433510085 shutdown_connections 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.809+0000 7f3d86c51740 1 -- 172.21.15.88:0/1433510085 wait complete. 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.810+0000 7f3d86c51740 1 -- start start 2024-09-21T07:47:44.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.810+0000 7f3d86c51740 1 Processor -- start 2024-09-21T07:47:44.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.811+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e9b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.811+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9e07c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.811+0000 7f3d86c51740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e08180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.811+0000 7f3d86c51740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55acf9e00fd0 con 0x55acf9a16590 2024-09-21T07:47:44.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.811+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55acf9ce7330 con 0x55acf9a0e3b0 2024-09-21T07:47:44.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.811+0000 7f3d86c51740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55acf9e354b0 con 0x55acf9a37270 2024-09-21T07:47:44.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d86c51740 10 client.0 _ll_register_callbacks cb 0x7f3d700632f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:47:44.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9e07c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9e07c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39224/0 (socket says 172.21.15.88:39224) 2024-09-21T07:47:44.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 -- 172.21.15.88:0/2437488061 learned_addr learned my addr 172.21.15.88:0/2437488061 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d7d000640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e9b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d7da00640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e08180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 -- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 msgr2=0x55acf9e08180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e08180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 -- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 msgr2=0x55acf9e9b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e9b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 -- 172.21.15.88:0/2437488061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55acf9dfe8d0 con 0x55acf9a16590 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d7da00640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e08180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d6fe00640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9e07c40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d600136e0 tx=0x7f3d60013ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ae5440ed81b1fa4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.812+0000 7f3d66000640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d66000640 1 -- 172.21.15.88:0/2437488061 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d60020060 con 0x55acf9a16590 2024-09-21T07:47:44.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d66000640 1 -- 172.21.15.88:0/2437488061 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f3d60011a70 con 0x55acf9a16590 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d70067f10 con 0x55acf9a16590 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d66000640 1 -- 172.21.15.88:0/2437488061 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) ==== 24+0+0 (secure 0 0 0) 0x7f3d60026050 con 0x55acf9a16590 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 10 client.5451 fetch_fsmap learned FSMap version 26 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) -- 0x55acf9ea1640 con 0x55acf9a16590 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d66000640 1 -- 172.21.15.88:0/2437488061 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3d70067f10 con 0x55acf9a16590 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d66000640 1 -- 172.21.15.88:0/2437488061 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 26) ==== 3502+0+0 (secure 0 0 0) 0x7f3d60029020 con 0x55acf9a16590 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 10 client.5451 fetch_fsmap finished waiting for FSMap version 26 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 10 client.5451 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 20 client.5451 populate_metadata read hostname 'smithi088' 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf9e23820 0x55acf9e23c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 4 client.5451 mds_command: new command op to 5337 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.813+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 --> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] -- command(tid 0: {"prefix": "status"}) -- 0x55acf9d4efd0 con 0x55acf9e23820 2024-09-21T07:47:44.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d7d000640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf9e23820 0x55acf9e23c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d7d000640 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf9e23820 0x55acf9e23c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0f4b307de5b9b04 server_cookie=6a6120dff3e7d61d in_seq=0 out_seq=0 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d66000640 10 client.5451 ms_handle_connect on v2:172.21.15.88:6833/909378044 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d66000640 1 -- 172.21.15.88:0/2437488061 <== mds.0 v2:172.21.15.88:6833/909378044 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x55acf9d4efd0 con 0x55acf9e23820 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d66000640 10 client.5451 handle_command_reply: tid=0 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d86c51740 1 client.5451 shutdown 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 msgr2=0x55acf9e07c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.814+0000 7f3d86c51740 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9e07c40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f3d600136e0 tx=0x7f3d60013ba0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 shutdown_connections 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:6833/909378044,v1:172.21.15.88:6835/909378044] conn(0x55acf9e23820 0x55acf9e23c30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55acf9a37270 0x55acf9e08180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55acf9a16590 0x55acf9e07c40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 --2- 172.21.15.88:0/2437488061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55acf9a0e3b0 0x55acf9e9b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d66000640 0 client.5451 ms_handle_reset on v2:172.21.15.88:6833/909378044 2024-09-21T07:47:44.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 >> 172.21.15.88:0/2437488061 conn(0x55acf9a2b3c0 msgr2=0x55acf9e45cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 shutdown_connections 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 1 -- 172.21.15.88:0/2437488061 wait complete. 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d86c51740 20 client.5451 trim_cache size 0 max 16384 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5337, 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:47:44.822 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:47:44.823 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.7414073439999997, 2024-09-21T07:47:44.823 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 26, 2024-09-21T07:47:44.823 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 47, 2024-09-21T07:47:44.823 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 47, 2024-09-21T07:47:44.823 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.789864443000001 2024-09-21T07:47:44.823 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3d54078d70 msgr2=0x7f3d5407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3d54078d70 0x7f3d5407b280 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f3d68033000 tx=0x7f3d68004730 comp rx=0 tx=0).stop 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 msgr2=0x7f3d78073f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 0x7f3d78073f70 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f3d7000e990 tx=0x7f3d7000ee50 comp rx=0 tx=0).stop 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.815+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 shutdown_connections 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3d54078d70 0x7f3d5407b280 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3d780cfc80 0x7f3d780c95f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d7815ce30 0x7f3d78073f70 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 --2- 172.21.15.88:0/3430345758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3d780705b0 0x7f3d780c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:44.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 >> 172.21.15.88:0/3430345758 conn(0x7f3d7806d3d0 msgr2=0x7f3d780c6970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:44.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 shutdown_connections 2024-09-21T07:47:44.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:44.816+0000 7f3d7fa00640 1 -- 172.21.15.88:0/3430345758 wait complete. 2024-09-21T07:47:44.842 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:47:45.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f8d200640 1 -- 172.21.15.88:0/528185578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f6c004800 msgr2=0x7f6f6c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f8d200640 1 --2- 172.21.15.88:0/528185578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f6c004800 0x7f6f6c004c10 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009b20 tx=0x7f6f6c005830 comp rx=0 tx=0).stop 2024-09-21T07:47:45.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f86800640 1 -- 172.21.15.88:0/528185578 reap_dead start 2024-09-21T07:47:45.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f8d200640 1 -- 172.21.15.88:0/528185578 shutdown_connections 2024-09-21T07:47:45.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f8d200640 1 -- 172.21.15.88:0/528185578 >> 172.21.15.88:0/528185578 conn(0x7f6f8806d730 msgr2=0x7f6f880b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:45.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f8d200640 1 -- 172.21.15.88:0/528185578 shutdown_connections 2024-09-21T07:47:45.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.059+0000 7f6f8d200640 1 -- 172.21.15.88:0/528185578 wait complete. 2024-09-21T07:47:45.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.061+0000 7f6f8d200640 1 Processor -- start 2024-09-21T07:47:45.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.062+0000 7f6f8d200640 1 -- start start 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f8d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c004800 0x7f6f88099270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f8d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 0x7f6f88155aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f8d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f88155fe0 0x7f6f881563f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f8d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f880b4da0 con 0x7f6f88149830 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f8d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f880b4c20 con 0x7f6f6c004800 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f8d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f880ad4f0 con 0x7f6f88155fe0 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f7d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 0x7f6f88155aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f7d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 0x7f6f88155aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39246/0 (socket says 172.21.15.88:39246) 2024-09-21T07:47:45.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f7d400640 1 -- 172.21.15.88:0/2860983402 learned_addr learned my addr 172.21.15.88:0/2860983402 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f85e00640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c004800 0x7f6f88099270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.063+0000 7f6f86800640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f88155fe0 0x7f6f881563f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7d400640 1 -- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f88155fe0 msgr2=0x7f6f881563f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7d400640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f88155fe0 0x7f6f881563f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7d400640 1 -- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c004800 msgr2=0x7f6f88099270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7d400640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c004800 0x7f6f88099270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7d400640 1 -- 172.21.15.88:0/2860983402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f8814a270 con 0x7f6f88149830 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f86800640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f88155fe0 0x7f6f881563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7d400640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 0x7f6f88155aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009c40 tx=0x7f6f70033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70038050 con 0x7f6f88149830 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f70044070 con 0x7f6f88149830 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f88154fc0 con 0x7f6f88149830 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f70033ce0 con 0x7f6f88149830 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.064+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f88155580 con 0x7f6f88149830 2024-09-21T07:47:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.065+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f881535c0 con 0x7f6f88149830 2024-09-21T07:47:45.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.066+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f6f7003f450 con 0x7f6f88149830 2024-09-21T07:47:45.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.067+0000 7f6f7ea00640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f64078d40 0x7f6f6407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:45.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.068+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5709+0+0 (secure 0 0 0) 0x7f6f700bbf60 con 0x7f6f88149830 2024-09-21T07:47:45.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.068+0000 7f6f85e00640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f64078d40 0x7f6f6407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:45.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.069+0000 7f6f85e00640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f64078d40 0x7f6f6407b250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f78000fb0 tx=0x7f6f780012b0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:45.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.072+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f7004d020 con 0x7f6f88149830 2024-09-21T07:47:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.296+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6f880cfcc0 con 0x7f6f88149830 2024-09-21T07:47:45.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.296+0000 7f6f7ea00640 1 -- 172.21.15.88:0/2860983402 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7f6f70084010 con 0x7f6f88149830 2024-09-21T07:47:45.298 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:47:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f64078d40 msgr2=0x7f6f6407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f64078d40 0x7f6f6407b250 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f6f78000fb0 tx=0x7f6f780012b0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 msgr2=0x7f6f88155aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 0x7f6f88155aa0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f6f70009c40 tx=0x7f6f70033670 comp rx=0 tx=0).stop 2024-09-21T07:47:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 shutdown_connections 2024-09-21T07:47:45.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f64078d40 0x7f6f6407b250 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f88155fe0 0x7f6f881563f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f88149830 0x7f6f88155aa0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 --2- 172.21.15.88:0/2860983402 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f6c004800 0x7f6f88099270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:45.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 >> 172.21.15.88:0/2860983402 conn(0x7f6f8806d730 msgr2=0x7f6f880b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:45.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 shutdown_connections 2024-09-21T07:47:45.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45.299+0000 7f6f8d200640 1 -- 172.21.15.88:0/2860983402 wait complete. 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:47:45.327 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:45.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:45.327 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:47:45.363 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:47:45.363 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:47:45.363 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:47:45.363 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:45.364 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42629sec preferred_lft 42629sec 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.365 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:45.366 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:47:45.366 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:47:45.366 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:45.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:45.366 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:47:45.457 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:47:45.460 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:45.461 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:47:45.496 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:47:45.497 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:47:45.497 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:47:45.497 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:47:45.497 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:47:45.497 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:47:45.497 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:47:45.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:47:45.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:47:45.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:47:45.498 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:47:45.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:47:45.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:45.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:45.695 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:47:45.730 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:47:45.731 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:47:45.731 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:45.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:45.731 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:47:45.827 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:45.827 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:45.862 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:47:45.863 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:45.863 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:45.937 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:47:45.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:45.937 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:46.136 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:46.136 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:47:46.136 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:47:46.156 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:46.157 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:47:46.157 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:47:46.157 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:46.157 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:46.185 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:47:46.185 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:47:46.186 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:47:46.186 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:46.186 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:46.186 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:47:46.186 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:47:46.186 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:47:46.187 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:47:46.187 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:47:46.187 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:47:46.187 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:47:46.188 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:47:46.190 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:47:46.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:46.191 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:46.246 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:47:46.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:46.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:46.342 INFO:teuthology.orchestra.run.smithi088.stdout:client5460 2024-09-21T07:47:46.342 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:46.342 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:46.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:47:46.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0da00640 1 -- 172.21.15.88:0/1532877479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadec003a00 msgr2=0x7fadec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0da00640 1 --2- 172.21.15.88:0/1532877479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fadec003a00 0x7fadec003e70 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fadf8009b10 tx=0x7fadec005830 comp rx=0 tx=0).stop 2024-09-21T07:47:46.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0d000640 1 -- 172.21.15.88:0/1532877479 reap_dead start 2024-09-21T07:47:46.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0da00640 1 -- 172.21.15.88:0/1532877479 shutdown_connections 2024-09-21T07:47:46.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0da00640 1 -- 172.21.15.88:0/1532877479 >> 172.21.15.88:0/1532877479 conn(0x7fae080d44a0 msgr2=0x7fae080d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0da00640 1 -- 172.21.15.88:0/1532877479 shutdown_connections 2024-09-21T07:47:46.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.643+0000 7fae0da00640 1 -- 172.21.15.88:0/1532877479 wait complete. 2024-09-21T07:47:46.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.645+0000 7fae0da00640 1 Processor -- start 2024-09-21T07:47:46.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.645+0000 7fae0da00640 1 -- start start 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadec003a00 0x7fae080cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0815ac60 0x7fae0815cb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 0x7fae080b5510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0da00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae080d5350 con 0x7fae0814e7e0 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae080d51d0 con 0x7fadec003a00 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae080d2b60 con 0x7fae0815ac60 2024-09-21T07:47:46.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 0x7fae080b5510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 0x7fae080b5510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44080/0 (socket says 172.21.15.88:44080) 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 -- 172.21.15.88:0/1775605557 learned_addr learned my addr 172.21.15.88:0/1775605557 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae07e00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadec003a00 0x7fae080cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fadff400640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0815ac60 0x7fae0815cb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 -- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0815ac60 msgr2=0x7fae0815cb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0815ac60 0x7fae0815cb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 -- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadec003a00 msgr2=0x7fae080cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadec003a00 0x7fae080cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.647+0000 7fae0d000640 1 -- 172.21.15.88:0/1775605557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae080ce1e0 con 0x7fae0814e7e0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.648+0000 7fae0d000640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 0x7fae080b5510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fae080d3830 tx=0x7fae00010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.648+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae0000ee50 con 0x7fae0814e7e0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.648+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae00015070 con 0x7fae0814e7e0 2024-09-21T07:47:46.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.648+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae00003d00 con 0x7fae0814e7e0 2024-09-21T07:47:46.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.648+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae080ce4c0 con 0x7fae0814e7e0 2024-09-21T07:47:46.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.648+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae0815b530 con 0x7fae0814e7e0 2024-09-21T07:47:46.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.649+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae0815d140 con 0x7fae0814e7e0 2024-09-21T07:47:46.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.651+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fae00004030 con 0x7fae0814e7e0 2024-09-21T07:47:46.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.651+0000 7fae04c00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fade4078d40 0x7fade407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:46.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.652+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fae0009bc10 con 0x7fae0814e7e0 2024-09-21T07:47:46.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.652+0000 7fae07e00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fade4078d40 0x7fade407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:46.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.653+0000 7fae07e00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fade4078d40 0x7fade407b250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadf8034000 tx=0x7fadf80046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:46.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.656+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae00063540 con 0x7fae0814e7e0 2024-09-21T07:47:46.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.881+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fae080cdf50 con 0x7fae0814e7e0 2024-09-21T07:47:46.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.881+0000 7fae04c00640 1 -- 172.21.15.88:0/1775605557 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) ==== 53+0+73 (secure 0 0 0) 0x7fae00067f70 con 0x7fae0814e7e0 2024-09-21T07:47:46.884 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:47:46.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fade4078d40 msgr2=0x7fade407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fade4078d40 0x7fade407b250 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fadf8034000 tx=0x7fadf80046c0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 msgr2=0x7fae080b5510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 0x7fae080b5510 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fae080d3830 tx=0x7fae00010f40 comp rx=0 tx=0).stop 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 shutdown_connections 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fade4078d40 0x7fade407b250 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae0815ac60 0x7fae0815cb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae0814e7e0 0x7fae080b5510 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 --2- 172.21.15.88:0/1775605557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fadec003a00 0x7fae080cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 >> 172.21.15.88:0/1775605557 conn(0x7fae080d44a0 msgr2=0x7fae080d22a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 shutdown_connections 2024-09-21T07:47:46.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:46.884+0000 7fae0da00640 1 -- 172.21.15.88:0/1775605557 wait complete. 2024-09-21T07:47:46.907 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:47:46.907 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:47:46.907 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:47:46.908 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:47:46.908 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:46.908 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:47:46.908 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:47:46.908 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:47:46.908 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:47:46.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:46.908 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:47:46.948 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:47:46.948 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:47:46.948 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:47:46.948 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:47:46.949 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42578sec preferred_lft 42578sec 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.950 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:47:46.951 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:47:46.951 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:47:46.951 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:47:46.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:46.952 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:47:47.049 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:47:47.052 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:47.052 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:47:47.089 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:47:47.089 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:47:47.089 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:47:47.089 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:47:47.089 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:47:47.090 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:47:47.090 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:47:47.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:47:47.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:47:47.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:47:47.090 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:47:47.207 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:47:47.278 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:47.286 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:47.286 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:47:47.321 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:47:47.322 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:47:47.322 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:47:47.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:47.322 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:47:47.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:47.419 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:47.454 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:47:47.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:47:47.455 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:47.531 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:47:47.531 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:47.531 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:47.729 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:47.729 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:47:47.729 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:47:47.750 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:47.750 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:47:47.750 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:47:47.750 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:47.750 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:47:47.781 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:47:47.783 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:47:47.783 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:47:47.783 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:47:47.783 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:47:47.783 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:47:47.786 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:47:47.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:47.786 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:47.842 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:47:47.929 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:47:47.933 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:47:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:47:47.941 INFO:teuthology.orchestra.run.smithi175.stdout:client5078 2024-09-21T07:47:47.941 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:47.942 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:47.977 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:47:48.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 -- 172.21.15.88:0/394434032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80af100 msgr2=0x7f3cd80af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 --2- 172.21.15.88:0/394434032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80af100 0x7f3cd80af510 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3cd80d0d00 tx=0x7f3cd002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 -- 172.21.15.88:0/394434032 shutdown_connections 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 --2- 172.21.15.88:0/394434032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80af100 0x7f3cd80af510 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 --2- 172.21.15.88:0/394434032 >> v2:172.21.15.175:3300/0 conn(0x7f3cd806d4b0 0x7f3cd80b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 -- 172.21.15.88:0/394434032 >> 172.21.15.88:0/394434032 conn(0x7f3cd80b2540 msgr2=0x7f3cd80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.205+0000 7f3cdf000640 1 -- 172.21.15.88:0/394434032 shutdown_connections 2024-09-21T07:47:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.206+0000 7f3cdf000640 1 -- 172.21.15.88:0/394434032 wait complete. 2024-09-21T07:47:48.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.208+0000 7f3cdf000640 1 Processor -- start 2024-09-21T07:47:48.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.208+0000 7f3cdf000640 1 -- start start 2024-09-21T07:47:48.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.209+0000 7f3cdf000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd80af100 0x7f3cd81f2b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.209+0000 7f3cdf000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cd80cc830 0x7f3cd8098840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.209+0000 7f3cdf000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 0x7f3cd8098d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.209+0000 7f3cdf000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80b3620 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.209+0000 7f3cdf000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3cd80b34a0 con 0x7f3cd80af100 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.209+0000 7f3cdf000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3cd80b37a0 con 0x7f3cd80cc830 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 0x7f3cd8098d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 0x7f3cd8098d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44098/0 (socket says 172.21.15.88:44098) 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 -- 172.21.15.88:0/2267784479 learned_addr learned my addr 172.21.15.88:0/2267784479 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cd7e00640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd80af100 0x7f3cd81f2b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cd7400640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cd80cc830 0x7f3cd8098840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 -- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cd80cc830 msgr2=0x7f3cd8098840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cd80cc830 0x7f3cd8098840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 -- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd80af100 msgr2=0x7f3cd81f2b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd80af100 0x7f3cd81f2b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 -- 172.21.15.88:0/2267784479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cd81f6620 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.210+0000 7f3cdd000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 0x7f3cd8098d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009ca0 tx=0x7f3cd0031760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.211+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0031990 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.211+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3cd0031c20 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.211+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00330d0 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.211+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cd81f3080 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.211+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cd81f5d80 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.212+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cd80b3cc0 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.213+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f3cd0033270 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.214+0000 7f3ccfe00640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3cb8078d40 0x7f3cb807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.214+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f3cd00bc790 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.214+0000 7f3cd7e00640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3cb8078d40 0x7f3cb807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.215+0000 7f3cd7e00640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3cb8078d40 0x7f3cb807b250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8000f60 tx=0x7f3cc8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.219+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3cd004a020 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.440+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f3cd80d1030 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.558+0000 7f3ccfe00640 1 -- 172.21.15.88:0/2267784479 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v27) ==== 136+0+0 (secure 0 0 0) 0x7f3cd003f030 con 0x7f3cd80cdfb0 2024-09-21T07:47:48.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3cb8078d40 msgr2=0x7f3cb807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3cb8078d40 0x7f3cb807b250 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f3cc8000f60 tx=0x7f3cc8009040 comp rx=0 tx=0).stop 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 msgr2=0x7f3cd8098d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 0x7f3cd8098d80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3cd0009ca0 tx=0x7f3cd0031760 comp rx=0 tx=0).stop 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 shutdown_connections 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3cb8078d40 0x7f3cb807b250 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cd80cc830 0x7f3cd8098840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cd80cdfb0 0x7f3cd8098d80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 --2- 172.21.15.88:0/2267784479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cd80af100 0x7f3cd81f2b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 >> 172.21.15.88:0/2267784479 conn(0x7f3cd80b2540 msgr2=0x7f3cd8099920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 shutdown_connections 2024-09-21T07:47:48.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.560+0000 7f3cdf000640 1 -- 172.21.15.88:0/2267784479 wait complete. 2024-09-21T07:47:48.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:47:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 -- 172.21.15.88:0/4157899406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a900b2f60 msgr2=0x7f7a9015a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 --2- 172.21.15.88:0/4157899406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a900b2f60 0x7f7a9015a0f0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7a90051590 tx=0x7f7a7802a560 comp rx=0 tx=0).stop 2024-09-21T07:47:48.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 -- 172.21.15.88:0/4157899406 shutdown_connections 2024-09-21T07:47:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 --2- 172.21.15.88:0/4157899406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a900b2f60 0x7f7a9015a0f0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 --2- 172.21.15.88:0/4157899406 >> v2:172.21.15.175:3300/0 conn(0x7f7a900ad9c0 0x7f7a900af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 -- 172.21.15.88:0/4157899406 >> 172.21.15.88:0/4157899406 conn(0x7f7a900b1e40 msgr2=0x7f7a900b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 -- 172.21.15.88:0/4157899406 shutdown_connections 2024-09-21T07:47:48.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.804+0000 7f7a94a00640 1 -- 172.21.15.88:0/4157899406 wait complete. 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.806+0000 7f7a94a00640 1 Processor -- start 2024-09-21T07:47:48.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.806+0000 7f7a94a00640 1 -- start start 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a94a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a900ad9c0 0x7f7a900cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a94a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a900b2f60 0x7f7a90158f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a94a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 0x7f7a900b0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a94a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7a9006f290 con 0x7f7a9015d670 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a94a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7a90070d50 con 0x7f7a900ad9c0 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a94a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7a9006f410 con 0x7f7a900b2f60 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a8f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 0x7f7a900b0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a8e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a900ad9c0 0x7f7a900cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a8f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 0x7f7a900b0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44114/0 (socket says 172.21.15.88:44114) 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a8f200640 1 -- 172.21.15.88:0/2224642772 learned_addr learned my addr 172.21.15.88:0/2224642772 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:48.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.808+0000 7f7a85e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a900b2f60 0x7f7a90158f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a8f200640 1 -- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a900b2f60 msgr2=0x7f7a90158f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a8f200640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a900b2f60 0x7f7a90158f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a8f200640 1 -- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a900ad9c0 msgr2=0x7f7a900cb7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a8f200640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a900ad9c0 0x7f7a900cb7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a8f200640 1 -- 172.21.15.88:0/2224642772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a901f8bb0 con 0x7f7a9015d670 2024-09-21T07:47:48.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a8f200640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 0x7f7a900b0c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c3c0 tx=0x7f7a8800c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88017070 con 0x7f7a9015d670 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7a880053e0 con 0x7f7a9015d670 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a88003850 con 0x7f7a9015d670 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a901f8e90 con 0x7f7a9015d670 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.809+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a901f9450 con 0x7f7a9015d670 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.810+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a90159840 con 0x7f7a9015d670 2024-09-21T07:47:48.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.812+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f7a8801a070 con 0x7f7a9015d670 2024-09-21T07:47:48.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.812+0000 7f7a87400640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7a6c078d40 0x7f7a6c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:48.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.813+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f7a8809a3d0 con 0x7f7a9015d670 2024-09-21T07:47:48.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.813+0000 7f7a8e800640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7a6c078d40 0x7f7a6c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:48.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.813+0000 7f7a8e800640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7a6c078d40 0x7f7a6c07b250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005fd0 tx=0x7f7a80009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:48.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:48.817+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7a88061ed0 con 0x7f7a9015d670 2024-09-21T07:47:49.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.025+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7a901f8960 con 0x7f7a6c078d40 2024-09-21T07:47:49.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.027+0000 7f7a87400640 1 -- 172.21.15.88:0/2224642772 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7a901f8960 con 0x7f7a6c078d40 2024-09-21T07:47:49.029 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:47:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:47:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:47:49.029 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:47:49.029 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:47:49.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7a6c078d40 msgr2=0x7f7a6c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7a6c078d40 0x7f7a6c07b250 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f7a80005fd0 tx=0x7f7a80009500 comp rx=0 tx=0).stop 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 msgr2=0x7f7a900b0c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 0x7f7a900b0c20 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7a8800c3c0 tx=0x7f7a8800c7c0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 shutdown_connections 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7a6c078d40 0x7f7a6c07b250 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7a900b2f60 0x7f7a90158f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7a9015d670 0x7f7a900b0c20 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 --2- 172.21.15.88:0/2224642772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7a900ad9c0 0x7f7a900cb7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 >> 172.21.15.88:0/2224642772 conn(0x7f7a900b1e40 msgr2=0x7f7a900cead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 shutdown_connections 2024-09-21T07:47:49.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.029+0000 7f7a94a00640 1 -- 172.21.15.88:0/2224642772 wait complete. 2024-09-21T07:47:49.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 -- 172.21.15.88:0/1468832731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 msgr2=0x7f523c0739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 --2- 172.21.15.88:0/1468832731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c0739f0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f523c072bd0 tx=0x7f522402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 -- 172.21.15.88:0/1468832731 shutdown_connections 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 --2- 172.21.15.88:0/1468832731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c0739f0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 --2- 172.21.15.88:0/1468832731 >> v2:172.21.15.175:3300/0 conn(0x7f523c145450 0x7f523c06f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 -- 172.21.15.88:0/1468832731 >> 172.21.15.88:0/1468832731 conn(0x7f523c06d1c0 msgr2=0x7f523c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 -- 172.21.15.88:0/1468832731 shutdown_connections 2024-09-21T07:47:49.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.278+0000 7f5243a00640 1 -- 172.21.15.88:0/1468832731 wait complete. 2024-09-21T07:47:49.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.281+0000 7f5243a00640 1 Processor -- start 2024-09-21T07:47:49.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.281+0000 7f5243a00640 1 -- start start 2024-09-21T07:47:49.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5243a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c15f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5243a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523c145450 0x7f523c15a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5243a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f523c160320 0x7f523c15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5243a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f523c0ce620 con 0x7f523c0735e0 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5243a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f523c071020 con 0x7f523c145450 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5243a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f523c0ce7a0 con 0x7f523c160320 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.282+0000 7f5241000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c15f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c15f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44134/0 (socket says 172.21.15.88:44134) 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 -- 172.21.15.88:0/2437386175 learned_addr learned my addr 172.21.15.88:0/2437386175 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5233e00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523c145450 0x7f523c15a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f523c160320 0x7f523c15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 -- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f523c160320 msgr2=0x7f523c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f523c160320 0x7f523c15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 -- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523c145450 msgr2=0x7f523c15a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523c145450 0x7f523c15a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 -- 172.21.15.88:0/2437386175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f523c0ae5d0 con 0x7f523c0735e0 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5233e00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523c145450 0x7f523c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f523c160320 0x7f523c15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.283+0000 7f5241000640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c15f880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f522c00e7c0 tx=0x7f522c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.284+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c00ccc0 con 0x7f523c0735e0 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.284+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f522c010430 con 0x7f523c0735e0 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.284+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f522c010860 con 0x7f523c0735e0 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.284+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f523c0ae850 con 0x7f523c0735e0 2024-09-21T07:47:49.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.284+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f523c0aee10 con 0x7f523c0735e0 2024-09-21T07:47:49.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.285+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f523c160f60 con 0x7f523c0735e0 2024-09-21T07:47:49.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.286+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f522c010a00 con 0x7f523c0735e0 2024-09-21T07:47:49.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.287+0000 7f5239600640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5218078d40 0x7f521807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.287+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f522c09bc10 con 0x7f523c0735e0 2024-09-21T07:47:49.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.288+0000 7f5233e00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5218078d40 0x7f521807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.288+0000 7f5233e00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5218078d40 0x7f521807b250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5224033000 tx=0x7f5224004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:49.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.291+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f523c0ae5d0 con 0x7f523c0735e0 2024-09-21T07:47:49.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.479+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f523c15a610 con 0x7f523c0735e0 2024-09-21T07:47:49.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.480+0000 7f5239600640 1 -- 172.21.15.88:0/2437386175 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) ==== 122+0+0 (secure 0 0 0) 0x7f522c063c20 con 0x7f523c0735e0 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5218078d40 msgr2=0x7f521807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5218078d40 0x7f521807b250 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f5224033000 tx=0x7f5224004730 comp rx=0 tx=0).stop 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 msgr2=0x7f523c15f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c15f880 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f522c00e7c0 tx=0x7f522c00ec80 comp rx=0 tx=0).stop 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 shutdown_connections 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5218078d40 0x7f521807b250 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f523c160320 0x7f523c15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f523c145450 0x7f523c15a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 --2- 172.21.15.88:0/2437386175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f523c0735e0 0x7f523c15f880 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.482+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 >> 172.21.15.88:0/2437386175 conn(0x7f523c06d1c0 msgr2=0x7f523c142eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.483+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 shutdown_connections 2024-09-21T07:47:49.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.483+0000 7f5243a00640 1 -- 172.21.15.88:0/2437386175 wait complete. 2024-09-21T07:47:49.507 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:49.507 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:49.542 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:47:49.542 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:49.619 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:47:49.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 -- 172.21.15.88:0/4000244080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0cfc80 msgr2=0x7f050c156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 --2- 172.21.15.88:0/4000244080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0cfc80 0x7f050c156e40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f0500009860 tx=0x7f050002a4f0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 -- 172.21.15.88:0/4000244080 shutdown_connections 2024-09-21T07:47:49.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 --2- 172.21.15.88:0/4000244080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0cfc80 0x7f050c156e40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 -- 172.21.15.88:0/4000244080 >> 172.21.15.88:0/4000244080 conn(0x7f050c06d3d0 msgr2=0x7f050c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:49.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 -- 172.21.15.88:0/4000244080 shutdown_connections 2024-09-21T07:47:49.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.879+0000 7f0514200640 1 -- 172.21.15.88:0/4000244080 wait complete. 2024-09-21T07:47:49.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.881+0000 7f0514200640 1 Processor -- start 2024-09-21T07:47:49.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.882+0000 7f0514200640 1 -- start start 2024-09-21T07:47:49.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0514200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0705b0 0x7f050c1f40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0514200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 0x7f050c156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0514200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f050c15ce30 0x7f050c0b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0514200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f050c0cd100 con 0x7f050c0cfc80 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0514200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f050c0ccf80 con 0x7f050c0705b0 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0514200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f050c0cd280 con 0x7f050c15ce30 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0510e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 0x7f050c156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0510e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 0x7f050c156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44146/0 (socket says 172.21.15.88:44146) 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.883+0000 7f0510e00640 1 -- 172.21.15.88:0/1369959327 learned_addr learned my addr 172.21.15.88:0/1369959327 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0511800640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0705b0 0x7f050c1f40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0512200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f050c15ce30 0x7f050c0b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0510e00640 1 -- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f050c15ce30 msgr2=0x7f050c0b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0510e00640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f050c15ce30 0x7f050c0b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0510e00640 1 -- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0705b0 msgr2=0x7f050c1f40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:49.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0510e00640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0705b0 0x7f050c1f40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0510e00640 1 -- 172.21.15.88:0/1369959327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f050c09c730 con 0x7f050c0cfc80 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0510e00640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 0x7f050c156400 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c480 tx=0x7f04f800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.884+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8017070 con 0x7f050c0cfc80 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.885+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04f80035f0 con 0x7f050c0cfc80 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.885+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04f8003a40 con 0x7f050c0cfc80 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.885+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f050c09c9a0 con 0x7f050c0cfc80 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.885+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f050c09cf60 con 0x7f050c0cfc80 2024-09-21T07:47:49.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.886+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f050c0cd7b0 con 0x7f050c0cfc80 2024-09-21T07:47:49.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.887+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f04f801a070 con 0x7f050c0cfc80 2024-09-21T07:47:49.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.888+0000 7f0509600640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f04ec078d40 0x7f04ec07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:49.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.888+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f04f809a520 con 0x7f050c0cfc80 2024-09-21T07:47:49.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.888+0000 7f0511800640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f04ec078d40 0x7f04ec07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:49.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.889+0000 7f0511800640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f04ec078d40 0x7f04ec07b250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0500034000 tx=0x7f0500004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:49.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:49.892+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04f8061f10 con 0x7f050c0cfc80 2024-09-21T07:47:50.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.090+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f050c09ad10 con 0x7f050c0cfc80 2024-09-21T07:47:50.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.090+0000 7f0509600640 1 -- 172.21.15.88:0/1369959327 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7f04f8066940 con 0x7f050c0cfc80 2024-09-21T07:47:50.093 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:47:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f04ec078d40 msgr2=0x7f04ec07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f04ec078d40 0x7f04ec07b250 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f0500034000 tx=0x7f0500004730 comp rx=0 tx=0).stop 2024-09-21T07:47:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 msgr2=0x7f050c156400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 0x7f050c156400 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f04f800c480 tx=0x7f04f800c940 comp rx=0 tx=0).stop 2024-09-21T07:47:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 shutdown_connections 2024-09-21T07:47:50.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f04ec078d40 0x7f04ec07b250 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f050c15ce30 0x7f050c0b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f050c0cfc80 0x7f050c156400 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 --2- 172.21.15.88:0/1369959327 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f050c0705b0 0x7f050c1f40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 >> 172.21.15.88:0/1369959327 conn(0x7f050c06d3d0 msgr2=0x7f050c0b31e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 shutdown_connections 2024-09-21T07:47:50.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.093+0000 7f0514200640 1 -- 172.21.15.88:0/1369959327 wait complete. 2024-09-21T07:47:50.117 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:47:50.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:50.117 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:47:50.179 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:47:50.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:50.180 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:47:50.215 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:50.215 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:50.256 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:47:50.256 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:47:50.335 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:47:50.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 -- 172.21.15.88:0/3285978418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840af170 msgr2=0x7f54840b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 --2- 172.21.15.88:0/3285978418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840af170 0x7f54840b2f60 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f546c005fd0 tx=0x7f546c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 -- 172.21.15.88:0/3285978418 shutdown_connections 2024-09-21T07:47:50.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 --2- 172.21.15.88:0/3285978418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840af170 0x7f54840b2f60 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 -- 172.21.15.88:0/3285978418 >> 172.21.15.88:0/3285978418 conn(0x7f54840b1e40 msgr2=0x7f54840b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:50.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 -- 172.21.15.88:0/3285978418 shutdown_connections 2024-09-21T07:47:50.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.554+0000 7f5483200640 1 -- 172.21.15.88:0/3285978418 wait complete. 2024-09-21T07:47:50.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.557+0000 7f5483200640 1 Processor -- start 2024-09-21T07:47:50.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.557+0000 7f5483200640 1 -- start start 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5483200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 0x7f54841f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5483200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54840af170 0x7f5484154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5483200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f548415a0f0 0x7f54840b0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5483200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f548406fb20 con 0x7f54840ad9c0 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5483200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f548406f9a0 con 0x7f54840af170 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5483200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f548415d670 con 0x7f548415a0f0 2024-09-21T07:47:50.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.558+0000 7f5481e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 0x7f54841f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 0x7f54841f9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44174/0 (socket says 172.21.15.88:44174) 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 -- 172.21.15.88:0/2130302932 learned_addr learned my addr 172.21.15.88:0/2130302932 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5479400640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54840af170 0x7f5484154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5482800640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f548415a0f0 0x7f54840b0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 -- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f548415a0f0 msgr2=0x7f54840b0930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f548415a0f0 0x7f54840b0930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 -- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54840af170 msgr2=0x7f5484154bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54840af170 0x7f5484154bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 -- 172.21.15.88:0/2130302932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54841f5e30 con 0x7f54840ad9c0 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.559+0000 7f5481e00640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 0x7f54841f9800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f547400ba50 tx=0x7f547400bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.560+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5474019070 con 0x7f54840ad9c0 2024-09-21T07:47:50.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.560+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f547400cd10 con 0x7f54840ad9c0 2024-09-21T07:47:50.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.560+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54740146a0 con 0x7f54840ad9c0 2024-09-21T07:47:50.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.560+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54841f6110 con 0x7f54840ad9c0 2024-09-21T07:47:50.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.560+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54841f66d0 con 0x7f54840ad9c0 2024-09-21T07:47:50.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.561+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5484069840 con 0x7f54840ad9c0 2024-09-21T07:47:50.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.562+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f547401c070 con 0x7f54840ad9c0 2024-09-21T07:47:50.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.563+0000 7f547aa00640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5460078d40 0x7f546007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:50.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.563+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f547409ae30 con 0x7f54840ad9c0 2024-09-21T07:47:50.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.564+0000 7f5479400640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5460078d40 0x7f546007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:50.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.565+0000 7f5479400640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5460078d40 0x7f546007b250 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f546c033000 tx=0x7f546c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:50.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.568+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5484069840 con 0x7f54840ad9c0 2024-09-21T07:47:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.765+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f54840b2f60 con 0x7f54840ad9c0 2024-09-21T07:47:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.765+0000 7f547aa00640 1 -- 172.21.15.88:0/2130302932 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v49) ==== 80+0+0 (secure 0 0 0) 0x7f54740628a0 con 0x7f54840ad9c0 2024-09-21T07:47:50.767 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5460078d40 msgr2=0x7f546007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5460078d40 0x7f546007b250 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f546c033000 tx=0x7f546c004730 comp rx=0 tx=0).stop 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 msgr2=0x7f54841f9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 0x7f54841f9800 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f547400ba50 tx=0x7f547400bf10 comp rx=0 tx=0).stop 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 shutdown_connections 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5460078d40 0x7f546007b250 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f548415a0f0 0x7f54840b0930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f54840af170 0x7f5484154bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 --2- 172.21.15.88:0/2130302932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54840ad9c0 0x7f54841f9800 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.767+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 >> 172.21.15.88:0/2130302932 conn(0x7f54840b1e40 msgr2=0x7f548415b4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:50.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.768+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 shutdown_connections 2024-09-21T07:47:50.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:50.768+0000 7f5483200640 1 -- 172.21.15.88:0/2130302932 wait complete. 2024-09-21T07:47:50.791 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:47:50.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:50.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:47:50.858 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:47:50.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:47:50.858 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:47:50.895 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 -- 172.21.15.88:0/250184252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40add10 msgr2=0x7f81d40b3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 --2- 172.21.15.88:0/250184252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40add10 0x7f81d40b3e10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f81d4070cd0 tx=0x7f81cc02a660 comp rx=0 tx=0).stop 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 -- 172.21.15.88:0/250184252 shutdown_connections 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 --2- 172.21.15.88:0/250184252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40add10 0x7f81d40b3e10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 --2- 172.21.15.88:0/250184252 >> v2:172.21.15.175:3300/0 conn(0x7f81d40ad3c0 0x7f81d40ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 --2- 172.21.15.88:0/250184252 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81d40704b0 0x7f81d4073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 -- 172.21.15.88:0/250184252 >> 172.21.15.88:0/250184252 conn(0x7f81d406d730 msgr2=0x7f81d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 -- 172.21.15.88:0/250184252 shutdown_connections 2024-09-21T07:47:51.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.112+0000 7f81db400640 1 -- 172.21.15.88:0/250184252 wait complete. 2024-09-21T07:47:51.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.115+0000 7f81db400640 1 Processor -- start 2024-09-21T07:47:51.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.115+0000 7f81db400640 1 -- start start 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.116+0000 7f81db400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81d40704b0 0x7f81d415dd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.116+0000 7f81db400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 0x7f81d41f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.116+0000 7f81db400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81d40add10 0x7f81d41f9140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.116+0000 7f81db400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81d40712b0 con 0x7f81d40ad3c0 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.116+0000 7f81db400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81d4071130 con 0x7f81d40add10 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.116+0000 7f81db400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81d40b4350 con 0x7f81d40704b0 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 0x7f81d41f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 0x7f81d41f8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44198/0 (socket says 172.21.15.88:44198) 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 -- 172.21.15.88:0/1684765707 learned_addr learned my addr 172.21.15.88:0/1684765707 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81d9400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81d40add10 0x7f81d41f9140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81d8a00640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81d40704b0 0x7f81d415dd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 -- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81d40704b0 msgr2=0x7f81d415dd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81d40704b0 0x7f81d415dd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 -- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81d40add10 msgr2=0x7f81d41f9140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81d40add10 0x7f81d41f9140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 -- 172.21.15.88:0/1684765707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81d4150b80 con 0x7f81d40ad3c0 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81d9400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81d40add10 0x7f81d41f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.117+0000 7f81cbe00640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 0x7f81d41f8c00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00d740 tx=0x7f81bc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.118+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc017070 con 0x7f81d40ad3c0 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.118+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81bc009cc0 con 0x7f81d40ad3c0 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.118+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81d4150e60 con 0x7f81d40ad3c0 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.118+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81bc0106c0 con 0x7f81d40ad3c0 2024-09-21T07:47:51.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.118+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81d4151420 con 0x7f81d40ad3c0 2024-09-21T07:47:51.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.119+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81d40ad7d0 con 0x7f81d40ad3c0 2024-09-21T07:47:51.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.120+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f81bc01a070 con 0x7f81d40ad3c0 2024-09-21T07:47:51.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.121+0000 7f81d1600640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f81b0078d70 0x7f81b007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.122+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f81bc09b310 con 0x7f81d40ad3c0 2024-09-21T07:47:51.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.122+0000 7f81d8a00640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f81b0078d70 0x7f81b007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.123+0000 7f81d8a00640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f81b0078d70 0x7f81b007b280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81c4005fd0 tx=0x7f81c400b360 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.126+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81bc062c40 con 0x7f81d40ad3c0 2024-09-21T07:47:51.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.347+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f81d415e420 con 0x7f81d40ad3c0 2024-09-21T07:47:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.348+0000 7f81d1600640 1 -- 172.21.15.88:0/1684765707 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f81bc067670 con 0x7f81d40ad3c0 2024-09-21T07:47:51.350 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:51.350 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:47:51.350 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:47:51.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f81b0078d70 msgr2=0x7f81b007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f81b0078d70 0x7f81b007b280 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f81c4005fd0 tx=0x7f81c400b360 comp rx=0 tx=0).stop 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 msgr2=0x7f81d41f8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 0x7f81d41f8c00 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00d740 tx=0x7f81bc00dc00 comp rx=0 tx=0).stop 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 shutdown_connections 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f81b0078d70 0x7f81b007b280 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81d40add10 0x7f81d41f9140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81d40ad3c0 0x7f81d41f8c00 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 --2- 172.21.15.88:0/1684765707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81d40704b0 0x7f81d415dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 >> 172.21.15.88:0/1684765707 conn(0x7f81d406d730 msgr2=0x7f81d414dbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 shutdown_connections 2024-09-21T07:47:51.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.350+0000 7f81db400640 1 -- 172.21.15.88:0/1684765707 wait complete. 2024-09-21T07:47:51.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:47:51.619 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-09-21T07:47:51.619 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T07:47:51.619 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol61930 2024-09-21T07:47:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 -- 172.21.15.88:0/390523528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe284003a50 msgr2=0x7fe284003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 --2- 172.21.15.88:0/390523528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe284003a50 0x7fe284003ec0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe29000c310 tx=0x7fe284005830 comp rx=0 tx=0).stop 2024-09-21T07:47:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 -- 172.21.15.88:0/390523528 shutdown_connections 2024-09-21T07:47:51.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 --2- 172.21.15.88:0/390523528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe284003a50 0x7fe284003ec0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 -- 172.21.15.88:0/390523528 >> 172.21.15.88:0/390523528 conn(0x7fe29c06d1c0 msgr2=0x7fe29c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 -- 172.21.15.88:0/390523528 shutdown_connections 2024-09-21T07:47:51.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.841+0000 7fe2a4800640 1 -- 172.21.15.88:0/390523528 wait complete. 2024-09-21T07:47:51.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.843+0000 7fe2a4800640 1 Processor -- start 2024-09-21T07:47:51.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.843+0000 7fe2a4800640 1 -- start start 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a4800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe284003a50 0x7fe29c138060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a4800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 0x7fe29c13c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a4800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe29c13ce50 0x7fe29c13f3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a4800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c073540 con 0x7fe29c136d90 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a4800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe29c070180 con 0x7fe29c13ce50 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a4800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe29c0736c0 con 0x7fe284003a50 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 0x7fe29c13c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 0x7fe29c13c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44210/0 (socket says 172.21.15.88:44210) 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 -- 172.21.15.88:0/3051905985 learned_addr learned my addr 172.21.15.88:0/3051905985 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a2800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe29c13ce50 0x7fe29c13f3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1e00640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe284003a50 0x7fe29c138060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 -- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe284003a50 msgr2=0x7fe29c138060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe284003a50 0x7fe29c138060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 -- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe29c13ce50 msgr2=0x7fe29c13f3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe29c13ce50 0x7fe29c13f3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 -- 172.21.15.88:0/3051905985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe29c13b1b0 con 0x7fe29c136d90 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.845+0000 7fe2a1400640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 0x7fe29c13c910 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe28c004d50 tx=0x7fe28c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.846+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c015020 con 0x7fe29c136d90 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.846+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe28c011960 con 0x7fe29c136d90 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.846+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe28c011d30 con 0x7fe29c136d90 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.846+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe29c13d720 con 0x7fe29c136d90 2024-09-21T07:47:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.846+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe29c15bfe0 con 0x7fe29c136d90 2024-09-21T07:47:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.847+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe29c073c20 con 0x7fe29c136d90 2024-09-21T07:47:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.848+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7fe28c018020 con 0x7fe29c136d90 2024-09-21T07:47:51.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.849+0000 7fe296000640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe280078d40 0x7fe28007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:51.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.849+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fe28c09bc00 con 0x7fe29c136d90 2024-09-21T07:47:51.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.850+0000 7fe2a1e00640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe280078d40 0x7fe28007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:51.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.850+0000 7fe2a1e00640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe280078d40 0x7fe28007b250 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe290033000 tx=0x7fe29c070bc0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:51.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:51.853+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe28c01f020 con 0x7fe29c136d90 2024-09-21T07:47:52.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.065+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol61930", "target": ["mon-mgr", ""]}) -- 0x7fe29c1e6720 con 0x7fe280078d40 2024-09-21T07:47:52.076 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:52.074+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:52.076 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:52.074+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:52.076 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:52.074+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:52.076 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:52.074+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:52.076 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:47:52.074+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:47:52.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.115+0000 7fe296000640 1 -- 172.21.15.88:0/3051905985 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe29c1e6720 con 0x7fe280078d40 2024-09-21T07:47:52.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe280078d40 msgr2=0x7fe28007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe280078d40 0x7fe28007b250 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fe290033000 tx=0x7fe29c070bc0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 msgr2=0x7fe29c13c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 0x7fe29c13c910 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe28c004d50 tx=0x7fe28c001630 comp rx=0 tx=0).stop 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 shutdown_connections 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe280078d40 0x7fe28007b250 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe29c13ce50 0x7fe29c13f3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe29c136d90 0x7fe29c13c910 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 --2- 172.21.15.88:0/3051905985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe284003a50 0x7fe29c138060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 >> 172.21.15.88:0/3051905985 conn(0x7fe29c06d1c0 msgr2=0x7fe29c0cea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 shutdown_connections 2024-09-21T07:47:52.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.117+0000 7fe2a4800640 1 -- 172.21.15.88:0/3051905985 wait complete. 2024-09-21T07:47:52.140 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.356+0000 7f91b6800640 1 -- 172.21.15.88:0/601480956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b0152920 msgr2=0x7f91b015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.356+0000 7f91b6800640 1 --2- 172.21.15.88:0/601480956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b0152920 0x7f91b015f3d0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f91b0160420 tx=0x7f919c02a580 comp rx=0 tx=0).stop 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.356+0000 7f91b6800640 1 -- 172.21.15.88:0/601480956 shutdown_connections 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.356+0000 7f91b6800640 1 --2- 172.21.15.88:0/601480956 >> v2:172.21.15.175:3300/0 conn(0x7f91b015f910 0x7f91b00b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.356+0000 7f91b6800640 1 --2- 172.21.15.88:0/601480956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b0152920 0x7f91b015f3d0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.356+0000 7f91b6800640 1 -- 172.21.15.88:0/601480956 >> 172.21.15.88:0/601480956 conn(0x7f91b006d730 msgr2=0x7f91b00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:52.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.357+0000 7f91b6800640 1 -- 172.21.15.88:0/601480956 shutdown_connections 2024-09-21T07:47:52.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.357+0000 7f91b6800640 1 -- 172.21.15.88:0/601480956 wait complete. 2024-09-21T07:47:52.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.359+0000 7f91b6800640 1 Processor -- start 2024-09-21T07:47:52.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.359+0000 7f91b6800640 1 -- start start 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.360+0000 7f91b6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b00aebc0 0x7f91b0152f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.360+0000 7f91b6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0152920 0x7f91b0147d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.360+0000 7f91b6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 0x7f91b0154770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.360+0000 7f91b6800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f91b00b40d0 con 0x7f91b015f910 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.360+0000 7f91b6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f91b00b3f50 con 0x7f91b0152920 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.360+0000 7f91b6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f91b00b4250 con 0x7f91b00aebc0 2024-09-21T07:47:52.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 0x7f91b0154770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 0x7f91b0154770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44234/0 (socket says 172.21.15.88:44234) 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 -- 172.21.15.88:0/2091825380 learned_addr learned my addr 172.21.15.88:0/2091825380 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91aea00640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0152920 0x7f91b0147d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91af400640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b00aebc0 0x7f91b0152f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 -- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b00aebc0 msgr2=0x7f91b0152f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b00aebc0 0x7f91b0152f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 -- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0152920 msgr2=0x7f91b0147d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0152920 0x7f91b0147d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 -- 172.21.15.88:0/2091825380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91b0153440 con 0x7f91b015f910 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.361+0000 7f91afe00640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 0x7f91b0154770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f91a800cad0 tx=0x7f91a800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.362+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800ae20 con 0x7f91b015f910 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.362+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f91a800f450 con 0x7f91b015f910 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.362+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91b0153950 con 0x7f91b015f910 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.362+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91a800f8c0 con 0x7f91b015f910 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.362+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91b014bc00 con 0x7f91b015f910 2024-09-21T07:47:52.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.363+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91b01536c0 con 0x7f91b015f910 2024-09-21T07:47:52.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.364+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7f91a800fa60 con 0x7f91b015f910 2024-09-21T07:47:52.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.365+0000 7f91a7400640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9190078d70 0x7f919007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.365+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f91a809bb60 con 0x7f91b015f910 2024-09-21T07:47:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.366+0000 7f91af400640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9190078d70 0x7f919007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.366+0000 7f91af400640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9190078d70 0x7f919007b280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91a0001720 tx=0x7f91a0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:52.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.370+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f91a8014030 con 0x7f91b015f910 2024-09-21T07:47:52.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.591+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f91b00b47e0 con 0x7f91b015f910 2024-09-21T07:47:52.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.592+0000 7f91a7400640 1 -- 172.21.15.88:0/2091825380 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f91a80635b0 con 0x7f91b015f910 2024-09-21T07:47:52.594 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:52.594 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:47:52.595 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.594+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9190078d70 msgr2=0x7f919007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.594+0000 7f91b6800640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9190078d70 0x7f919007b280 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f91a0001720 tx=0x7f91a0009040 comp rx=0 tx=0).stop 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.594+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 msgr2=0x7f91b0154770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.594+0000 7f91b6800640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 0x7f91b0154770 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f91a800cad0 tx=0x7f91a800cf90 comp rx=0 tx=0).stop 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 shutdown_connections 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9190078d70 0x7f919007b280 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91b015f910 0x7f91b0154770 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91b0152920 0x7f91b0147d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 --2- 172.21.15.88:0/2091825380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f91b00aebc0 0x7f91b0152f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 >> 172.21.15.88:0/2091825380 conn(0x7f91b006d730 msgr2=0x7f91b015dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 shutdown_connections 2024-09-21T07:47:52.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:52.595+0000 7f91b6800640 1 -- 172.21.15.88:0/2091825380 wait complete. 2024-09-21T07:47:52.618 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:47:52.858 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.75998524700000003,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.962139000378414,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:47:52.859 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.962139000378414 }, "state": "open", "uptime": 0.759985247 } ] 2024-09-21T07:47:52.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 -- 172.21.15.88:0/3610437034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7700af630 msgr2=0x7ff770073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 --2- 172.21.15.88:0/3610437034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7700af630 0x7ff770073500 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7ff760005fd0 tx=0x7ff76002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 -- 172.21.15.88:0/3610437034 shutdown_connections 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 --2- 172.21.15.88:0/3610437034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7700af630 0x7ff770073500 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 -- 172.21.15.88:0/3610437034 >> 172.21.15.88:0/3610437034 conn(0x7ff77006d730 msgr2=0x7ff7700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 -- 172.21.15.88:0/3610437034 shutdown_connections 2024-09-21T07:47:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.076+0000 7ff774e00640 1 -- 172.21.15.88:0/3610437034 wait complete. 2024-09-21T07:47:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.079+0000 7ff774e00640 1 Processor -- start 2024-09-21T07:47:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.079+0000 7ff774e00640 1 -- start start 2024-09-21T07:47:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff774e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77006f7b0 0x7ff7701f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff774e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 0x7ff770147010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff774e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7700af630 0x7ff770147550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff774e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700b3d90 con 0x7ff770073a40 2024-09-21T07:47:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff774e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff7700b3c10 con 0x7ff7700af630 2024-09-21T07:47:53.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff774e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff7700b3f10 con 0x7ff77006f7b0 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff76d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 0x7ff770147010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff76d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 0x7ff770147010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44242/0 (socket says 172.21.15.88:44242) 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.080+0000 7ff76d400640 1 -- 172.21.15.88:0/2161163776 learned_addr learned my addr 172.21.15.88:0/2161163776 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76de00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77006f7b0 0x7ff7701f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76e800640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7700af630 0x7ff770147550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76d400640 1 -- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77006f7b0 msgr2=0x7ff7701f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76d400640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77006f7b0 0x7ff7701f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76d400640 1 -- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7700af630 msgr2=0x7ff770147550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76d400640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7700af630 0x7ff770147550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76d400640 1 -- 172.21.15.88:0/2161163776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff77009a500 con 0x7ff770073a40 2024-09-21T07:47:53.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.081+0000 7ff76d400640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 0x7ff770147010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c3a0 tx=0x7ff75c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.082+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c017070 con 0x7ff770073a40 2024-09-21T07:47:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.082+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff75c0053c0 con 0x7ff770073a40 2024-09-21T07:47:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.082+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff75c003850 con 0x7ff770073a40 2024-09-21T07:47:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.082+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff770155830 con 0x7ff770073a40 2024-09-21T07:47:53.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.082+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff77014b1b0 con 0x7ff770073a40 2024-09-21T07:47:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.083+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7700b4420 con 0x7ff770073a40 2024-09-21T07:47:53.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.084+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 7) ==== 101594+0+0 (secure 0 0 0) 0x7ff75c01a070 con 0x7ff770073a40 2024-09-21T07:47:53.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.086+0000 7ff766000640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff750078d40 0x7ff75007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:53.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.086+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7ff75c09a480 con 0x7ff770073a40 2024-09-21T07:47:53.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.086+0000 7ff76de00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff750078d40 0x7ff75007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:53.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.087+0000 7ff76de00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff750078d40 0x7ff75007b250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff760033000 tx=0x7ff760004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:53.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.090+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff770073e50 con 0x7ff770073a40 2024-09-21T07:47:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.315+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff7701555c0 con 0x7ff770073a40 2024-09-21T07:47:53.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.315+0000 7ff766000640 1 -- 172.21.15.88:0/2161163776 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7ff75c061f10 con 0x7ff770073a40 2024-09-21T07:47:53.317 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:53.318 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:47:53.318 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.317+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff750078d40 msgr2=0x7ff75007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.317+0000 7ff774e00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff750078d40 0x7ff75007b250 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff760033000 tx=0x7ff760004730 comp rx=0 tx=0).stop 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.317+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 msgr2=0x7ff770147010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.317+0000 7ff774e00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 0x7ff770147010 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7ff75c00c3a0 tx=0x7ff75c00c860 comp rx=0 tx=0).stop 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 shutdown_connections 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff750078d40 0x7ff75007b250 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff7700af630 0x7ff770147550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff770073a40 0x7ff770147010 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 --2- 172.21.15.88:0/2161163776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff77006f7b0 0x7ff7701f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 >> 172.21.15.88:0/2161163776 conn(0x7ff77006d730 msgr2=0x7ff7700aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 shutdown_connections 2024-09-21T07:47:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:53.318+0000 7ff774e00640 1 -- 172.21.15.88:0/2161163776 wait complete. 2024-09-21T07:47:53.347 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:47:53.586 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":1.4879711149999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.92522963317446,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:47:53.586 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.92522963317446 }, "state": "open", "uptime": 1.487971115 } ] 2024-09-21T07:47:53.586 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:47:56.134 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:56.139 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:47:58.587 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:47:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 -- 172.21.15.88:0/543109434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60d0005250 msgr2=0x7f60d000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 --2- 172.21.15.88:0/543109434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60d0005250 0x7f60d000b9c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f60e000c470 tx=0x7f60d0005830 comp rx=0 tx=0).stop 2024-09-21T07:47:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 -- 172.21.15.88:0/543109434 shutdown_connections 2024-09-21T07:47:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 --2- 172.21.15.88:0/543109434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60d0005250 0x7f60d000b9c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 -- 172.21.15.88:0/543109434 >> 172.21.15.88:0/543109434 conn(0x7f60e80ceef0 msgr2=0x7f60e80cf300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:58.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 -- 172.21.15.88:0/543109434 shutdown_connections 2024-09-21T07:47:58.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.817+0000 7f60f0400640 1 -- 172.21.15.88:0/543109434 wait complete. 2024-09-21T07:47:58.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.819+0000 7f60f0400640 1 Processor -- start 2024-09-21T07:47:58.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.819+0000 7f60f0400640 1 -- start start 2024-09-21T07:47:58.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60f0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60d0005250 0x7f60e81d7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60f0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 0x7f60e81da340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60f0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e81da880 0x7f60e81dac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60f0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60e80d20b0 con 0x7f60e81e6360 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60f0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60e809ba60 con 0x7f60e81da880 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60f0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60e81d9cb0 con 0x7f60d0005250 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60ed000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 0x7f60e81da340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60ed000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 0x7f60e81da340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51834/0 (socket says 172.21.15.88:51834) 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60ed000640 1 -- 172.21.15.88:0/234348048 learned_addr learned my addr 172.21.15.88:0/234348048 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60ee400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e81da880 0x7f60e81dac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60eda00640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60d0005250 0x7f60e81d7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60ed000640 1 -- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60d0005250 msgr2=0x7f60e81d7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.821+0000 7f60ed000640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60d0005250 0x7f60e81d7ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60ed000640 1 -- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e81da880 msgr2=0x7f60e81dac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60ed000640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e81da880 0x7f60e81dac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60ed000640 1 -- 172.21.15.88:0/234348048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60e81e6be0 con 0x7f60e81e6360 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60ed000640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 0x7f60e81da340 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f60d4011ae0 tx=0x7f60d4009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d401e070 con 0x7f60e81e6360 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f60d4004480 con 0x7f60e81e6360 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60e809f390 con 0x7f60e81e6360 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60d400fc60 con 0x7f60e81e6360 2024-09-21T07:47:58.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.822+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60e809f950 con 0x7f60e81e6360 2024-09-21T07:47:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.824+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60e80ccac0 con 0x7f60e81e6360 2024-09-21T07:47:58.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.825+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f60d4021070 con 0x7f60e81e6360 2024-09-21T07:47:58.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.826+0000 7f60e5600640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f60c8078dd0 0x7f60c807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:47:58.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.826+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f60d409bcf0 con 0x7f60e81e6360 2024-09-21T07:47:58.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.826+0000 7f60eda00640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f60c8078dd0 0x7f60c807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:47:58.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.827+0000 7f60eda00640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f60c8078dd0 0x7f60c807b2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f60dc004fd0 tx=0x7f60dc004ab0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:47:58.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:58.830+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f60d40a0070 con 0x7f60e81e6360 2024-09-21T07:47:59.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.053+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60e80b3d40 con 0x7f60e81e6360 2024-09-21T07:47:59.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.053+0000 7f60e5600640 1 -- 172.21.15.88:0/234348048 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f60e81e6be0 con 0x7f60e81e6360 2024-09-21T07:47:59.056 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:47:59.056 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:47:59.056 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:47:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f60c8078dd0 msgr2=0x7f60c807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f60c8078dd0 0x7f60c807b2e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f60dc004fd0 tx=0x7f60dc004ab0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 msgr2=0x7f60e81da340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:47:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 0x7f60e81da340 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f60d4011ae0 tx=0x7f60d4009e40 comp rx=0 tx=0).stop 2024-09-21T07:47:59.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 shutdown_connections 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f60c8078dd0 0x7f60c807b2e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60e81da880 0x7f60e81dac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60e81e6360 0x7f60e81da340 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 --2- 172.21.15.88:0/234348048 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60d0005250 0x7f60e81d7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 >> 172.21.15.88:0/234348048 conn(0x7f60e80ceef0 msgr2=0x7f60e81e6fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 shutdown_connections 2024-09-21T07:47:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:47:59.056+0000 7f60f0400640 1 -- 172.21.15.88:0/234348048 wait complete. 2024-09-21T07:47:59.084 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:47:59.323 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":14,"uptime":7.2248597510000003,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.636527719157947,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:47:59.324 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 14, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.636527719157947 }, "state": "open", "uptime": 7.224859751 } ] 2024-09-21T07:47:59.324 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:04.325 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.555+0000 7f7551800640 1 -- 172.21.15.88:0/1089481542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 msgr2=0x7f754c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.555+0000 7f7551800640 1 --2- 172.21.15.88:0/1089481542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c15a0f0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f754c051590 tx=0x7f753802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 -- 172.21.15.88:0/1089481542 shutdown_connections 2024-09-21T07:48:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 --2- 172.21.15.88:0/1089481542 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f754c15d670 0x7f754c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 --2- 172.21.15.88:0/1089481542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c15a0f0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 --2- 172.21.15.88:0/1089481542 >> v2:172.21.15.175:3300/0 conn(0x7f754c0ad9c0 0x7f754c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 -- 172.21.15.88:0/1089481542 >> 172.21.15.88:0/1089481542 conn(0x7f754c0b1e40 msgr2=0x7f754c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:04.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 -- 172.21.15.88:0/1089481542 shutdown_connections 2024-09-21T07:48:04.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.556+0000 7f7551800640 1 -- 172.21.15.88:0/1089481542 wait complete. 2024-09-21T07:48:04.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.558+0000 7f7551800640 1 Processor -- start 2024-09-21T07:48:04.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.558+0000 7f7551800640 1 -- start start 2024-09-21T07:48:04.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7551800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f754c0ad9c0 0x7f754c1454a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7551800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c1459e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7551800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f754c15d670 0x7f754c146160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7551800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f754c070c80 con 0x7f754c0b2f60 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7551800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f754c06f290 con 0x7f754c15d670 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7551800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f754c146ff0 con 0x7f754c0ad9c0 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c1459e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c1459e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51840/0 (socket says 172.21.15.88:51840) 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 -- 172.21.15.88:0/3748609743 learned_addr learned my addr 172.21.15.88:0/3748609743 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754be00640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f754c0ad9c0 0x7f754c1454a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f7550e00640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f754c15d670 0x7f754c146160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 -- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f754c0ad9c0 msgr2=0x7f754c1454a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f754c0ad9c0 0x7f754c1454a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 -- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f754c15d670 msgr2=0x7f754c146160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f754c15d670 0x7f754c146160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.560+0000 7f754b400640 1 -- 172.21.15.88:0/3748609743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f754c09d900 con 0x7f754c0b2f60 2024-09-21T07:48:04.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.561+0000 7f754b400640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c1459e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7538034040 tx=0x7f75380038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:04.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.561+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f753803e030 con 0x7f754c0b2f60 2024-09-21T07:48:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.561+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7538003ca0 con 0x7f754c0b2f60 2024-09-21T07:48:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.561+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7538035040 con 0x7f754c0b2f60 2024-09-21T07:48:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.561+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f754c09ddf0 con 0x7f754c0b2f60 2024-09-21T07:48:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.561+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f754c158500 con 0x7f754c0b2f60 2024-09-21T07:48:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.562+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f754c15a0f0 con 0x7f754c0b2f60 2024-09-21T07:48:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.563+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f7538041030 con 0x7f754c0b2f60 2024-09-21T07:48:04.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.564+0000 7f7543e00640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f752c078dd0 0x7f752c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:04.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.564+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f75380bb8c0 con 0x7f754c0b2f60 2024-09-21T07:48:04.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.565+0000 7f754be00640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f752c078dd0 0x7f752c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:04.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.566+0000 7f754be00640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f752c078dd0 0x7f752c07b2e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f753c005fd0 tx=0x7f753c009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:04.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.569+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7538083270 con 0x7f754c0b2f60 2024-09-21T07:48:04.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.790+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f754c09db80 con 0x7f754c0b2f60 2024-09-21T07:48:04.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.791+0000 7f7543e00640 1 -- 172.21.15.88:0/3748609743 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f7538087ca0 con 0x7f754c0b2f60 2024-09-21T07:48:04.793 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:04.793 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:04.793 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:04.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f752c078dd0 msgr2=0x7f752c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f752c078dd0 0x7f752c07b2e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f753c005fd0 tx=0x7f753c009500 comp rx=0 tx=0).stop 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 msgr2=0x7f754c1459e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c1459e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f7538034040 tx=0x7f75380038a0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 shutdown_connections 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f752c078dd0 0x7f752c07b2e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f754c15d670 0x7f754c146160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f754c0b2f60 0x7f754c1459e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 --2- 172.21.15.88:0/3748609743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f754c0ad9c0 0x7f754c1454a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 >> 172.21.15.88:0/3748609743 conn(0x7f754c0b1e40 msgr2=0x7f754c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 shutdown_connections 2024-09-21T07:48:04.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:04.793+0000 7f7551800640 1 -- 172.21.15.88:0/3748609743 wait complete. 2024-09-21T07:48:04.818 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:05.060 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":13,"uptime":12.961748387,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.351627306724374,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:05.061 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 13, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.351627306724374 }, "state": "open", "uptime": 12.961748387 } ] 2024-09-21T07:48:05.061 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:10.062 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:10.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 -- 172.21.15.88:0/3377896894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d8152920 msgr2=0x7fe6d815f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 --2- 172.21.15.88:0/3377896894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d8152920 0x7fe6d815f3d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4005fd0 tx=0x7fe6c402a540 comp rx=0 tx=0).stop 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 -- 172.21.15.88:0/3377896894 shutdown_connections 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 --2- 172.21.15.88:0/3377896894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6d815f910 0x7fe6d80b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 --2- 172.21.15.88:0/3377896894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d8152920 0x7fe6d815f3d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 -- 172.21.15.88:0/3377896894 >> 172.21.15.88:0/3377896894 conn(0x7fe6d806d730 msgr2=0x7fe6d80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 -- 172.21.15.88:0/3377896894 shutdown_connections 2024-09-21T07:48:10.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.292+0000 7fe6de000640 1 -- 172.21.15.88:0/3377896894 wait complete. 2024-09-21T07:48:10.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.294+0000 7fe6de000640 1 Processor -- start 2024-09-21T07:48:10.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.294+0000 7fe6de000640 1 -- start start 2024-09-21T07:48:10.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6de000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6d80aebc0 0x7fe6d8151100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:10.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6de000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 0x7fe6d80ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:10.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6de000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d815f910 0x7fe6d80cd490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:10.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6de000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d80b4190 con 0x7fe6d8152920 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6de000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6d80b4010 con 0x7fe6d815f910 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6de000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6d80b4310 con 0x7fe6d80aebc0 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 0x7fe6d80ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6d6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 0x7fe6d80ce800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51728/0 (socket says 172.21.15.88:51728) 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6d6a00640 1 -- 172.21.15.88:0/3528819745 learned_addr learned my addr 172.21.15.88:0/3528819745 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6d7400640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6d80aebc0 0x7fe6d8151100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:10.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.296+0000 7fe6d7e00640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d815f910 0x7fe6d80cd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:10.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6d6a00640 1 -- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6d80aebc0 msgr2=0x7fe6d8151100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6d6a00640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6d80aebc0 0x7fe6d8151100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6d6a00640 1 -- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d815f910 msgr2=0x7fe6d80cd490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6d6a00640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d815f910 0x7fe6d80cd490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6d6a00640 1 -- 172.21.15.88:0/3528819745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6d81500c0 con 0x7fe6d8152920 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6d6a00640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 0x7fe6d80ce800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4006010 tx=0x7fe6c40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c403e030 con 0x7fe6d8152920 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6c4003d90 con 0x7fe6d8152920 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6c4035040 con 0x7fe6d8152920 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6d8150340 con 0x7fe6d8152920 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.297+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6d8150900 con 0x7fe6d8152920 2024-09-21T07:48:10.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.298+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6d8142080 con 0x7fe6d8152920 2024-09-21T07:48:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.300+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fe6c4041030 con 0x7fe6d8152920 2024-09-21T07:48:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.300+0000 7fe6cf400640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6b8078dd0 0x7fe6b807b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:10.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.301+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fe6c40bb8e0 con 0x7fe6d8152920 2024-09-21T07:48:10.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.301+0000 7fe6d7400640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6b8078dd0 0x7fe6b807b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:10.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.302+0000 7fe6d7400640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6b8078dd0 0x7fe6b807b2e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8000f60 tx=0x7fe6c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:10.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.305+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6d81500c0 con 0x7fe6d8152920 2024-09-21T07:48:10.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.530+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe6d80b48a0 con 0x7fe6d8152920 2024-09-21T07:48:10.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.530+0000 7fe6cf400640 1 -- 172.21.15.88:0/3528819745 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fe6d815fd20 con 0x7fe6d8152920 2024-09-21T07:48:10.531 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:10.531 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:10.532 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6b8078dd0 msgr2=0x7fe6b807b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6b8078dd0 0x7fe6b807b2e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe6c8000f60 tx=0x7fe6c8009040 comp rx=0 tx=0).stop 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 msgr2=0x7fe6d80ce800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 0x7fe6d80ce800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe6c4006010 tx=0x7fe6c40038a0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 shutdown_connections 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe6b8078dd0 0x7fe6b807b2e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6d815f910 0x7fe6d80cd490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6d8152920 0x7fe6d80ce800 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 --2- 172.21.15.88:0/3528819745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6d80aebc0 0x7fe6d8151100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 >> 172.21.15.88:0/3528819745 conn(0x7fe6d806d730 msgr2=0x7fe6d815a4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 shutdown_connections 2024-09-21T07:48:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:10.533+0000 7fe6de000640 1 -- 172.21.15.88:0/3528819745 wait complete. 2024-09-21T07:48:10.559 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:10.800 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":18.701636961999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.070332293399034,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:10.800 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 21.070332293399034 }, "state": "open", "uptime": 18.701636962 } ] 2024-09-21T07:48:10.800 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:15.802 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:16.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 -- 172.21.15.88:0/2631060008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 msgr2=0x7fa2400b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:16.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 --2- 172.21.15.88:0/2631060008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2400b3960 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fa238005fd0 tx=0x7fa23802aa00 comp rx=0 tx=0).stop 2024-09-21T07:48:16.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 -- 172.21.15.88:0/2631060008 shutdown_connections 2024-09-21T07:48:16.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 --2- 172.21.15.88:0/2631060008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2400b3960 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 --2- 172.21.15.88:0/2631060008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa24015d760 0x7fa2401535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 -- 172.21.15.88:0/2631060008 >> 172.21.15.88:0/2631060008 conn(0x7fa24006d730 msgr2=0x7fa2400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:16.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 -- 172.21.15.88:0/2631060008 shutdown_connections 2024-09-21T07:48:16.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.032+0000 7fa245a00640 1 -- 172.21.15.88:0/2631060008 wait complete. 2024-09-21T07:48:16.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.034+0000 7fa245a00640 1 Processor -- start 2024-09-21T07:48:16.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.034+0000 7fa245a00640 1 -- start start 2024-09-21T07:48:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa245a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2400ad0e0 0x7fa240153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa245a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2401485c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa245a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa24015d760 0x7fa240159c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa245a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2400b4da0 con 0x7fa2400b3550 2024-09-21T07:48:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa245a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2400b4c20 con 0x7fa2400ad0e0 2024-09-21T07:48:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa245a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2400b3ea0 con 0x7fa24015d760 2024-09-21T07:48:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa236000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2401485c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa236000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2401485c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43634/0 (socket says 172.21.15.88:43634) 2024-09-21T07:48:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa236000640 1 -- 172.21.15.88:0/3858450236 learned_addr learned my addr 172.21.15.88:0/3858450236 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa23ea00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2400ad0e0 0x7fa240153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:16.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.036+0000 7fa23f400640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa24015d760 0x7fa240159c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa236000640 1 -- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa24015d760 msgr2=0x7fa240159c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa236000640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa24015d760 0x7fa240159c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa236000640 1 -- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2400ad0e0 msgr2=0x7fa240153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa236000640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2400ad0e0 0x7fa240153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa236000640 1 -- 172.21.15.88:0/3858450236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa24015a140 con 0x7fa2400b3550 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa236000640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2401485c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa22800c370 tx=0x7fa22800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228017070 con 0x7fa2400b3550 2024-09-21T07:48:16.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa2280053c0 con 0x7fa2400b3550 2024-09-21T07:48:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa24014be50 con 0x7fa2400b3550 2024-09-21T07:48:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa228003850 con 0x7fa2400b3550 2024-09-21T07:48:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.037+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa240158490 con 0x7fa2400b3550 2024-09-21T07:48:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.038+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2400b4330 con 0x7fa2400b3550 2024-09-21T07:48:16.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.040+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fa22801a070 con 0x7fa2400b3550 2024-09-21T07:48:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.041+0000 7fa237400640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa21c078dd0 0x7fa21c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:16.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.041+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fa22809a4f0 con 0x7fa2400b3550 2024-09-21T07:48:16.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.041+0000 7fa23ea00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa21c078dd0 0x7fa21c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:16.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.042+0000 7fa23ea00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa21c078dd0 0x7fa21c07b2e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa23000bb50 tx=0x7fa23000b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:16.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.045+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa228061f40 con 0x7fa2400b3550 2024-09-21T07:48:16.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.264+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa24015a310 con 0x7fa2400b3550 2024-09-21T07:48:16.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.265+0000 7fa237400640 1 -- 172.21.15.88:0/3858450236 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fa228066970 con 0x7fa2400b3550 2024-09-21T07:48:16.266 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:16.266 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:16.267 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa21c078dd0 msgr2=0x7fa21c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa21c078dd0 0x7fa21c07b2e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa23000bb50 tx=0x7fa23000b2e0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 msgr2=0x7fa2401485c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2401485c0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa22800c370 tx=0x7fa22800c830 comp rx=0 tx=0).stop 2024-09-21T07:48:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 shutdown_connections 2024-09-21T07:48:16.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa21c078dd0 0x7fa21c07b2e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa24015d760 0x7fa240159c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2400b3550 0x7fa2401485c0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 --2- 172.21.15.88:0/3858450236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa2400ad0e0 0x7fa240153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:16.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 >> 172.21.15.88:0/3858450236 conn(0x7fa24006d730 msgr2=0x7fa2400cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:16.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 shutdown_connections 2024-09-21T07:48:16.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:16.267+0000 7fa245a00640 1 -- 172.21.15.88:0/3858450236 wait complete. 2024-09-21T07:48:16.294 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:16.535 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":12,"uptime":24.436525630999999,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.79298337892968,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:16.535 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 1, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 12, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.79298337892968 }, "state": "open", "uptime": 24.436525631 } ] 2024-09-21T07:48:16.535 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:21.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 -- 172.21.15.88:0/1946511633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31000ad0e0 msgr2=0x7f31000ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 --2- 172.21.15.88:0/1946511633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31000ad0e0 0x7f31000ad4f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f3100072d90 tx=0x7f30e802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 -- 172.21.15.88:0/1946511633 shutdown_connections 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 --2- 172.21.15.88:0/1946511633 >> v2:172.21.15.175:3300/0 conn(0x7f3100160310 0x7f3100160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 --2- 172.21.15.88:0/1946511633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31000ad0e0 0x7f31000ad4f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 --2- 172.21.15.88:0/1946511633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f310015d760 0x7f31001535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 -- 172.21.15.88:0/1946511633 >> 172.21.15.88:0/1946511633 conn(0x7f310006d730 msgr2=0x7f31000b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 -- 172.21.15.88:0/1946511633 shutdown_connections 2024-09-21T07:48:21.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.765+0000 7f3107c00640 1 -- 172.21.15.88:0/1946511633 wait complete. 2024-09-21T07:48:21.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.768+0000 7f3107c00640 1 Processor -- start 2024-09-21T07:48:21.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.768+0000 7f3107c00640 1 -- start start 2024-09-21T07:48:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f3107c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31000ad0e0 0x7f31001f9af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f3107c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 0x7f3100149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f3107c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3100160310 0x7f310009c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f3107c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31000b4da0 con 0x7f310015d760 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f3107c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31000b4c20 con 0x7f3100160310 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f3107c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3100160c60 con 0x7f31000ad0e0 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.769+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 0x7f3100149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 0x7f3100149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43654/0 (socket says 172.21.15.88:43654) 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 -- 172.21.15.88:0/3988869025 learned_addr learned my addr 172.21.15.88:0/3988869025 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f3105200640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31000ad0e0 0x7f31001f9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f3105c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3100160310 0x7f310009c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 -- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31000ad0e0 msgr2=0x7f31001f9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31000ad0e0 0x7f31001f9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 -- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3100160310 msgr2=0x7f310009c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3100160310 0x7f310009c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 -- 172.21.15.88:0/3988869025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3100156b50 con 0x7f310015d760 2024-09-21T07:48:21.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.770+0000 7f30f7e00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 0x7f3100149890 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30e8000c00 tx=0x7f30e8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.771+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e803e030 con 0x7f310015d760 2024-09-21T07:48:21.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.771+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30e8003e00 con 0x7f310015d760 2024-09-21T07:48:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.771+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30e8035040 con 0x7f310015d760 2024-09-21T07:48:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.771+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f310009a400 con 0x7f310015d760 2024-09-21T07:48:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.771+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3100156670 con 0x7f310015d760 2024-09-21T07:48:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.772+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31001568a0 con 0x7f310015d760 2024-09-21T07:48:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.773+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f30e8041030 con 0x7f310015d760 2024-09-21T07:48:21.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.774+0000 7f30fd600640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30dc078dd0 0x7f30dc07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:21.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.775+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f30e80bba60 con 0x7f310015d760 2024-09-21T07:48:21.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.775+0000 7f3105200640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30dc078dd0 0x7f30dc07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:21.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.776+0000 7f3105200640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30dc078dd0 0x7f30dc07b2e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005fd0 tx=0x7f30f0005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:21.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:21.779+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30e8083a60 con 0x7f310015d760 2024-09-21T07:48:22.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.003+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f31001535c0 con 0x7f310015d760 2024-09-21T07:48:22.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.003+0000 7f30fd600640 1 -- 172.21.15.88:0/3988869025 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f30e8083360 con 0x7f310015d760 2024-09-21T07:48:22.005 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:22.005 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:22.005 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30dc078dd0 msgr2=0x7f30dc07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30dc078dd0 0x7f30dc07b2e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f30f0005fd0 tx=0x7f30f0005850 comp rx=0 tx=0).stop 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 msgr2=0x7f3100149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 0x7f3100149890 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f30e8000c00 tx=0x7f30e8003910 comp rx=0 tx=0).stop 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 shutdown_connections 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30dc078dd0 0x7f30dc07b2e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:22.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3100160310 0x7f310009c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f310015d760 0x7f3100149890 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 --2- 172.21.15.88:0/3988869025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31000ad0e0 0x7f31001f9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 >> 172.21.15.88:0/3988869025 conn(0x7f310006d730 msgr2=0x7f31000b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 shutdown_connections 2024-09-21T07:48:22.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:22.006+0000 7f3107c00640 1 -- 172.21.15.88:0/3988869025 wait complete. 2024-09-21T07:48:22.029 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:22.269 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":11,"uptime":30.170414326,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.51933261837689,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:22.269 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 11, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.51933261837689 }, "state": "open", "uptime": 30.170414326 } ] 2024-09-21T07:48:22.269 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:26.231 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:26.236 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:27.270 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 -- 172.21.15.88:0/1689660634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 msgr2=0x7fa6fc0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 --2- 172.21.15.88:0/1689660634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 0x7fa6fc0b3960 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005fd0 tx=0x7fa6f402a810 comp rx=0 tx=0).stop 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 -- 172.21.15.88:0/1689660634 shutdown_connections 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 --2- 172.21.15.88:0/1689660634 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 0x7fa6fc0b3960 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 --2- 172.21.15.88:0/1689660634 >> v2:172.21.15.175:3300/0 conn(0x7fa6fc0ad0e0 0x7fa6fc0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 -- 172.21.15.88:0/1689660634 >> 172.21.15.88:0/1689660634 conn(0x7fa6fc06d730 msgr2=0x7fa6fc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:27.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 -- 172.21.15.88:0/1689660634 shutdown_connections 2024-09-21T07:48:27.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.499+0000 7fa702000640 1 -- 172.21.15.88:0/1689660634 wait complete. 2024-09-21T07:48:27.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.501+0000 7fa702000640 1 Processor -- start 2024-09-21T07:48:27.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.501+0000 7fa702000640 1 -- start start 2024-09-21T07:48:27.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa702000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6fc0ad0e0 0x7fa6fc154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa702000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 0x7fa6fc099980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa702000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 0x7fa6fc1f3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa702000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0b4da0 con 0x7fa6fc15d760 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa702000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6fc0b4c20 con 0x7fa6fc0ad0e0 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa702000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6fc0b3ea0 con 0x7fa6fc0b3550 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa6fbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 0x7fa6fc1f3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa6fbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 0x7fa6fc1f3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33940/0 (socket says 172.21.15.88:33940) 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa6fbe00640 1 -- 172.21.15.88:0/472880593 learned_addr learned my addr 172.21.15.88:0/472880593 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa6faa00640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 0x7fa6fc099980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:27.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.503+0000 7fa6fb400640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6fc0ad0e0 0x7fa6fc154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:27.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6fbe00640 1 -- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 msgr2=0x7fa6fc099980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:27.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6fbe00640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 0x7fa6fc099980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6fbe00640 1 -- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6fc0ad0e0 msgr2=0x7fa6fc154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:27.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6fbe00640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6fc0ad0e0 0x7fa6fc154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6fbe00640 1 -- 172.21.15.88:0/472880593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6fc1f48d0 con 0x7fa6fc15d760 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6fbe00640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 0x7fa6fc1f3900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005fd0 tx=0x7fa6f4031650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4041070 con 0x7fa6fc15d760 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f4031a50 con 0x7fa6fc15d760 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4037040 con 0x7fa6fc15d760 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6fc099ec0 con 0x7fa6fc15d760 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.504+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6fc153f50 con 0x7fa6fc15d760 2024-09-21T07:48:27.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.506+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6fc0b4360 con 0x7fa6fc15d760 2024-09-21T07:48:27.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.507+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fa6f4044070 con 0x7fa6fc15d760 2024-09-21T07:48:27.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.507+0000 7fa6f3400640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa6dc078e00 0x7fa6dc07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:27.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.508+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fa6f40bc100 con 0x7fa6fc15d760 2024-09-21T07:48:27.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.508+0000 7fa6fb400640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa6dc078e00 0x7fa6dc07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:27.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.509+0000 7fa6fb400640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa6dc078e00 0x7fa6dc07b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007d60 tx=0x7fa6ec007cf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:27.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.512+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6f4083ab0 con 0x7fa6fc15d760 2024-09-21T07:48:27.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.732+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa6fc1f4380 con 0x7fa6fc15d760 2024-09-21T07:48:27.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.732+0000 7fa6f3400640 1 -- 172.21.15.88:0/472880593 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fa6f40884e0 con 0x7fa6fc15d760 2024-09-21T07:48:27.734 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:27.735 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:27.735 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:27.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.734+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa6dc078e00 msgr2=0x7fa6dc07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:27.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.734+0000 7fa702000640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa6dc078e00 0x7fa6dc07b310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa6ec007d60 tx=0x7fa6ec007cf0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.734+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 msgr2=0x7fa6fc1f3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:27.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.734+0000 7fa702000640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 0x7fa6fc1f3900 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa6f4005fd0 tx=0x7fa6f4031650 comp rx=0 tx=0).stop 2024-09-21T07:48:27.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 shutdown_connections 2024-09-21T07:48:27.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa6dc078e00 0x7fa6dc07b310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6fc0b3550 0x7fa6fc099980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6fc15d760 0x7fa6fc1f3900 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 --2- 172.21.15.88:0/472880593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6fc0ad0e0 0x7fa6fc154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:27.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 >> 172.21.15.88:0/472880593 conn(0x7fa6fc06d730 msgr2=0x7fa6fc1535c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:27.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 shutdown_connections 2024-09-21T07:48:27.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:27.735+0000 7fa702000640 1 -- 172.21.15.88:0/472880593 wait complete. 2024-09-21T07:48:27.759 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:28.000 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":10,"uptime":35.902303058000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":20.249376870593185,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:28.000 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 10, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 20.249376870593185 }, "state": "open", "uptime": 35.902303058 } ] 2024-09-21T07:48:28.001 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:33.002 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fc2200640 1 -- 172.21.15.88:0/103722889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 msgr2=0x7f4fa0005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fc2200640 1 --2- 172.21.15.88:0/103722889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fa0005150 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4fb4005fd0 tx=0x7f4fa0005830 comp rx=0 tx=0).stop 2024-09-21T07:48:33.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fbbe00640 1 -- 172.21.15.88:0/103722889 reap_dead start 2024-09-21T07:48:33.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fc2200640 1 -- 172.21.15.88:0/103722889 shutdown_connections 2024-09-21T07:48:33.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fc2200640 1 -- 172.21.15.88:0/103722889 >> 172.21.15.88:0/103722889 conn(0x7f4fbc06d730 msgr2=0x7f4fbc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:33.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fc2200640 1 -- 172.21.15.88:0/103722889 shutdown_connections 2024-09-21T07:48:33.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.234+0000 7f4fc2200640 1 -- 172.21.15.88:0/103722889 wait complete. 2024-09-21T07:48:33.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.236+0000 7f4fc2200640 1 Processor -- start 2024-09-21T07:48:33.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.236+0000 7f4fc2200640 1 -- start start 2024-09-21T07:48:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fc2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fbc1468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fc2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fbc156810 0x7f4fbc0b13a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fc2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fbc14f890 0x7f4fbc0b18e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fc2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc0b4da0 con 0x7f4fa0004d40 2024-09-21T07:48:33.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fc2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4fbc0b4c20 con 0x7f4fbc14f890 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fc2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4fbc0b3960 con 0x7f4fbc156810 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fbc1468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fbc1468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:33954/0 (socket says 172.21.15.88:33954) 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 -- 172.21.15.88:0/871430508 learned_addr learned my addr 172.21.15.88:0/871430508 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbaa00640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fbc156810 0x7f4fbc0b13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbbe00640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fbc14f890 0x7f4fbc0b18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 -- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fbc156810 msgr2=0x7f4fbc0b13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fbc156810 0x7f4fbc0b13a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 -- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fbc14f890 msgr2=0x7f4fbc0b18e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fbc14f890 0x7f4fbc0b18e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbb400640 1 -- 172.21.15.88:0/871430508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fbc158580 con 0x7f4fa0004d40 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.238+0000 7f4fbbe00640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fbc14f890 0x7f4fbc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.239+0000 7f4fbb400640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fbc1468f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4fac006740 tx=0x7f4fac003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:33.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.239+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac015070 con 0x7f4fa0004d40 2024-09-21T07:48:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.239+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4fac00f920 con 0x7f4fa0004d40 2024-09-21T07:48:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.239+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fbc158860 con 0x7f4fa0004d40 2024-09-21T07:48:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.239+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fac00fd70 con 0x7f4fa0004d40 2024-09-21T07:48:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.239+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fbc158e20 con 0x7f4fa0004d40 2024-09-21T07:48:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.240+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fbc0ad4f0 con 0x7f4fa0004d40 2024-09-21T07:48:33.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.241+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f4fac011430 con 0x7f4fa0004d40 2024-09-21T07:48:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.242+0000 7f4fb3400640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f9c078dd0 0x7f4f9c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:33.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.242+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f4fac09c190 con 0x7f4fa0004d40 2024-09-21T07:48:33.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.243+0000 7f4fbaa00640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f9c078dd0 0x7f4f9c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:33.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.244+0000 7f4fbaa00640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f9c078dd0 0x7f4f9c07b2e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc15f4b0 tx=0x7f4fa8005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:33.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.247+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4fbc0ada30 con 0x7f4fa0004d40 2024-09-21T07:48:33.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.467+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4fbc15c480 con 0x7f4fa0004d40 2024-09-21T07:48:33.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.468+0000 7f4fb3400640 1 -- 172.21.15.88:0/871430508 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f4fac064030 con 0x7f4fa0004d40 2024-09-21T07:48:33.471 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:33.471 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:33.471 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f9c078dd0 msgr2=0x7f4f9c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f9c078dd0 0x7f4f9c07b2e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc15f4b0 tx=0x7f4fa8005f90 comp rx=0 tx=0).stop 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 msgr2=0x7f4fbc1468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fbc1468f0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f4fac006740 tx=0x7f4fac003a80 comp rx=0 tx=0).stop 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 shutdown_connections 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4f9c078dd0 0x7f4f9c07b2e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4fbc156810 0x7f4fbc0b13a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4fbc14f890 0x7f4fbc0b18e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 --2- 172.21.15.88:0/871430508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4fa0004d40 0x7f4fbc1468f0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:33.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 >> 172.21.15.88:0/871430508 conn(0x7f4fbc06d730 msgr2=0x7f4fbc06fb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 shutdown_connections 2024-09-21T07:48:33.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:33.470+0000 7f4fc2200640 1 -- 172.21.15.88:0/871430508 wait complete. 2024-09-21T07:48:33.496 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:33.736 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":41.638191712000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.982788028033777,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:33.737 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.982788028033777 }, "state": "open", "uptime": 41.638191712 } ] 2024-09-21T07:48:33.737 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:38.738 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 -- 172.21.15.88:0/1610854553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9500b41f0 msgr2=0x7ff95006f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 --2- 172.21.15.88:0/1610854553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9500b41f0 0x7ff95006f7b0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7ff948009ca0 tx=0x7ff94802aac0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 -- 172.21.15.88:0/1610854553 shutdown_connections 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 --2- 172.21.15.88:0/1610854553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9500b41f0 0x7ff95006f7b0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 -- 172.21.15.88:0/1610854553 >> 172.21.15.88:0/1610854553 conn(0x7ff95006d730 msgr2=0x7ff9500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 -- 172.21.15.88:0/1610854553 shutdown_connections 2024-09-21T07:48:38.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.966+0000 7ff957c00640 1 -- 172.21.15.88:0/1610854553 wait complete. 2024-09-21T07:48:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.968+0000 7ff957c00640 1 Processor -- start 2024-09-21T07:48:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.969+0000 7ff957c00640 1 -- start start 2024-09-21T07:48:38.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff957c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9500af630 0x7ff950148ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff957c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9500b41f0 0x7ff95014de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff957c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 0x7ff950153be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff957c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff950169d50 con 0x7ff9501f98e0 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff957c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff950169bd0 con 0x7ff9500b41f0 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff957c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff950169ed0 con 0x7ff9500af630 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff955c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 0x7ff950153be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff955c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 0x7ff950153be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54614/0 (socket says 172.21.15.88:54614) 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff955c00640 1 -- 172.21.15.88:0/562823815 learned_addr learned my addr 172.21.15.88:0/562823815 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff947e00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9500b41f0 0x7ff95014de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.970+0000 7ff955200640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9500af630 0x7ff950148ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955c00640 1 -- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9500af630 msgr2=0x7ff950148ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9500af630 0x7ff950148ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955c00640 1 -- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9500b41f0 msgr2=0x7ff95014de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9500b41f0 0x7ff95014de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955c00640 1 -- 172.21.15.88:0/562823815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff950149310 con 0x7ff9501f98e0 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955200640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9500af630 0x7ff950148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff955c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 0x7ff950153be0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff948009bf0 tx=0x7ff948003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff948044070 con 0x7ff9501f98e0 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff948004250 con 0x7ff9501f98e0 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.971+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9501f8e00 con 0x7ff9501f98e0 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.972+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9480046a0 con 0x7ff9501f98e0 2024-09-21T07:48:38.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.972+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9501f93c0 con 0x7ff9501f98e0 2024-09-21T07:48:38.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.973+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff950149080 con 0x7ff9501f98e0 2024-09-21T07:48:38.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.974+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7ff948047070 con 0x7ff9501f98e0 2024-09-21T07:48:38.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.975+0000 7ff94d600640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff92c078e00 0x7ff92c07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:38.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.976+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7ff9480bc120 con 0x7ff9501f98e0 2024-09-21T07:48:38.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.976+0000 7ff955200640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff92c078e00 0x7ff92c07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:38.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.977+0000 7ff955200640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff92c078e00 0x7ff92c07b310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff940006fd0 tx=0x7ff940008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:38.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:38.980+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff948032030 con 0x7ff9501f98e0 2024-09-21T07:48:39.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.202+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff950073500 con 0x7ff9501f98e0 2024-09-21T07:48:39.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.203+0000 7ff94d600640 1 -- 172.21.15.88:0/562823815 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7ff948083aa0 con 0x7ff9501f98e0 2024-09-21T07:48:39.207 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:39.207 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.205+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff92c078e00 msgr2=0x7ff92c07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.205+0000 7ff957c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff92c078e00 0x7ff92c07b310 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7ff940006fd0 tx=0x7ff940008040 comp rx=0 tx=0).stop 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.205+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 msgr2=0x7ff950153be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.205+0000 7ff957c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 0x7ff950153be0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7ff948009bf0 tx=0x7ff948003f10 comp rx=0 tx=0).stop 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 shutdown_connections 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff92c078e00 0x7ff92c07b310 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9501f98e0 0x7ff950153be0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9500b41f0 0x7ff95014de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 --2- 172.21.15.88:0/562823815 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9500af630 0x7ff950148ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:39.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 >> 172.21.15.88:0/562823815 conn(0x7ff95006d730 msgr2=0x7ff9500b0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:39.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 shutdown_connections 2024-09-21T07:48:39.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:39.206+0000 7ff957c00640 1 -- 172.21.15.88:0/562823815 wait complete. 2024-09-21T07:48:39.233 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:39.475 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":9,"uptime":47.375080310000001,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.719663344295448,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:39.475 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 9, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.719663344295448 }, "state": "open", "uptime": 47.37508031 } ] 2024-09-21T07:48:39.476 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:44.477 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:44.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 -- 172.21.15.88:0/2840129614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c0ad0e0 msgr2=0x7f1d6c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 --2- 172.21.15.88:0/2840129614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c0ad4f0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1d54009830 tx=0x7f1d5402a530 comp rx=0 tx=0).stop 2024-09-21T07:48:44.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 -- 172.21.15.88:0/2840129614 shutdown_connections 2024-09-21T07:48:44.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 --2- 172.21.15.88:0/2840129614 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c0ad4f0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 -- 172.21.15.88:0/2840129614 >> 172.21.15.88:0/2840129614 conn(0x7f1d6c06d730 msgr2=0x7f1d6c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:44.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 -- 172.21.15.88:0/2840129614 shutdown_connections 2024-09-21T07:48:44.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.706+0000 7f1d70c00640 1 -- 172.21.15.88:0/2840129614 wait complete. 2024-09-21T07:48:44.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.708+0000 7f1d70c00640 1 Processor -- start 2024-09-21T07:48:44.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.708+0000 7f1d70c00640 1 -- start start 2024-09-21T07:48:44.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.709+0000 7f1d70c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c15bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.709+0000 7f1d70c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c0b3550 0x7f1d6c159920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.709+0000 7f1d70c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c15d760 0x7f1d6c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d70c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c0b4da0 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d70c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c0b4c20 con 0x7f1d6c15d760 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d70c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d6c0b4070 con 0x7f1d6c0b3550 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c15bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c15bc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54630/0 (socket says 172.21.15.88:54630) 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 -- 172.21.15.88:0/3855397559 learned_addr learned my addr 172.21.15.88:0/3855397559 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d63e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c0b3550 0x7f1d6c159920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d6a800640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c15d760 0x7f1d6c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 -- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c0b3550 msgr2=0x7f1d6c159920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c0b3550 0x7f1d6c159920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 -- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c15d760 msgr2=0x7f1d6c159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c15d760 0x7f1d6c159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d69e00640 1 -- 172.21.15.88:0/3855397559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c157e50 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d63e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c0b3550 0x7f1d6c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:44.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.710+0000 7f1d6a800640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c15d760 0x7f1d6c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:44.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.711+0000 7f1d69e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c15bc80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c430 tx=0x7f1d5c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.711+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c017070 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.711+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d5c0053c0 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.711+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c003850 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.711+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c157510 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.711+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c158130 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.712+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6c0ad4f0 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.713+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f1d5c01a070 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.715+0000 7f1d62000640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1d48078e00 0x7f1d4807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:44.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.715+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f1d5c09a610 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.715+0000 7f1d63e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1d48078e00 0x7f1d4807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:44.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.716+0000 7f1d63e00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1d48078e00 0x7f1d4807b310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d54034000 tx=0x7f1d54004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:44.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.720+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d5c062620 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.942+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1d6c0b3cc0 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.943+0000 7f1d62000640 1 -- 172.21.15.88:0/3855397559 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f1d5c061f20 con 0x7f1d6c0ad0e0 2024-09-21T07:48:44.945 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:44.945 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:44.946 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:44.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1d48078e00 msgr2=0x7f1d4807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1d48078e00 0x7f1d4807b310 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f1d54034000 tx=0x7f1d54004730 comp rx=0 tx=0).stop 2024-09-21T07:48:44.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 msgr2=0x7f1d6c15bc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:44.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c15bc80 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00c430 tx=0x7f1d5c00c8f0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 shutdown_connections 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1d48078e00 0x7f1d4807b310 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d6c0b3550 0x7f1d6c159920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d6c15d760 0x7f1d6c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 --2- 172.21.15.88:0/3855397559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d6c0ad0e0 0x7f1d6c15bc80 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.945+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 >> 172.21.15.88:0/3855397559 conn(0x7f1d6c06d730 msgr2=0x7f1d6c15f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.946+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 shutdown_connections 2024-09-21T07:48:44.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:44.946+0000 7f1d70c00640 1 -- 172.21.15.88:0/3855397559 wait complete. 2024-09-21T07:48:44.974 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:45.215 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":53.114968851,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.45986848880835,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:45.216 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.45986848880835 }, "state": "open", "uptime": 53.114968851 } ] 2024-09-21T07:48:45.216 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:50.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:50.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 -- 172.21.15.88:0/1035214398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0310152920 msgr2=0x7f031015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 --2- 172.21.15.88:0/1035214398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0310152920 0x7f031015f3d0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02f8005fd0 tx=0x7f02f802a540 comp rx=0 tx=0).stop 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 -- 172.21.15.88:0/1035214398 shutdown_connections 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 --2- 172.21.15.88:0/1035214398 >> v2:172.21.15.175:3300/0 conn(0x7f031015f910 0x7f03100b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 --2- 172.21.15.88:0/1035214398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0310152920 0x7f031015f3d0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 -- 172.21.15.88:0/1035214398 >> 172.21.15.88:0/1035214398 conn(0x7f031006d730 msgr2=0x7f03100b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 -- 172.21.15.88:0/1035214398 shutdown_connections 2024-09-21T07:48:50.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.443+0000 7f0315400640 1 -- 172.21.15.88:0/1035214398 wait complete. 2024-09-21T07:48:50.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.445+0000 7f0315400640 1 Processor -- start 2024-09-21T07:48:50.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.446+0000 7f0315400640 1 -- start start 2024-09-21T07:48:50.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f0315400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 0x7f03100cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f0315400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0310152920 0x7f0310150b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f0315400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031015f910 0x7f03100ccc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f0315400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03100b4130 con 0x7f03100aebc0 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f0315400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03100b3fb0 con 0x7f0310152920 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f0315400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03100b42b0 con 0x7f031015f910 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f030e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 0x7f03100cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f030e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 0x7f03100cc440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:59378/0 (socket says 172.21.15.88:59378) 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.447+0000 7f030e800640 1 -- 172.21.15.88:0/1973232902 learned_addr learned my addr 172.21.15.88:0/1973232902 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030f200640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031015f910 0x7f03100ccc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f0305e00640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0310152920 0x7f0310150b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030e800640 1 -- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031015f910 msgr2=0x7f03100ccc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030e800640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031015f910 0x7f03100ccc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030e800640 1 -- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0310152920 msgr2=0x7f0310150b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030e800640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0310152920 0x7f0310150b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030e800640 1 -- 172.21.15.88:0/1973232902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0310154d70 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f030e800640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 0x7f03100cc440 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f030000c730 tx=0x7f030000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0300003b70 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0310155050 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0300003d30 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f030000aa50 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.448+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0310155610 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.449+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03100b4840 con 0x7f03100aebc0 2024-09-21T07:48:50.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.451+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f030000abf0 con 0x7f03100aebc0 2024-09-21T07:48:50.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.452+0000 7f0307400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f02ec078e00 0x7f02ec07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.452+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f030009b170 con 0x7f03100aebc0 2024-09-21T07:48:50.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.452+0000 7f0305e00640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f02ec078e00 0x7f02ec07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:50.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.453+0000 7f0305e00640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f02ec078e00 0x7f02ec07b310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02f8033000 tx=0x7f02f8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:50.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.457+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0300063020 con 0x7f03100aebc0 2024-09-21T07:48:50.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.681+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03100b4af0 con 0x7f03100aebc0 2024-09-21T07:48:50.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.681+0000 7f0307400640 1 -- 172.21.15.88:0/1973232902 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f0300062920 con 0x7f03100aebc0 2024-09-21T07:48:50.684 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:50.684 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:50.684 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:50.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f02ec078e00 msgr2=0x7f02ec07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f02ec078e00 0x7f02ec07b310 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f02f8033000 tx=0x7f02f8004730 comp rx=0 tx=0).stop 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 msgr2=0x7f03100cc440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 0x7f03100cc440 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f030000c730 tx=0x7f030000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 shutdown_connections 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f02ec078e00 0x7f02ec07b310 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f031015f910 0x7f03100ccc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0310152920 0x7f0310150b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 --2- 172.21.15.88:0/1973232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03100aebc0 0x7f03100cc440 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 >> 172.21.15.88:0/1973232902 conn(0x7f031006d730 msgr2=0x7f031006f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 shutdown_connections 2024-09-21T07:48:50.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:50.684+0000 7f0315400640 1 -- 172.21.15.88:0/1973232902 wait complete. 2024-09-21T07:48:50.711 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:50.952 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":8,"uptime":58.852857428,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":19.203585013885267,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:50.953 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 8, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 19.203585013885267 }, "state": "open", "uptime": 58.852857428 } ] 2024-09-21T07:48:50.953 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:48:55.954 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 -- 172.21.15.88:0/2825203601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480ad3c0 msgr2=0x7fc4480ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 --2- 172.21.15.88:0/2825203601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480ad3c0 0x7fc4480ad7d0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fc434005fd0 tx=0x7fc43402a540 comp rx=0 tx=0).stop 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 -- 172.21.15.88:0/2825203601 shutdown_connections 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 --2- 172.21.15.88:0/2825203601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480ad3c0 0x7fc4480ad7d0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 --2- 172.21.15.88:0/2825203601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480704b0 0x7fc448073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 -- 172.21.15.88:0/2825203601 >> 172.21.15.88:0/2825203601 conn(0x7fc44806d730 msgr2=0x7fc4480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 -- 172.21.15.88:0/2825203601 shutdown_connections 2024-09-21T07:48:56.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.183+0000 7fc44d600640 1 -- 172.21.15.88:0/2825203601 wait complete. 2024-09-21T07:48:56.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.185+0000 7fc44d600640 1 Processor -- start 2024-09-21T07:48:56.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.186+0000 7fc44d600640 1 -- start start 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc44d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 0x7fc4480b1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc44d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ad3c0 0x7fc4481f5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc44d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4480add10 0x7fc448154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc44d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4480712b0 con 0x7fc4480704b0 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc44d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc448071130 con 0x7fc4480add10 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc44d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4480b4520 con 0x7fc4480ad3c0 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc446a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 0x7fc4480b1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:56.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc446a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 0x7fc4480b1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55016/0 (socket says 172.21.15.88:55016) 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.187+0000 7fc446a00640 1 -- 172.21.15.88:0/3578869901 learned_addr learned my addr 172.21.15.88:0/3578869901 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446000640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ad3c0 0x7fc4481f5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446a00640 1 -- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ad3c0 msgr2=0x7fc4481f5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446a00640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ad3c0 0x7fc4481f5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc447400640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4480add10 0x7fc448154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446a00640 1 -- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4480add10 msgr2=0x7fc448154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446a00640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4480add10 0x7fc448154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446a00640 1 -- 172.21.15.88:0/3578869901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc44815c980 con 0x7fc4480704b0 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc447400640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4480add10 0x7fc448154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:48:56.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc446a00640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 0x7fc4480b1ee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b470 tx=0x7fc43800b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438015020 con 0x7fc4480704b0 2024-09-21T07:48:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc438011940 con 0x7fc4480704b0 2024-09-21T07:48:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438011d90 con 0x7fc4480704b0 2024-09-21T07:48:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc44815cc60 con 0x7fc4480704b0 2024-09-21T07:48:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.188+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc44815d220 con 0x7fc4480704b0 2024-09-21T07:48:56.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.189+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4480b49f0 con 0x7fc4480704b0 2024-09-21T07:48:56.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.191+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fc438018020 con 0x7fc4480704b0 2024-09-21T07:48:56.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.192+0000 7fc43ea00640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc428078e00 0x7fc42807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:48:56.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.192+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fc43809b2b0 con 0x7fc4480704b0 2024-09-21T07:48:56.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.193+0000 7fc446000640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc428078e00 0x7fc42807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:48:56.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.194+0000 7fc446000640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc428078e00 0x7fc42807b310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc434033000 tx=0x7fc434004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:48:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.197+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc43801f020 con 0x7fc4480704b0 2024-09-21T07:48:56.344 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:56.348 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:48:56.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.419+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc44815e420 con 0x7fc4480704b0 2024-09-21T07:48:56.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.420+0000 7fc43ea00640 1 -- 172.21.15.88:0/3578869901 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fc438063150 con 0x7fc4480704b0 2024-09-21T07:48:56.422 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:48:56.422 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:48:56.423 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.422+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc428078e00 msgr2=0x7fc42807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.422+0000 7fc44d600640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc428078e00 0x7fc42807b310 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc434033000 tx=0x7fc434004730 comp rx=0 tx=0).stop 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.422+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 msgr2=0x7fc4480b1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.422+0000 7fc44d600640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 0x7fc4480b1ee0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fc43800b470 tx=0x7fc43800b930 comp rx=0 tx=0).stop 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 shutdown_connections 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc428078e00 0x7fc42807b310 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4480ad3c0 0x7fc4481f5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4480add10 0x7fc448154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 --2- 172.21.15.88:0/3578869901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4480704b0 0x7fc4480b1ee0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:48:56.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 >> 172.21.15.88:0/3578869901 conn(0x7fc44806d730 msgr2=0x7fc448151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:48:56.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 shutdown_connections 2024-09-21T07:48:56.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:48:56.423+0000 7fc44d600640 1 -- 172.21.15.88:0/3578869901 wait complete. 2024-09-21T07:48:56.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:48:56.729 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":64.630745224999998,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.948925453566588,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:48:56.730 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.948925453566588 }, "state": "open", "uptime": 64.630745225 } ] 2024-09-21T07:48:56.730 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:49:01.731 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.962+0000 7f6687200640 1 -- 172.21.15.88:0/692731033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 msgr2=0x7f66801535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.962+0000 7f6687200640 1 --2- 172.21.15.88:0/692731033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 0x7f66801535c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6670009860 tx=0x7f667002a510 comp rx=0 tx=0).stop 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.962+0000 7f6687200640 1 -- 172.21.15.88:0/692731033 shutdown_connections 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.962+0000 7f6687200640 1 --2- 172.21.15.88:0/692731033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 0x7f66801535c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.962+0000 7f6687200640 1 -- 172.21.15.88:0/692731033 >> 172.21.15.88:0/692731033 conn(0x7f668006d730 msgr2=0x7f66800b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.962+0000 7f6687200640 1 -- 172.21.15.88:0/692731033 shutdown_connections 2024-09-21T07:49:01.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.963+0000 7f6687200640 1 -- 172.21.15.88:0/692731033 wait complete. 2024-09-21T07:49:01.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.965+0000 7f6687200640 1 Processor -- start 2024-09-21T07:49:01.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.965+0000 7f6687200640 1 -- start start 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6687200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66800ad0e0 0x7f668009de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6687200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 0x7f668014a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6687200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 0x7f66801565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6687200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66800b4da0 con 0x7f66800b3550 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6687200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66800b4c20 con 0x7f66800ad0e0 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6687200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66800b4070 con 0x7f668015d760 2024-09-21T07:49:01.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6677400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 0x7f668014a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.966+0000 7f6677400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 0x7f668014a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55028/0 (socket says 172.21.15.88:55028) 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 -- 172.21.15.88:0/3926480523 learned_addr learned my addr 172.21.15.88:0/3926480523 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f667fe00640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66800ad0e0 0x7f668009de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6685200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 0x7f66801565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 -- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 msgr2=0x7f66801565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 0x7f66801565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 -- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66800ad0e0 msgr2=0x7f668009de00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66800ad0e0 0x7f668009de00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 -- 172.21.15.88:0/3926480523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66801f9120 con 0x7f66800b3550 2024-09-21T07:49:01.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f6677400640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 0x7f668014a710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f666800c400 tx=0x7f666800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.967+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668017070 con 0x7f66800b3550 2024-09-21T07:49:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.968+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f66680053c0 con 0x7f66800b3550 2024-09-21T07:49:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.968+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6668003850 con 0x7f66800b3550 2024-09-21T07:49:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.968+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66801f9400 con 0x7f66800b3550 2024-09-21T07:49:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.968+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66801f99c0 con 0x7f66800b3550 2024-09-21T07:49:01.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.969+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66801f9da0 con 0x7f66800b3550 2024-09-21T07:49:01.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.970+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f666801a070 con 0x7f66800b3550 2024-09-21T07:49:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.971+0000 7f667cc00640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f665c078dd0 0x7f665c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:01.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.972+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f666809a700 con 0x7f66800b3550 2024-09-21T07:49:01.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.972+0000 7f667fe00640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f665c078dd0 0x7f665c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:01.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.973+0000 7f667fe00640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f665c078dd0 0x7f665c07b2e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6670034000 tx=0x7f6670004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:01.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:01.976+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66801f9da0 con 0x7f66800b3550 2024-09-21T07:49:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.200+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f66801535c0 con 0x7f66800b3550 2024-09-21T07:49:02.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.200+0000 7f667cc00640 1 -- 172.21.15.88:0/3926480523 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f6668062670 con 0x7f66800b3550 2024-09-21T07:49:02.201 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:02.202 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:02.202 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:02.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f665c078dd0 msgr2=0x7f665c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f665c078dd0 0x7f665c07b2e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f6670034000 tx=0x7f6670004730 comp rx=0 tx=0).stop 2024-09-21T07:49:02.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 msgr2=0x7f668014a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 0x7f668014a710 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f666800c400 tx=0x7f666800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 shutdown_connections 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f665c078dd0 0x7f665c07b2e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f668015d760 0x7f66801565d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66800b3550 0x7f668014a710 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 --2- 172.21.15.88:0/3926480523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66800ad0e0 0x7f668009de00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 >> 172.21.15.88:0/3926480523 conn(0x7f668006d730 msgr2=0x7f66801602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 shutdown_connections 2024-09-21T07:49:02.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:02.203+0000 7f6687200640 1 -- 172.21.15.88:0/3926480523 wait complete. 2024-09-21T07:49:02.229 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:49:02.468 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":7,"uptime":70.369633777999994,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.699327816196561,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:49:02.468 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 7, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.69932781619656 }, "state": "open", "uptime": 70.369633778 } ] 2024-09-21T07:49:02.468 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:49:07.469 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:07.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f539bc00640 1 -- 172.21.15.88:0/1429097660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5378004d90 msgr2=0x7f5378005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f539bc00640 1 --2- 172.21.15.88:0/1429097660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5378004d90 0x7f5378005220 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c610 tx=0x7f5378005830 comp rx=0 tx=0).stop 2024-09-21T07:49:07.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f5399c00640 1 -- 172.21.15.88:0/1429097660 reap_dead start 2024-09-21T07:49:07.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f539bc00640 1 -- 172.21.15.88:0/1429097660 shutdown_connections 2024-09-21T07:49:07.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f539bc00640 1 -- 172.21.15.88:0/1429097660 >> 172.21.15.88:0/1429097660 conn(0x7f539406d730 msgr2=0x7f53940b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f539bc00640 1 -- 172.21.15.88:0/1429097660 shutdown_connections 2024-09-21T07:49:07.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.698+0000 7f539bc00640 1 -- 172.21.15.88:0/1429097660 wait complete. 2024-09-21T07:49:07.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.700+0000 7f539bc00640 1 Processor -- start 2024-09-21T07:49:07.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.700+0000 7f539bc00640 1 -- start start 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f539bc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5378004d90 0x7f539409da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f539bc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539409df40 0x7f539409c520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f539bc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 0x7f53941f3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f539bc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53940b4070 con 0x7f539409ca60 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f539bc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53940b4e90 con 0x7f539409df40 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f539bc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53940b41f0 con 0x7f5378004d90 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f5399c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 0x7f53941f3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f5399c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 0x7f53941f3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39320/0 (socket says 172.21.15.88:39320) 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f5399c00640 1 -- 172.21.15.88:0/593596333 learned_addr learned my addr 172.21.15.88:0/593596333 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f5399200640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5378004d90 0x7f539409da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.702+0000 7f538be00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539409df40 0x7f539409c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5399c00640 1 -- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5378004d90 msgr2=0x7f539409da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5399c00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5378004d90 0x7f539409da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5399c00640 1 -- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539409df40 msgr2=0x7f539409c520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5399c00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539409df40 0x7f539409c520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5399c00640 1 -- 172.21.15.88:0/593596333 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539409a960 con 0x7f539409ca60 2024-09-21T07:49:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5399c00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 0x7f53941f3bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c610 tx=0x7f538c004bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c02b900 con 0x7f539409ca60 2024-09-21T07:49:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f538c02bb90 con 0x7f539409ca60 2024-09-21T07:49:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538c0060a0 con 0x7f539409ca60 2024-09-21T07:49:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539409ac40 con 0x7f539409ca60 2024-09-21T07:49:07.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.703+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539409b200 con 0x7f539409ca60 2024-09-21T07:49:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.704+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53940b3960 con 0x7f539409ca60 2024-09-21T07:49:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.706+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f538c006240 con 0x7f539409ca60 2024-09-21T07:49:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.707+0000 7f5391600640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5370078dd0 0x7f537007b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.707+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f538c0bb480 con 0x7f539409ca60 2024-09-21T07:49:07.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.707+0000 7f5399200640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5370078dd0 0x7f537007b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:07.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.708+0000 7f5399200640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5370078dd0 0x7f537007b2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5384009c50 tx=0x7f5384005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:07.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.712+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f538c082cd0 con 0x7f539409ca60 2024-09-21T07:49:07.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.934+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f539409a6f0 con 0x7f539409ca60 2024-09-21T07:49:07.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.934+0000 7f5391600640 1 -- 172.21.15.88:0/593596333 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f538c087700 con 0x7f539409ca60 2024-09-21T07:49:07.935 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:07.936 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:07.936 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.936+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5370078dd0 msgr2=0x7f537007b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.936+0000 7f539bc00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5370078dd0 0x7f537007b2e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f5384009c50 tx=0x7f5384005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 msgr2=0x7f53941f3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 0x7f53941f3bc0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f538c00c610 tx=0x7f538c004bb0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 shutdown_connections 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5370078dd0 0x7f537007b2e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539409ca60 0x7f53941f3bc0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f539409df40 0x7f539409c520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 --2- 172.21.15.88:0/593596333 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5378004d90 0x7f539409da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 >> 172.21.15.88:0/593596333 conn(0x7f539406d730 msgr2=0x7f53940b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 shutdown_connections 2024-09-21T07:49:07.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:07.937+0000 7f539bc00640 1 -- 172.21.15.88:0/593596333 wait complete. 2024-09-21T07:49:07.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:49:08.205 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":5108,"entity":{"name":{"type":"client","num":5108},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":24040253}},"state":"open","num_leases":0,"num_caps":7,"request_load_avg":6,"uptime":76.106522374999997,"requests_in_flight":0,"num_completed_requests":2,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":18.453103179774384,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.5108 172.21.15.175:0/24040253","completed_requests":[{"tid":15,"created_ino":"0x0"},{"tid":16,"created_ino":"0x100000001f9"}],"prealloc_inos":[{"start":"0x10000000005","length":500},{"start":"0x100000001fa","length":1000}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:49:08.205 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [ { "created_ino": "0x0", "tid": 15 }, { "created_ino": "0x100000001f9", "tid": 16 } ], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 24040253, "type": "any" }, "name": { "num": 5108, "type": "client" } }, "id": 5108, "inst": "client.5108 172.21.15.175:0/24040253", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 2, "num_leases": 0, "prealloc_inos": [ { "length": 500, "start": "0x10000000005" }, { "length": 1000, "start": "0x100000001fa" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 6, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 18.453103179774384 }, "state": "open", "uptime": 76.106522375 } ] 2024-09-21T07:49:08.206 DEBUG:tasks.ceph_test_case:wait_until_true: waiting (timeout=90 retry_count=0)... 2024-09-21T07:49:13.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:13.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 -- 172.21.15.88:0/482071461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 msgr2=0x7faf900ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:13.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 --2- 172.21.15.88:0/482071461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf900ad4f0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7faf78005fd0 tx=0x7faf7802a540 comp rx=0 tx=0).stop 2024-09-21T07:49:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 -- 172.21.15.88:0/482071461 shutdown_connections 2024-09-21T07:49:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 --2- 172.21.15.88:0/482071461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf900b3550 0x7faf900b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 --2- 172.21.15.88:0/482071461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf900ad4f0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 -- 172.21.15.88:0/482071461 >> 172.21.15.88:0/482071461 conn(0x7faf9006d730 msgr2=0x7faf900b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 -- 172.21.15.88:0/482071461 shutdown_connections 2024-09-21T07:49:13.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.436+0000 7faf96200640 1 -- 172.21.15.88:0/482071461 wait complete. 2024-09-21T07:49:13.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.438+0000 7faf96200640 1 Processor -- start 2024-09-21T07:49:13.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.438+0000 7faf96200640 1 -- start start 2024-09-21T07:49:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf96200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf9014a650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf96200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf900b3550 0x7faf901602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf96200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 0x7faf901607e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf96200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf900b4da0 con 0x7faf9015d760 2024-09-21T07:49:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf96200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf900b4c20 con 0x7faf900ad0e0 2024-09-21T07:49:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf96200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf900b3ea0 con 0x7faf900b3550 2024-09-21T07:49:13.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 0x7faf901607e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:13.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 0x7faf901607e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39326/0 (socket says 172.21.15.88:39326) 2024-09-21T07:49:13.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf8fe00640 1 -- 172.21.15.88:0/455431876 learned_addr learned my addr 172.21.15.88:0/455431876 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:13.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf86a00640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf900b3550 0x7faf901602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:13.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.440+0000 7faf8f400640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf9014a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8fe00640 1 -- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf900b3550 msgr2=0x7faf901602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8fe00640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf900b3550 0x7faf901602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8fe00640 1 -- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 msgr2=0x7faf9014a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8fe00640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf9014a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8fe00640 1 -- 172.21.15.88:0/455431876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf901551b0 con 0x7faf9015d760 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8f400640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf9014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf8fe00640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 0x7faf901607e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf8800eab0 tx=0x7faf8800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:13.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf8800cde0 con 0x7faf9015d760 2024-09-21T07:49:13.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf90155490 con 0x7faf9015d760 2024-09-21T07:49:13.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf88004b20 con 0x7faf9015d760 2024-09-21T07:49:13.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf88005020 con 0x7faf9015d760 2024-09-21T07:49:13.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.441+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf90155a50 con 0x7faf9015d760 2024-09-21T07:49:13.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.442+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf900b4370 con 0x7faf9015d760 2024-09-21T07:49:13.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.444+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7faf88005290 con 0x7faf9015d760 2024-09-21T07:49:13.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.444+0000 7faf87e00640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faf6c078dd0 0x7faf6c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:13.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.445+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7faf8809bf40 con 0x7faf9015d760 2024-09-21T07:49:13.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.445+0000 7faf8f400640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faf6c078dd0 0x7faf6c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:13.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.446+0000 7faf8f400640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faf6c078dd0 0x7faf6c07b2e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faf80007fd0 tx=0x7faf80009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:13.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.449+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf88063df0 con 0x7faf9015d760 2024-09-21T07:49:13.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.667+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faf901582d0 con 0x7faf9015d760 2024-09-21T07:49:13.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.668+0000 7faf87e00640 1 -- 172.21.15.88:0/455431876 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7faf880636f0 con 0x7faf9015d760 2024-09-21T07:49:13.669 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:13.669 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:13.670 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.670+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faf6c078dd0 msgr2=0x7faf6c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faf6c078dd0 0x7faf6c07b2e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7faf80007fd0 tx=0x7faf80009040 comp rx=0 tx=0).stop 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 msgr2=0x7faf901607e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 0x7faf901607e0 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7faf8800eab0 tx=0x7faf8800ef70 comp rx=0 tx=0).stop 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 shutdown_connections 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7faf6c078dd0 0x7faf6c07b2e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf900b3550 0x7faf901602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf9015d760 0x7faf901607e0 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 --2- 172.21.15.88:0/455431876 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf900ad0e0 0x7faf9014a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:13.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 >> 172.21.15.88:0/455431876 conn(0x7faf9006d730 msgr2=0x7faf900cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 shutdown_connections 2024-09-21T07:49:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:13.671+0000 7faf96200640 1 -- 172.21.15.88:0/455431876 wait complete. 2024-09-21T07:49:13.694 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:49:13.972 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-09-21T07:49:13.973 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T07:49:13.973 DEBUG:tasks.ceph_test_case:wait_until_true: success in 70s and 0 retries 2024-09-21T07:49:13.973 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:49:13.973 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:14.063 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:14.155 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:49:14.155 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:14.156 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:49:14.156 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:14.191 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:14.191 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:14.192 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:49:14.192 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:14.228 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:14.228 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:14.228 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:49:14.229 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:14.268 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:14.269 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:49:14.269 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:49:14.269 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:14.304 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:14.304 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:14.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.527+0000 7f15af600640 1 -- 172.21.15.88:0/2472947206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 msgr2=0x7f158c003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.527+0000 7f15af600640 1 --2- 172.21.15.88:0/2472947206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f158c003e70 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f159800c4a0 tx=0x7f158c005830 comp rx=0 tx=0).stop 2024-09-21T07:49:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.528+0000 7f15ad600640 1 -- 172.21.15.88:0/2472947206 reap_dead start 2024-09-21T07:49:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.528+0000 7f15af600640 1 -- 172.21.15.88:0/2472947206 shutdown_connections 2024-09-21T07:49:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.528+0000 7f15af600640 1 -- 172.21.15.88:0/2472947206 >> 172.21.15.88:0/2472947206 conn(0x7f15a806d730 msgr2=0x7f15a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:14.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.528+0000 7f15af600640 1 -- 172.21.15.88:0/2472947206 shutdown_connections 2024-09-21T07:49:14.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.528+0000 7f15af600640 1 -- 172.21.15.88:0/2472947206 wait complete. 2024-09-21T07:49:14.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.530+0000 7f15af600640 1 Processor -- start 2024-09-21T07:49:14.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.530+0000 7f15af600640 1 -- start start 2024-09-21T07:49:14.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.531+0000 7f15af600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f15a8159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15af600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a8074190 0x7f15a806f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15af600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a806f960 0x7f15a8142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15af600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15a80b4da0 con 0x7f158c003a00 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15af600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15a80b4c20 con 0x7f15a806f960 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15af600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15a80b3960 con 0x7f15a8074190 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f15a8159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f15a8159fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39346/0 (socket says 172.21.15.88:39346) 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 -- 172.21.15.88:0/3924108195 learned_addr learned my addr 172.21.15.88:0/3924108195 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15ad600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a806f960 0x7f15a8142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:14.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f159fe00640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a8074190 0x7f15a806f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 -- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a8074190 msgr2=0x7f15a806f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a8074190 0x7f15a806f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 -- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a806f960 msgr2=0x7f15a8142e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a806f960 0x7f15a8142e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.532+0000 7f15acc00640 1 -- 172.21.15.88:0/3924108195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15a815a4e0 con 0x7f158c003a00 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.533+0000 7f15acc00640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f15a8159fa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f159800c590 tx=0x7f1598006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.533+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15980065b0 con 0x7f158c003a00 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.533+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f159802b660 con 0x7f158c003a00 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.533+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f159802bab0 con 0x7f158c003a00 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.533+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15a806fd70 con 0x7f158c003a00 2024-09-21T07:49:14.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.533+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15a81f9480 con 0x7f158c003a00 2024-09-21T07:49:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.534+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15a80ad4f0 con 0x7f158c003a00 2024-09-21T07:49:14.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.535+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f1598047020 con 0x7f158c003a00 2024-09-21T07:49:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.537+0000 7f15a5600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1584078dd0 0x7f158407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.537+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f15980bb370 con 0x7f158c003a00 2024-09-21T07:49:14.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.537+0000 7f159fe00640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1584078dd0 0x7f158407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:14.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.538+0000 7f159fe00640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1584078dd0 0x7f158407b2e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f15a814df20 tx=0x7f1590005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:14.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.542+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15a806fd70 con 0x7f158c003a00 2024-09-21T07:49:14.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.767+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15a80b2210 con 0x7f158c003a00 2024-09-21T07:49:14.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.767+0000 7f15a5600640 1 -- 172.21.15.88:0/3924108195 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f15a80ada30 con 0x7f158c003a00 2024-09-21T07:49:14.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:14.769 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:14.769 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:14.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1584078dd0 msgr2=0x7f158407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1584078dd0 0x7f158407b2e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f15a814df20 tx=0x7f1590005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 msgr2=0x7f15a8159fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f15a8159fa0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f159800c590 tx=0x7f1598006e20 comp rx=0 tx=0).stop 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 shutdown_connections 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1584078dd0 0x7f158407b2e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15a8074190 0x7f15a806f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15a806f960 0x7f15a8142e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 --2- 172.21.15.88:0/3924108195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f158c003a00 0x7f15a8159fa0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 >> 172.21.15.88:0/3924108195 conn(0x7f15a806d730 msgr2=0x7f15a815ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 shutdown_connections 2024-09-21T07:49:14.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:14.770+0000 7f15af600640 1 -- 172.21.15.88:0/3924108195 wait complete. 2024-09-21T07:49:14.794 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:15.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.014+0000 7ff3d9e00640 1 -- 172.21.15.88:0/580488917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d415d760 msgr2=0x7ff3d41535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.015+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/580488917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d415d760 0x7ff3d41535c0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7ff3c4009830 tx=0x7ff3c402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.015+0000 7ff3d9e00640 1 -- 172.21.15.88:0/580488917 shutdown_connections 2024-09-21T07:49:15.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.015+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/580488917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d415d760 0x7ff3d41535c0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.015+0000 7ff3d9e00640 1 -- 172.21.15.88:0/580488917 >> 172.21.15.88:0/580488917 conn(0x7ff3d406d730 msgr2=0x7ff3d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.015+0000 7ff3d9e00640 1 -- 172.21.15.88:0/580488917 shutdown_connections 2024-09-21T07:49:15.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.015+0000 7ff3d9e00640 1 -- 172.21.15.88:0/580488917 wait complete. 2024-09-21T07:49:15.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.017+0000 7ff3d9e00640 1 Processor -- start 2024-09-21T07:49:15.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.017+0000 7ff3d9e00640 1 -- start start 2024-09-21T07:49:15.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.018+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 0x7ff3d41f6500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d40b3550 0x7ff3d41f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3d415d760 0x7ff3d41f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d9e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d406f240 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d9e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3d40b4e30 con 0x7ff3d40b3550 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d9e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3d40b4070 con 0x7ff3d415d760 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 0x7ff3d41f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 0x7ff3d41f6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39362/0 (socket says 172.21.15.88:39362) 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 -- 172.21.15.88:0/2857870432 learned_addr learned my addr 172.21.15.88:0/2857870432 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2000640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d40b3550 0x7ff3d41f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d3400640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3d415d760 0x7ff3d41f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 -- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3d415d760 msgr2=0x7ff3d41f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3d415d760 0x7ff3d41f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 -- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d40b3550 msgr2=0x7ff3d41f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d40b3550 0x7ff3d41f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2a00640 1 -- 172.21.15.88:0/2857870432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3d41f9b30 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d2000640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d40b3550 0x7ff3d41f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.019+0000 7ff3d3400640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3d415d760 0x7ff3d41f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.020+0000 7ff3d2a00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 0x7ff3d41f6500 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff3c402aaa0 tx=0x7ff3c4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.020+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4005c90 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.020+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff3c4005020 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.020+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3d41f9e10 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.020+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3c4005450 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.020+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3d41f5fe0 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.021+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3d41535c0 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.022+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7ff3c40317c0 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.023+0000 7ff3cb400640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff3b4078dd0 0x7ff3b407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.023+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7ff3c40bc220 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.024+0000 7ff3d2000640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff3b4078dd0 0x7ff3b407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.024+0000 7ff3d2000640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff3b4078dd0 0x7ff3b407b2e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40699f0 tx=0x7ff3c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.027+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff3c403f030 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.247+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff3d40b0740 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.247+0000 7ff3cb400640 1 -- 172.21.15.88:0/2857870432 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7ff3c4084230 con 0x7ff3d40ad0e0 2024-09-21T07:49:15.248 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:15.249 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:15.249 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff3b4078dd0 msgr2=0x7ff3b407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff3b4078dd0 0x7ff3b407b2e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff3d40699f0 tx=0x7ff3c0009040 comp rx=0 tx=0).stop 2024-09-21T07:49:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 msgr2=0x7ff3d41f6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 0x7ff3d41f6500 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff3c402aaa0 tx=0x7ff3c4003c10 comp rx=0 tx=0).stop 2024-09-21T07:49:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 shutdown_connections 2024-09-21T07:49:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff3b4078dd0 0x7ff3b407b2e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3d415d760 0x7ff3d41f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3d40b3550 0x7ff3d41f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 --2- 172.21.15.88:0/2857870432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3d40ad0e0 0x7ff3d41f6500 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 >> 172.21.15.88:0/2857870432 conn(0x7ff3d406d730 msgr2=0x7ff3d40b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 shutdown_connections 2024-09-21T07:49:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.250+0000 7ff3d9e00640 1 -- 172.21.15.88:0/2857870432 wait complete. 2024-09-21T07:49:15.276 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 -- 172.21.15.88:0/3484900814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900ad3c0 msgr2=0x7f8f900ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 --2- 172.21.15.88:0/3484900814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900ad3c0 0x7f8f900ad7d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f8f900afd40 tx=0x7f8f7c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 -- 172.21.15.88:0/3484900814 shutdown_connections 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 --2- 172.21.15.88:0/3484900814 >> v2:172.21.15.175:3300/0 conn(0x7f8f900add10 0x7f8f900b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 --2- 172.21.15.88:0/3484900814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900ad3c0 0x7f8f900ad7d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 --2- 172.21.15.88:0/3484900814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f900704b0 0x7f8f90073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 -- 172.21.15.88:0/3484900814 >> 172.21.15.88:0/3484900814 conn(0x7f8f9006d730 msgr2=0x7f8f900b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 -- 172.21.15.88:0/3484900814 shutdown_connections 2024-09-21T07:49:15.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.494+0000 7f8f95c00640 1 -- 172.21.15.88:0/3484900814 wait complete. 2024-09-21T07:49:15.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.497+0000 7f8f95c00640 1 Processor -- start 2024-09-21T07:49:15.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.497+0000 7f8f95c00640 1 -- start start 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.498+0000 7f8f95c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f900704b0 0x7f8f90154200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.498+0000 7f8f95c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f900ad3c0 0x7f8f901f33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.498+0000 7f8f95c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 0x7f8f901f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.498+0000 7f8f95c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f900712b0 con 0x7f8f900add10 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.498+0000 7f8f95c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f90071130 con 0x7f8f900ad3c0 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.498+0000 7f8f95c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f900b4350 con 0x7f8f900704b0 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 0x7f8f901f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 0x7f8f901f71b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39376/0 (socket says 172.21.15.88:39376) 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 -- 172.21.15.88:0/2246533599 learned_addr learned my addr 172.21.15.88:0/2246533599 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:15.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8e000640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f900ad3c0 0x7f8f901f33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8ea00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f900704b0 0x7f8f90154200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 -- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f900704b0 msgr2=0x7f8f90154200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f900704b0 0x7f8f90154200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 -- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f900ad3c0 msgr2=0x7f8f901f33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f900ad3c0 0x7f8f901f33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 -- 172.21.15.88:0/2246533599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f9015c1e0 con 0x7f8f900add10 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.499+0000 7f8f8f400640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 0x7f8f901f71b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800ebb0 tx=0x7f8f8800c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.500+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f88004d20 con 0x7f8f900add10 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.500+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8f8800b570 con 0x7f8f900add10 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.500+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f90151b30 con 0x7f8f900add10 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.500+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f8800b9c0 con 0x7f8f900add10 2024-09-21T07:49:15.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.500+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f9015c760 con 0x7f8f900add10 2024-09-21T07:49:15.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.501+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f90154a10 con 0x7f8f900add10 2024-09-21T07:49:15.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.502+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f8f8800bb60 con 0x7f8f900add10 2024-09-21T07:49:15.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.503+0000 7f8f86a00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8f70078e00 0x7f8f7007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.504+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f8f8809be10 con 0x7f8f900add10 2024-09-21T07:49:15.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.504+0000 7f8f8ea00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8f70078e00 0x7f8f7007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.505+0000 7f8f8ea00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8f70078e00 0x7f8f7007b310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f80005fd0 tx=0x7f8f8000b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.508+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f8809f030 con 0x7f8f900add10 2024-09-21T07:49:15.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.693+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8f900ad7d0 con 0x7f8f900add10 2024-09-21T07:49:15.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.694+0000 7f8f86a00640 1 -- 172.21.15.88:0/2246533599 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16973 (secure 0 0 0) 0x7f8f880636f0 con 0x7f8f900add10 2024-09-21T07:49:15.696 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:15.696 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":49,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:47:46.542029+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:35.498681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:36.524995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":7}]},{"pgid":"7.28","mappings":[{"from":5,"to":0}]},{"pgid":"7.30","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:15.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.696+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8f70078e00 msgr2=0x7f8f7007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.696+0000 7f8f95c00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8f70078e00 0x7f8f7007b310 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8f80005fd0 tx=0x7f8f8000b260 comp rx=0 tx=0).stop 2024-09-21T07:49:15.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 msgr2=0x7f8f901f71b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 0x7f8f901f71b0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f8f8800ebb0 tx=0x7f8f8800c7a0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 shutdown_connections 2024-09-21T07:49:15.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8f70078e00 0x7f8f7007b310 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f900add10 0x7f8f901f71b0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f900ad3c0 0x7f8f901f33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 --2- 172.21.15.88:0/2246533599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8f900704b0 0x7f8f90154200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 >> 172.21.15.88:0/2246533599 conn(0x7f8f9006d730 msgr2=0x7f8f9014daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 shutdown_connections 2024-09-21T07:49:15.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.697+0000 7f8f95c00640 1 -- 172.21.15.88:0/2246533599 wait complete. 2024-09-21T07:49:15.721 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:49:15.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:15.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.941+0000 7fdb92200640 1 -- 172.21.15.88:0/4269745846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c15fae0 msgr2=0x7fdb8c0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.941+0000 7fdb92200640 1 --2- 172.21.15.88:0/4269745846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c15fae0 0x7fdb8c0b3750 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c072810 tx=0x7fdb8402a980 comp rx=0 tx=0).stop 2024-09-21T07:49:15.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.941+0000 7fdb92200640 1 -- 172.21.15.88:0/4269745846 shutdown_connections 2024-09-21T07:49:15.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.941+0000 7fdb92200640 1 --2- 172.21.15.88:0/4269745846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c15fae0 0x7fdb8c0b3750 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.941+0000 7fdb92200640 1 --2- 172.21.15.88:0/4269745846 >> v2:172.21.15.175:3300/0 conn(0x7fdb8c0aebc0 0x7fdb8c152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.941+0000 7fdb92200640 1 -- 172.21.15.88:0/4269745846 >> 172.21.15.88:0/4269745846 conn(0x7fdb8c06d730 msgr2=0x7fdb8c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:15.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.942+0000 7fdb92200640 1 -- 172.21.15.88:0/4269745846 shutdown_connections 2024-09-21T07:49:15.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.942+0000 7fdb92200640 1 -- 172.21.15.88:0/4269745846 wait complete. 2024-09-21T07:49:15.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.944+0000 7fdb92200640 1 Processor -- start 2024-09-21T07:49:15.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.944+0000 7fdb92200640 1 -- start start 2024-09-21T07:49:15.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.945+0000 7fdb92200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb8c0aebc0 0x7fdb8c156cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.945+0000 7fdb92200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8c0b2750 0x7fdb8c1f6c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.945+0000 7fdb92200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 0x7fdb8c1f7160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.945+0000 7fdb92200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c0b40d0 con 0x7fdb8c1f6730 2024-09-21T07:49:15.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.945+0000 7fdb92200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8c0b3f50 con 0x7fdb8c0aebc0 2024-09-21T07:49:15.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.945+0000 7fdb92200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb8c0b4250 con 0x7fdb8c0b2750 2024-09-21T07:49:15.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 0x7fdb8c1f7160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 0x7fdb8c1f7160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50230/0 (socket says 172.21.15.88:50230) 2024-09-21T07:49:15.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 -- 172.21.15.88:0/743192945 learned_addr learned my addr 172.21.15.88:0/743192945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:15.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8b400640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb8c0aebc0 0x7fdb8c156cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8aa00640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8c0b2750 0x7fdb8c1f6c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 -- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8c0b2750 msgr2=0x7fdb8c1f6c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8c0b2750 0x7fdb8c1f6c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 -- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb8c0aebc0 msgr2=0x7fdb8c156cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:15.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb8c0aebc0 0x7fdb8c156cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:15.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 -- 172.21.15.88:0/743192945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8c155820 con 0x7fdb8c1f6730 2024-09-21T07:49:15.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.946+0000 7fdb8be00640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 0x7fdb8c1f7160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009d60 tx=0x7fdb84003ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.947+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84034040 con 0x7fdb8c1f6730 2024-09-21T07:49:15.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.947+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdb84004640 con 0x7fdb8c1f6730 2024-09-21T07:49:15.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.947+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84045480 con 0x7fdb8c1f6730 2024-09-21T07:49:15.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.947+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb8c155b00 con 0x7fdb8c1f6730 2024-09-21T07:49:15.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.947+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb8c0ce8b0 con 0x7fdb8c1f6730 2024-09-21T07:49:15.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.948+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb8c0ceb10 con 0x7fdb8c1f6730 2024-09-21T07:49:15.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.949+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7fdb84037020 con 0x7fdb8c1f6730 2024-09-21T07:49:15.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.950+0000 7fdb83400640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdb6c078dd0 0x7fdb6c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:15.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.950+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7fdb840bcb80 con 0x7fdb8c1f6730 2024-09-21T07:49:15.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.950+0000 7fdb8b400640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdb6c078dd0 0x7fdb6c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:15.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.951+0000 7fdb8b400640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdb6c078dd0 0x7fdb6c07b2e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0098e0 tx=0x7fdb7c009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:15.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:15.954+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdb84084b40 con 0x7fdb8c1f6730 2024-09-21T07:49:16.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.175+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdb8c153470 con 0x7fdb8c1f6730 2024-09-21T07:49:16.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.176+0000 7fdb83400640 1 -- 172.21.15.88:0/743192945 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7fdb84084440 con 0x7fdb8c1f6730 2024-09-21T07:49:16.260 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:16.260 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:16.261 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:16.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdb6c078dd0 msgr2=0x7fdb6c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdb6c078dd0 0x7fdb6c07b2e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c0098e0 tx=0x7fdb7c009060 comp rx=0 tx=0).stop 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 msgr2=0x7fdb8c1f7160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 0x7fdb8c1f7160 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009d60 tx=0x7fdb84003ef0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 shutdown_connections 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fdb6c078dd0 0x7fdb6c07b2e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb8c0b2750 0x7fdb8c1f6c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb8c1f6730 0x7fdb8c1f7160 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 --2- 172.21.15.88:0/743192945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb8c0aebc0 0x7fdb8c156cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 >> 172.21.15.88:0/743192945 conn(0x7fdb8c06d730 msgr2=0x7fdb8c15c550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 shutdown_connections 2024-09-21T07:49:16.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.178+0000 7fdb92200640 1 -- 172.21.15.88:0/743192945 wait complete. 2024-09-21T07:49:16.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406f000640 1 -- 172.21.15.88:0/500862241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 msgr2=0x7f404c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406f000640 1 --2- 172.21.15.88:0/500862241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f404c004c10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4054006f80 tx=0x7f404c005830 comp rx=0 tx=0).stop 2024-09-21T07:49:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406d000640 1 -- 172.21.15.88:0/500862241 reap_dead start 2024-09-21T07:49:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406f000640 1 -- 172.21.15.88:0/500862241 shutdown_connections 2024-09-21T07:49:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406f000640 1 -- 172.21.15.88:0/500862241 >> 172.21.15.88:0/500862241 conn(0x7f406806d730 msgr2=0x7f40680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406f000640 1 -- 172.21.15.88:0/500862241 shutdown_connections 2024-09-21T07:49:16.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.486+0000 7f406f000640 1 -- 172.21.15.88:0/500862241 wait complete. 2024-09-21T07:49:16.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.488+0000 7f406f000640 1 Processor -- start 2024-09-21T07:49:16.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.488+0000 7f406f000640 1 -- start start 2024-09-21T07:49:16.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406f000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f406815c040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4068149f40 0x7f406814ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406814f1c0 0x7f40681f9200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406f000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40680b41f0 con 0x7f404c004800 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40680b4070 con 0x7f406814f1c0 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40680b4370 con 0x7f4068149f40 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f406815c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f406815c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50250/0 (socket says 172.21.15.88:50250) 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 -- 172.21.15.88:0/1019884518 learned_addr learned my addr 172.21.15.88:0/1019884518 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f406d000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406814f1c0 0x7f40681f9200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067400640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4068149f40 0x7f406814ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 -- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4068149f40 msgr2=0x7f406814ec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4068149f40 0x7f406814ec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 -- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406814f1c0 msgr2=0x7f40681f9200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406814f1c0 0x7f40681f9200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.490+0000 7f4067e00640 1 -- 172.21.15.88:0/1019884518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f406814f5d0 con 0x7f404c004800 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.491+0000 7f4067e00640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f406815c040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4058011b10 tx=0x7f4058009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.491+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405801e070 con 0x7f404c004800 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.491+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4058004050 con 0x7f404c004800 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.491+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f406814a990 con 0x7f404c004800 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.491+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f405800fea0 con 0x7f404c004800 2024-09-21T07:49:16.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.491+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f406814af50 con 0x7f404c004800 2024-09-21T07:49:16.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.492+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40680b3960 con 0x7f404c004800 2024-09-21T07:49:16.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.493+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 8) ==== 101645+0+0 (secure 0 0 0) 0x7f4058021070 con 0x7f404c004800 2024-09-21T07:49:16.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.495+0000 7f405fe00640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4048078e00 0x7f404807b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.495+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f405809bd40 con 0x7f404c004800 2024-09-21T07:49:16.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.495+0000 7f4067400640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4048078e00 0x7f404807b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.496+0000 7f4067400640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4048078e00 0x7f404807b310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4054033000 tx=0x7f405402d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.499+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f406814f5d0 con 0x7f404c004800 2024-09-21T07:49:16.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.721+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f40680d0bb0 con 0x7f404c004800 2024-09-21T07:49:16.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.722+0000 7f405fe00640 1 -- 172.21.15.88:0/1019884518 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 27 v27) ==== 94+0+5226 (secure 0 0 0) 0x7f4058063650 con 0x7f404c004800 2024-09-21T07:49:16.723 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:16.723 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":27,"btime":"2024-09-21T07:47:48:552505+0000","default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:47:37.572652+0000","modified":"2024-09-21T07:47:40.630013+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5337},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5337":{"gid":5337,"name":"c","rank":0,"incarnation":23,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/909378044","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":909378044},{"type":"v1","addr":"172.21.15.88:6835","nonce":909378044}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5337,"qdb_cluster":[5337]},"id":3}]} 2024-09-21T07:49:16.723 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 27 2024-09-21T07:49:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4048078e00 msgr2=0x7f404807b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4048078e00 0x7f404807b310 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f4054033000 tx=0x7f405402d000 comp rx=0 tx=0).stop 2024-09-21T07:49:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 msgr2=0x7f406815c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f406815c040 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f4058011b10 tx=0x7f4058009e90 comp rx=0 tx=0).stop 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 shutdown_connections 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4048078e00 0x7f404807b310 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4068149f40 0x7f406814ec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f406814f1c0 0x7f40681f9200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 --2- 172.21.15.88:0/1019884518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f404c004800 0x7f406815c040 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 >> 172.21.15.88:0/1019884518 conn(0x7f406806d730 msgr2=0x7f40680b1900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 shutdown_connections 2024-09-21T07:49:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.724+0000 7f406f000640 1 -- 172.21.15.88:0/1019884518 wait complete. 2024-09-21T07:49:16.749 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.969+0000 7f2d6f200640 1 -- 172.21.15.88:0/1801083328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680af170 msgr2=0x7f2d680b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.969+0000 7f2d6f200640 1 --2- 172.21.15.88:0/1801083328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680af170 0x7f2d680b2f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2d58009900 tx=0x7f2d5802a580 comp rx=0 tx=0).stop 2024-09-21T07:49:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.969+0000 7f2d6f200640 1 -- 172.21.15.88:0/1801083328 shutdown_connections 2024-09-21T07:49:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.969+0000 7f2d6f200640 1 --2- 172.21.15.88:0/1801083328 >> v2:172.21.15.175:3300/0 conn(0x7f2d6815a0f0 0x7f2d6815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.969+0000 7f2d6f200640 1 --2- 172.21.15.88:0/1801083328 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680af170 0x7f2d680b2f60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.969+0000 7f2d6f200640 1 -- 172.21.15.88:0/1801083328 >> 172.21.15.88:0/1801083328 conn(0x7f2d680b1e40 msgr2=0x7f2d680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:16.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.970+0000 7f2d6f200640 1 -- 172.21.15.88:0/1801083328 shutdown_connections 2024-09-21T07:49:16.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.970+0000 7f2d6f200640 1 -- 172.21.15.88:0/1801083328 wait complete. 2024-09-21T07:49:16.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.972+0000 7f2d6f200640 1 Processor -- start 2024-09-21T07:49:16.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.972+0000 7f2d6f200640 1 -- start start 2024-09-21T07:49:16.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.973+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680ad9c0 0x7f2d6809de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.973+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680af170 0x7f2d681486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.973+0000 7f2d6f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 0x7f2d6814b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.973+0000 7f2d6f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6806f410 con 0x7f2d6815a0f0 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.973+0000 7f2d6f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d6806f290 con 0x7f2d680af170 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.973+0000 7f2d6f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d6806f590 con 0x7f2d680ad9c0 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 0x7f2d6814b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 0x7f2d6814b090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50254/0 (socket says 172.21.15.88:50254) 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 -- 172.21.15.88:0/2285578565 learned_addr learned my addr 172.21.15.88:0/2285578565 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6de00640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680ad9c0 0x7f2d6809de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6d400640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680af170 0x7f2d681486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 -- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680ad9c0 msgr2=0x7f2d6809de60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680ad9c0 0x7f2d6809de60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 -- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680af170 msgr2=0x7f2d681486b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:16.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680af170 0x7f2d681486b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.974+0000 7f2d6e800640 1 -- 172.21.15.88:0/2285578565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d68149210 con 0x7f2d6815a0f0 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.975+0000 7f2d6e800640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 0x7f2d6814b090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400ec20 tx=0x7f2d6400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.975+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64003b70 con 0x7f2d6815a0f0 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.975+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2d64003e00 con 0x7f2d6815a0f0 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.975+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d681494f0 con 0x7f2d6815a0f0 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.975+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64004280 con 0x7f2d6815a0f0 2024-09-21T07:49:16.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.975+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d6809b500 con 0x7f2d6815a0f0 2024-09-21T07:49:16.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.976+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d6806fab0 con 0x7f2d6815a0f0 2024-09-21T07:49:16.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.977+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f2d64004420 con 0x7f2d6815a0f0 2024-09-21T07:49:16.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.978+0000 7f2d62000640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2d4c078d40 0x7f2d4c07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:16.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.978+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f2d6409bff0 con 0x7f2d6815a0f0 2024-09-21T07:49:16.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.979+0000 7f2d6de00640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2d4c078d40 0x7f2d4c07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:16.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.979+0000 7f2d6de00640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2d4c078d40 0x7f2d4c07b250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c001720 tx=0x7f2d5c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:16.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:16.983+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2d64063950 con 0x7f2d6815a0f0 2024-09-21T07:49:17.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.171+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2d680b06f0 con 0x7f2d6815a0f0 2024-09-21T07:49:17.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.172+0000 7f2d62000640 1 -- 172.21.15.88:0/2285578565 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v49) ==== 74+0+16973 (secure 0 0 0) 0x7f2d64068380 con 0x7f2d6815a0f0 2024-09-21T07:49:17.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:17.174 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":49,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:47:46.542029+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:47:35.498681+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"46","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2024-09-21T07:47:36.524995+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.5","mappings":[{"from":5,"to":7}]},{"pgid":"7.1b","mappings":[{"from":5,"to":7}]},{"pgid":"7.28","mappings":[{"from":5,"to":0}]},{"pgid":"7.30","mappings":[{"from":5,"to":7}]},{"pgid":"7.32","mappings":[{"from":5,"to":7}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:17.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2d4c078d40 msgr2=0x7f2d4c07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2d4c078d40 0x7f2d4c07b250 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d5c001720 tx=0x7f2d5c009040 comp rx=0 tx=0).stop 2024-09-21T07:49:17.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 msgr2=0x7f2d6814b090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 0x7f2d6814b090 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400ec20 tx=0x7f2d6400c6a0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 shutdown_connections 2024-09-21T07:49:17.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2d4c078d40 0x7f2d4c07b250 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d6815a0f0 0x7f2d6814b090 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d680af170 0x7f2d681486b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 --2- 172.21.15.88:0/2285578565 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d680ad9c0 0x7f2d6809de60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.174+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 >> 172.21.15.88:0/2285578565 conn(0x7f2d680b1e40 msgr2=0x7f2d680b12d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.175+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 shutdown_connections 2024-09-21T07:49:17.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.175+0000 7f2d6f200640 1 -- 172.21.15.88:0/2285578565 wait complete. 2024-09-21T07:49:17.200 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:49:17.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 -- 172.21.15.88:0/3349230328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7328003c30 msgr2=0x7f7328004080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 --2- 172.21.15.88:0/3349230328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7328003c30 0x7f7328004080 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f73400cf0d0 tx=0x7f7328005830 comp rx=0 tx=0).stop 2024-09-21T07:49:17.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 -- 172.21.15.88:0/3349230328 shutdown_connections 2024-09-21T07:49:17.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 --2- 172.21.15.88:0/3349230328 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7328003c30 0x7f7328004080 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 -- 172.21.15.88:0/3349230328 >> 172.21.15.88:0/3349230328 conn(0x7f734006d3d0 msgr2=0x7f73400b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 -- 172.21.15.88:0/3349230328 shutdown_connections 2024-09-21T07:49:17.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.421+0000 7f7348800640 1 -- 172.21.15.88:0/3349230328 wait complete. 2024-09-21T07:49:17.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.423+0000 7f7348800640 1 Processor -- start 2024-09-21T07:49:17.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.423+0000 7f7348800640 1 -- start start 2024-09-21T07:49:17.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7348800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7328003c30 0x7f73400b0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7348800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 0x7f73401e73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7348800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73400afd50 0x7f73401e7920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7348800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f734015ce30 con 0x7f73400b4280 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7348800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7340157210 con 0x7f73400afd50 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7348800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f734015cfb0 con 0x7f7328003c30 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 0x7f73401e73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 0x7f73401e73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50268/0 (socket says 172.21.15.88:50268) 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 -- 172.21.15.88:0/1631522761 learned_addr learned my addr 172.21.15.88:0/1631522761 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7346800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73400afd50 0x7f73401e7920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345e00640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7328003c30 0x7f73400b0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 -- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7328003c30 msgr2=0x7f73400b0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7328003c30 0x7f73400b0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 -- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73400afd50 msgr2=0x7f73401e7920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73400afd50 0x7f73401e7920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.425+0000 7f7345400640 1 -- 172.21.15.88:0/1631522761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73400b11d0 con 0x7f73400b4280 2024-09-21T07:49:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.426+0000 7f7345400640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 0x7f73401e73e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7330003cd0 tx=0x7f7330001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.426+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330011640 con 0x7f73400b4280 2024-09-21T07:49:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.426+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7330016020 con 0x7f73400b4280 2024-09-21T07:49:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.426+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7330011a60 con 0x7f73400b4280 2024-09-21T07:49:17.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.426+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7340072fc0 con 0x7f73400b4280 2024-09-21T07:49:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.426+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7340074880 con 0x7f73400b4280 2024-09-21T07:49:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.427+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73401f4020 con 0x7f73400b4280 2024-09-21T07:49:17.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.428+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f7330011c00 con 0x7f73400b4280 2024-09-21T07:49:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.429+0000 7f733a000640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7324078d40 0x7f732407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.429+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5854+0+0 (secure 0 0 0) 0x7f733009be40 con 0x7f73400b4280 2024-09-21T07:49:17.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.430+0000 7f7345e00640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7324078d40 0x7f732407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.430+0000 7f7345e00640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7324078d40 0x7f732407b250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7334033000 tx=0x7f733402d000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.434+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7330020020 con 0x7f73400b4280 2024-09-21T07:49:17.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.656+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f73400742f0 con 0x7f73400b4280 2024-09-21T07:49:17.658 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:49:17.656+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:49:17.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.658+0000 7f733a000640 1 -- 172.21.15.88:0/1631522761 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v28) ==== 161+0+0 (secure 0 0 0) 0x7f7330063f20 con 0x7f73400b4280 2024-09-21T07:49:17.660 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:49:17.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.662+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7324078d40 msgr2=0x7f732407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.662+0000 7f7348800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7324078d40 0x7f732407b250 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f7334033000 tx=0x7f733402d000 comp rx=0 tx=0).stop 2024-09-21T07:49:17.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.662+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 msgr2=0x7f73401e73e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.662+0000 7f7348800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 0x7f73401e73e0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f7330003cd0 tx=0x7f7330001800 comp rx=0 tx=0).stop 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.662+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 shutdown_connections 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7324078d40 0x7f732407b250 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73400afd50 0x7f73401e7920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73400b4280 0x7f73401e73e0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 --2- 172.21.15.88:0/1631522761 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7328003c30 0x7f73400b0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 >> 172.21.15.88:0/1631522761 conn(0x7f734006d3d0 msgr2=0x7f7340155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 shutdown_connections 2024-09-21T07:49:17.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.663+0000 7f7348800640 1 -- 172.21.15.88:0/1631522761 wait complete. 2024-09-21T07:49:17.689 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:49:17.803 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 -- 172.21.15.88:0/274628901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc005280 msgr2=0x7fb5cc00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 --2- 172.21.15.88:0/274628901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc005280 0x7fb5cc00b9c0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb5e40b0070 tx=0x7fb5cc005830 comp rx=0 tx=0).stop 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 -- 172.21.15.88:0/274628901 shutdown_connections 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 --2- 172.21.15.88:0/274628901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc005280 0x7fb5cc00b9c0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 -- 172.21.15.88:0/274628901 >> 172.21.15.88:0/274628901 conn(0x7fb5e406d1c0 msgr2=0x7fb5e406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 -- 172.21.15.88:0/274628901 shutdown_connections 2024-09-21T07:49:17.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.914+0000 7fb5eba00640 1 -- 172.21.15.88:0/274628901 wait complete. 2024-09-21T07:49:17.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.917+0000 7fb5eba00640 1 Processor -- start 2024-09-21T07:49:17.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.917+0000 7fb5eba00640 1 -- start start 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.918+0000 7fb5eba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5cc005280 0x7fb5e41f7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5eba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e41f28c0 0x7fb5e413d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5eba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 0x7fb5e413d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5eba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4073540 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5eba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4070130 con 0x7fb5cc005280 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5eba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e40736c0 con 0x7fb5e41f28c0 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 0x7fb5e413d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 0x7fb5e413d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50288/0 (socket says 172.21.15.88:50288) 2024-09-21T07:49:17.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 -- 172.21.15.88:0/3582259695 learned_addr learned my addr 172.21.15.88:0/3582259695 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:17.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9000640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5cc005280 0x7fb5e41f7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e3e00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e41f28c0 0x7fb5e413d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 -- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e41f28c0 msgr2=0x7fb5e413d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e41f28c0 0x7fb5e413d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 -- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5cc005280 msgr2=0x7fb5e41f7110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5cc005280 0x7fb5e41f7110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.919+0000 7fb5e9a00640 1 -- 172.21.15.88:0/3582259695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e41f7650 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.920+0000 7fb5e9a00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 0x7fb5e413d750 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc02b4d0 tx=0x7fb5dc02caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.920+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc005020 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.920+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5dc004aa0 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.920+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5dc03c440 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.920+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e4143d70 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.920+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e4138a80 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.921+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4073bb0 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.922+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fb5dc044020 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.923+0000 7fb5e0c00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb5c4078d40 0x7fb5c407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:17.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.923+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5940+0+0 (secure 0 0 0) 0x7fb5dc0bba20 con 0x7fb5e41f2cd0 2024-09-21T07:49:17.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.924+0000 7fb5e9000640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb5c4078d40 0x7fb5c407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:17.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.924+0000 7fb5e9000640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb5c4078d40 0x7fb5c407b250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8006fd0 tx=0x7fb5d8006aa0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:17.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:17.928+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5dc04b020 con 0x7fb5e41f2cd0 2024-09-21T07:49:18.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.150+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fb5e41e8e20 con 0x7fb5e41f2cd0 2024-09-21T07:49:18.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.750+0000 7fb5e0c00640 1 -- 172.21.15.88:0/3582259695 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v29) ==== 104+0+0 (secure 0 0 0) 0x7fb5e41f7650 con 0x7fb5e41f2cd0 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.752+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb5c4078d40 msgr2=0x7fb5c407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.752+0000 7fb5eba00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb5c4078d40 0x7fb5c407b250 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8006fd0 tx=0x7fb5d8006aa0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.752+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 msgr2=0x7fb5e413d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.752+0000 7fb5eba00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 0x7fb5e413d750 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fb5dc02b4d0 tx=0x7fb5dc02caa0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 shutdown_connections 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb5c4078d40 0x7fb5c407b250 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5e41f28c0 0x7fb5e413d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e41f2cd0 0x7fb5e413d750 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 --2- 172.21.15.88:0/3582259695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5cc005280 0x7fb5e41f7110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 >> 172.21.15.88:0/3582259695 conn(0x7fb5e406d1c0 msgr2=0x7fb5e40b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 shutdown_connections 2024-09-21T07:49:18.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:18.753+0000 7fb5eba00640 1 -- 172.21.15.88:0/3582259695 wait complete. 2024-09-21T07:49:18.779 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:49:19.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 -- 172.21.15.88:0/2789627547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80cc7d0 msgr2=0x7efee80b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 --2- 172.21.15.88:0/2789627547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80cc7d0 0x7efee80b3120 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7efed8009830 tx=0x7efed802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 -- 172.21.15.88:0/2789627547 shutdown_connections 2024-09-21T07:49:19.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 --2- 172.21.15.88:0/2789627547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee80cc7d0 0x7efee80b3120 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 -- 172.21.15.88:0/2789627547 >> 172.21.15.88:0/2789627547 conn(0x7efee80b2540 msgr2=0x7efee80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:19.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 -- 172.21.15.88:0/2789627547 shutdown_connections 2024-09-21T07:49:19.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.005+0000 7efeee200640 1 -- 172.21.15.88:0/2789627547 wait complete. 2024-09-21T07:49:19.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.007+0000 7efeee200640 1 Processor -- start 2024-09-21T07:49:19.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.007+0000 7efeee200640 1 -- start start 2024-09-21T07:49:19.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efeee200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee80b36f0 0x7efee81570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efeee200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee80cc7d0 0x7efee80cb250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efeee200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 0x7efee80ad0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efeee200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efee806fab0 con 0x7efee815ba50 2024-09-21T07:49:19.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efeee200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efee806f930 con 0x7efee80b36f0 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efeee200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efee806fc30 con 0x7efee80cc7d0 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efee7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 0x7efee80ad0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efee7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 0x7efee80ad0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50298/0 (socket says 172.21.15.88:50298) 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efee7e00640 1 -- 172.21.15.88:0/1882485654 learned_addr learned my addr 172.21.15.88:0/1882485654 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efee7400640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee80b36f0 0x7efee81570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.009+0000 7efedea00640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee80cc7d0 0x7efee80cb250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7e00640 1 -- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee80cc7d0 msgr2=0x7efee80cb250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7e00640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee80cc7d0 0x7efee80cb250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7e00640 1 -- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee80b36f0 msgr2=0x7efee81570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7e00640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee80b36f0 0x7efee81570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7e00640 1 -- 172.21.15.88:0/1882485654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee8156320 con 0x7efee815ba50 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7400640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee80b36f0 0x7efee81570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:19.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efee7e00640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 0x7efee80ad0e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efee000cad0 tx=0x7efee000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0016020 con 0x7efee815ba50 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efee000ad40 con 0x7efee815ba50 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0012680 con 0x7efee815ba50 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee8156600 con 0x7efee815ba50 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.010+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee815eeb0 con 0x7efee815ba50 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.011+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efee815f170 con 0x7efee815ba50 2024-09-21T07:49:19.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.013+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7efee0019020 con 0x7efee815ba50 2024-09-21T07:49:19.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.013+0000 7efedfe00640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efec4078d70 0x7efec407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:19.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.014+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(50..50 src has 1..50) ==== 5940+0+0 (secure 0 0 0) 0x7efee009b750 con 0x7efee815ba50 2024-09-21T07:49:19.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.014+0000 7efee7400640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efec4078d70 0x7efec407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:19.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.015+0000 7efee7400640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efec4078d70 0x7efec407b280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efed8034000 tx=0x7efed8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:19.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.018+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efee0020020 con 0x7efee815ba50 2024-09-21T07:49:19.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.204+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7efee81f5da0 con 0x7efee815ba50 2024-09-21T07:49:19.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.759+0000 7efedfe00640 1 -- 172.21.15.88:0/1882485654 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) ==== 181+0+0 (secure 0 0 0) 0x7efee00637a0 con 0x7efee815ba50 2024-09-21T07:49:19.761 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:49:19.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efec4078d70 msgr2=0x7efec407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efec4078d70 0x7efec407b280 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7efed8034000 tx=0x7efed8004730 comp rx=0 tx=0).stop 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 msgr2=0x7efee80ad0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 0x7efee80ad0e0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7efee000cad0 tx=0x7efee000cf90 comp rx=0 tx=0).stop 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 shutdown_connections 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efec4078d70 0x7efec407b280 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efee80cc7d0 0x7efee80cb250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee815ba50 0x7efee80ad0e0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 --2- 172.21.15.88:0/1882485654 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efee80b36f0 0x7efee81570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 >> 172.21.15.88:0/1882485654 conn(0x7efee80b2540 msgr2=0x7efee80741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 shutdown_connections 2024-09-21T07:49:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:19.761+0000 7efeee200640 1 -- 172.21.15.88:0/1882485654 wait complete. 2024-09-21T07:49:19.787 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:49:20.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 -- 172.21.15.88:0/163004675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 msgr2=0x7fdde80af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 --2- 172.21.15.88:0/163004675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde80af3c0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009830 tx=0x7fddd402a480 comp rx=0 tx=0).stop 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 -- 172.21.15.88:0/163004675 shutdown_connections 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 --2- 172.21.15.88:0/163004675 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde80af3c0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 --2- 172.21.15.88:0/163004675 >> v2:172.21.15.175:3300/0 conn(0x7fdde80b3ec0 0x7fdde80b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 -- 172.21.15.88:0/163004675 >> 172.21.15.88:0/163004675 conn(0x7fdde80b1dd0 msgr2=0x7fdde80b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 -- 172.21.15.88:0/163004675 shutdown_connections 2024-09-21T07:49:20.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.010+0000 7fddeee00640 1 -- 172.21.15.88:0/163004675 wait complete. 2024-09-21T07:49:20.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.012+0000 7fddeee00640 1 Processor -- start 2024-09-21T07:49:20.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.012+0000 7fddeee00640 1 -- start start 2024-09-21T07:49:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fddeee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde8148e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fddeee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 0x7fdde8155c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fddeee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde81532e0 0x7fdde81580c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fddeee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8153a80 con 0x7fdde80b3ec0 2024-09-21T07:49:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fddeee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdde8153900 con 0x7fdde81532e0 2024-09-21T07:49:20.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fddeee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdde8153c00 con 0x7fdde80705b0 2024-09-21T07:49:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fdded000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 0x7fdde8155c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fdded000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 0x7fdde8155c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50314/0 (socket says 172.21.15.88:50314) 2024-09-21T07:49:20.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.014+0000 7fdded000640 1 -- 172.21.15.88:0/985790349 learned_addr learned my addr 172.21.15.88:0/985790349 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fddeda00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde8148e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fddee400640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde81532e0 0x7fdde81580c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdded000640 1 -- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 msgr2=0x7fdde8148e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdded000640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde8148e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdded000640 1 -- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde81532e0 msgr2=0x7fdde81580c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdded000640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde81532e0 0x7fdde81580c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdded000640 1 -- 172.21.15.88:0/985790349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdde8157cd0 con 0x7fdde80b3ec0 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fddeda00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde8148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:20.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdded000640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 0x7fdde8155c60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009950 tx=0x7fddd4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4031d60 con 0x7fdde80b3ec0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddd4033e10 con 0x7fdde80b3ec0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdde8157730 con 0x7fdde80b3ec0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.015+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddd4033310 con 0x7fdde80b3ec0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.016+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdde81579f0 con 0x7fdde80b3ec0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.017+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdde80af3c0 con 0x7fdde80b3ec0 2024-09-21T07:49:20.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.018+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fddd40334b0 con 0x7fdde80b3ec0 2024-09-21T07:49:20.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.019+0000 7fdde5600640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fddc8078d70 0x7fddc807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:20.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.019+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(51..51 src has 1..51) ==== 5502+0+0 (secure 0 0 0) 0x7fddd40bc520 con 0x7fdde80b3ec0 2024-09-21T07:49:20.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.019+0000 7fddeda00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fddc8078d70 0x7fddc807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:20.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.020+0000 7fddeda00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fddc8078d70 0x7fddc807b280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005fd0 tx=0x7fdddc009260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:20.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.023+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fddd403f030 con 0x7fdde80b3ec0 2024-09-21T07:49:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.214+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdde8157ea0 con 0x7fdde80b3ec0 2024-09-21T07:49:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.765+0000 7fdde5600640 1 -- 172.21.15.88:0/985790349 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) ==== 169+0+0 (secure 0 0 0) 0x7fddd4084600 con 0x7fdde80b3ec0 2024-09-21T07:49:20.767 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fddc8078d70 msgr2=0x7fddc807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fddc8078d70 0x7fddc807b280 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fdddc005fd0 tx=0x7fdddc009260 comp rx=0 tx=0).stop 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 msgr2=0x7fdde8155c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 0x7fdde8155c60 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fddd4009950 tx=0x7fddd4033c20 comp rx=0 tx=0).stop 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 shutdown_connections 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fddc8078d70 0x7fddc807b280 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdde81532e0 0x7fdde81580c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdde80b3ec0 0x7fdde8155c60 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 --2- 172.21.15.88:0/985790349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdde80705b0 0x7fdde8148e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.767+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 >> 172.21.15.88:0/985790349 conn(0x7fdde80b1dd0 msgr2=0x7fdde8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:20.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.768+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 shutdown_connections 2024-09-21T07:49:20.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:20.768+0000 7fddeee00640 1 -- 172.21.15.88:0/985790349 wait complete. 2024-09-21T07:49:20.794 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.022+0000 7f0f31c00640 1 -- 172.21.15.88:0/43083738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f10004800 msgr2=0x7f0f10004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.022+0000 7f0f31c00640 1 --2- 172.21.15.88:0/43083738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f10004800 0x7f0f10004c10 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0f18009b90 tx=0x7f0f10005830 comp rx=0 tx=0).stop 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.022+0000 7f0f2b400640 1 -- 172.21.15.88:0/43083738 reap_dead start 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.022+0000 7f0f31c00640 1 -- 172.21.15.88:0/43083738 shutdown_connections 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.022+0000 7f0f31c00640 1 -- 172.21.15.88:0/43083738 >> 172.21.15.88:0/43083738 conn(0x7f0f2c06d3d0 msgr2=0x7f0f2c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.023+0000 7f0f31c00640 1 -- 172.21.15.88:0/43083738 shutdown_connections 2024-09-21T07:49:21.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.023+0000 7f0f31c00640 1 -- 172.21.15.88:0/43083738 wait complete. 2024-09-21T07:49:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.025+0000 7f0f31c00640 1 Processor -- start 2024-09-21T07:49:21.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.025+0000 7f0f31c00640 1 -- start start 2024-09-21T07:49:21.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.026+0000 7f0f31c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f10004800 0x7f0f2c1f6fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.026+0000 7f0f31c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f2c1f40e0 0x7f0f2c1560e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.026+0000 7f0f31c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 0x7f0f2c0744b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.026+0000 7f0f31c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c156e40 con 0x7f0f2c0740a0 2024-09-21T07:49:21.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.026+0000 7f0f31c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f2c070ed0 con 0x7f0f10004800 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.026+0000 7f0f31c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f2c156fc0 con 0x7f0f2c1f40e0 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 0x7f0f2c0744b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 0x7f0f2c0744b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50338/0 (socket says 172.21.15.88:50338) 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 -- 172.21.15.88:0/427831288 learned_addr learned my addr 172.21.15.88:0/427831288 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2a000640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f2c1f40e0 0x7f0f2c1560e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2aa00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f10004800 0x7f0f2c1f6fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 -- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f2c1f40e0 msgr2=0x7f0f2c1560e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f2c1f40e0 0x7f0f2c1560e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 -- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f10004800 msgr2=0x7f0f2c1f6fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f10004800 0x7f0f2c1f6fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 -- 172.21.15.88:0/427831288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f2c0749f0 con 0x7f0f2c0740a0 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2a000640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f2c1f40e0 0x7f0f2c1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:21.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.027+0000 7f0f2b400640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 0x7f0f2c0744b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0cf7a0 tx=0x7f0f2400e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:21.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.028+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f2400eea0 con 0x7f0f2c0740a0 2024-09-21T07:49:21.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.028+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f24015070 con 0x7f0f2c0740a0 2024-09-21T07:49:21.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.028+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f24003dd0 con 0x7f0f2c0740a0 2024-09-21T07:49:21.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.028+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f2c09df20 con 0x7f0f2c0740a0 2024-09-21T07:49:21.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.028+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f2c0c4b50 con 0x7f0f2c0740a0 2024-09-21T07:49:21.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.029+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f2c15d240 con 0x7f0f2c0740a0 2024-09-21T07:49:21.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.030+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f0f24004030 con 0x7f0f2c0740a0 2024-09-21T07:49:21.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.031+0000 7f0f22a00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f0c078d70 0x7f0f0c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.031+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7f0f2409bbd0 con 0x7f0f2c0740a0 2024-09-21T07:49:21.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.032+0000 7f0f2aa00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f0c078d70 0x7f0f0c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.033+0000 7f0f2aa00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f0c078d70 0x7f0f0c07b280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c09e510 tx=0x7f0f1c006c10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:21.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.035+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f2c0749f0 con 0x7f0f2c0740a0 2024-09-21T07:49:21.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.222+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f0f2c0afa70 con 0x7f0f2c0740a0 2024-09-21T07:49:21.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.227+0000 7f0f22a00640 1 -- 172.21.15.88:0/427831288 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v35) ==== 71+0+0 (secure 0 0 0) 0x7f0f24063ee0 con 0x7f0f2c0740a0 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f0c078d70 msgr2=0x7f0f0c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f0c078d70 0x7f0f0c07b280 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c09e510 tx=0x7f0f1c006c10 comp rx=0 tx=0).stop 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 msgr2=0x7f0f2c0744b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 0x7f0f2c0744b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f2c0cf7a0 tx=0x7f0f2400e720 comp rx=0 tx=0).stop 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 shutdown_connections 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0f0c078d70 0x7f0f0c07b280 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f2c1f40e0 0x7f0f2c1560e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f2c0740a0 0x7f0f2c0744b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 --2- 172.21.15.88:0/427831288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f10004800 0x7f0f2c1f6fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 >> 172.21.15.88:0/427831288 conn(0x7f0f2c06d3d0 msgr2=0x7f0f2c1603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 shutdown_connections 2024-09-21T07:49:21.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.229+0000 7f0f31c00640 1 -- 172.21.15.88:0/427831288 wait complete. 2024-09-21T07:49:21.257 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration' 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.474+0000 7f9395800640 1 -- 172.21.15.88:0/2179179852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 msgr2=0x7f93900ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.474+0000 7f9395800640 1 --2- 172.21.15.88:0/2179179852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93900ad4f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f937c009830 tx=0x7f937c02a480 comp rx=0 tx=0).stop 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.474+0000 7f9395800640 1 -- 172.21.15.88:0/2179179852 shutdown_connections 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.474+0000 7f9395800640 1 --2- 172.21.15.88:0/2179179852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93900b3550 0x7f93900b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.474+0000 7f9395800640 1 --2- 172.21.15.88:0/2179179852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93900ad4f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.474+0000 7f9395800640 1 -- 172.21.15.88:0/2179179852 >> 172.21.15.88:0/2179179852 conn(0x7f939006d730 msgr2=0x7f93900b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.475+0000 7f9395800640 1 -- 172.21.15.88:0/2179179852 shutdown_connections 2024-09-21T07:49:21.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.475+0000 7f9395800640 1 -- 172.21.15.88:0/2179179852 wait complete. 2024-09-21T07:49:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.477+0000 7f9395800640 1 Processor -- start 2024-09-21T07:49:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.477+0000 7f9395800640 1 -- start start 2024-09-21T07:49:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.478+0000 7f9395800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93901573e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.478+0000 7f9395800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93900b3550 0x7f9390153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.478+0000 7f9395800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f939015d760 0x7f939014a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.478+0000 7f9395800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93900b4da0 con 0x7f93900ad0e0 2024-09-21T07:49:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.478+0000 7f9395800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93900b4c20 con 0x7f93900b3550 2024-09-21T07:49:21.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.478+0000 7f9395800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93900b3ea0 con 0x7f939015d760 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93901573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93901573e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50364/0 (socket says 172.21.15.88:50364) 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 -- 172.21.15.88:0/1937347152 learned_addr learned my addr 172.21.15.88:0/1937347152 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938f400640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f939015d760 0x7f939014a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938e000640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93900b3550 0x7f9390153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 -- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f939015d760 msgr2=0x7f939014a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f939015d760 0x7f939014a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 -- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93900b3550 msgr2=0x7f9390153e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93900b3550 0x7f9390153e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 -- 172.21.15.88:0/1937347152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9390154380 con 0x7f93900ad0e0 2024-09-21T07:49:21.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.479+0000 7f938ea00640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93901573e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f938000c3f0 tx=0x7f938000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.480+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380017070 con 0x7f93900ad0e0 2024-09-21T07:49:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.480+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93800035f0 con 0x7f93900ad0e0 2024-09-21T07:49:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.480+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380003a40 con 0x7f93900ad0e0 2024-09-21T07:49:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.480+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93901f5980 con 0x7f93900ad0e0 2024-09-21T07:49:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.480+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9390155800 con 0x7f93900ad0e0 2024-09-21T07:49:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.481+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93900b4370 con 0x7f93900ad0e0 2024-09-21T07:49:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.482+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f938001a070 con 0x7f93900ad0e0 2024-09-21T07:49:21.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.483+0000 7f9386a00640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9370078d70 0x7f937007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:21.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.484+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7f9380099ff0 con 0x7f93900ad0e0 2024-09-21T07:49:21.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.484+0000 7f938e000640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9370078d70 0x7f937007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:21.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.485+0000 7f938e000640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9370078d70 0x7f937007b280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f937c034000 tx=0x7f937c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:21.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.488+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93900ada30 con 0x7f93900ad0e0 2024-09-21T07:49:21.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.746+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]} v 0) -- 0x7f93900aeb20 con 0x7f93900ad0e0 2024-09-21T07:49:21.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.767+0000 7f9386a00640 1 -- 172.21.15.88:0/1937347152 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_connection_expiration"]}]=0 v188) ==== 140+0+0 (secure 0 0 0) 0x7f9380061ea0 con 0x7f93900ad0e0 2024-09-21T07:49:21.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9370078d70 msgr2=0x7f937007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9370078d70 0x7f937007b280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f937c034000 tx=0x7f937c004730 comp rx=0 tx=0).stop 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 msgr2=0x7f93901573e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93901573e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f938000c3f0 tx=0x7f938000c8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 shutdown_connections 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9370078d70 0x7f937007b280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f939015d760 0x7f939014a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93900b3550 0x7f9390153e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 --2- 172.21.15.88:0/1937347152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93900ad0e0 0x7f93901573e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 >> 172.21.15.88:0/1937347152 conn(0x7f939006d730 msgr2=0x7f93900cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 shutdown_connections 2024-09-21T07:49:21.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:21.769+0000 7f9395800640 1 -- 172.21.15.88:0/1937347152 wait complete. 2024-09-21T07:49:21.793 INFO:tasks.cephfs_test_runner:test_connection_expiration (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-21T07:49:21.793 INFO:tasks.cephfs_test_runner:Starting test: test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate handcrafted .meta file on legacy subvol root doesn't break the system 2024-09-21T07:49:21.793 INFO:tasks.cephfs_test_runner:test_malicious_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:49:21.794 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:22.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc255e00640 1 -- 172.21.15.88:0/3521404928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc234004d40 msgr2=0x7fc234005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc255e00640 1 --2- 172.21.15.88:0/3521404928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc234004d40 0x7fc234005150 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7fc2500aea60 tx=0x7fc234005830 comp rx=0 tx=0).stop 2024-09-21T07:49:22.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc24f400640 1 -- 172.21.15.88:0/3521404928 reap_dead start 2024-09-21T07:49:22.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc255e00640 1 -- 172.21.15.88:0/3521404928 shutdown_connections 2024-09-21T07:49:22.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc255e00640 1 -- 172.21.15.88:0/3521404928 >> 172.21.15.88:0/3521404928 conn(0x7fc25006d730 msgr2=0x7fc2500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc255e00640 1 -- 172.21.15.88:0/3521404928 shutdown_connections 2024-09-21T07:49:22.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.012+0000 7fc255e00640 1 -- 172.21.15.88:0/3521404928 wait complete. 2024-09-21T07:49:22.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.014+0000 7fc255e00640 1 Processor -- start 2024-09-21T07:49:22.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.014+0000 7fc255e00640 1 -- start start 2024-09-21T07:49:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc255e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc234004d40 0x7fc250073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc255e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2501f4d20 0x7fc25014d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc255e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 0x7fc2501f5be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc255e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc25006f220 con 0x7fc2501f57d0 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc255e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc25006f0a0 con 0x7fc2501f4d20 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc255e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2500b4c20 con 0x7fc234004d40 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc24f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 0x7fc2501f5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc24f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 0x7fc2501f5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50392/0 (socket says 172.21.15.88:50392) 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc24f400640 1 -- 172.21.15.88:0/4008359739 learned_addr learned my addr 172.21.15.88:0/4008359739 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc24e000640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2501f4d20 0x7fc25014d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.016+0000 7fc24ea00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc234004d40 0x7fc250073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24f400640 1 -- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc234004d40 msgr2=0x7fc250073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24f400640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc234004d40 0x7fc250073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24f400640 1 -- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2501f4d20 msgr2=0x7fc25014d220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24f400640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2501f4d20 0x7fc25014d220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24f400640 1 -- 172.21.15.88:0/4008359739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2501f8d00 con 0x7fc2501f57d0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24ea00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc234004d40 0x7fc250073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc24f400640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 0x7fc2501f5be0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc24802b2d0 tx=0x7fc248006e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248006590 con 0x7fc2501f57d0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc24802b710 con 0x7fc2501f57d0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24802bb60 con 0x7fc2501f57d0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2501f8fe0 con 0x7fc2501f57d0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.017+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2501f95a0 con 0x7fc2501f57d0 2024-09-21T07:49:22.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.019+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc250160f50 con 0x7fc2501f57d0 2024-09-21T07:49:22.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.020+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fc24802bd00 con 0x7fc2501f57d0 2024-09-21T07:49:22.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.021+0000 7fc247400640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc230078d70 0x7fc23007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.021+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7fc2480ba7d0 con 0x7fc2501f57d0 2024-09-21T07:49:22.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.021+0000 7fc24ea00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc230078d70 0x7fc23007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.022+0000 7fc24ea00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc230078d70 0x7fc23007b280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc240005fd0 tx=0x7fc240009500 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.025+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc2500ada30 con 0x7fc2501f57d0 2024-09-21T07:49:22.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.211+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc2501f8a20 con 0x7fc2501f57d0 2024-09-21T07:49:22.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.212+0000 7fc247400640 1 -- 172.21.15.88:0/4008359739 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v52) ==== 74+0+13042 (secure 0 0 0) 0x7fc248082490 con 0x7fc2501f57d0 2024-09-21T07:49:22.214 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:22.214 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":52,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:49:20.760829+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/909378044":"2024-09-22T07:49:17.657866+0000","172.21.15.88:6833/909378044":"2024-09-22T07:49:17.657866+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc230078d70 msgr2=0x7fc23007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc230078d70 0x7fc23007b280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc240005fd0 tx=0x7fc240009500 comp rx=0 tx=0).stop 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 msgr2=0x7fc2501f5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 0x7fc2501f5be0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7fc24802b2d0 tx=0x7fc248006e20 comp rx=0 tx=0).stop 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 shutdown_connections 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc230078d70 0x7fc23007b280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2501f57d0 0x7fc2501f5be0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc2501f4d20 0x7fc25014d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 --2- 172.21.15.88:0/4008359739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc234004d40 0x7fc250073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.214+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 >> 172.21.15.88:0/4008359739 conn(0x7fc25006d730 msgr2=0x7fc25015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.215+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 shutdown_connections 2024-09-21T07:49:22.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.215+0000 7fc255e00640 1 -- 172.21.15.88:0/4008359739 wait complete. 2024-09-21T07:49:22.240 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:49:22.240 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:49:22.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 -- 172.21.15.88:0/343781822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 msgr2=0x7f5168073870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 --2- 172.21.15.88:0/343781822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f5168073870 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f5150009860 tx=0x7f515002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 -- 172.21.15.88:0/343781822 shutdown_connections 2024-09-21T07:49:22.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 --2- 172.21.15.88:0/343781822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f5168073870 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 -- 172.21.15.88:0/343781822 >> 172.21.15.88:0/343781822 conn(0x7f516806d710 msgr2=0x7f51680b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 -- 172.21.15.88:0/343781822 shutdown_connections 2024-09-21T07:49:22.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.463+0000 7f516f600640 1 -- 172.21.15.88:0/343781822 wait complete. 2024-09-21T07:49:22.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.465+0000 7f516f600640 1 Processor -- start 2024-09-21T07:49:22.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.465+0000 7f516f600640 1 -- start start 2024-09-21T07:49:22.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f51681f6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51680b5000 0x7f51680cf0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516f600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 0x7f51681f5110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516f600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f51680b4670 con 0x7f5168155090 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f51680b44f0 con 0x7f51680b5000 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5168155bb0 con 0x7f5168073460 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 0x7f51681f5110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 0x7f51681f5110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50402/0 (socket says 172.21.15.88:50402) 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 -- 172.21.15.88:0/1859495191 learned_addr learned my addr 172.21.15.88:0/1859495191 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f515fe00640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51680b5000 0x7f51680cf0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516cc00640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f51681f6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 -- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 msgr2=0x7f51681f6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f51681f6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 -- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51680b5000 msgr2=0x7f51680cf0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51680b5000 0x7f51680cf0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516d600640 1 -- 172.21.15.88:0/1859495191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51681f9180 con 0x7f5168155090 2024-09-21T07:49:22.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.467+0000 7f516cc00640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f51681f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:22.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.468+0000 7f516d600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 0x7f51681f5110 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f516000cfb0 tx=0x7f516000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.468+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160016020 con 0x7f5168155090 2024-09-21T07:49:22.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.468+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5160012420 con 0x7f5168155090 2024-09-21T07:49:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.468+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5160012870 con 0x7f5168155090 2024-09-21T07:49:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.468+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51681f9460 con 0x7f5168155090 2024-09-21T07:49:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.468+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51681f9a20 con 0x7f5168155090 2024-09-21T07:49:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.469+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51680ad830 con 0x7f5168155090 2024-09-21T07:49:22.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.470+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f5160019020 con 0x7f5168155090 2024-09-21T07:49:22.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.471+0000 7f5165600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5144078d70 0x7f514407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.472+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7f516009b450 con 0x7f5168155090 2024-09-21T07:49:22.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.472+0000 7f516cc00640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5144078d70 0x7f514407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.473+0000 7f516cc00640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5144078d70 0x7f514407b280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5158007fd0 tx=0x7f5158009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.476+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f51680ad830 con 0x7f5168155090 2024-09-21T07:49:22.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.662+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5168073870 con 0x7f5168155090 2024-09-21T07:49:22.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.662+0000 7f5165600640 1 -- 172.21.15.88:0/1859495191 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v52) ==== 93+0+10 (secure 0 0 0) 0x7f5160020020 con 0x7f5168155090 2024-09-21T07:49:22.665 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:49:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5144078d70 msgr2=0x7f514407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5144078d70 0x7f514407b280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5158007fd0 tx=0x7f5158009040 comp rx=0 tx=0).stop 2024-09-21T07:49:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 msgr2=0x7f51681f5110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 0x7f51681f5110 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f516000cfb0 tx=0x7f516000a430 comp rx=0 tx=0).stop 2024-09-21T07:49:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 shutdown_connections 2024-09-21T07:49:22.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5144078d70 0x7f514407b280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5168155090 0x7f51681f5110 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f51680b5000 0x7f51680cf0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 --2- 172.21.15.88:0/1859495191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5168073460 0x7f51681f6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 >> 172.21.15.88:0/1859495191 conn(0x7f516806d710 msgr2=0x7f51681554a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 shutdown_connections 2024-09-21T07:49:22.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.665+0000 7f516f600640 1 -- 172.21.15.88:0/1859495191 wait complete. 2024-09-21T07:49:22.689 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 -- 172.21.15.88:0/4065698751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 msgr2=0x7fd8040ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 --2- 172.21.15.88:0/4065698751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd8040ad4f0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fd804075250 tx=0x7fd7f002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 -- 172.21.15.88:0/4065698751 shutdown_connections 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 --2- 172.21.15.88:0/4065698751 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd804160310 0x7fd804160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 --2- 172.21.15.88:0/4065698751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd8040ad4f0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 --2- 172.21.15.88:0/4065698751 >> v2:172.21.15.175:3300/0 conn(0x7fd80415d760 0x7fd8041535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.908+0000 7fd808c00640 1 -- 172.21.15.88:0/4065698751 >> 172.21.15.88:0/4065698751 conn(0x7fd80406d730 msgr2=0x7fd8040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:22.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.909+0000 7fd808c00640 1 -- 172.21.15.88:0/4065698751 shutdown_connections 2024-09-21T07:49:22.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.909+0000 7fd808c00640 1 -- 172.21.15.88:0/4065698751 wait complete. 2024-09-21T07:49:22.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.911+0000 7fd808c00640 1 Processor -- start 2024-09-21T07:49:22.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.911+0000 7fd808c00640 1 -- start start 2024-09-21T07:49:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd808c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd804153d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd808c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80415d760 0x7fd80414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd808c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd804160310 0x7fd8041f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd808c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040b4da0 con 0x7fd8040ad0e0 2024-09-21T07:49:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd808c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd8040b4c20 con 0x7fd80415d760 2024-09-21T07:49:22.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd808c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd804160c60 con 0x7fd804160310 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd804153d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd804153d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50416/0 (socket says 172.21.15.88:50416) 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 -- 172.21.15.88:0/2975640588 learned_addr learned my addr 172.21.15.88:0/2975640588 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd802800640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd804160310 0x7fd8041f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801400640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80415d760 0x7fd80414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 -- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd804160310 msgr2=0x7fd8041f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd804160310 0x7fd8041f5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 -- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80415d760 msgr2=0x7fd80414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80415d760 0x7fd80414f170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801e00640 1 -- 172.21.15.88:0/2975640588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd804154290 con 0x7fd8040ad0e0 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.913+0000 7fd801400640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80415d760 0x7fd80414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.914+0000 7fd801e00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd804153d50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400d740 tx=0x7fd7f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.914+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4017070 con 0x7fd8040ad0e0 2024-09-21T07:49:22.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.914+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd7f4009cc0 con 0x7fd8040ad0e0 2024-09-21T07:49:22.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.914+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd7f40106c0 con 0x7fd8040ad0e0 2024-09-21T07:49:22.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.914+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8041f55b0 con 0x7fd8040ad0e0 2024-09-21T07:49:22.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.914+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8041f9690 con 0x7fd8040ad0e0 2024-09-21T07:49:22.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.915+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8041535c0 con 0x7fd8040ad0e0 2024-09-21T07:49:22.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.916+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fd7f401a070 con 0x7fd8040ad0e0 2024-09-21T07:49:22.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.917+0000 7fd7fa000640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd7e4078d40 0x7fd7e407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:22.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.918+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7fd7f409afe0 con 0x7fd8040ad0e0 2024-09-21T07:49:22.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.918+0000 7fd801400640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd7e4078d40 0x7fd7e407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:22.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.919+0000 7fd801400640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd7e4078d40 0x7fd7e407b250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0033000 tx=0x7fd7f0004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:22.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:22.922+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd7f40633d0 con 0x7fd8040ad0e0 2024-09-21T07:49:23.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.183+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fd80415e4b0 con 0x7fd8040ad0e0 2024-09-21T07:49:23.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.776+0000 7fd7fa000640 1 -- 172.21.15.88:0/2975640588 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v190) ==== 164+0+0 (secure 0 0 0) 0x7fd7f4062cd0 con 0x7fd8040ad0e0 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.779+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd7e4078d40 msgr2=0x7fd7e407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.779+0000 7fd808c00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd7e4078d40 0x7fd7e407b250 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd7f0033000 tx=0x7fd7f0004730 comp rx=0 tx=0).stop 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.779+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 msgr2=0x7fd804153d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.779+0000 7fd808c00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd804153d50 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7fd7f400d740 tx=0x7fd7f400dc00 comp rx=0 tx=0).stop 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 shutdown_connections 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd7e4078d40 0x7fd7e407b250 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd804160310 0x7fd8041f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd80415d760 0x7fd80414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 --2- 172.21.15.88:0/2975640588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8040ad0e0 0x7fd804153d50 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:23.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 >> 172.21.15.88:0/2975640588 conn(0x7fd80406d730 msgr2=0x7fd804152af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:23.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 shutdown_connections 2024-09-21T07:49:23.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:23.780+0000 7fd808c00640 1 -- 172.21.15.88:0/2975640588 wait complete. 2024-09-21T07:49:23.805 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 -- 172.21.15.88:0/2077271181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3728160310 msgr2=0x7f3728160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 --2- 172.21.15.88:0/2077271181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3728160310 0x7f3728160720 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f3728075440 tx=0x7f372002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 -- 172.21.15.88:0/2077271181 shutdown_connections 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 --2- 172.21.15.88:0/2077271181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3728160310 0x7f3728160720 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 --2- 172.21.15.88:0/2077271181 >> v2:172.21.15.175:3300/0 conn(0x7f37280ad0e0 0x7f37280ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 --2- 172.21.15.88:0/2077271181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f372815d760 0x7f37281535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 -- 172.21.15.88:0/2077271181 >> 172.21.15.88:0/2077271181 conn(0x7f372806d730 msgr2=0x7f37280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.022+0000 7f372fe00640 1 -- 172.21.15.88:0/2077271181 shutdown_connections 2024-09-21T07:49:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.023+0000 7f372fe00640 1 -- 172.21.15.88:0/2077271181 wait complete. 2024-09-21T07:49:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.025+0000 7f372fe00640 1 Processor -- start 2024-09-21T07:49:24.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.025+0000 7f372fe00640 1 -- start start 2024-09-21T07:49:24.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 0x7f3728154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372815d760 0x7f3728151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3728160310 0x7f37281595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37280b4da0 con 0x7f37280ad0e0 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37280b4c20 con 0x7f372815d760 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3728160c60 con 0x7f3728160310 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 0x7f3728154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 0x7f3728154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50428/0 (socket says 172.21.15.88:50428) 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.026+0000 7f372d400640 1 -- 172.21.15.88:0/1000245455 learned_addr learned my addr 172.21.15.88:0/1000245455 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372ca00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372815d760 0x7f3728151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372de00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3728160310 0x7f37281595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372d400640 1 -- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3728160310 msgr2=0x7f37281595b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372d400640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3728160310 0x7f37281595b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372d400640 1 -- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372815d760 msgr2=0x7f3728151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372d400640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372815d760 0x7f3728151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372d400640 1 -- 172.21.15.88:0/1000245455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3728151ec0 con 0x7f37280ad0e0 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372ca00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372815d760 0x7f3728151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372d400640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 0x7f3728154700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f371c00d700 tx=0x7f371c00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c017070 con 0x7f37280ad0e0 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f371c005310 con 0x7f37280ad0e0 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f371c0097c0 con 0x7f37280ad0e0 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.027+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3728154f30 con 0x7f37280ad0e0 2024-09-21T07:49:24.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.028+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f372809c230 con 0x7f37280ad0e0 2024-09-21T07:49:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.029+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37281535c0 con 0x7f37280ad0e0 2024-09-21T07:49:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.030+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f371c009960 con 0x7f37280ad0e0 2024-09-21T07:49:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.031+0000 7f3725600640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3708078d70 0x7f370807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.031+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7f371c09af50 con 0x7f37280ad0e0 2024-09-21T07:49:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.031+0000 7f372ca00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3708078d70 0x7f370807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.032+0000 7f372ca00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3708078d70 0x7f370807b280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3728157390 tx=0x7f3714005f10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.035+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f371c063260 con 0x7f37280ad0e0 2024-09-21T07:49:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.220+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f37280702c0 con 0x7f37280ad0e0 2024-09-21T07:49:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.221+0000 7f3725600640 1 -- 172.21.15.88:0/1000245455 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v35) ==== 59+0+2294 (secure 0 0 0) 0x7f371c062b60 con 0x7f37280ad0e0 2024-09-21T07:49:24.223 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:49:24.223 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:49:24.224 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:49:24.225 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:49:24.226 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:49:24.226 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:49:24.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3708078d70 msgr2=0x7f370807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3708078d70 0x7f370807b280 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f3728157390 tx=0x7f3714005f10 comp rx=0 tx=0).stop 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 msgr2=0x7f3728154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 0x7f3728154700 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f371c00d700 tx=0x7f371c00dbc0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 shutdown_connections 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3708078d70 0x7f370807b280 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3728160310 0x7f37281595b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f372815d760 0x7f3728151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 --2- 172.21.15.88:0/1000245455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37280ad0e0 0x7f3728154700 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 >> 172.21.15.88:0/1000245455 conn(0x7f372806d730 msgr2=0x7f37281527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 shutdown_connections 2024-09-21T07:49:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.223+0000 7f372fe00640 1 -- 172.21.15.88:0/1000245455 wait complete. 2024-09-21T07:49:24.252 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.470+0000 7f4705200640 1 -- 172.21.15.88:0/3486999746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700073c50 msgr2=0x7f47001674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.470+0000 7f4705200640 1 --2- 172.21.15.88:0/3486999746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700073c50 0x7f47001674a0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f46f8009830 tx=0x7f46f802a640 comp rx=0 tx=0).stop 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.470+0000 7f4705200640 1 -- 172.21.15.88:0/3486999746 shutdown_connections 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.470+0000 7f4705200640 1 --2- 172.21.15.88:0/3486999746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4700073c50 0x7f47001674a0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.470+0000 7f4705200640 1 -- 172.21.15.88:0/3486999746 >> 172.21.15.88:0/3486999746 conn(0x7f470006d1c0 msgr2=0x7f470006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.470+0000 7f4705200640 1 -- 172.21.15.88:0/3486999746 shutdown_connections 2024-09-21T07:49:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.471+0000 7f4705200640 1 -- 172.21.15.88:0/3486999746 wait complete. 2024-09-21T07:49:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.473+0000 7f4705200640 1 Processor -- start 2024-09-21T07:49:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.473+0000 7f4705200640 1 -- start start 2024-09-21T07:49:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f4705200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4700145550 0x7f47000b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f4705200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4700145a40 0x7f4700140e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f4705200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 0x7f47001413d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f4705200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4700167d70 con 0x7f47000b4380 2024-09-21T07:49:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f4705200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4700167bf0 con 0x7f4700145550 2024-09-21T07:49:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f4705200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4700167ef0 con 0x7f4700145a40 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.474+0000 7f46fe800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 0x7f47001413d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 0x7f47001413d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50454/0 (socket says 172.21.15.88:50454) 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 -- 172.21.15.88:0/555490639 learned_addr learned my addr 172.21.15.88:0/555490639 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46f5400640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4700145a40 0x7f4700140e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fde00640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4700145550 0x7f47000b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 -- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4700145a40 msgr2=0x7f4700140e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4700145a40 0x7f4700140e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 -- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4700145550 msgr2=0x7f47000b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4700145550 0x7f47000b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 -- 172.21.15.88:0/555490639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f470015c560 con 0x7f47000b4380 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46f5400640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4700145a40 0x7f4700140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.475+0000 7f46fe800640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 0x7f47001413d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46f802ac30 tx=0x7f46f8033df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.476+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f80332e0 con 0x7f47000b4380 2024-09-21T07:49:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.476+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f46f8033570 con 0x7f47000b4380 2024-09-21T07:49:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.476+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47000b2270 con 0x7f47000b4380 2024-09-21T07:49:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.476+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f46f8005260 con 0x7f47000b4380 2024-09-21T07:49:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.476+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47000b2830 con 0x7f47000b4380 2024-09-21T07:49:24.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.477+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47000cf310 con 0x7f47000b4380 2024-09-21T07:49:24.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.478+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f46f8005400 con 0x7f47000b4380 2024-09-21T07:49:24.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.479+0000 7f46f6a00640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46dc078d40 0x7f46dc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:24.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.480+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7f46f80bbef0 con 0x7f47000b4380 2024-09-21T07:49:24.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.480+0000 7f46fde00640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46dc078d40 0x7f46dc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:24.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.481+0000 7f46fde00640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46dc078d40 0x7f46dc07b250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46f0000f60 tx=0x7f46f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:24.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.484+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f47000cf310 con 0x7f47000b4380 2024-09-21T07:49:24.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.669+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f470006fb40 con 0x7f47000b4380 2024-09-21T07:49:24.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.674+0000 7f46f6a00640 1 -- 172.21.15.88:0/555490639 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f46f803f030 con 0x7f47000b4380 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46dc078d40 msgr2=0x7f46dc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46dc078d40 0x7f46dc07b250 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f46f0000f60 tx=0x7f46f0009040 comp rx=0 tx=0).stop 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 msgr2=0x7f47001413d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 0x7f47001413d0 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f46f802ac30 tx=0x7f46f8033df0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 shutdown_connections 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f46dc078d40 0x7f46dc07b250 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4700145a40 0x7f4700140e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47000b4380 0x7f47001413d0 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 --2- 172.21.15.88:0/555490639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4700145550 0x7f47000b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:24.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.676+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 >> 172.21.15.88:0/555490639 conn(0x7f470006d1c0 msgr2=0x7f470013a1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:24.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.677+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 shutdown_connections 2024-09-21T07:49:24.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:24.677+0000 7f4705200640 1 -- 172.21.15.88:0/555490639 wait complete. 2024-09-21T07:49:24.700 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:49:24.700 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:24.794 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:49:24.895 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:49:24.895 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:49:24.895 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:24.932 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:24.932 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:49:24.932 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:24.969 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:24.969 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:49:24.969 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:25.007 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:49:25.007 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:49:25.007 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:25.045 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:49:25.046 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:49:25.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:25.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 -- 172.21.15.88:0/2077830053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc164004800 msgr2=0x7fc164004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 --2- 172.21.15.88:0/2077830053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc164004800 0x7fc164004c10 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc16800c2e0 tx=0x7fc164005830 comp rx=0 tx=0).stop 2024-09-21T07:49:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 -- 172.21.15.88:0/2077830053 shutdown_connections 2024-09-21T07:49:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 --2- 172.21.15.88:0/2077830053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc164004800 0x7fc164004c10 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 -- 172.21.15.88:0/2077830053 >> 172.21.15.88:0/2077830053 conn(0x7fc1800b1e40 msgr2=0x7fc1800b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 -- 172.21.15.88:0/2077830053 shutdown_connections 2024-09-21T07:49:25.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.265+0000 7fc185c00640 1 -- 172.21.15.88:0/2077830053 wait complete. 2024-09-21T07:49:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.267+0000 7fc185c00640 1 Processor -- start 2024-09-21T07:49:25.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.267+0000 7fc185c00640 1 -- start start 2024-09-21T07:49:25.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc164004800 0x7fc1801f3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1801f3670 0x7fc1800cd9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 0x7fc1800cdf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800af500 con 0x7fc1801f3ff0 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc1800af380 con 0x7fc1801f3670 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc18015d670 con 0x7fc164004800 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 0x7fc1800cdf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 0x7fc1800cdf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50476/0 (socket says 172.21.15.88:50476) 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc185200640 1 -- 172.21.15.88:0/2326276015 learned_addr learned my addr 172.21.15.88:0/2326276015 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc177400640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1801f3670 0x7fc1800cd9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.269+0000 7fc17fe00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc164004800 0x7fc1801f3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185200640 1 -- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc164004800 msgr2=0x7fc1801f3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185200640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc164004800 0x7fc1801f3130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185200640 1 -- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1801f3670 msgr2=0x7fc1800cd9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185200640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1801f3670 0x7fc1800cd9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185200640 1 -- 172.21.15.88:0/2326276015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1800ccc60 con 0x7fc1801f3ff0 2024-09-21T07:49:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc177400640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1801f3670 0x7fc1800cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185200640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 0x7fc1800cdf20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1800cfe50 tx=0x7fc17800e6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178011870 con 0x7fc1801f3ff0 2024-09-21T07:49:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc178018070 con 0x7fc1801f3ff0 2024-09-21T07:49:25.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1800ccf00 con 0x7fc1801f3ff0 2024-09-21T07:49:25.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc178011dc0 con 0x7fc1801f3ff0 2024-09-21T07:49:25.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.270+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1801f44a0 con 0x7fc1801f3ff0 2024-09-21T07:49:25.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.271+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc1800cf6e0 con 0x7fc1801f3ff0 2024-09-21T07:49:25.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.272+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fc17801d460 con 0x7fc1801f3ff0 2024-09-21T07:49:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.273+0000 7fc17cc00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc15c078d70 0x7fc15c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.273+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7fc17809b270 con 0x7fc1801f3ff0 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.274+0000 7fc17fe00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc15c078d70 0x7fc15c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.274+0000 7fc17fe00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc15c078d70 0x7fc15c07b280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc170004700 tx=0x7fc170005aa0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.278+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc178063650 con 0x7fc1801f3ff0 2024-09-21T07:49:25.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.499+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc1800b2610 con 0x7fc1801f3ff0 2024-09-21T07:49:25.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.500+0000 7fc17cc00640 1 -- 172.21.15.88:0/2326276015 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+3917 (secure 0 0 0) 0x7fc1800b2610 con 0x7fc1801f3ff0 2024-09-21T07:49:25.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:25.502 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":29,"btime":"2024-09-21T07:49:18:744600+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4973,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2149521460","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2149521460},{"type":"v1","addr":"172.21.15.175:6837","nonce":2149521460}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5278,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/3650207225","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3650207225},{"type":"v1","addr":"172.21.15.175:6836","nonce":3650207225}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":22},{"gid":5334,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2743589105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2743589105},{"type":"v1","addr":"172.21.15.88:6834","nonce":2743589105}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":21},{"gid":5664,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3934648909","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3934648909},{"type":"v1","addr":"172.21.15.88:6835","nonce":3934648909}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":29}],"filesystems":[]} 2024-09-21T07:49:25.503 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 29 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.502+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc15c078d70 msgr2=0x7fc15c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.502+0000 7fc185c00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc15c078d70 0x7fc15c07b280 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fc170004700 tx=0x7fc170005aa0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.502+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 msgr2=0x7fc1800cdf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.502+0000 7fc185c00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 0x7fc1800cdf20 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1800cfe50 tx=0x7fc17800e6b0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 shutdown_connections 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc15c078d70 0x7fc15c07b280 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc1801f3ff0 0x7fc1800cdf20 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc1801f3670 0x7fc1800cd9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 --2- 172.21.15.88:0/2326276015 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc164004800 0x7fc1801f3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 >> 172.21.15.88:0/2326276015 conn(0x7fc1800b1e40 msgr2=0x7fc1800b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 shutdown_connections 2024-09-21T07:49:25.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.503+0000 7fc185c00640 1 -- 172.21.15.88:0/2326276015 wait complete. 2024-09-21T07:49:25.530 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:49:25.530 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:49:25.531 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:49:25.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:25.531 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:49:25.531 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:49:25.531 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:49:25.531 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:25.531 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:49:25.531 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:49:25.532 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:49:25.532 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:25.532 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:49:25.532 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:49:25.532 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:49:25.532 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:49:25.555 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:49:25.555 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:49:25.557 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:49:25.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:49:25.568 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:49:25.568 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:49:25.569 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:49:25.569 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:49:25.600 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:49:25.601 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:25.603 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:49:25.604 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:25.613 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:49:25.613 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:25.615 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:49:25.616 DEBUG:teuthology.parallel:result is None 2024-09-21T07:49:25.616 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:49:25.795 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:49:25.797 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:49:25.812 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:49:25.813 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 -- 172.21.15.88:0/4104077042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500b4d60 msgr2=0x7f795006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 --2- 172.21.15.88:0/4104077042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500b4d60 0x7f795006ddb0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f793c009830 tx=0x7f793c02a510 comp rx=0 tx=0).stop 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 -- 172.21.15.88:0/4104077042 shutdown_connections 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 --2- 172.21.15.88:0/4104077042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f795006e2f0 0x7f795006f9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 --2- 172.21.15.88:0/4104077042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500b4d60 0x7f795006ddb0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 -- 172.21.15.88:0/4104077042 >> 172.21.15.88:0/4104077042 conn(0x7f7950073940 msgr2=0x7f7950073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 -- 172.21.15.88:0/4104077042 shutdown_connections 2024-09-21T07:49:25.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.847+0000 7f7955c00640 1 -- 172.21.15.88:0/4104077042 wait complete. 2024-09-21T07:49:25.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.850+0000 7f7955c00640 1 Processor -- start 2024-09-21T07:49:25.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.850+0000 7f7955c00640 1 -- start start 2024-09-21T07:49:25.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f7955c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f795006e2f0 0x7f795014c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f7955c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 0x7f7950158790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f7955c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79500b4d60 0x7f7950158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f7955c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7950070460 con 0x7f79500749a0 2024-09-21T07:49:25.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f7955c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79500702e0 con 0x7f79500b4d60 2024-09-21T07:49:25.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f7955c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79500705e0 con 0x7f795006e2f0 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f794e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 0x7f7950158790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f794e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 0x7f7950158790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55312/0 (socket says 172.21.15.88:55312) 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.851+0000 7f794e000640 1 -- 172.21.15.88:0/3460084989 learned_addr learned my addr 172.21.15.88:0/3460084989 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794ea00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f795006e2f0 0x7f795014c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794f400640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79500b4d60 0x7f7950158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794e000640 1 -- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f795006e2f0 msgr2=0x7f795014c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794e000640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f795006e2f0 0x7f795014c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794e000640 1 -- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79500b4d60 msgr2=0x7f7950158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794e000640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79500b4d60 0x7f7950158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794e000640 1 -- 172.21.15.88:0/3460084989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f795014e930 con 0x7f79500749a0 2024-09-21T07:49:25.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.852+0000 7f794e000640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 0x7f7950158790 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f793c02ab00 tx=0x7f793c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.853+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c005c90 con 0x7f79500749a0 2024-09-21T07:49:25.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.853+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f793c031e50 con 0x7f79500749a0 2024-09-21T07:49:25.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.853+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f793c005260 con 0x7f79500749a0 2024-09-21T07:49:25.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.853+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f795014e090 con 0x7f79500749a0 2024-09-21T07:49:25.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.853+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f795014e650 con 0x7f79500749a0 2024-09-21T07:49:25.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.854+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7950070a90 con 0x7f79500749a0 2024-09-21T07:49:25.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.855+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f793c005400 con 0x7f79500749a0 2024-09-21T07:49:25.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.856+0000 7f7946a00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7930078d40 0x7f793007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:25.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.856+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(52..52 src has 1..52) ==== 4943+0+0 (secure 0 0 0) 0x7f793c0bbdd0 con 0x7f79500749a0 2024-09-21T07:49:25.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.857+0000 7f794ea00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7930078d40 0x7f793007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:25.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.857+0000 7f794ea00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7930078d40 0x7f793007b250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7940007d60 tx=0x7f7940007cf0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:25.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:25.860+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f793c0c0070 con 0x7f79500749a0 2024-09-21T07:49:26.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.062+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f79500713f0 con 0x7f79500749a0 2024-09-21T07:49:26.440 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:26.449 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:26.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.797+0000 7f7946a00640 1 -- 172.21.15.88:0/3460084989 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v54) ==== 97+0+0 (secure 0 0 0) 0x7f793c03f030 con 0x7f79500749a0 2024-09-21T07:49:26.799 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7930078d40 msgr2=0x7f793007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7930078d40 0x7f793007b250 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f7940007d60 tx=0x7f7940007cf0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 msgr2=0x7f7950158790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 0x7f7950158790 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f793c02ab00 tx=0x7f793c033c20 comp rx=0 tx=0).stop 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 shutdown_connections 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7930078d40 0x7f793007b250 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79500b4d60 0x7f7950158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79500749a0 0x7f7950158790 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 --2- 172.21.15.88:0/3460084989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f795006e2f0 0x7f795014c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.800+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 >> 172.21.15.88:0/3460084989 conn(0x7f7950073940 msgr2=0x7f79500b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.801+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 shutdown_connections 2024-09-21T07:49:26.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:26.801+0000 7f7955c00640 1 -- 172.21.15.88:0/3460084989 wait complete. 2024-09-21T07:49:26.828 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 -- 172.21.15.88:0/685581854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 msgr2=0x7fb6d8164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 --2- 172.21.15.88:0/685581854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d8164af0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0005fd0 tx=0x7fb6d002aa10 comp rx=0 tx=0).stop 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 -- 172.21.15.88:0/685581854 shutdown_connections 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 --2- 172.21.15.88:0/685581854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d8164af0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 --2- 172.21.15.88:0/685581854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6d80704b0 0x7fb6d8073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 -- 172.21.15.88:0/685581854 >> 172.21.15.88:0/685581854 conn(0x7fb6d806d730 msgr2=0x7fb6d80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.046+0000 7fb6dd200640 1 -- 172.21.15.88:0/685581854 shutdown_connections 2024-09-21T07:49:27.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.047+0000 7fb6dd200640 1 -- 172.21.15.88:0/685581854 wait complete. 2024-09-21T07:49:27.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.049+0000 7fb6dd200640 1 Processor -- start 2024-09-21T07:49:27.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.049+0000 7fb6dd200640 1 -- start start 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6dd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6d80704b0 0x7fb6d814eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6dd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6d80ad3c0 0x7fb6d80ae420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6dd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d80ae960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6dd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d80ae060 con 0x7fb6d81626d0 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6dd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6d80adee0 con 0x7fb6d80ad3c0 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6dd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6d8071130 con 0x7fb6d80704b0 2024-09-21T07:49:27.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.050+0000 7fb6d6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d80ae960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d80ae960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55316/0 (socket says 172.21.15.88:55316) 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 -- 172.21.15.88:0/2419537632 learned_addr learned my addr 172.21.15.88:0/2419537632 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d5e00640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6d80704b0 0x7fb6d814eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d5400640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6d80ad3c0 0x7fb6d80ae420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 -- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6d80704b0 msgr2=0x7fb6d814eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6d80704b0 0x7fb6d814eb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 -- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6d80ad3c0 msgr2=0x7fb6d80ae420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6d80ad3c0 0x7fb6d80ae420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 -- 172.21.15.88:0/2419537632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6d81504e0 con 0x7fb6d81626d0 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d5400640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6d80ad3c0 0x7fb6d80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:27.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6d6800640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d80ae960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006010 tx=0x7fb6d0009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.051+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0046070 con 0x7fb6d81626d0 2024-09-21T07:49:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.052+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb6d0004540 con 0x7fb6d81626d0 2024-09-21T07:49:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.052+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6d81f9430 con 0x7fb6d81626d0 2024-09-21T07:49:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.052+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6d0033070 con 0x7fb6d81626d0 2024-09-21T07:49:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.052+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6d81f99f0 con 0x7fb6d81626d0 2024-09-21T07:49:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.053+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6d8073040 con 0x7fb6d81626d0 2024-09-21T07:49:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.054+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fb6d0049070 con 0x7fb6d81626d0 2024-09-21T07:49:27.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.055+0000 7fb6ce000640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb6b8078d40 0x7fb6b807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.055+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7fb6d00bbe90 con 0x7fb6d81626d0 2024-09-21T07:49:27.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.055+0000 7fb6d5e00640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb6b8078d40 0x7fb6b807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.056+0000 7fb6d5e00640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb6b8078d40 0x7fb6b807b250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009b30 tx=0x7fb6c8005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.059+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6d81504e0 con 0x7fb6d81626d0 2024-09-21T07:49:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.313+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fb6d81f91c0 con 0x7fb6d81626d0 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.314+0000 7fb6ce000640 1 -- 172.21.15.88:0/2419537632 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v10) ==== 80+0+5304 (secure 0 0 0) 0x7fb6d003e070 con 0x7fb6d81626d0 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.316 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:49:27.318 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.320 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.321 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.322 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.323 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.324 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:49:27.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.326 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.327 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.328 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:49:27.329 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb6b8078d40 msgr2=0x7fb6b807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb6b8078d40 0x7fb6b807b250 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fb6c8009b30 tx=0x7fb6c8005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 msgr2=0x7fb6d80ae960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d80ae960 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb6d0006010 tx=0x7fb6d0009360 comp rx=0 tx=0).stop 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 shutdown_connections 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb6b8078d40 0x7fb6b807b250 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6d81626d0 0x7fb6d80ae960 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6d80ad3c0 0x7fb6d80ae420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 --2- 172.21.15.88:0/2419537632 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6d80704b0 0x7fb6d814eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 >> 172.21.15.88:0/2419537632 conn(0x7fb6d806d730 msgr2=0x7fb6d8151ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 shutdown_connections 2024-09-21T07:49:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.316+0000 7fb6dd200640 1 -- 172.21.15.88:0/2419537632 wait complete. 2024-09-21T07:49:27.344 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.560+0000 7f9d39a00640 1 -- 172.21.15.88:0/2834940878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340b3550 msgr2=0x7f9d340b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.560+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2834940878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340b3550 0x7f9d340b3960 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f9d34072d60 tx=0x7f9d2c02a820 comp rx=0 tx=0).stop 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.561+0000 7f9d39a00640 1 -- 172.21.15.88:0/2834940878 shutdown_connections 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.561+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2834940878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340b3550 0x7f9d340b3960 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.561+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2834940878 >> v2:172.21.15.175:3300/0 conn(0x7f9d340ad0e0 0x7f9d340ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.561+0000 7f9d39a00640 1 -- 172.21.15.88:0/2834940878 >> 172.21.15.88:0/2834940878 conn(0x7f9d3406d730 msgr2=0x7f9d340b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.561+0000 7f9d39a00640 1 -- 172.21.15.88:0/2834940878 shutdown_connections 2024-09-21T07:49:27.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.561+0000 7f9d39a00640 1 -- 172.21.15.88:0/2834940878 wait complete. 2024-09-21T07:49:27.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.563+0000 7f9d39a00640 1 Processor -- start 2024-09-21T07:49:27.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.563+0000 7f9d39a00640 1 -- start start 2024-09-21T07:49:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.564+0000 7f9d39a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 0x7f9d3409dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.564+0000 7f9d39a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3550 0x7f9d34155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d39a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415d760 0x7f9d341562d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d39a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340b4da0 con 0x7f9d340ad0e0 2024-09-21T07:49:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d39a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d340b4c20 con 0x7f9d3415d760 2024-09-21T07:49:27.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d39a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d340b3ea0 con 0x7f9d340b3550 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 0x7f9d3409dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 0x7f9d3409dc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55334/0 (socket says 172.21.15.88:55334) 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 -- 172.21.15.88:0/2702059321 learned_addr learned my addr 172.21.15.88:0/2702059321 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d33400640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415d760 0x7f9d341562d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d2a000640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3550 0x7f9d34155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 -- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3550 msgr2=0x7f9d34155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3550 0x7f9d34155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 -- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415d760 msgr2=0x7f9d341562d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415d760 0x7f9d341562d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 -- 172.21.15.88:0/2702059321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d3409b910 con 0x7f9d340ad0e0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.565+0000 7f9d32a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 0x7f9d3409dc00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c3c0 tx=0x7f9d2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.566+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24017070 con 0x7f9d340ad0e0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.566+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d240035f0 con 0x7f9d340ad0e0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.566+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d24003a80 con 0x7f9d340ad0e0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.566+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d341f5e00 con 0x7f9d340ad0e0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.566+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d341f63c0 con 0x7f9d340ad0e0 2024-09-21T07:49:27.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.567+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d340b43c0 con 0x7f9d340ad0e0 2024-09-21T07:49:27.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.568+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f9d2401a070 con 0x7f9d340ad0e0 2024-09-21T07:49:27.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.569+0000 7f9d2b400640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9d10078d40 0x7f9d1007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:27.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.569+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f9d2409a1c0 con 0x7f9d340ad0e0 2024-09-21T07:49:27.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.569+0000 7f9d2a000640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9d10078d40 0x7f9d1007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:27.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.570+0000 7f9d2a000640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9d10078d40 0x7f9d1007b250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d340ae820 tx=0x7f9d1c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:27.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.574+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d340ada30 con 0x7f9d340ad0e0 2024-09-21T07:49:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.760+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f9d34153810 con 0x7f9d340ad0e0 2024-09-21T07:49:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.760+0000 7f9d2b400640 1 -- 172.21.15.88:0/2702059321 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v56) ==== 84+0+65 (secure 0 0 0) 0x7f9d24062520 con 0x7f9d340ad0e0 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:27.763 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:49:27.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9d10078d40 msgr2=0x7f9d1007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9d10078d40 0x7f9d1007b250 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f9d340ae820 tx=0x7f9d1c005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:27.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 msgr2=0x7f9d3409dc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:27.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 0x7f9d3409dc00 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f9d2400c3c0 tx=0x7f9d2400c880 comp rx=0 tx=0).stop 2024-09-21T07:49:27.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 shutdown_connections 2024-09-21T07:49:27.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9d10078d40 0x7f9d1007b250 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d340b3550 0x7f9d34155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d3415d760 0x7f9d341562d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 --2- 172.21.15.88:0/2702059321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d340ad0e0 0x7f9d3409dc00 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 >> 172.21.15.88:0/2702059321 conn(0x7f9d3406d730 msgr2=0x7f9d340b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 shutdown_connections 2024-09-21T07:49:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:27.763+0000 7f9d39a00640 1 -- 172.21.15.88:0/2702059321 wait complete. 2024-09-21T07:49:27.786 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:49:27.787 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:49:28.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 -- 172.21.15.88:0/700874435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f445006d4b0 msgr2=0x7f44500b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 --2- 172.21.15.88:0/700874435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f445006d4b0 0x7f44500b4620 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f443c005fd0 tx=0x7f443c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 -- 172.21.15.88:0/700874435 shutdown_connections 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 --2- 172.21.15.88:0/700874435 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44500af100 0x7f44500af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 --2- 172.21.15.88:0/700874435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f445006d4b0 0x7f44500b4620 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 -- 172.21.15.88:0/700874435 >> 172.21.15.88:0/700874435 conn(0x7f44500b2540 msgr2=0x7f44500d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 -- 172.21.15.88:0/700874435 shutdown_connections 2024-09-21T07:49:28.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.011+0000 7f4456600640 1 -- 172.21.15.88:0/700874435 wait complete. 2024-09-21T07:49:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.014+0000 7f4456600640 1 Processor -- start 2024-09-21T07:49:28.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.014+0000 7f4456600640 1 -- start start 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f4456600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44500af100 0x7f445009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f4456600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44500cc830 0x7f44501f3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f4456600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 0x7f44501f6e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f4456600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44500b36a0 con 0x7f4450153390 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f4456600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f44500b3520 con 0x7f44500af100 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f4456600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f44500b3820 con 0x7f44500cc830 2024-09-21T07:49:28.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f444fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 0x7f44501f6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f444fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 0x7f44501f6e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55342/0 (socket says 172.21.15.88:55342) 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.015+0000 7f444fe00640 1 -- 172.21.15.88:0/3265960895 learned_addr learned my addr 172.21.15.88:0/3265960895 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444f400640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44500af100 0x7f445009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444ea00640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44500cc830 0x7f44501f3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444fe00640 1 -- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44500cc830 msgr2=0x7f44501f3210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444fe00640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44500cc830 0x7f44501f3210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444fe00640 1 -- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44500af100 msgr2=0x7f445009a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444fe00640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44500af100 0x7f445009a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444fe00640 1 -- 172.21.15.88:0/3265960895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44501f7cd0 con 0x7f4450153390 2024-09-21T07:49:28.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f444fe00640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 0x7f44501f6e00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f444800ea40 tx=0x7f444800ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f444800ce40 con 0x7f4450153390 2024-09-21T07:49:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4448004c10 con 0x7f4450153390 2024-09-21T07:49:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4450156d20 con 0x7f4450153390 2024-09-21T07:49:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.016+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4448005020 con 0x7f4450153390 2024-09-21T07:49:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.017+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4450154b40 con 0x7f4450153390 2024-09-21T07:49:28.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.017+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4450157240 con 0x7f4450153390 2024-09-21T07:49:28.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.019+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f44480051c0 con 0x7f4450153390 2024-09-21T07:49:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.020+0000 7f4447400640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4430078d70 0x7f443007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.020+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 4857+0+0 (secure 0 0 0) 0x7f444809bbc0 con 0x7f4450153390 2024-09-21T07:49:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.020+0000 7f444f400640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4430078d70 0x7f443007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:28.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.021+0000 7f444f400640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4430078d70 0x7f443007b280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4440001720 tx=0x7f4440009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:28.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.024+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f44480638d0 con 0x7f4450153390 2024-09-21T07:49:28.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.208+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f44500b3d40 con 0x7f4450153390 2024-09-21T07:49:28.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.807+0000 7f4447400640 1 -- 172.21.15.88:0/3265960895 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v57) ==== 138+0+0 (secure 0 0 0) 0x7f4448068300 con 0x7f4450153390 2024-09-21T07:49:28.809 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:49:28.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4430078d70 msgr2=0x7f443007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4430078d70 0x7f443007b280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f4440001720 tx=0x7f4440009040 comp rx=0 tx=0).stop 2024-09-21T07:49:28.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 msgr2=0x7f44501f6e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:28.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 0x7f44501f6e00 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f444800ea40 tx=0x7f444800ef00 comp rx=0 tx=0).stop 2024-09-21T07:49:28.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 shutdown_connections 2024-09-21T07:49:28.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4430078d70 0x7f443007b280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f44500cc830 0x7f44501f3210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4450153390 0x7f44501f6e00 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 --2- 172.21.15.88:0/3265960895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f44500af100 0x7f445009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 >> 172.21.15.88:0/3265960895 conn(0x7f44500b2540 msgr2=0x7f44500afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 shutdown_connections 2024-09-21T07:49:28.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:28.811+0000 7f4456600640 1 -- 172.21.15.88:0/3265960895 wait complete. 2024-09-21T07:49:28.843 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:49:29.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2007600640 1 -- 172.21.15.88:0/3491324078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 msgr2=0x7f1fe400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2007600640 1 --2- 172.21.15.88:0/3491324078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f1fe400b9c0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1ff8005fd0 tx=0x7f1fe4005830 comp rx=0 tx=0).stop 2024-09-21T07:49:29.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2005600640 1 -- 172.21.15.88:0/3491324078 reap_dead start 2024-09-21T07:49:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2007600640 1 -- 172.21.15.88:0/3491324078 shutdown_connections 2024-09-21T07:49:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2007600640 1 -- 172.21.15.88:0/3491324078 >> 172.21.15.88:0/3491324078 conn(0x7f200006d1c0 msgr2=0x7f200006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2007600640 1 -- 172.21.15.88:0/3491324078 shutdown_connections 2024-09-21T07:49:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.070+0000 7f2007600640 1 -- 172.21.15.88:0/3491324078 wait complete. 2024-09-21T07:49:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.072+0000 7f2007600640 1 Processor -- start 2024-09-21T07:49:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.072+0000 7f2007600640 1 -- start start 2024-09-21T07:49:29.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2007600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f200013dff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2007600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200013bb60 0x7f200013b170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2007600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2000139b10 0x7f2000139f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2007600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20000cd790 con 0x7f1fe4005150 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2007600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20000cd610 con 0x7f200013bb60 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2007600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20000cd910 con 0x7f2000139b10 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f200013dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f200013dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55360/0 (socket says 172.21.15.88:55360) 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 -- 172.21.15.88:0/1047773353 learned_addr learned my addr 172.21.15.88:0/1047773353 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 -- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2000139b10 msgr2=0x7f2000139f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2000139b10 0x7f2000139f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 -- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200013bb60 msgr2=0x7f200013b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200013bb60 0x7f200013b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.074+0000 7f2004c00640 1 -- 172.21.15.88:0/1047773353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f200013a460 con 0x7f1fe4005150 2024-09-21T07:49:29.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.075+0000 7f2004c00640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f200013dff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0011a60 tx=0x7f1ff0011f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:29.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.075+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff000fc40 con 0x7f1fe4005150 2024-09-21T07:49:29.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.075+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ff000fde0 con 0x7f1fe4005150 2024-09-21T07:49:29.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.075+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f200013d230 con 0x7f1fe4005150 2024-09-21T07:49:29.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.075+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ff0004280 con 0x7f1fe4005150 2024-09-21T07:49:29.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.075+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f200013d7f0 con 0x7f1fe4005150 2024-09-21T07:49:29.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.076+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f200006fb40 con 0x7f1fe4005150 2024-09-21T07:49:29.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.078+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f1ff00035f0 con 0x7f1fe4005150 2024-09-21T07:49:29.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.078+0000 7f1ffd600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fdc078d70 0x7f1fdc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:29.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.079+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 5227+0+0 (secure 0 0 0) 0x7f1ff009ba40 con 0x7f1fe4005150 2024-09-21T07:49:29.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.079+0000 7f1ff7e00640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fdc078d70 0x7f1fdc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:29.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.080+0000 7f1ff7e00640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fdc078d70 0x7f1fdc07b280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8001c00 tx=0x7f1fe8001b90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:29.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.084+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ff0063c30 con 0x7f1fe4005150 2024-09-21T07:49:29.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.270+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2000141500 con 0x7f1fe4005150 2024-09-21T07:49:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.810+0000 7f1ffd600640 1 -- 172.21.15.88:0/1047773353 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v58) ==== 170+0+0 (secure 0 0 0) 0x7f1ff0063530 con 0x7f1fe4005150 2024-09-21T07:49:29.813 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:49:29.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.814+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fdc078d70 msgr2=0x7f1fdc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.814+0000 7f2007600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fdc078d70 0x7f1fdc07b280 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f1fe8001c00 tx=0x7f1fe8001b90 comp rx=0 tx=0).stop 2024-09-21T07:49:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.814+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 msgr2=0x7f200013dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.814+0000 7f2007600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f200013dff0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1ff0011a60 tx=0x7f1ff0011f20 comp rx=0 tx=0).stop 2024-09-21T07:49:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 shutdown_connections 2024-09-21T07:49:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1fdc078d70 0x7f1fdc07b280 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2000139b10 0x7f2000139f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f200013bb60 0x7f200013b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 --2- 172.21.15.88:0/1047773353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe4005150 0x7f200013dff0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 >> 172.21.15.88:0/1047773353 conn(0x7f200006d1c0 msgr2=0x7f200015bbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 shutdown_connections 2024-09-21T07:49:29.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:29.815+0000 7f2007600640 1 -- 172.21.15.88:0/1047773353 wait complete. 2024-09-21T07:49:29.854 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 -- 172.21.15.88:0/1185936309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c8073a80 msgr2=0x7f30c806f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 --2- 172.21.15.88:0/1185936309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c8073a80 0x7f30c806f7d0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f30c81600d0 tx=0x7f30c002a980 comp rx=0 tx=0).stop 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 -- 172.21.15.88:0/1185936309 shutdown_connections 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 --2- 172.21.15.88:0/1185936309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c8073a80 0x7f30c806f7d0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 --2- 172.21.15.88:0/1185936309 >> v2:172.21.15.175:3300/0 conn(0x7f30c8147b90 0x7f30c8073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 -- 172.21.15.88:0/1185936309 >> 172.21.15.88:0/1185936309 conn(0x7f30c806d1c0 msgr2=0x7f30c806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 -- 172.21.15.88:0/1185936309 shutdown_connections 2024-09-21T07:49:30.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.075+0000 7f30cfc00640 1 -- 172.21.15.88:0/1185936309 wait complete. 2024-09-21T07:49:30.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.078+0000 7f30cfc00640 1 Processor -- start 2024-09-21T07:49:30.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.078+0000 7f30cfc00640 1 -- start start 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cfc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30c8073a80 0x7f30c81366a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cfc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30c8147b90 0x7f30c81ea340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cfc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 0x7f30c81ea880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cfc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f30c8148470 con 0x7f30c815f9e0 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cfc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30c81482f0 con 0x7f30c8147b90 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cfc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30c806fd10 con 0x7f30c8073a80 2024-09-21T07:49:30.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cdc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 0x7f30c81ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cdc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 0x7f30c81ea880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55384/0 (socket says 172.21.15.88:55384) 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.079+0000 7f30cdc00640 1 -- 172.21.15.88:0/614736465 learned_addr learned my addr 172.21.15.88:0/614736465 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30bfe00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30c8147b90 0x7f30c81ea340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cd200640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30c8073a80 0x7f30c81366a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cdc00640 1 -- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30c8073a80 msgr2=0x7f30c81366a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cdc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30c8073a80 0x7f30c81366a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cdc00640 1 -- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30c8147b90 msgr2=0x7f30c81ea340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cdc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30c8147b90 0x7f30c81ea340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cdc00640 1 -- 172.21.15.88:0/614736465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30c81e9580 con 0x7f30c815f9e0 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cd200640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30c8073a80 0x7f30c81366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:30.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cdc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 0x7f30c81ea880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f30c8070440 tx=0x7f30c0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0034040 con 0x7f30c815f9e0 2024-09-21T07:49:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30c0004500 con 0x7f30c815f9e0 2024-09-21T07:49:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.080+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30c81e9860 con 0x7f30c815f9e0 2024-09-21T07:49:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.081+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c0045480 con 0x7f30c815f9e0 2024-09-21T07:49:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.081+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30c81eadc0 con 0x7f30c815f9e0 2024-09-21T07:49:30.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.082+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30c806f7d0 con 0x7f30c815f9e0 2024-09-21T07:49:30.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.083+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f30c0037020 con 0x7f30c815f9e0 2024-09-21T07:49:30.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.084+0000 7f30c5600640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30a4078d70 0x7f30a407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:30.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.084+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(58..58 src has 1..58) ==== 5609+0+0 (secure 0 0 0) 0x7f30c00bc940 con 0x7f30c815f9e0 2024-09-21T07:49:30.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.086+0000 7f30cd200640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30a4078d70 0x7f30a407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:30.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.089+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30c00849c0 con 0x7f30c815f9e0 2024-09-21T07:49:30.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.090+0000 7f30cd200640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30a4078d70 0x7f30a407b280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000f60 tx=0x7f30b8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.310+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f30c8147950 con 0x7f30c815f9e0 2024-09-21T07:49:30.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.836+0000 7f30c5600640 1 -- 172.21.15.88:0/614736465 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v32) ==== 172+0+0 (secure 0 0 0) 0x7f30c00842c0 con 0x7f30c815f9e0 2024-09-21T07:49:30.839 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 8 and data pool 9 2024-09-21T07:49:30.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.841+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30a4078d70 msgr2=0x7f30a407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.841+0000 7f30cfc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30a4078d70 0x7f30a407b280 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f30b8000f60 tx=0x7f30b8009040 comp rx=0 tx=0).stop 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 msgr2=0x7f30c81ea880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 0x7f30c81ea880 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f30c8070440 tx=0x7f30c0003f90 comp rx=0 tx=0).stop 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 shutdown_connections 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f30a4078d70 0x7f30a407b280 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30c815f9e0 0x7f30c81ea880 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30c8147b90 0x7f30c81ea340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 --2- 172.21.15.88:0/614736465 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30c8073a80 0x7f30c81366a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:30.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.842+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 >> 172.21.15.88:0/614736465 conn(0x7f30c806d1c0 msgr2=0x7f30c8146290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:30.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.843+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 shutdown_connections 2024-09-21T07:49:30.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:30.845+0000 7f30cfc00640 1 -- 172.21.15.88:0/614736465 wait complete. 2024-09-21T07:49:30.886 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:31.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 -- 172.21.15.88:0/1076035166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40b3550 msgr2=0x7f20c40b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 --2- 172.21.15.88:0/1076035166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40b3550 0x7f20c40b3960 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f20bc005fd0 tx=0x7f20bc02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 -- 172.21.15.88:0/1076035166 shutdown_connections 2024-09-21T07:49:31.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 --2- 172.21.15.88:0/1076035166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40b3550 0x7f20c40b3960 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 --2- 172.21.15.88:0/1076035166 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40ad0e0 0x7f20c40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 -- 172.21.15.88:0/1076035166 >> 172.21.15.88:0/1076035166 conn(0x7f20c406d730 msgr2=0x7f20c40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 -- 172.21.15.88:0/1076035166 shutdown_connections 2024-09-21T07:49:31.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.102+0000 7f20cc600640 1 -- 172.21.15.88:0/1076035166 wait complete. 2024-09-21T07:49:31.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.104+0000 7f20cc600640 1 Processor -- start 2024-09-21T07:49:31.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.105+0000 7f20cc600640 1 -- start start 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20cc600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 0x7f20c41f9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20cc600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40b3550 0x7f20c4147800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20cc600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c415d760 0x7f20c406f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20cc600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40b4da0 con 0x7f20c40ad0e0 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20cc600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f20c40b4c20 con 0x7f20c415d760 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20cc600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f20c40b3ea0 con 0x7f20c40b3550 2024-09-21T07:49:31.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20c9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 0x7f20c41f9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20c9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 0x7f20c41f9bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55404/0 (socket says 172.21.15.88:55404) 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.106+0000 7f20c9c00640 1 -- 172.21.15.88:0/1690940267 learned_addr learned my addr 172.21.15.88:0/1690940267 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9200640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40b3550 0x7f20c4147800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20ca600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c415d760 0x7f20c406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9c00640 1 -- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40b3550 msgr2=0x7f20c4147800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9c00640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40b3550 0x7f20c4147800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9c00640 1 -- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c415d760 msgr2=0x7f20c406f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9c00640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c415d760 0x7f20c406f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9c00640 1 -- 172.21.15.88:0/1690940267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c41fa130 con 0x7f20c40ad0e0 2024-09-21T07:49:31.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c9c00640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 0x7f20c41f9bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f20b800ba80 tx=0x7f20b800bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b8019070 con 0x7f20c40ad0e0 2024-09-21T07:49:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f20b800cd80 con 0x7f20c40ad0e0 2024-09-21T07:49:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20b80146a0 con 0x7f20c40ad0e0 2024-09-21T07:49:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.107+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c4147d40 con 0x7f20c40ad0e0 2024-09-21T07:49:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.108+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c41f61e0 con 0x7f20c40ad0e0 2024-09-21T07:49:31.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.109+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c40b4350 con 0x7f20c40ad0e0 2024-09-21T07:49:31.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.110+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f20b801c070 con 0x7f20c40ad0e0 2024-09-21T07:49:31.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.111+0000 7f20c1600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f20a4078d40 0x7f20a407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.111+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f20b809b0c0 con 0x7f20c40ad0e0 2024-09-21T07:49:31.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.111+0000 7f20c9200640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f20a4078d40 0x7f20a407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.112+0000 7f20c9200640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f20a4078d40 0x7f20a407b250 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f20c414b370 tx=0x7f20b0005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.116+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f20c4147d40 con 0x7f20c40ad0e0 2024-09-21T07:49:31.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.301+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f20c4153840 con 0x7f20c40ad0e0 2024-09-21T07:49:31.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.302+0000 7f20c1600640 1 -- 172.21.15.88:0/1690940267 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16750 (secure 0 0 0) 0x7f20c41fa130 con 0x7f20c40ad0e0 2024-09-21T07:49:31.305 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:31.305 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":59,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:49:30.312183+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:28.221593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:29.283162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f20a4078d40 msgr2=0x7f20a407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f20a4078d40 0x7f20a407b250 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f20c414b370 tx=0x7f20b0005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 msgr2=0x7f20c41f9bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 0x7f20c41f9bf0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f20b800ba80 tx=0x7f20b800bf40 comp rx=0 tx=0).stop 2024-09-21T07:49:31.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 shutdown_connections 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f20a4078d40 0x7f20a407b250 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f20c40b3550 0x7f20c4147800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f20c415d760 0x7f20c406f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 --2- 172.21.15.88:0/1690940267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c40ad0e0 0x7f20c41f9bf0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 >> 172.21.15.88:0/1690940267 conn(0x7f20c406d730 msgr2=0x7f20c415f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 shutdown_connections 2024-09-21T07:49:31.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.305+0000 7f20cc600640 1 -- 172.21.15.88:0/1690940267 wait complete. 2024-09-21T07:49:31.329 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:31.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 -- 172.21.15.88:0/920949720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 msgr2=0x7f79801535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 --2- 172.21.15.88:0/920949720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f79801535c0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f7970009860 tx=0x7f797002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 -- 172.21.15.88:0/920949720 shutdown_connections 2024-09-21T07:49:31.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 --2- 172.21.15.88:0/920949720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f79801535c0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 -- 172.21.15.88:0/920949720 >> 172.21.15.88:0/920949720 conn(0x7f798006d730 msgr2=0x7f79800b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 -- 172.21.15.88:0/920949720 shutdown_connections 2024-09-21T07:49:31.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.549+0000 7f7984c00640 1 -- 172.21.15.88:0/920949720 wait complete. 2024-09-21T07:49:31.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.551+0000 7f7984c00640 1 Processor -- start 2024-09-21T07:49:31.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.551+0000 7f7984c00640 1 -- start start 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f7984c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79800ad0e0 0x7f79801f5240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f7984c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 0x7f798014ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f7984c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f798014e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f7984c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79800b4da0 con 0x7f79800b3550 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f7984c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f79800b4c20 con 0x7f79800ad0e0 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f7984c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f79800b4070 con 0x7f798015d760 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f797d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 0x7f798014ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f797d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 0x7f798014ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55416/0 (socket says 172.21.15.88:55416) 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f797d400640 1 -- 172.21.15.88:0/3408364789 learned_addr learned my addr 172.21.15.88:0/3408364789 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f797de00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79800ad0e0 0x7f79801f5240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f797e800640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f798014e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.553+0000 7f797d400640 1 -- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 msgr2=0x7f798014e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797d400640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f798014e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797d400640 1 -- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79800ad0e0 msgr2=0x7f79801f5240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797d400640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79800ad0e0 0x7f79801f5240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797d400640 1 -- 172.21.15.88:0/3408364789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7980151290 con 0x7f79800b3550 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797e800640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f798014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797de00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79800ad0e0 0x7f79801f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f797d400640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 0x7f798014ded0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c430 tx=0x7f796c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c017070 con 0x7f79800b3550 2024-09-21T07:49:31.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f796c0053c0 con 0x7f79800b3550 2024-09-21T07:49:31.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7980150cd0 con 0x7f79800b3550 2024-09-21T07:49:31.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f796c003850 con 0x7f79800b3550 2024-09-21T07:49:31.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.554+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f798014d6d0 con 0x7f79800b3550 2024-09-21T07:49:31.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.555+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7980150a40 con 0x7f79800b3550 2024-09-21T07:49:31.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.557+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f796c01a070 con 0x7f79800b3550 2024-09-21T07:49:31.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.558+0000 7f7976000640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7960078d40 0x7f796007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:31.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.558+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(59..59 src has 1..59) ==== 5709+0+0 (secure 0 0 0) 0x7f796c09a540 con 0x7f79800b3550 2024-09-21T07:49:31.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.558+0000 7f797de00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7960078d40 0x7f796007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:31.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.559+0000 7f797de00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7960078d40 0x7f796007b250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7970034000 tx=0x7f7970033040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:31.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.562+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f796c0625f0 con 0x7f79800b3550 2024-09-21T07:49:31.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.749+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f79801535c0 con 0x7f79800b3550 2024-09-21T07:49:31.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.750+0000 7f7976000640 1 -- 172.21.15.88:0/3408364789 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) ==== 74+0+16750 (secure 0 0 0) 0x7f796c061ef0 con 0x7f79800b3550 2024-09-21T07:49:31.753 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:31.753 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":59,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:49:30.312183+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:28.221593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:29.283162+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.752+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7960078d40 msgr2=0x7f796007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.752+0000 7f7984c00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7960078d40 0x7f796007b250 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f7970034000 tx=0x7f7970033040 comp rx=0 tx=0).stop 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.752+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 msgr2=0x7f798014ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.752+0000 7f7984c00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 0x7f798014ded0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f796c00c430 tx=0x7f796c00c8f0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 shutdown_connections 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7960078d40 0x7f796007b250 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f798015d760 0x7f798014e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79800b3550 0x7f798014ded0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 --2- 172.21.15.88:0/3408364789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f79800ad0e0 0x7f79801f5240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:31.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 >> 172.21.15.88:0/3408364789 conn(0x7f798006d730 msgr2=0x7f798006f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:31.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 shutdown_connections 2024-09-21T07:49:31.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:31.753+0000 7f7984c00640 1 -- 172.21.15.88:0/3408364789 wait complete. 2024-09-21T07:49:31.777 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:49:32.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 -- 172.21.15.88:0/3815127053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb54070290 msgr2=0x7ffb540706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 --2- 172.21.15.88:0/3815127053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb54070290 0x7ffb540706a0 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7ffb54070d90 tx=0x7ffb4c02aa10 comp rx=0 tx=0).stop 2024-09-21T07:49:32.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 -- 172.21.15.88:0/3815127053 shutdown_connections 2024-09-21T07:49:32.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 --2- 172.21.15.88:0/3815127053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb54070290 0x7ffb540706a0 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 --2- 172.21.15.88:0/3815127053 >> v2:172.21.15.175:3300/0 conn(0x7ffb540cf7c0 0x7ffb540cfbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 --2- 172.21.15.88:0/3815127053 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb540ccc30 0x7ffb540b4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 -- 172.21.15.88:0/3815127053 >> 172.21.15.88:0/3815127053 conn(0x7ffb5406d1c0 msgr2=0x7ffb5406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:32.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 -- 172.21.15.88:0/3815127053 shutdown_connections 2024-09-21T07:49:32.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.005+0000 7ffb5a600640 1 -- 172.21.15.88:0/3815127053 wait complete. 2024-09-21T07:49:32.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.007+0000 7ffb5a600640 1 Processor -- start 2024-09-21T07:49:32.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.007+0000 7ffb5a600640 1 -- start start 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb5a600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb54070290 0x7ffb54155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb5a600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb540ccc30 0x7ffb5414b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb5a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 0x7ffb540b20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb5a600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5415c1f0 con 0x7ffb540cf7c0 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb5a600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffb5415c070 con 0x7ffb540ccc30 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb5a600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffb5415c370 con 0x7ffb54070290 2024-09-21T07:49:32.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 0x7ffb540b20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 0x7ffb540b20f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55432/0 (socket says 172.21.15.88:55432) 2024-09-21T07:49:32.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 -- 172.21.15.88:0/63970848 learned_addr learned my addr 172.21.15.88:0/63970848 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:32.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb4aa00640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb540ccc30 0x7ffb5414b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53400640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb54070290 0x7ffb54155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 -- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb54070290 msgr2=0x7ffb54155350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb54070290 0x7ffb54155350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 -- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb540ccc30 msgr2=0x7ffb5414b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb540ccc30 0x7ffb5414b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53e00640 1 -- 172.21.15.88:0/63970848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb5414f140 con 0x7ffb540cf7c0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.009+0000 7ffb53400640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb54070290 0x7ffb54155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.010+0000 7ffb53e00640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 0x7ffb540b20f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c02b000 tx=0x7ffb4c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.010+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c033050 con 0x7ffb540cf7c0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.010+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffb4c004450 con 0x7ffb540cf7c0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.010+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb4c038450 con 0x7ffb540cf7c0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.010+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb54154050 con 0x7ffb540cf7c0 2024-09-21T07:49:32.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.010+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb54154610 con 0x7ffb540cf7c0 2024-09-21T07:49:32.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.011+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb54153dc0 con 0x7ffb540cf7c0 2024-09-21T07:49:32.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.012+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7ffb4c036070 con 0x7ffb540cf7c0 2024-09-21T07:49:32.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.013+0000 7ffb4be00640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffb30078d40 0x7ffb3007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:32.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.013+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7ffb4c0bc5b0 con 0x7ffb540cf7c0 2024-09-21T07:49:32.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.014+0000 7ffb53400640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffb30078d40 0x7ffb3007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:32.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.015+0000 7ffb53400640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffb30078d40 0x7ffb3007b250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffb44005fd0 tx=0x7ffb4400b530 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:32.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.018+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffb4c084650 con 0x7ffb540cf7c0 2024-09-21T07:49:32.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.239+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7ffb540690d0 con 0x7ffb540cf7c0 2024-09-21T07:49:32.811 INFO:tasks.ceph.mds.d.smithi175.stderr:2024-09-21T07:49:32.809+0000 7f90e6a00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:49:32.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.852+0000 7ffb4be00640 1 -- 172.21.15.88:0/63970848 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v35) ==== 118+0+0 (secure 0 0 0) 0x7ffb4c083f50 con 0x7ffb540cf7c0 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.854+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffb30078d40 msgr2=0x7ffb3007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.854+0000 7ffb5a600640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffb30078d40 0x7ffb3007b250 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7ffb44005fd0 tx=0x7ffb4400b530 comp rx=0 tx=0).stop 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.854+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 msgr2=0x7ffb540b20f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.854+0000 7ffb5a600640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 0x7ffb540b20f0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c02b000 tx=0x7ffb4c003f10 comp rx=0 tx=0).stop 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 shutdown_connections 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ffb30078d40 0x7ffb3007b250 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb540cf7c0 0x7ffb540b20f0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffb540ccc30 0x7ffb5414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 --2- 172.21.15.88:0/63970848 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffb54070290 0x7ffb54155350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:32.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 >> 172.21.15.88:0/63970848 conn(0x7ffb5406d1c0 msgr2=0x7ffb5415e750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:32.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 shutdown_connections 2024-09-21T07:49:32.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:32.855+0000 7ffb5a600640 1 -- 172.21.15.88:0/63970848 wait complete. 2024-09-21T07:49:32.883 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:49:32.883 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 -- 172.21.15.88:0/3334238795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 msgr2=0x7f4798073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 --2- 172.21.15.88:0/3334238795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f4798073540 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f4798072bd0 tx=0x7f478402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 -- 172.21.15.88:0/3334238795 shutdown_connections 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 --2- 172.21.15.88:0/3334238795 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4798073a80 0x7f4798160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 --2- 172.21.15.88:0/3334238795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f4798073540 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 --2- 172.21.15.88:0/3334238795 >> v2:172.21.15.175:3300/0 conn(0x7f4798145550 0x7f479806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 -- 172.21.15.88:0/3334238795 >> 172.21.15.88:0/3334238795 conn(0x7f479806d1c0 msgr2=0x7f479806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.104+0000 7f479d200640 1 -- 172.21.15.88:0/3334238795 shutdown_connections 2024-09-21T07:49:33.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.105+0000 7f479d200640 1 -- 172.21.15.88:0/3334238795 wait complete. 2024-09-21T07:49:33.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.107+0000 7f479d200640 1 Processor -- start 2024-09-21T07:49:33.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.107+0000 7f479d200640 1 -- start start 2024-09-21T07:49:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f479d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4798073a80 0x7f47980adbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f479d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4798145550 0x7f47980b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f479d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f47980b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f479d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4798160c30 con 0x7f4798145a40 2024-09-21T07:49:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f479d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4798160ab0 con 0x7f4798073a80 2024-09-21T07:49:33.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f479d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4798160db0 con 0x7f4798145550 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.108+0000 7f4796800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f47980b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f47980b4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55454/0 (socket says 172.21.15.88:55454) 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 -- 172.21.15.88:0/1416256443 learned_addr learned my addr 172.21.15.88:0/1416256443 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4795e00640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4798073a80 0x7f47980adbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4795400640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4798145550 0x7f47980b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 -- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4798145550 msgr2=0x7f47980b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4798145550 0x7f47980b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 -- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4798073a80 msgr2=0x7f47980adbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4798073a80 0x7f47980adbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 -- 172.21.15.88:0/1416256443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47980af4e0 con 0x7f4798145a40 2024-09-21T07:49:33.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f4796800640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f47980b4370 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f479000e960 tx=0x7f479000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790004d20 con 0x7f4798145a40 2024-09-21T07:49:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.109+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f479000ca60 con 0x7f4798145a40 2024-09-21T07:49:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.110+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47980ae440 con 0x7f4798145a40 2024-09-21T07:49:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.110+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4790005020 con 0x7f4798145a40 2024-09-21T07:49:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.110+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47980aea00 con 0x7f4798145a40 2024-09-21T07:49:33.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.111+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47980ae1b0 con 0x7f4798145a40 2024-09-21T07:49:33.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.112+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f47900051c0 con 0x7f4798145a40 2024-09-21T07:49:33.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.113+0000 7f478e000640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4778078d70 0x7f477807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:33.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.113+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f479009b920 con 0x7f4798145a40 2024-09-21T07:49:33.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.113+0000 7f4795e00640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4778078d70 0x7f477807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:33.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.114+0000 7f4795e00640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4778078d70 0x7f477807b280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4788005fd0 tx=0x7f478800b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:33.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.117+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4790016030 con 0x7f4798145a40 2024-09-21T07:49:33.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.338+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f47980cc690 con 0x7f4798145a40 2024-09-21T07:49:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.855+0000 7f478e000640 1 -- 172.21.15.88:0/1416256443 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7f4790063390 con 0x7f4798145a40 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.857+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4778078d70 msgr2=0x7f477807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.857+0000 7f479d200640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4778078d70 0x7f477807b280 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4788005fd0 tx=0x7f478800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 msgr2=0x7f47980b4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f47980b4370 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f479000e960 tx=0x7f479000ee20 comp rx=0 tx=0).stop 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 shutdown_connections 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4778078d70 0x7f477807b280 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4798145550 0x7f47980b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4798145a40 0x7f47980b4370 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 --2- 172.21.15.88:0/1416256443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4798073a80 0x7f47980adbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 >> 172.21.15.88:0/1416256443 conn(0x7f479806d1c0 msgr2=0x7f4798143ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 shutdown_connections 2024-09-21T07:49:33.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:33.858+0000 7f479d200640 1 -- 172.21.15.88:0/1416256443 wait complete. 2024-09-21T07:49:33.884 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:49:33.884 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 -- 172.21.15.88:0/3192536640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 msgr2=0x7f38e8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 --2- 172.21.15.88:0/3192536640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f38e8004c10 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f38ec00c4a0 tx=0x7f38e8005830 comp rx=0 tx=0).stop 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 -- 172.21.15.88:0/3192536640 shutdown_connections 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 --2- 172.21.15.88:0/3192536640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f38e8004c10 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 -- 172.21.15.88:0/3192536640 >> 172.21.15.88:0/3192536640 conn(0x7f390406d730 msgr2=0x7f39040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 -- 172.21.15.88:0/3192536640 shutdown_connections 2024-09-21T07:49:34.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.103+0000 7f390ac00640 1 -- 172.21.15.88:0/3192536640 wait complete. 2024-09-21T07:49:34.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.106+0000 7f390ac00640 1 Processor -- start 2024-09-21T07:49:34.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.106+0000 7f390ac00640 1 -- start start 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f390ac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f39040aea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f390ac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39041f4ed0 0x7f39041f52e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f390ac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3904151c80 0x7f3904152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f390ac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f390406f0a0 con 0x7f38e8004800 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f390ac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3904073760 con 0x7f3904151c80 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f390ac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f390406f220 con 0x7f39041f4ed0 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f3903e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f39040aea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f3903e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f39040aea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55468/0 (socket says 172.21.15.88:55468) 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.107+0000 7f3903e00640 1 -- 172.21.15.88:0/2511459389 learned_addr learned my addr 172.21.15.88:0/2511459389 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:34.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3903e00640 1 -- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39041f4ed0 msgr2=0x7f39041f52e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3903e00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39041f4ed0 0x7f39041f52e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3903e00640 1 -- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3904151c80 msgr2=0x7f3904152090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3903e00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3904151c80 0x7f3904152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3903e00640 1 -- 172.21.15.88:0/2511459389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39041525d0 con 0x7f38e8004800 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3903e00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f39040aea80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011ab0 tx=0x7f38f4011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:34.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f401e070 con 0x7f38e8004800 2024-09-21T07:49:34.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38f400fbf0 con 0x7f38e8004800 2024-09-21T07:49:34.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38f4019410 con 0x7f38e8004800 2024-09-21T07:49:34.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39041e9280 con 0x7f38e8004800 2024-09-21T07:49:34.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.108+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39041e9840 con 0x7f38e8004800 2024-09-21T07:49:34.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.110+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39041e9ef0 con 0x7f38e8004800 2024-09-21T07:49:34.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.111+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f38f4021070 con 0x7f38e8004800 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.112+0000 7f3900c00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f38e0078da0 0x7f38e007b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.112+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f38f409b8c0 con 0x7f38e8004800 2024-09-21T07:49:34.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.112+0000 7f38fb400640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f38e0078da0 0x7f38e007b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.113+0000 7f38fb400640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f38e0078da0 0x7f38e007b2b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f38ec033000 tx=0x7f3904070560 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.116+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f39041525d0 con 0x7f38e8004800 2024-09-21T07:49:34.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.337+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3904162560 con 0x7f38e8004800 2024-09-21T07:49:34.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.338+0000 7f3900c00640 1 -- 172.21.15.88:0/2511459389 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5226 (secure 0 0 0) 0x7f38f4063310 con 0x7f38e8004800 2024-09-21T07:49:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:34.340 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":36,"btime":"2024-09-21T07:49:33:853295+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:30.312143+0000","modified":"2024-09-21T07:49:33.340468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2365988221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2365988221},{"type":"v1","addr":"172.21.15.175:6837","nonce":2365988221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5763,"qdb_cluster":[5763]},"id":4}]} 2024-09-21T07:49:34.340 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 36 2024-09-21T07:49:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f38e0078da0 msgr2=0x7f38e007b2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f38e0078da0 0x7f38e007b2b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f38ec033000 tx=0x7f3904070560 comp rx=0 tx=0).stop 2024-09-21T07:49:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 msgr2=0x7f39040aea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f39040aea80 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f38f4011ab0 tx=0x7f38f4011f70 comp rx=0 tx=0).stop 2024-09-21T07:49:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 shutdown_connections 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f38e0078da0 0x7f38e007b2b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f39041f4ed0 0x7f39041f52e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3904151c80 0x7f3904152090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 --2- 172.21.15.88:0/2511459389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e8004800 0x7f39040aea80 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 >> 172.21.15.88:0/2511459389 conn(0x7f390406d730 msgr2=0x7f39040b1600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 shutdown_connections 2024-09-21T07:49:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.340+0000 7f390ac00640 1 -- 172.21.15.88:0/2511459389 wait complete. 2024-09-21T07:49:34.368 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:49:34.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 -- 172.21.15.88:0/1299589948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140ad0e0 msgr2=0x7f3f140ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 --2- 172.21.15.88:0/1299589948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140ad0e0 0x7f3f140ad4f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f3f00005fd0 tx=0x7f3f0002a540 comp rx=0 tx=0).stop 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 -- 172.21.15.88:0/1299589948 shutdown_connections 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 --2- 172.21.15.88:0/1299589948 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f140b3550 0x7f3f140b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 --2- 172.21.15.88:0/1299589948 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140ad0e0 0x7f3f140ad4f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 -- 172.21.15.88:0/1299589948 >> 172.21.15.88:0/1299589948 conn(0x7f3f1406d730 msgr2=0x7f3f140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 -- 172.21.15.88:0/1299589948 shutdown_connections 2024-09-21T07:49:34.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.587+0000 7f3f19a00640 1 -- 172.21.15.88:0/1299589948 wait complete. 2024-09-21T07:49:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.589+0000 7f3f19a00640 1 Processor -- start 2024-09-21T07:49:34.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.590+0000 7f3f19a00640 1 -- start start 2024-09-21T07:49:34.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f19a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 0x7f3f14156420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f19a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140b3550 0x7f3f1409dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f19a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1415d760 0x7f3f1413d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f19a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f140b4da0 con 0x7f3f140ad0e0 2024-09-21T07:49:34.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f19a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f140b4c20 con 0x7f3f140b3550 2024-09-21T07:49:34.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f19a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f140b3ea0 con 0x7f3f1415d760 2024-09-21T07:49:34.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f12a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 0x7f3f14156420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f12a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 0x7f3f14156420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55484/0 (socket says 172.21.15.88:55484) 2024-09-21T07:49:34.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.591+0000 7f3f12a00640 1 -- 172.21.15.88:0/2294173831 learned_addr learned my addr 172.21.15.88:0/2294173831 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f13400640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1415d760 0x7f3f1413d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12000640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140b3550 0x7f3f1409dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12a00640 1 -- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1415d760 msgr2=0x7f3f1413d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1415d760 0x7f3f1413d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12a00640 1 -- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140b3550 msgr2=0x7f3f1409dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140b3550 0x7f3f1409dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12a00640 1 -- 172.21.15.88:0/2294173831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f14150250 con 0x7f3f140ad0e0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f12a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 0x7f3f14156420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c3a0 tx=0x7f3f0400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04017070 con 0x7f3f140ad0e0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f040053c0 con 0x7f3f140ad0e0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f04003850 con 0x7f3f140ad0e0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f141507e0 con 0x7f3f140ad0e0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.592+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f14155940 con 0x7f3f140ad0e0 2024-09-21T07:49:34.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.594+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f140b4370 con 0x7f3f140ad0e0 2024-09-21T07:49:34.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.595+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f3f0401a070 con 0x7f3f140ad0e0 2024-09-21T07:49:34.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.596+0000 7f3f0aa00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef4078d70 0x7f3ef407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:34.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.597+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f3f0409a4d0 con 0x7f3f140ad0e0 2024-09-21T07:49:34.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.597+0000 7f3f12000640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef4078d70 0x7f3ef407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:34.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.598+0000 7f3f12000640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef4078d70 0x7f3ef407b280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3f00033000 tx=0x7f3f00004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:34.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.601+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f140ada30 con 0x7f3f140ad0e0 2024-09-21T07:49:34.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.788+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f14069700 con 0x7f3f140ad0e0 2024-09-21T07:49:34.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.789+0000 7f3f0aa00640 1 -- 172.21.15.88:0/2294173831 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) ==== 74+0+16737 (secure 0 0 0) 0x7f3f04098e20 con 0x7f3f140ad0e0 2024-09-21T07:49:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:34.791 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":60,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:49:31.837248+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:28.221593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:29.283162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:49:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef4078d70 msgr2=0x7f3ef407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef4078d70 0x7f3ef407b280 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f3f00033000 tx=0x7f3f00004730 comp rx=0 tx=0).stop 2024-09-21T07:49:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 msgr2=0x7f3f14156420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 0x7f3f14156420 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3f0400c3a0 tx=0x7f3f0400c860 comp rx=0 tx=0).stop 2024-09-21T07:49:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 shutdown_connections 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3ef4078d70 0x7f3ef407b280 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f1415d760 0x7f3f1413d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f140b3550 0x7f3f1409dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 --2- 172.21.15.88:0/2294173831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f140ad0e0 0x7f3f14156420 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 >> 172.21.15.88:0/2294173831 conn(0x7f3f1406d730 msgr2=0x7f3f1415dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 shutdown_connections 2024-09-21T07:49:34.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:34.792+0000 7f3f19a00640 1 -- 172.21.15.88:0/2294173831 wait complete. 2024-09-21T07:49:34.820 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:49:34.820 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.037+0000 7ff9f4a00640 1 -- 172.21.15.88:0/984370646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f00b2750 msgr2=0x7ff9f015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.037+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/984370646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f00b2750 0x7ff9f015f3d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff9f0072810 tx=0x7ff9dc02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.037+0000 7ff9f4a00640 1 -- 172.21.15.88:0/984370646 shutdown_connections 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.037+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/984370646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f00b2750 0x7ff9f015f3d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.037+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/984370646 >> v2:172.21.15.175:3300/0 conn(0x7ff9f00aebc0 0x7ff9f0152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.037+0000 7ff9f4a00640 1 -- 172.21.15.88:0/984370646 >> 172.21.15.88:0/984370646 conn(0x7ff9f006d730 msgr2=0x7ff9f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.038+0000 7ff9f4a00640 1 -- 172.21.15.88:0/984370646 shutdown_connections 2024-09-21T07:49:35.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.038+0000 7ff9f4a00640 1 -- 172.21.15.88:0/984370646 wait complete. 2024-09-21T07:49:35.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.040+0000 7ff9f4a00640 1 Processor -- start 2024-09-21T07:49:35.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.040+0000 7ff9f4a00640 1 -- start start 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.041+0000 7ff9f4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f00aebc0 0x7ff9f014d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.041+0000 7ff9f4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f00b2750 0x7ff9f01e9620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.041+0000 7ff9f4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 0x7ff9f01e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.041+0000 7ff9f4a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f015c4a0 con 0x7ff9f015f910 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.041+0000 7ff9f4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f00b4970 con 0x7ff9f00b2750 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.041+0000 7ff9f4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9f015c620 con 0x7ff9f00aebc0 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 0x7ff9f01e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 0x7ff9f01e9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55504/0 (socket says 172.21.15.88:55504) 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 -- 172.21.15.88:0/3476673703 learned_addr learned my addr 172.21.15.88:0/3476673703 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ed400640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f00b2750 0x7ff9f01e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ede00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f00aebc0 0x7ff9f014d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 -- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f00aebc0 msgr2=0x7ff9f014d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f00aebc0 0x7ff9f014d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 -- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f00b2750 msgr2=0x7ff9f01e9620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f00b2750 0x7ff9f01e9620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 -- 172.21.15.88:0/3476673703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f014f6e0 con 0x7ff9f015f910 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.042+0000 7ff9ee800640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 0x7ff9f01e9b60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800b970 tx=0x7ff9e800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.043+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e8019070 con 0x7ff9f015f910 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.043+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9e800cc30 con 0x7ff9f015f910 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.043+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f014e220 con 0x7ff9f015f910 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.043+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e80146a0 con 0x7ff9f015f910 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.043+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f01f63b0 con 0x7ff9f015f910 2024-09-21T07:49:35.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.044+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9f01f5e20 con 0x7ff9f015f910 2024-09-21T07:49:35.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.045+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7ff9e801c070 con 0x7ff9f015f910 2024-09-21T07:49:35.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.046+0000 7ff9e6000640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff9d0078d70 0x7ff9d007b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.046+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7ff9e809adc0 con 0x7ff9f015f910 2024-09-21T07:49:35.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.047+0000 7ff9ede00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff9d0078d70 0x7ff9d007b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.047+0000 7ff9ede00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff9d0078d70 0x7ff9d007b280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0005fd0 tx=0x7ff9e0005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.050+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9f014e220 con 0x7ff9f015f910 2024-09-21T07:49:35.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.232+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff9f015cab0 con 0x7ff9d0078d70 2024-09-21T07:49:35.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.237+0000 7ff9e6000640 1 -- 172.21.15.88:0/3476673703 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244742 (secure 0 0 0) 0x7ff9f015cab0 con 0x7ff9d0078d70 2024-09-21T07:49:35.242 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:35.243 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff9d0078d70 msgr2=0x7ff9d007b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff9d0078d70 0x7ff9d007b280 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7ff9e0005fd0 tx=0x7ff9e0005850 comp rx=0 tx=0).stop 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 msgr2=0x7ff9f01e9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 0x7ff9f01e9b60 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff9e800b970 tx=0x7ff9e800be30 comp rx=0 tx=0).stop 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 shutdown_connections 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff9d0078d70 0x7ff9d007b280 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9f015f910 0x7ff9f01e9b60 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9f00b2750 0x7ff9f01e9620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 --2- 172.21.15.88:0/3476673703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9f00aebc0 0x7ff9f014d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 >> 172.21.15.88:0/3476673703 conn(0x7ff9f006d730 msgr2=0x7ff9f00ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 shutdown_connections 2024-09-21T07:49:35.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.241+0000 7ff9f4a00640 1 -- 172.21.15.88:0/3476673703 wait complete. 2024-09-21T07:49:35.266 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-21T07:49:33.912034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":140,"acting":140,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243548,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731172,"statfs":{"total":773094113280,"available":772844720128,"internally_reserved":0,"allocated":6631424,"data_stored":3406800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.102038"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898139+0000","last_change":"2024-09-21T07:49:30.866746+0000","last_active":"2024-09-21T07:49:31.898139+0000","last_peered":"2024-09-21T07:49:31.898139+0000","last_clean":"2024-09-21T07:49:31.898139+0000","last_became_active":"2024-09-21T07:49:30.866386+0000","last_became_peered":"2024-09-21T07:49:30.866386+0000","last_unstale":"2024-09-21T07:49:31.898139+0000","last_undegraded":"2024-09-21T07:49:31.898139+0000","last_fullsized":"2024-09-21T07:49:31.898139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:41.251356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.830568+0000","last_change":"2024-09-21T07:49:29.830568+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.830568+0000","last_undegraded":"2024-09-21T07:49:29.830568+0000","last_fullsized":"2024-09-21T07:49:29.830568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.821239+0000","last_change":"2024-09-21T07:49:29.821239+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.821239+0000","last_undegraded":"2024-09-21T07:49:29.821239+0000","last_fullsized":"2024-09-21T07:49:29.821239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.823537+0000","last_change":"2024-09-21T07:49:29.823537+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.823537+0000","last_undegraded":"2024-09-21T07:49:29.823537+0000","last_fullsized":"2024-09-21T07:49:29.823537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820108+0000","last_change":"2024-09-21T07:49:29.820108+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820108+0000","last_undegraded":"2024-09-21T07:49:29.820108+0000","last_fullsized":"2024-09-21T07:49:29.820108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.819998+0000","last_change":"2024-09-21T07:49:29.819998+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.819998+0000","last_undegraded":"2024-09-21T07:49:29.819998+0000","last_fullsized":"2024-09-21T07:49:29.819998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.828713+0000","last_change":"2024-09-21T07:49:29.828713+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.828713+0000","last_undegraded":"2024-09-21T07:49:29.828713+0000","last_fullsized":"2024-09-21T07:49:29.828713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.825135+0000","last_change":"2024-09-21T07:49:29.825135+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.825135+0000","last_undegraded":"2024-09-21T07:49:29.825135+0000","last_fullsized":"2024-09-21T07:49:29.825135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.828250+0000","last_change":"2024-09-21T07:49:29.828250+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.828250+0000","last_undegraded":"2024-09-21T07:49:29.828250+0000","last_fullsized":"2024-09-21T07:49:29.828250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.826636+0000","last_change":"2024-09-21T07:49:29.826636+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.826636+0000","last_undegraded":"2024-09-21T07:49:29.826636+0000","last_fullsized":"2024-09-21T07:49:29.826636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897965+0000","last_change":"2024-09-21T07:49:30.870745+0000","last_active":"2024-09-21T07:49:31.897965+0000","last_peered":"2024-09-21T07:49:31.897965+0000","last_clean":"2024-09-21T07:49:31.897965+0000","last_became_active":"2024-09-21T07:49:30.870361+0000","last_became_peered":"2024-09-21T07:49:30.870361+0000","last_unstale":"2024-09-21T07:49:31.897965+0000","last_undegraded":"2024-09-21T07:49:31.897965+0000","last_fullsized":"2024-09-21T07:49:31.897965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:18.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.828162+0000","last_change":"2024-09-21T07:49:29.828162+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.828162+0000","last_undegraded":"2024-09-21T07:49:29.828162+0000","last_fullsized":"2024-09-21T07:49:29.828162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.822253+0000","last_change":"2024-09-21T07:49:29.822253+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.822253+0000","last_undegraded":"2024-09-21T07:49:29.822253+0000","last_fullsized":"2024-09-21T07:49:29.822253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.824004+0000","last_change":"2024-09-21T07:49:29.824004+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.824004+0000","last_undegraded":"2024-09-21T07:49:29.824004+0000","last_fullsized":"2024-09-21T07:49:29.824004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898160+0000","last_change":"2024-09-21T07:49:30.866983+0000","last_active":"2024-09-21T07:49:31.898160+0000","last_peered":"2024-09-21T07:49:31.898160+0000","last_clean":"2024-09-21T07:49:31.898160+0000","last_became_active":"2024-09-21T07:49:30.866405+0000","last_became_peered":"2024-09-21T07:49:30.866405+0000","last_unstale":"2024-09-21T07:49:31.898160+0000","last_undegraded":"2024-09-21T07:49:31.898160+0000","last_fullsized":"2024-09-21T07:49:31.898160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:33.824401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.827426+0000","last_change":"2024-09-21T07:49:29.827426+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.827426+0000","last_undegraded":"2024-09-21T07:49:29.827426+0000","last_fullsized":"2024-09-21T07:49:29.827426+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898197+0000","last_change":"2024-09-21T07:49:30.868732+0000","last_active":"2024-09-21T07:49:31.898197+0000","last_peered":"2024-09-21T07:49:31.898197+0000","last_clean":"2024-09-21T07:49:31.898197+0000","last_became_active":"2024-09-21T07:49:30.868358+0000","last_became_peered":"2024-09-21T07:49:30.868358+0000","last_unstale":"2024-09-21T07:49:31.898197+0000","last_undegraded":"2024-09-21T07:49:31.898197+0000","last_fullsized":"2024-09-21T07:49:31.898197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:09.597807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.821842+0000","last_change":"2024-09-21T07:49:29.821842+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.821842+0000","last_undegraded":"2024-09-21T07:49:29.821842+0000","last_fullsized":"2024-09-21T07:49:29.821842+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897737+0000","last_change":"2024-09-21T07:49:30.862236+0000","last_active":"2024-09-21T07:49:31.897737+0000","last_peered":"2024-09-21T07:49:31.897737+0000","last_clean":"2024-09-21T07:49:31.897737+0000","last_became_active":"2024-09-21T07:49:30.861285+0000","last_became_peered":"2024-09-21T07:49:30.861285+0000","last_unstale":"2024-09-21T07:49:31.897737+0000","last_undegraded":"2024-09-21T07:49:31.897737+0000","last_fullsized":"2024-09-21T07:49:31.897737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.866684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.813446+0000","last_change":"2024-09-21T07:49:28.813446+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.813446+0000","last_undegraded":"2024-09-21T07:49:28.813446+0000","last_fullsized":"2024-09-21T07:49:28.813446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898195+0000","last_change":"2024-09-21T07:49:29.844161+0000","last_active":"2024-09-21T07:49:31.898195+0000","last_peered":"2024-09-21T07:49:31.898195+0000","last_clean":"2024-09-21T07:49:31.898195+0000","last_became_active":"2024-09-21T07:49:29.843722+0000","last_became_peered":"2024-09-21T07:49:29.843722+0000","last_unstale":"2024-09-21T07:49:31.898195+0000","last_undegraded":"2024-09-21T07:49:31.898195+0000","last_fullsized":"2024-09-21T07:49:31.898195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:56.739749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817430+0000","last_change":"2024-09-21T07:49:28.817430+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817430+0000","last_undegraded":"2024-09-21T07:49:28.817430+0000","last_fullsized":"2024-09-21T07:49:28.817430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.842132+0000","last_change":"2024-09-21T07:49:29.839504+0000","last_active":"2024-09-21T07:49:29.842132+0000","last_peered":"2024-09-21T07:49:29.842132+0000","last_clean":"2024-09-21T07:49:29.842132+0000","last_became_active":"2024-09-21T07:49:29.838264+0000","last_became_peered":"2024-09-21T07:49:29.838264+0000","last_unstale":"2024-09-21T07:49:29.842132+0000","last_undegraded":"2024-09-21T07:49:29.842132+0000","last_fullsized":"2024-09-21T07:49:29.842132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:44.500806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.842818+0000","last_change":"2024-09-21T07:49:29.835030+0000","last_active":"2024-09-21T07:49:29.842818+0000","last_peered":"2024-09-21T07:49:29.842818+0000","last_clean":"2024-09-21T07:49:29.842818+0000","last_became_active":"2024-09-21T07:49:29.832816+0000","last_became_peered":"2024-09-21T07:49:29.832816+0000","last_unstale":"2024-09-21T07:49:29.842818+0000","last_undegraded":"2024-09-21T07:49:29.842818+0000","last_fullsized":"2024-09-21T07:49:29.842818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:46.877552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817381+0000","last_change":"2024-09-21T07:49:28.817381+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817381+0000","last_undegraded":"2024-09-21T07:49:28.817381+0000","last_fullsized":"2024-09-21T07:49:28.817381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.823219+0000","last_change":"2024-09-21T07:49:29.823219+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.823219+0000","last_undegraded":"2024-09-21T07:49:29.823219+0000","last_fullsized":"2024-09-21T07:49:29.823219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.837330+0000","last_change":"2024-09-21T07:49:29.835362+0000","last_active":"2024-09-21T07:49:29.837330+0000","last_peered":"2024-09-21T07:49:29.837330+0000","last_clean":"2024-09-21T07:49:29.837330+0000","last_became_active":"2024-09-21T07:49:29.834971+0000","last_became_peered":"2024-09-21T07:49:29.834971+0000","last_unstale":"2024-09-21T07:49:29.837330+0000","last_undegraded":"2024-09-21T07:49:29.837330+0000","last_fullsized":"2024-09-21T07:49:29.837330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:59.735243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.834824+0000","last_change":"2024-09-21T07:49:29.830256+0000","last_active":"2024-09-21T07:49:29.834824+0000","last_peered":"2024-09-21T07:49:29.834824+0000","last_clean":"2024-09-21T07:49:29.834824+0000","last_became_active":"2024-09-21T07:49:29.829961+0000","last_became_peered":"2024-09-21T07:49:29.829961+0000","last_unstale":"2024-09-21T07:49:29.834824+0000","last_undegraded":"2024-09-21T07:49:29.834824+0000","last_fullsized":"2024-09-21T07:49:29.834824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:50.193927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.822717+0000","last_change":"2024-09-21T07:49:29.822717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.822717+0000","last_undegraded":"2024-09-21T07:49:29.822717+0000","last_fullsized":"2024-09-21T07:49:29.822717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.845698+0000","last_change":"2024-09-21T07:49:29.841680+0000","last_active":"2024-09-21T07:49:29.845698+0000","last_peered":"2024-09-21T07:49:29.845698+0000","last_clean":"2024-09-21T07:49:29.845698+0000","last_became_active":"2024-09-21T07:49:29.841344+0000","last_became_peered":"2024-09-21T07:49:29.841344+0000","last_unstale":"2024-09-21T07:49:29.845698+0000","last_undegraded":"2024-09-21T07:49:29.845698+0000","last_fullsized":"2024-09-21T07:49:29.845698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:32.379324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.830128+0000","last_change":"2024-09-21T07:49:29.830128+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.830128+0000","last_undegraded":"2024-09-21T07:49:29.830128+0000","last_fullsized":"2024-09-21T07:49:29.830128+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821980+0000","last_change":"2024-09-21T07:49:28.821980+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821980+0000","last_undegraded":"2024-09-21T07:49:28.821980+0000","last_fullsized":"2024-09-21T07:49:28.821980+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.843477+0000","last_change":"2024-09-21T07:49:29.842330+0000","last_active":"2024-09-21T07:49:29.843477+0000","last_peered":"2024-09-21T07:49:29.843477+0000","last_clean":"2024-09-21T07:49:29.843477+0000","last_became_active":"2024-09-21T07:49:29.842047+0000","last_became_peered":"2024-09-21T07:49:29.842047+0000","last_unstale":"2024-09-21T07:49:29.843477+0000","last_undegraded":"2024-09-21T07:49:29.843477+0000","last_fullsized":"2024-09-21T07:49:29.843477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:04.477193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.821103+0000","last_change":"2024-09-21T07:49:29.821103+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.821103+0000","last_undegraded":"2024-09-21T07:49:29.821103+0000","last_fullsized":"2024-09-21T07:49:29.821103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.844213+0000","last_change":"2024-09-21T07:49:29.842473+0000","last_active":"2024-09-21T07:49:29.844213+0000","last_peered":"2024-09-21T07:49:29.844213+0000","last_clean":"2024-09-21T07:49:29.844213+0000","last_became_active":"2024-09-21T07:49:29.842149+0000","last_became_peered":"2024-09-21T07:49:29.842149+0000","last_unstale":"2024-09-21T07:49:29.844213+0000","last_undegraded":"2024-09-21T07:49:29.844213+0000","last_fullsized":"2024-09-21T07:49:29.844213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:23.324347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820635+0000","last_change":"2024-09-21T07:49:29.820635+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820635+0000","last_undegraded":"2024-09-21T07:49:29.820635+0000","last_fullsized":"2024-09-21T07:49:29.820635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821391+0000","last_change":"2024-09-21T07:49:28.821391+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821391+0000","last_undegraded":"2024-09-21T07:49:28.821391+0000","last_fullsized":"2024-09-21T07:49:28.821391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.824711+0000","last_change":"2024-09-21T07:49:29.824711+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.824711+0000","last_undegraded":"2024-09-21T07:49:29.824711+0000","last_fullsized":"2024-09-21T07:49:29.824711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.840466+0000","last_change":"2024-09-21T07:49:29.835475+0000","last_active":"2024-09-21T07:49:29.840466+0000","last_peered":"2024-09-21T07:49:29.840466+0000","last_clean":"2024-09-21T07:49:29.840466+0000","last_became_active":"2024-09-21T07:49:29.835130+0000","last_became_peered":"2024-09-21T07:49:29.835130+0000","last_unstale":"2024-09-21T07:49:29.840466+0000","last_undegraded":"2024-09-21T07:49:29.840466+0000","last_fullsized":"2024-09-21T07:49:29.840466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:24.392164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.822092+0000","last_change":"2024-09-21T07:49:29.822092+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.822092+0000","last_undegraded":"2024-09-21T07:49:29.822092+0000","last_fullsized":"2024-09-21T07:49:29.822092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.828256+0000","last_change":"2024-09-21T07:49:29.825131+0000","last_active":"2024-09-21T07:49:29.828256+0000","last_peered":"2024-09-21T07:49:29.828256+0000","last_clean":"2024-09-21T07:49:29.828256+0000","last_became_active":"2024-09-21T07:49:29.824617+0000","last_became_peered":"2024-09-21T07:49:29.824617+0000","last_unstale":"2024-09-21T07:49:29.828256+0000","last_undegraded":"2024-09-21T07:49:29.828256+0000","last_fullsized":"2024-09-21T07:49:29.828256+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:39.318132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820093+0000","last_change":"2024-09-21T07:49:29.820093+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820093+0000","last_undegraded":"2024-09-21T07:49:29.820093+0000","last_fullsized":"2024-09-21T07:49:29.820093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.839701+0000","last_change":"2024-09-21T07:49:29.836030+0000","last_active":"2024-09-21T07:49:29.839701+0000","last_peered":"2024-09-21T07:49:29.839701+0000","last_clean":"2024-09-21T07:49:29.839701+0000","last_became_active":"2024-09-21T07:49:29.835703+0000","last_became_peered":"2024-09-21T07:49:29.835703+0000","last_unstale":"2024-09-21T07:49:29.839701+0000","last_undegraded":"2024-09-21T07:49:29.839701+0000","last_fullsized":"2024-09-21T07:49:29.839701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:44.963378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.826148+0000","last_change":"2024-09-21T07:49:29.826148+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.826148+0000","last_undegraded":"2024-09-21T07:49:29.826148+0000","last_fullsized":"2024-09-21T07:49:29.826148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:29.843999+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:29.843591+0000","last_became_peered":"2024-09-21T07:49:29.843591+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:56.147012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.829668+0000","last_change":"2024-09-21T07:49:29.829668+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.829668+0000","last_undegraded":"2024-09-21T07:49:29.829668+0000","last_fullsized":"2024-09-21T07:49:29.829668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821309+0000","last_change":"2024-09-21T07:49:28.821309+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821309+0000","last_undegraded":"2024-09-21T07:49:28.821309+0000","last_fullsized":"2024-09-21T07:49:28.821309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897841+0000","last_change":"2024-09-21T07:49:30.861727+0000","last_active":"2024-09-21T07:49:31.897841+0000","last_peered":"2024-09-21T07:49:31.897841+0000","last_clean":"2024-09-21T07:49:31.897841+0000","last_became_active":"2024-09-21T07:49:30.861401+0000","last_became_peered":"2024-09-21T07:49:30.861401+0000","last_unstale":"2024-09-21T07:49:31.897841+0000","last_undegraded":"2024-09-21T07:49:31.897841+0000","last_fullsized":"2024-09-21T07:49:31.897841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:59.043011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.845663+0000","last_change":"2024-09-21T07:49:29.843565+0000","last_active":"2024-09-21T07:49:29.845663+0000","last_peered":"2024-09-21T07:49:29.845663+0000","last_clean":"2024-09-21T07:49:29.845663+0000","last_became_active":"2024-09-21T07:49:29.843265+0000","last_became_peered":"2024-09-21T07:49:29.843265+0000","last_unstale":"2024-09-21T07:49:29.845663+0000","last_undegraded":"2024-09-21T07:49:29.845663+0000","last_fullsized":"2024-09-21T07:49:29.845663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:40.743393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820577+0000","last_change":"2024-09-21T07:49:29.820577+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820577+0000","last_undegraded":"2024-09-21T07:49:29.820577+0000","last_fullsized":"2024-09-21T07:49:29.820577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821343+0000","last_change":"2024-09-21T07:49:28.821343+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821343+0000","last_undegraded":"2024-09-21T07:49:28.821343+0000","last_fullsized":"2024-09-21T07:49:28.821343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820172+0000","last_change":"2024-09-21T07:49:29.820172+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820172+0000","last_undegraded":"2024-09-21T07:49:29.820172+0000","last_fullsized":"2024-09-21T07:49:29.820172+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.813545+0000","last_change":"2024-09-21T07:49:28.813545+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.813545+0000","last_undegraded":"2024-09-21T07:49:28.813545+0000","last_fullsized":"2024-09-21T07:49:28.813545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821392+0000","last_change":"2024-09-21T07:49:28.821392+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821392+0000","last_undegraded":"2024-09-21T07:49:28.821392+0000","last_fullsized":"2024-09-21T07:49:28.821392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.834989+0000","last_change":"2024-09-21T07:49:29.832146+0000","last_active":"2024-09-21T07:49:29.834989+0000","last_peered":"2024-09-21T07:49:29.834989+0000","last_clean":"2024-09-21T07:49:29.834989+0000","last_became_active":"2024-09-21T07:49:29.831853+0000","last_became_peered":"2024-09-21T07:49:29.831853+0000","last_unstale":"2024-09-21T07:49:29.834989+0000","last_undegraded":"2024-09-21T07:49:29.834989+0000","last_fullsized":"2024-09-21T07:49:29.834989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:15.388689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.825669+0000","last_change":"2024-09-21T07:49:29.825669+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.825669+0000","last_undegraded":"2024-09-21T07:49:29.825669+0000","last_fullsized":"2024-09-21T07:49:29.825669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898152+0000","last_change":"2024-09-21T07:49:29.843093+0000","last_active":"2024-09-21T07:49:31.898152+0000","last_peered":"2024-09-21T07:49:31.898152+0000","last_clean":"2024-09-21T07:49:31.898152+0000","last_became_active":"2024-09-21T07:49:29.842766+0000","last_became_peered":"2024-09-21T07:49:29.842766+0000","last_unstale":"2024-09-21T07:49:31.898152+0000","last_undegraded":"2024-09-21T07:49:31.898152+0000","last_fullsized":"2024-09-21T07:49:31.898152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:50.990394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.827205+0000","last_change":"2024-09-21T07:49:29.827205+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.827205+0000","last_undegraded":"2024-09-21T07:49:29.827205+0000","last_fullsized":"2024-09-21T07:49:29.827205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817386+0000","last_change":"2024-09-21T07:49:28.817386+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817386+0000","last_undegraded":"2024-09-21T07:49:28.817386+0000","last_fullsized":"2024-09-21T07:49:28.817386+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:30.868811+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:30.868487+0000","last_became_peered":"2024-09-21T07:49:30.868487+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:14.218794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.841761+0000","last_change":"2024-09-21T07:49:29.840533+0000","last_active":"2024-09-21T07:49:29.841761+0000","last_peered":"2024-09-21T07:49:29.841761+0000","last_clean":"2024-09-21T07:49:29.841761+0000","last_became_active":"2024-09-21T07:49:29.840191+0000","last_became_peered":"2024-09-21T07:49:29.840191+0000","last_unstale":"2024-09-21T07:49:29.841761+0000","last_undegraded":"2024-09-21T07:49:29.841761+0000","last_fullsized":"2024-09-21T07:49:29.841761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:11:56.199141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.843001+0000","last_change":"2024-09-21T07:49:29.838408+0000","last_active":"2024-09-21T07:49:29.843001+0000","last_peered":"2024-09-21T07:49:29.843001+0000","last_clean":"2024-09-21T07:49:29.843001+0000","last_became_active":"2024-09-21T07:49:29.838107+0000","last_became_peered":"2024-09-21T07:49:29.838107+0000","last_unstale":"2024-09-21T07:49:29.843001+0000","last_undegraded":"2024-09-21T07:49:29.843001+0000","last_fullsized":"2024-09-21T07:49:29.843001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:58.652422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897754+0000","last_change":"2024-09-21T07:49:29.842367+0000","last_active":"2024-09-21T07:49:31.897754+0000","last_peered":"2024-09-21T07:49:31.897754+0000","last_clean":"2024-09-21T07:49:31.897754+0000","last_became_active":"2024-09-21T07:49:29.842035+0000","last_became_peered":"2024-09-21T07:49:29.842035+0000","last_unstale":"2024-09-21T07:49:31.897754+0000","last_undegraded":"2024-09-21T07:49:31.897754+0000","last_fullsized":"2024-09-21T07:49:31.897754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:53.958285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.826591+0000","last_change":"2024-09-21T07:49:29.826591+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.826591+0000","last_undegraded":"2024-09-21T07:49:29.826591+0000","last_fullsized":"2024-09-21T07:49:29.826591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.844126+0000","last_change":"2024-09-21T07:49:29.841930+0000","last_active":"2024-09-21T07:49:29.844126+0000","last_peered":"2024-09-21T07:49:29.844126+0000","last_clean":"2024-09-21T07:49:29.844126+0000","last_became_active":"2024-09-21T07:49:29.841523+0000","last_became_peered":"2024-09-21T07:49:29.841523+0000","last_unstale":"2024-09-21T07:49:29.844126+0000","last_undegraded":"2024-09-21T07:49:29.844126+0000","last_fullsized":"2024-09-21T07:49:29.844126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:35.269020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817411+0000","last_change":"2024-09-21T07:49:28.817411+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817411+0000","last_undegraded":"2024-09-21T07:49:28.817411+0000","last_fullsized":"2024-09-21T07:49:28.817411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897800+0000","last_change":"2024-09-21T07:49:30.866291+0000","last_active":"2024-09-21T07:49:31.897800+0000","last_peered":"2024-09-21T07:49:31.897800+0000","last_clean":"2024-09-21T07:49:31.897800+0000","last_became_active":"2024-09-21T07:49:30.863674+0000","last_became_peered":"2024-09-21T07:49:30.863674+0000","last_unstale":"2024-09-21T07:49:31.897800+0000","last_undegraded":"2024-09-21T07:49:31.897800+0000","last_fullsized":"2024-09-21T07:49:31.897800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:02.165356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821318+0000","last_change":"2024-09-21T07:49:28.821318+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821318+0000","last_undegraded":"2024-09-21T07:49:28.821318+0000","last_fullsized":"2024-09-21T07:49:28.821318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.843674+0000","last_change":"2024-09-21T07:49:29.840701+0000","last_active":"2024-09-21T07:49:29.843674+0000","last_peered":"2024-09-21T07:49:29.843674+0000","last_clean":"2024-09-21T07:49:29.843674+0000","last_became_active":"2024-09-21T07:49:29.838599+0000","last_became_peered":"2024-09-21T07:49:29.838599+0000","last_unstale":"2024-09-21T07:49:29.843674+0000","last_undegraded":"2024-09-21T07:49:29.843674+0000","last_fullsized":"2024-09-21T07:49:29.843674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:14.988738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.815627+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:49:29.815627+0000","last_peered":"2024-09-21T07:49:29.815627+0000","last_clean":"2024-09-21T07:49:29.815627+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:49:29.815627+0000","last_undegraded":"2024-09-21T07:49:29.815627+0000","last_fullsized":"2024-09-21T07:49:29.815627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":74,"acting":74,"num_store_stats":5},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":280534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476785,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":280534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476782,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1265664,"data_stored":864627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476782,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":274259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":274259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1265664,"data_stored":864627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":671744,"data_stored":274259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":293701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:35.270 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:35.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.523+0000 7efefb200640 1 -- 172.21.15.88:0/4234085468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 msgr2=0x7efef40b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.523+0000 7efefb200640 1 --2- 172.21.15.88:0/4234085468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef40b3750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7efeec005fd0 tx=0x7efeec02a960 comp rx=0 tx=0).stop 2024-09-21T07:49:35.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.524+0000 7efefb200640 1 -- 172.21.15.88:0/4234085468 shutdown_connections 2024-09-21T07:49:35.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.524+0000 7efefb200640 1 --2- 172.21.15.88:0/4234085468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef40b3750 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.524+0000 7efefb200640 1 --2- 172.21.15.88:0/4234085468 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef4152920 0x7efef415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.524+0000 7efefb200640 1 -- 172.21.15.88:0/4234085468 >> 172.21.15.88:0/4234085468 conn(0x7efef406d730 msgr2=0x7efef40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.524+0000 7efefb200640 1 -- 172.21.15.88:0/4234085468 shutdown_connections 2024-09-21T07:49:35.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.524+0000 7efefb200640 1 -- 172.21.15.88:0/4234085468 wait complete. 2024-09-21T07:49:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.526+0000 7efefb200640 1 Processor -- start 2024-09-21T07:49:35.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.526+0000 7efefb200640 1 -- start start 2024-09-21T07:49:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efefb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef40aebc0 0x7efef41516d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efefb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef4152920 0x7efef4149920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efefb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef4154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efefb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efef40b41e0 con 0x7efef415f910 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efefb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efef40b4060 con 0x7efef40aebc0 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efefb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efef40b4360 con 0x7efef4152920 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef4154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef40aebc0 0x7efef41516d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef4154a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55530/0 (socket says 172.21.15.88:55530) 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 -- 172.21.15.88:0/439386910 learned_addr learned my addr 172.21.15.88:0/439386910 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef3400640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef4152920 0x7efef4149920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 -- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef4152920 msgr2=0x7efef4149920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef4152920 0x7efef4149920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 -- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef40aebc0 msgr2=0x7efef41516d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef40aebc0 0x7efef41516d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.528+0000 7efef9200640 1 -- 172.21.15.88:0/439386910 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef40cc690 con 0x7efef415f910 2024-09-21T07:49:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.529+0000 7efef9200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef4154a00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d90 tx=0x7efeec031f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.529+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec034040 con 0x7efef415f910 2024-09-21T07:49:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.529+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efeec004560 con 0x7efef415f910 2024-09-21T07:49:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.529+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efeec045480 con 0x7efef415f910 2024-09-21T07:49:35.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.529+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef40ccfc0 con 0x7efef415f910 2024-09-21T07:49:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.529+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef4150260 con 0x7efef415f910 2024-09-21T07:49:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.530+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef40ccdb0 con 0x7efef415f910 2024-09-21T07:49:35.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.531+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7efeec037020 con 0x7efef415f910 2024-09-21T07:49:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.532+0000 7efeebe00640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efed4078d70 0x7efed407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:35.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.532+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7efeec0bc7c0 con 0x7efef415f910 2024-09-21T07:49:35.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.533+0000 7efef3e00640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efed4078d70 0x7efed407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:35.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.534+0000 7efef3e00640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efed4078d70 0x7efed407b280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee4006fd0 tx=0x7efee4009060 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:35.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.537+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efef415fd20 con 0x7efef415f910 2024-09-21T07:49:35.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.720+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efef40cc410 con 0x7efed4078d70 2024-09-21T07:49:35.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.725+0000 7efeebe00640 1 -- 172.21.15.88:0/439386910 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+244742 (secure 0 0 0) 0x7efef40cc410 con 0x7efed4078d70 2024-09-21T07:49:35.730 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:35.731 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efed4078d70 msgr2=0x7efed407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efed4078d70 0x7efed407b280 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7efee4006fd0 tx=0x7efee4009060 comp rx=0 tx=0).stop 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 msgr2=0x7efef4154a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef4154a00 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d90 tx=0x7efeec031f80 comp rx=0 tx=0).stop 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 shutdown_connections 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7efed4078d70 0x7efed407b280 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efef4152920 0x7efef4149920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efef415f910 0x7efef4154a00 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 --2- 172.21.15.88:0/439386910 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efef40aebc0 0x7efef41516d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 >> 172.21.15.88:0/439386910 conn(0x7efef406d730 msgr2=0x7efef415f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 shutdown_connections 2024-09-21T07:49:35.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:35.729+0000 7efefb200640 1 -- 172.21.15.88:0/439386910 wait complete. 2024-09-21T07:49:35.756 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":166,"stamp":"2024-09-21T07:49:33.912034+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590390,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":105,"num_write_kb":1629,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":140,"acting":140,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":15,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":243548,"kb_used_data":6476,"kb_used_omap":12,"kb_used_meta":236851,"kb_avail":754731172,"statfs":{"total":773094113280,"available":772844720128,"internally_reserved":0,"allocated":6631424,"data_stored":3406800,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":242535985},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.102038"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898139+0000","last_change":"2024-09-21T07:49:30.866746+0000","last_active":"2024-09-21T07:49:31.898139+0000","last_peered":"2024-09-21T07:49:31.898139+0000","last_clean":"2024-09-21T07:49:31.898139+0000","last_became_active":"2024-09-21T07:49:30.866386+0000","last_became_peered":"2024-09-21T07:49:30.866386+0000","last_unstale":"2024-09-21T07:49:31.898139+0000","last_undegraded":"2024-09-21T07:49:31.898139+0000","last_fullsized":"2024-09-21T07:49:31.898139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:41.251356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.830568+0000","last_change":"2024-09-21T07:49:29.830568+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.830568+0000","last_undegraded":"2024-09-21T07:49:29.830568+0000","last_fullsized":"2024-09-21T07:49:29.830568+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.821239+0000","last_change":"2024-09-21T07:49:29.821239+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.821239+0000","last_undegraded":"2024-09-21T07:49:29.821239+0000","last_fullsized":"2024-09-21T07:49:29.821239+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.823537+0000","last_change":"2024-09-21T07:49:29.823537+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.823537+0000","last_undegraded":"2024-09-21T07:49:29.823537+0000","last_fullsized":"2024-09-21T07:49:29.823537+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820108+0000","last_change":"2024-09-21T07:49:29.820108+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820108+0000","last_undegraded":"2024-09-21T07:49:29.820108+0000","last_fullsized":"2024-09-21T07:49:29.820108+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.819998+0000","last_change":"2024-09-21T07:49:29.819998+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.819998+0000","last_undegraded":"2024-09-21T07:49:29.819998+0000","last_fullsized":"2024-09-21T07:49:29.819998+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.828713+0000","last_change":"2024-09-21T07:49:29.828713+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.828713+0000","last_undegraded":"2024-09-21T07:49:29.828713+0000","last_fullsized":"2024-09-21T07:49:29.828713+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.825135+0000","last_change":"2024-09-21T07:49:29.825135+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.825135+0000","last_undegraded":"2024-09-21T07:49:29.825135+0000","last_fullsized":"2024-09-21T07:49:29.825135+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.828250+0000","last_change":"2024-09-21T07:49:29.828250+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.828250+0000","last_undegraded":"2024-09-21T07:49:29.828250+0000","last_fullsized":"2024-09-21T07:49:29.828250+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.826636+0000","last_change":"2024-09-21T07:49:29.826636+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.826636+0000","last_undegraded":"2024-09-21T07:49:29.826636+0000","last_fullsized":"2024-09-21T07:49:29.826636+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897965+0000","last_change":"2024-09-21T07:49:30.870745+0000","last_active":"2024-09-21T07:49:31.897965+0000","last_peered":"2024-09-21T07:49:31.897965+0000","last_clean":"2024-09-21T07:49:31.897965+0000","last_became_active":"2024-09-21T07:49:30.870361+0000","last_became_peered":"2024-09-21T07:49:30.870361+0000","last_unstale":"2024-09-21T07:49:31.897965+0000","last_undegraded":"2024-09-21T07:49:31.897965+0000","last_fullsized":"2024-09-21T07:49:31.897965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:18.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.828162+0000","last_change":"2024-09-21T07:49:29.828162+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.828162+0000","last_undegraded":"2024-09-21T07:49:29.828162+0000","last_fullsized":"2024-09-21T07:49:29.828162+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.822253+0000","last_change":"2024-09-21T07:49:29.822253+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.822253+0000","last_undegraded":"2024-09-21T07:49:29.822253+0000","last_fullsized":"2024-09-21T07:49:29.822253+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.824004+0000","last_change":"2024-09-21T07:49:29.824004+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.824004+0000","last_undegraded":"2024-09-21T07:49:29.824004+0000","last_fullsized":"2024-09-21T07:49:29.824004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898160+0000","last_change":"2024-09-21T07:49:30.866983+0000","last_active":"2024-09-21T07:49:31.898160+0000","last_peered":"2024-09-21T07:49:31.898160+0000","last_clean":"2024-09-21T07:49:31.898160+0000","last_became_active":"2024-09-21T07:49:30.866405+0000","last_became_peered":"2024-09-21T07:49:30.866405+0000","last_unstale":"2024-09-21T07:49:31.898160+0000","last_undegraded":"2024-09-21T07:49:31.898160+0000","last_fullsized":"2024-09-21T07:49:31.898160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:33.824401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.827426+0000","last_change":"2024-09-21T07:49:29.827426+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.827426+0000","last_undegraded":"2024-09-21T07:49:29.827426+0000","last_fullsized":"2024-09-21T07:49:29.827426+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898197+0000","last_change":"2024-09-21T07:49:30.868732+0000","last_active":"2024-09-21T07:49:31.898197+0000","last_peered":"2024-09-21T07:49:31.898197+0000","last_clean":"2024-09-21T07:49:31.898197+0000","last_became_active":"2024-09-21T07:49:30.868358+0000","last_became_peered":"2024-09-21T07:49:30.868358+0000","last_unstale":"2024-09-21T07:49:31.898197+0000","last_undegraded":"2024-09-21T07:49:31.898197+0000","last_fullsized":"2024-09-21T07:49:31.898197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:09.597807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.821842+0000","last_change":"2024-09-21T07:49:29.821842+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.821842+0000","last_undegraded":"2024-09-21T07:49:29.821842+0000","last_fullsized":"2024-09-21T07:49:29.821842+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897737+0000","last_change":"2024-09-21T07:49:30.862236+0000","last_active":"2024-09-21T07:49:31.897737+0000","last_peered":"2024-09-21T07:49:31.897737+0000","last_clean":"2024-09-21T07:49:31.897737+0000","last_became_active":"2024-09-21T07:49:30.861285+0000","last_became_peered":"2024-09-21T07:49:30.861285+0000","last_unstale":"2024-09-21T07:49:31.897737+0000","last_undegraded":"2024-09-21T07:49:31.897737+0000","last_fullsized":"2024-09-21T07:49:31.897737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.866684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1f","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.813446+0000","last_change":"2024-09-21T07:49:28.813446+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.813446+0000","last_undegraded":"2024-09-21T07:49:28.813446+0000","last_fullsized":"2024-09-21T07:49:28.813446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898195+0000","last_change":"2024-09-21T07:49:29.844161+0000","last_active":"2024-09-21T07:49:31.898195+0000","last_peered":"2024-09-21T07:49:31.898195+0000","last_clean":"2024-09-21T07:49:31.898195+0000","last_became_active":"2024-09-21T07:49:29.843722+0000","last_became_peered":"2024-09-21T07:49:29.843722+0000","last_unstale":"2024-09-21T07:49:31.898195+0000","last_undegraded":"2024-09-21T07:49:31.898195+0000","last_fullsized":"2024-09-21T07:49:31.898195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:56.739749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1d","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817430+0000","last_change":"2024-09-21T07:49:28.817430+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817430+0000","last_undegraded":"2024-09-21T07:49:28.817430+0000","last_fullsized":"2024-09-21T07:49:28.817430+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.842132+0000","last_change":"2024-09-21T07:49:29.839504+0000","last_active":"2024-09-21T07:49:29.842132+0000","last_peered":"2024-09-21T07:49:29.842132+0000","last_clean":"2024-09-21T07:49:29.842132+0000","last_became_active":"2024-09-21T07:49:29.838264+0000","last_became_peered":"2024-09-21T07:49:29.838264+0000","last_unstale":"2024-09-21T07:49:29.842132+0000","last_undegraded":"2024-09-21T07:49:29.842132+0000","last_fullsized":"2024-09-21T07:49:29.842132+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:44.500806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.842818+0000","last_change":"2024-09-21T07:49:29.835030+0000","last_active":"2024-09-21T07:49:29.842818+0000","last_peered":"2024-09-21T07:49:29.842818+0000","last_clean":"2024-09-21T07:49:29.842818+0000","last_became_active":"2024-09-21T07:49:29.832816+0000","last_became_peered":"2024-09-21T07:49:29.832816+0000","last_unstale":"2024-09-21T07:49:29.842818+0000","last_undegraded":"2024-09-21T07:49:29.842818+0000","last_fullsized":"2024-09-21T07:49:29.842818+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:59:46.877552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817381+0000","last_change":"2024-09-21T07:49:28.817381+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817381+0000","last_undegraded":"2024-09-21T07:49:28.817381+0000","last_fullsized":"2024-09-21T07:49:28.817381+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.823219+0000","last_change":"2024-09-21T07:49:29.823219+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.823219+0000","last_undegraded":"2024-09-21T07:49:29.823219+0000","last_fullsized":"2024-09-21T07:49:29.823219+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.837330+0000","last_change":"2024-09-21T07:49:29.835362+0000","last_active":"2024-09-21T07:49:29.837330+0000","last_peered":"2024-09-21T07:49:29.837330+0000","last_clean":"2024-09-21T07:49:29.837330+0000","last_became_active":"2024-09-21T07:49:29.834971+0000","last_became_peered":"2024-09-21T07:49:29.834971+0000","last_unstale":"2024-09-21T07:49:29.837330+0000","last_undegraded":"2024-09-21T07:49:29.837330+0000","last_fullsized":"2024-09-21T07:49:29.837330+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:59.735243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.834824+0000","last_change":"2024-09-21T07:49:29.830256+0000","last_active":"2024-09-21T07:49:29.834824+0000","last_peered":"2024-09-21T07:49:29.834824+0000","last_clean":"2024-09-21T07:49:29.834824+0000","last_became_active":"2024-09-21T07:49:29.829961+0000","last_became_peered":"2024-09-21T07:49:29.829961+0000","last_unstale":"2024-09-21T07:49:29.834824+0000","last_undegraded":"2024-09-21T07:49:29.834824+0000","last_fullsized":"2024-09-21T07:49:29.834824+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:50.193927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.822717+0000","last_change":"2024-09-21T07:49:29.822717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.822717+0000","last_undegraded":"2024-09-21T07:49:29.822717+0000","last_fullsized":"2024-09-21T07:49:29.822717+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.845698+0000","last_change":"2024-09-21T07:49:29.841680+0000","last_active":"2024-09-21T07:49:29.845698+0000","last_peered":"2024-09-21T07:49:29.845698+0000","last_clean":"2024-09-21T07:49:29.845698+0000","last_became_active":"2024-09-21T07:49:29.841344+0000","last_became_peered":"2024-09-21T07:49:29.841344+0000","last_unstale":"2024-09-21T07:49:29.845698+0000","last_undegraded":"2024-09-21T07:49:29.845698+0000","last_fullsized":"2024-09-21T07:49:29.845698+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:32.379324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.830128+0000","last_change":"2024-09-21T07:49:29.830128+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.830128+0000","last_undegraded":"2024-09-21T07:49:29.830128+0000","last_fullsized":"2024-09-21T07:49:29.830128+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821980+0000","last_change":"2024-09-21T07:49:28.821980+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821980+0000","last_undegraded":"2024-09-21T07:49:28.821980+0000","last_fullsized":"2024-09-21T07:49:28.821980+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.843477+0000","last_change":"2024-09-21T07:49:29.842330+0000","last_active":"2024-09-21T07:49:29.843477+0000","last_peered":"2024-09-21T07:49:29.843477+0000","last_clean":"2024-09-21T07:49:29.843477+0000","last_became_active":"2024-09-21T07:49:29.842047+0000","last_became_peered":"2024-09-21T07:49:29.842047+0000","last_unstale":"2024-09-21T07:49:29.843477+0000","last_undegraded":"2024-09-21T07:49:29.843477+0000","last_fullsized":"2024-09-21T07:49:29.843477+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:04.477193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.821103+0000","last_change":"2024-09-21T07:49:29.821103+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.821103+0000","last_undegraded":"2024-09-21T07:49:29.821103+0000","last_fullsized":"2024-09-21T07:49:29.821103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.844213+0000","last_change":"2024-09-21T07:49:29.842473+0000","last_active":"2024-09-21T07:49:29.844213+0000","last_peered":"2024-09-21T07:49:29.844213+0000","last_clean":"2024-09-21T07:49:29.844213+0000","last_became_active":"2024-09-21T07:49:29.842149+0000","last_became_peered":"2024-09-21T07:49:29.842149+0000","last_unstale":"2024-09-21T07:49:29.844213+0000","last_undegraded":"2024-09-21T07:49:29.844213+0000","last_fullsized":"2024-09-21T07:49:29.844213+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:23.324347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820635+0000","last_change":"2024-09-21T07:49:29.820635+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820635+0000","last_undegraded":"2024-09-21T07:49:29.820635+0000","last_fullsized":"2024-09-21T07:49:29.820635+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821391+0000","last_change":"2024-09-21T07:49:28.821391+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821391+0000","last_undegraded":"2024-09-21T07:49:28.821391+0000","last_fullsized":"2024-09-21T07:49:28.821391+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.824711+0000","last_change":"2024-09-21T07:49:29.824711+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.824711+0000","last_undegraded":"2024-09-21T07:49:29.824711+0000","last_fullsized":"2024-09-21T07:49:29.824711+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.840466+0000","last_change":"2024-09-21T07:49:29.835475+0000","last_active":"2024-09-21T07:49:29.840466+0000","last_peered":"2024-09-21T07:49:29.840466+0000","last_clean":"2024-09-21T07:49:29.840466+0000","last_became_active":"2024-09-21T07:49:29.835130+0000","last_became_peered":"2024-09-21T07:49:29.835130+0000","last_unstale":"2024-09-21T07:49:29.840466+0000","last_undegraded":"2024-09-21T07:49:29.840466+0000","last_fullsized":"2024-09-21T07:49:29.840466+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:24.392164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.822092+0000","last_change":"2024-09-21T07:49:29.822092+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.822092+0000","last_undegraded":"2024-09-21T07:49:29.822092+0000","last_fullsized":"2024-09-21T07:49:29.822092+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.828256+0000","last_change":"2024-09-21T07:49:29.825131+0000","last_active":"2024-09-21T07:49:29.828256+0000","last_peered":"2024-09-21T07:49:29.828256+0000","last_clean":"2024-09-21T07:49:29.828256+0000","last_became_active":"2024-09-21T07:49:29.824617+0000","last_became_peered":"2024-09-21T07:49:29.824617+0000","last_unstale":"2024-09-21T07:49:29.828256+0000","last_undegraded":"2024-09-21T07:49:29.828256+0000","last_fullsized":"2024-09-21T07:49:29.828256+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:39.318132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820093+0000","last_change":"2024-09-21T07:49:29.820093+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820093+0000","last_undegraded":"2024-09-21T07:49:29.820093+0000","last_fullsized":"2024-09-21T07:49:29.820093+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.839701+0000","last_change":"2024-09-21T07:49:29.836030+0000","last_active":"2024-09-21T07:49:29.839701+0000","last_peered":"2024-09-21T07:49:29.839701+0000","last_clean":"2024-09-21T07:49:29.839701+0000","last_became_active":"2024-09-21T07:49:29.835703+0000","last_became_peered":"2024-09-21T07:49:29.835703+0000","last_unstale":"2024-09-21T07:49:29.839701+0000","last_undegraded":"2024-09-21T07:49:29.839701+0000","last_fullsized":"2024-09-21T07:49:29.839701+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:44.963378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.826148+0000","last_change":"2024-09-21T07:49:29.826148+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.826148+0000","last_undegraded":"2024-09-21T07:49:29.826148+0000","last_fullsized":"2024-09-21T07:49:29.826148+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:29.843999+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:29.843591+0000","last_became_peered":"2024-09-21T07:49:29.843591+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:56.147012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.829668+0000","last_change":"2024-09-21T07:49:29.829668+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.829668+0000","last_undegraded":"2024-09-21T07:49:29.829668+0000","last_fullsized":"2024-09-21T07:49:29.829668+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821309+0000","last_change":"2024-09-21T07:49:28.821309+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821309+0000","last_undegraded":"2024-09-21T07:49:28.821309+0000","last_fullsized":"2024-09-21T07:49:28.821309+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897841+0000","last_change":"2024-09-21T07:49:30.861727+0000","last_active":"2024-09-21T07:49:31.897841+0000","last_peered":"2024-09-21T07:49:31.897841+0000","last_clean":"2024-09-21T07:49:31.897841+0000","last_became_active":"2024-09-21T07:49:30.861401+0000","last_became_peered":"2024-09-21T07:49:30.861401+0000","last_unstale":"2024-09-21T07:49:31.897841+0000","last_undegraded":"2024-09-21T07:49:31.897841+0000","last_fullsized":"2024-09-21T07:49:31.897841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:59.043011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.845663+0000","last_change":"2024-09-21T07:49:29.843565+0000","last_active":"2024-09-21T07:49:29.845663+0000","last_peered":"2024-09-21T07:49:29.845663+0000","last_clean":"2024-09-21T07:49:29.845663+0000","last_became_active":"2024-09-21T07:49:29.843265+0000","last_became_peered":"2024-09-21T07:49:29.843265+0000","last_unstale":"2024-09-21T07:49:29.845663+0000","last_undegraded":"2024-09-21T07:49:29.845663+0000","last_fullsized":"2024-09-21T07:49:29.845663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:01:40.743393+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820577+0000","last_change":"2024-09-21T07:49:29.820577+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820577+0000","last_undegraded":"2024-09-21T07:49:29.820577+0000","last_fullsized":"2024-09-21T07:49:29.820577+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821343+0000","last_change":"2024-09-21T07:49:28.821343+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821343+0000","last_undegraded":"2024-09-21T07:49:28.821343+0000","last_fullsized":"2024-09-21T07:49:28.821343+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.820172+0000","last_change":"2024-09-21T07:49:29.820172+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.820172+0000","last_undegraded":"2024-09-21T07:49:29.820172+0000","last_fullsized":"2024-09-21T07:49:29.820172+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.813545+0000","last_change":"2024-09-21T07:49:28.813545+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.813545+0000","last_undegraded":"2024-09-21T07:49:28.813545+0000","last_fullsized":"2024-09-21T07:49:28.813545+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821392+0000","last_change":"2024-09-21T07:49:28.821392+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821392+0000","last_undegraded":"2024-09-21T07:49:28.821392+0000","last_fullsized":"2024-09-21T07:49:28.821392+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.834989+0000","last_change":"2024-09-21T07:49:29.832146+0000","last_active":"2024-09-21T07:49:29.834989+0000","last_peered":"2024-09-21T07:49:29.834989+0000","last_clean":"2024-09-21T07:49:29.834989+0000","last_became_active":"2024-09-21T07:49:29.831853+0000","last_became_peered":"2024-09-21T07:49:29.831853+0000","last_unstale":"2024-09-21T07:49:29.834989+0000","last_undegraded":"2024-09-21T07:49:29.834989+0000","last_fullsized":"2024-09-21T07:49:29.834989+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:15.388689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.825669+0000","last_change":"2024-09-21T07:49:29.825669+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.825669+0000","last_undegraded":"2024-09-21T07:49:29.825669+0000","last_fullsized":"2024-09-21T07:49:29.825669+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898152+0000","last_change":"2024-09-21T07:49:29.843093+0000","last_active":"2024-09-21T07:49:31.898152+0000","last_peered":"2024-09-21T07:49:31.898152+0000","last_clean":"2024-09-21T07:49:31.898152+0000","last_became_active":"2024-09-21T07:49:29.842766+0000","last_became_peered":"2024-09-21T07:49:29.842766+0000","last_unstale":"2024-09-21T07:49:31.898152+0000","last_undegraded":"2024-09-21T07:49:31.898152+0000","last_fullsized":"2024-09-21T07:49:31.898152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:50.990394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.827205+0000","last_change":"2024-09-21T07:49:29.827205+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.827205+0000","last_undegraded":"2024-09-21T07:49:29.827205+0000","last_fullsized":"2024-09-21T07:49:29.827205+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817386+0000","last_change":"2024-09-21T07:49:28.817386+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817386+0000","last_undegraded":"2024-09-21T07:49:28.817386+0000","last_fullsized":"2024-09-21T07:49:28.817386+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:30.868811+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:30.868487+0000","last_became_peered":"2024-09-21T07:49:30.868487+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:14.218794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.841761+0000","last_change":"2024-09-21T07:49:29.840533+0000","last_active":"2024-09-21T07:49:29.841761+0000","last_peered":"2024-09-21T07:49:29.841761+0000","last_clean":"2024-09-21T07:49:29.841761+0000","last_became_active":"2024-09-21T07:49:29.840191+0000","last_became_peered":"2024-09-21T07:49:29.840191+0000","last_unstale":"2024-09-21T07:49:29.841761+0000","last_undegraded":"2024-09-21T07:49:29.841761+0000","last_fullsized":"2024-09-21T07:49:29.841761+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:11:56.199141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.843001+0000","last_change":"2024-09-21T07:49:29.838408+0000","last_active":"2024-09-21T07:49:29.843001+0000","last_peered":"2024-09-21T07:49:29.843001+0000","last_clean":"2024-09-21T07:49:29.843001+0000","last_became_active":"2024-09-21T07:49:29.838107+0000","last_became_peered":"2024-09-21T07:49:29.838107+0000","last_unstale":"2024-09-21T07:49:29.843001+0000","last_undegraded":"2024-09-21T07:49:29.843001+0000","last_fullsized":"2024-09-21T07:49:29.843001+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:04:58.652422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897754+0000","last_change":"2024-09-21T07:49:29.842367+0000","last_active":"2024-09-21T07:49:31.897754+0000","last_peered":"2024-09-21T07:49:31.897754+0000","last_clean":"2024-09-21T07:49:31.897754+0000","last_became_active":"2024-09-21T07:49:29.842035+0000","last_became_peered":"2024-09-21T07:49:29.842035+0000","last_unstale":"2024-09-21T07:49:31.897754+0000","last_undegraded":"2024-09-21T07:49:31.897754+0000","last_fullsized":"2024-09-21T07:49:31.897754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:53.958285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":1,"reported_epoch":58,"state":"creating+peering","last_fresh":"2024-09-21T07:49:29.826591+0000","last_change":"2024-09-21T07:49:29.826591+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.826591+0000","last_undegraded":"2024-09-21T07:49:29.826591+0000","last_fullsized":"2024-09-21T07:49:29.826591+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.844126+0000","last_change":"2024-09-21T07:49:29.841930+0000","last_active":"2024-09-21T07:49:29.844126+0000","last_peered":"2024-09-21T07:49:29.844126+0000","last_clean":"2024-09-21T07:49:29.844126+0000","last_became_active":"2024-09-21T07:49:29.841523+0000","last_became_peered":"2024-09-21T07:49:29.841523+0000","last_unstale":"2024-09-21T07:49:29.844126+0000","last_undegraded":"2024-09-21T07:49:29.844126+0000","last_fullsized":"2024-09-21T07:49:29.844126+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:35.269020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.817411+0000","last_change":"2024-09-21T07:49:28.817411+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.817411+0000","last_undegraded":"2024-09-21T07:49:28.817411+0000","last_fullsized":"2024-09-21T07:49:28.817411+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897800+0000","last_change":"2024-09-21T07:49:30.866291+0000","last_active":"2024-09-21T07:49:31.897800+0000","last_peered":"2024-09-21T07:49:31.897800+0000","last_clean":"2024-09-21T07:49:31.897800+0000","last_became_active":"2024-09-21T07:49:30.863674+0000","last_became_peered":"2024-09-21T07:49:30.863674+0000","last_unstale":"2024-09-21T07:49:31.897800+0000","last_undegraded":"2024-09-21T07:49:31.897800+0000","last_fullsized":"2024-09-21T07:49:31.897800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:02.165356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":1,"reported_epoch":57,"state":"creating+peering","last_fresh":"2024-09-21T07:49:28.821318+0000","last_change":"2024-09-21T07:49:28.821318+0000","last_active":"2024-09-21T07:49:28.801255+0000","last_peered":"2024-09-21T07:49:28.801255+0000","last_clean":"2024-09-21T07:49:28.801255+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:28.821318+0000","last_undegraded":"2024-09-21T07:49:28.821318+0000","last_fullsized":"2024-09-21T07:49:28.821318+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:49:29.808717+0000","last_change":"2024-09-21T07:49:29.808717+0000","last_active":"2024-09-21T07:49:29.808717+0000","last_peered":"2024-09-21T07:49:29.808717+0000","last_clean":"2024-09-21T07:49:29.808717+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:49:29.808717+0000","last_undegraded":"2024-09-21T07:49:29.808717+0000","last_fullsized":"2024-09-21T07:49:29.808717+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":10,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.843674+0000","last_change":"2024-09-21T07:49:29.840701+0000","last_active":"2024-09-21T07:49:29.843674+0000","last_peered":"2024-09-21T07:49:29.843674+0000","last_clean":"2024-09-21T07:49:29.843674+0000","last_became_active":"2024-09-21T07:49:29.838599+0000","last_became_peered":"2024-09-21T07:49:29.838599+0000","last_unstale":"2024-09-21T07:49:29.843674+0000","last_undegraded":"2024-09-21T07:49:29.843674+0000","last_fullsized":"2024-09-21T07:49:29.843674+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:31:14.988738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":178,"reported_epoch":58,"state":"active+clean","last_fresh":"2024-09-21T07:49:29.815627+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:49:29.815627+0000","last_peered":"2024-09-21T07:49:29.815627+0000","last_clean":"2024-09-21T07:49:29.815627+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:49:29.815627+0000","last_undegraded":"2024-09-21T07:49:29.815627+0000","last_fullsized":"2024-09-21T07:49:29.815627+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":74,"acting":74,"num_store_stats":5},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476785,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":280534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476785,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31856,"kb_used_data":668,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339984,"statfs":{"total":96636764160,"available":96604143616,"internally_reserved":0,"allocated":684032,"data_stored":280534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476782,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1265664,"data_stored":864627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476782,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":274259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509485,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":671744,"data_stored":274259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509486,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1265664,"data_stored":864627,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509486,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27700,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344140,"statfs":{"total":96636764160,"available":96608399360,"internally_reserved":0,"allocated":671744,"data_stored":274259,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":293701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:35.760 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15bec00640 1 -- 172.21.15.88:0/3465078518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 msgr2=0x7f15a0005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15bec00640 1 --2- 172.21.15.88:0/3465078518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15a0005240 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c610 tx=0x7f15a0005830 comp rx=0 tx=0).stop 2024-09-21T07:49:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15be200640 1 -- 172.21.15.88:0/3465078518 reap_dead start 2024-09-21T07:49:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15bec00640 1 -- 172.21.15.88:0/3465078518 shutdown_connections 2024-09-21T07:49:36.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15bec00640 1 -- 172.21.15.88:0/3465078518 >> 172.21.15.88:0/3465078518 conn(0x7f15b808bfe0 msgr2=0x7f15b809d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:36.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15bec00640 1 -- 172.21.15.88:0/3465078518 shutdown_connections 2024-09-21T07:49:36.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.017+0000 7f15bec00640 1 -- 172.21.15.88:0/3465078518 wait complete. 2024-09-21T07:49:36.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.019+0000 7f15bec00640 1 Processor -- start 2024-09-21T07:49:36.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.019+0000 7f15bec00640 1 -- start start 2024-09-21T07:49:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.020+0000 7f15bec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15b814f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15b8152410 0x7f15b80a1660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15b80a1ba0 0x7f15b809f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15b80d2830 con 0x7f15a0004d90 2024-09-21T07:49:36.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15b80d26b0 con 0x7f15b8152410 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15b80d4330 con 0x7f15b80a1ba0 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15b814f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15b814f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45670/0 (socket says 172.21.15.88:45670) 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 -- 172.21.15.88:0/3630102943 learned_addr learned my addr 172.21.15.88:0/3630102943 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bce00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15b8152410 0x7f15b80a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15be200640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15b80a1ba0 0x7f15b809f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 -- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15b80a1ba0 msgr2=0x7f15b809f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15b80a1ba0 0x7f15b809f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 -- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15b8152410 msgr2=0x7f15b80a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15b8152410 0x7f15b80a1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.021+0000 7f15bd800640 1 -- 172.21.15.88:0/3630102943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15b815e6f0 con 0x7f15a0004d90 2024-09-21T07:49:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.022+0000 7f15bd800640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15b814f360 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f15ac003ec0 tx=0x7f15ac001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.022+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac015070 con 0x7f15a0004d90 2024-09-21T07:49:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.022+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15ac011920 con 0x7f15a0004d90 2024-09-21T07:49:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.022+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15b815e9d0 con 0x7f15a0004d90 2024-09-21T07:49:36.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.022+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15ac011d70 con 0x7f15a0004d90 2024-09-21T07:49:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.022+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15b815ef90 con 0x7f15a0004d90 2024-09-21T07:49:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.023+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15b80d4890 con 0x7f15a0004d90 2024-09-21T07:49:36.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.024+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f15ac018070 con 0x7f15a0004d90 2024-09-21T07:49:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.025+0000 7f15b5600640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1598078d40 0x7f159807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.025+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f15ac09be80 con 0x7f15a0004d90 2024-09-21T07:49:36.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.026+0000 7f15bce00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1598078d40 0x7f159807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.027+0000 7f15bce00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1598078d40 0x7f159807b250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f15b815e430 tx=0x7f15a4005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:36.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.030+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15ac063840 con 0x7f15a0004d90 2024-09-21T07:49:36.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.213+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f15b8162000 con 0x7f1598078d40 2024-09-21T07:49:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.218+0000 7f15b5600640 1 -- 172.21.15.88:0/3630102943 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f15a405e840 con 0x7f1598078d40 2024-09-21T07:49:36.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:36.224 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.222+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1598078d40 msgr2=0x7f159807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.222+0000 7f15bec00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1598078d40 0x7f159807b250 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f15b815e430 tx=0x7f15a4005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.222+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 msgr2=0x7f15b814f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.222+0000 7f15bec00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15b814f360 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f15ac003ec0 tx=0x7f15ac001800 comp rx=0 tx=0).stop 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.222+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 shutdown_connections 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1598078d40 0x7f159807b250 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15b80a1ba0 0x7f15b809f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15b8152410 0x7f15b80a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 --2- 172.21.15.88:0/3630102943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15a0004d90 0x7f15b814f360 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 >> 172.21.15.88:0/3630102943 conn(0x7f15b808bfe0 msgr2=0x7f15b8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:36.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 shutdown_connections 2024-09-21T07:49:36.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.223+0000 7f15bec00640 1 -- 172.21.15.88:0/3630102943 wait complete. 2024-09-21T07:49:36.250 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-21T07:49:35.912601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252144,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722576,"statfs":{"total":773094113280,"available":772835917824,"internally_reserved":0,"allocated":6922240,"data_stored":3534428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.098966"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858884+0000","last_change":"2024-09-21T07:49:30.865835+0000","last_active":"2024-09-21T07:49:31.858884+0000","last_peered":"2024-09-21T07:49:31.858884+0000","last_clean":"2024-09-21T07:49:31.858884+0000","last_became_active":"2024-09-21T07:49:30.863575+0000","last_became_peered":"2024-09-21T07:49:30.863575+0000","last_unstale":"2024-09-21T07:49:31.858884+0000","last_undegraded":"2024-09-21T07:49:31.858884+0000","last_fullsized":"2024-09-21T07:49:31.858884+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:26.732169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898139+0000","last_change":"2024-09-21T07:49:30.866746+0000","last_active":"2024-09-21T07:49:31.898139+0000","last_peered":"2024-09-21T07:49:31.898139+0000","last_clean":"2024-09-21T07:49:31.898139+0000","last_became_active":"2024-09-21T07:49:30.866386+0000","last_became_peered":"2024-09-21T07:49:30.866386+0000","last_unstale":"2024-09-21T07:49:31.898139+0000","last_undegraded":"2024-09-21T07:49:31.898139+0000","last_fullsized":"2024-09-21T07:49:31.898139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:41.251356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861624+0000","last_change":"2024-09-21T07:49:30.842935+0000","last_active":"2024-09-21T07:49:31.861624+0000","last_peered":"2024-09-21T07:49:31.861624+0000","last_clean":"2024-09-21T07:49:31.861624+0000","last_became_active":"2024-09-21T07:49:30.841630+0000","last_became_peered":"2024-09-21T07:49:30.841630+0000","last_unstale":"2024-09-21T07:49:31.861624+0000","last_undegraded":"2024-09-21T07:49:31.861624+0000","last_fullsized":"2024-09-21T07:49:31.861624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:22.601026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898018+0000","last_change":"2024-09-21T07:49:30.862748+0000","last_active":"2024-09-21T07:49:31.898018+0000","last_peered":"2024-09-21T07:49:31.898018+0000","last_clean":"2024-09-21T07:49:31.898018+0000","last_became_active":"2024-09-21T07:49:30.862527+0000","last_became_peered":"2024-09-21T07:49:30.862527+0000","last_unstale":"2024-09-21T07:49:31.898018+0000","last_undegraded":"2024-09-21T07:49:31.898018+0000","last_fullsized":"2024-09-21T07:49:31.898018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:17.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858588+0000","last_change":"2024-09-21T07:49:30.857817+0000","last_active":"2024-09-21T07:49:31.858588+0000","last_peered":"2024-09-21T07:49:31.858588+0000","last_clean":"2024-09-21T07:49:31.858588+0000","last_became_active":"2024-09-21T07:49:30.857065+0000","last_became_peered":"2024-09-21T07:49:30.857065+0000","last_unstale":"2024-09-21T07:49:31.858588+0000","last_undegraded":"2024-09-21T07:49:31.858588+0000","last_fullsized":"2024-09-21T07:49:31.858588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:39.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858182+0000","last_change":"2024-09-21T07:49:30.850469+0000","last_active":"2024-09-21T07:49:31.858182+0000","last_peered":"2024-09-21T07:49:31.858182+0000","last_clean":"2024-09-21T07:49:31.858182+0000","last_became_active":"2024-09-21T07:49:30.850119+0000","last_became_peered":"2024-09-21T07:49:30.850119+0000","last_unstale":"2024-09-21T07:49:31.858182+0000","last_undegraded":"2024-09-21T07:49:31.858182+0000","last_fullsized":"2024-09-21T07:49:31.858182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:23.015017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897597+0000","last_change":"2024-09-21T07:49:30.845351+0000","last_active":"2024-09-21T07:49:31.897597+0000","last_peered":"2024-09-21T07:49:31.897597+0000","last_clean":"2024-09-21T07:49:31.897597+0000","last_became_active":"2024-09-21T07:49:30.843920+0000","last_became_peered":"2024-09-21T07:49:30.843920+0000","last_unstale":"2024-09-21T07:49:31.897597+0000","last_undegraded":"2024-09-21T07:49:31.897597+0000","last_fullsized":"2024-09-21T07:49:31.897597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:25.494394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851362+0000","last_change":"2024-09-21T07:49:30.850184+0000","last_active":"2024-09-21T07:49:31.851362+0000","last_peered":"2024-09-21T07:49:31.851362+0000","last_clean":"2024-09-21T07:49:31.851362+0000","last_became_active":"2024-09-21T07:49:30.849571+0000","last_became_peered":"2024-09-21T07:49:30.849571+0000","last_unstale":"2024-09-21T07:49:31.851362+0000","last_undegraded":"2024-09-21T07:49:31.851362+0000","last_fullsized":"2024-09-21T07:49:31.851362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:25.897438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849745+0000","last_change":"2024-09-21T07:49:30.858444+0000","last_active":"2024-09-21T07:49:31.849745+0000","last_peered":"2024-09-21T07:49:31.849745+0000","last_clean":"2024-09-21T07:49:31.849745+0000","last_became_active":"2024-09-21T07:49:30.857207+0000","last_became_peered":"2024-09-21T07:49:30.857207+0000","last_unstale":"2024-09-21T07:49:31.849745+0000","last_undegraded":"2024-09-21T07:49:31.849745+0000","last_fullsized":"2024-09-21T07:49:31.849745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:43.445886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858632+0000","last_change":"2024-09-21T07:49:30.849692+0000","last_active":"2024-09-21T07:49:31.858632+0000","last_peered":"2024-09-21T07:49:31.858632+0000","last_clean":"2024-09-21T07:49:31.858632+0000","last_became_active":"2024-09-21T07:49:30.849329+0000","last_became_peered":"2024-09-21T07:49:30.849329+0000","last_unstale":"2024-09-21T07:49:31.858632+0000","last_undegraded":"2024-09-21T07:49:31.858632+0000","last_fullsized":"2024-09-21T07:49:31.858632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:50.221480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852174+0000","last_change":"2024-09-21T07:49:30.857819+0000","last_active":"2024-09-21T07:49:31.852174+0000","last_peered":"2024-09-21T07:49:31.852174+0000","last_clean":"2024-09-21T07:49:31.852174+0000","last_became_active":"2024-09-21T07:49:30.857491+0000","last_became_peered":"2024-09-21T07:49:30.857491+0000","last_unstale":"2024-09-21T07:49:31.852174+0000","last_undegraded":"2024-09-21T07:49:31.852174+0000","last_fullsized":"2024-09-21T07:49:31.852174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:22.601026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849627+0000","last_change":"2024-09-21T07:49:30.845589+0000","last_active":"2024-09-21T07:49:31.849627+0000","last_peered":"2024-09-21T07:49:31.849627+0000","last_clean":"2024-09-21T07:49:31.849627+0000","last_became_active":"2024-09-21T07:49:30.845259+0000","last_became_peered":"2024-09-21T07:49:30.845259+0000","last_unstale":"2024-09-21T07:49:31.849627+0000","last_undegraded":"2024-09-21T07:49:31.849627+0000","last_fullsized":"2024-09-21T07:49:31.849627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:18.427024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897362+0000","last_change":"2024-09-21T07:49:30.850524+0000","last_active":"2024-09-21T07:49:31.897362+0000","last_peered":"2024-09-21T07:49:31.897362+0000","last_clean":"2024-09-21T07:49:31.897362+0000","last_became_active":"2024-09-21T07:49:30.849502+0000","last_became_peered":"2024-09-21T07:49:30.849502+0000","last_unstale":"2024-09-21T07:49:31.897362+0000","last_undegraded":"2024-09-21T07:49:31.897362+0000","last_fullsized":"2024-09-21T07:49:31.897362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:19.251551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861401+0000","last_change":"2024-09-21T07:49:30.855844+0000","last_active":"2024-09-21T07:49:31.861401+0000","last_peered":"2024-09-21T07:49:31.861401+0000","last_clean":"2024-09-21T07:49:31.861401+0000","last_became_active":"2024-09-21T07:49:30.854339+0000","last_became_peered":"2024-09-21T07:49:30.854339+0000","last_unstale":"2024-09-21T07:49:31.861401+0000","last_undegraded":"2024-09-21T07:49:31.861401+0000","last_fullsized":"2024-09-21T07:49:31.861401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.682369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897940+0000","last_change":"2024-09-21T07:49:30.856897+0000","last_active":"2024-09-21T07:49:31.897940+0000","last_peered":"2024-09-21T07:49:31.897940+0000","last_clean":"2024-09-21T07:49:31.897940+0000","last_became_active":"2024-09-21T07:49:30.856435+0000","last_became_peered":"2024-09-21T07:49:30.856435+0000","last_unstale":"2024-09-21T07:49:31.897940+0000","last_undegraded":"2024-09-21T07:49:31.897940+0000","last_fullsized":"2024-09-21T07:49:31.897940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:50.221480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858315+0000","last_change":"2024-09-21T07:49:30.849113+0000","last_active":"2024-09-21T07:49:31.858315+0000","last_peered":"2024-09-21T07:49:31.858315+0000","last_clean":"2024-09-21T07:49:31.858315+0000","last_became_active":"2024-09-21T07:49:30.846401+0000","last_became_peered":"2024-09-21T07:49:30.846401+0000","last_unstale":"2024-09-21T07:49:31.858315+0000","last_undegraded":"2024-09-21T07:49:31.858315+0000","last_fullsized":"2024-09-21T07:49:31.858315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:19.251551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858976+0000","last_change":"2024-09-21T07:49:30.855205+0000","last_active":"2024-09-21T07:49:31.858976+0000","last_peered":"2024-09-21T07:49:31.858976+0000","last_clean":"2024-09-21T07:49:31.858976+0000","last_became_active":"2024-09-21T07:49:30.853927+0000","last_became_peered":"2024-09-21T07:49:30.853927+0000","last_unstale":"2024-09-21T07:49:31.858976+0000","last_undegraded":"2024-09-21T07:49:31.858976+0000","last_fullsized":"2024-09-21T07:49:31.858976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:54.485238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861697+0000","last_change":"2024-09-21T07:49:30.851466+0000","last_active":"2024-09-21T07:49:31.861697+0000","last_peered":"2024-09-21T07:49:31.861697+0000","last_clean":"2024-09-21T07:49:31.861697+0000","last_became_active":"2024-09-21T07:49:30.851064+0000","last_became_peered":"2024-09-21T07:49:30.851064+0000","last_unstale":"2024-09-21T07:49:31.861697+0000","last_undegraded":"2024-09-21T07:49:31.861697+0000","last_fullsized":"2024-09-21T07:49:31.861697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:40.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897965+0000","last_change":"2024-09-21T07:49:30.870745+0000","last_active":"2024-09-21T07:49:31.897965+0000","last_peered":"2024-09-21T07:49:31.897965+0000","last_clean":"2024-09-21T07:49:31.897965+0000","last_became_active":"2024-09-21T07:49:30.870361+0000","last_became_peered":"2024-09-21T07:49:30.870361+0000","last_unstale":"2024-09-21T07:49:31.897965+0000","last_undegraded":"2024-09-21T07:49:31.897965+0000","last_fullsized":"2024-09-21T07:49:31.897965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:18.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852286+0000","last_change":"2024-09-21T07:49:30.847507+0000","last_active":"2024-09-21T07:49:31.852286+0000","last_peered":"2024-09-21T07:49:31.852286+0000","last_clean":"2024-09-21T07:49:31.852286+0000","last_became_active":"2024-09-21T07:49:30.847122+0000","last_became_peered":"2024-09-21T07:49:30.847122+0000","last_unstale":"2024-09-21T07:49:31.852286+0000","last_undegraded":"2024-09-21T07:49:31.852286+0000","last_fullsized":"2024-09-21T07:49:31.852286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:40.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897475+0000","last_change":"2024-09-21T07:49:30.845594+0000","last_active":"2024-09-21T07:49:31.897475+0000","last_peered":"2024-09-21T07:49:31.897475+0000","last_clean":"2024-09-21T07:49:31.897475+0000","last_became_active":"2024-09-21T07:49:30.844794+0000","last_became_peered":"2024-09-21T07:49:30.844794+0000","last_unstale":"2024-09-21T07:49:31.897475+0000","last_undegraded":"2024-09-21T07:49:31.897475+0000","last_fullsized":"2024-09-21T07:49:31.897475+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:39.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851646+0000","last_change":"2024-09-21T07:49:30.862336+0000","last_active":"2024-09-21T07:49:31.851646+0000","last_peered":"2024-09-21T07:49:31.851646+0000","last_clean":"2024-09-21T07:49:31.851646+0000","last_became_active":"2024-09-21T07:49:30.858105+0000","last_became_peered":"2024-09-21T07:49:30.858105+0000","last_unstale":"2024-09-21T07:49:31.851646+0000","last_undegraded":"2024-09-21T07:49:31.851646+0000","last_fullsized":"2024-09-21T07:49:31.851646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.363283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897813+0000","last_change":"2024-09-21T07:49:30.852704+0000","last_active":"2024-09-21T07:49:31.897813+0000","last_peered":"2024-09-21T07:49:31.897813+0000","last_clean":"2024-09-21T07:49:31.897813+0000","last_became_active":"2024-09-21T07:49:30.851579+0000","last_became_peered":"2024-09-21T07:49:30.851579+0000","last_unstale":"2024-09-21T07:49:31.897813+0000","last_undegraded":"2024-09-21T07:49:31.897813+0000","last_fullsized":"2024-09-21T07:49:31.897813+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:29.279321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849130+0000","last_change":"2024-09-21T07:49:30.852598+0000","last_active":"2024-09-21T07:49:31.849130+0000","last_peered":"2024-09-21T07:49:31.849130+0000","last_clean":"2024-09-21T07:49:31.849130+0000","last_became_active":"2024-09-21T07:49:30.852201+0000","last_became_peered":"2024-09-21T07:49:30.852201+0000","last_unstale":"2024-09-21T07:49:31.849130+0000","last_undegraded":"2024-09-21T07:49:31.849130+0000","last_fullsized":"2024-09-21T07:49:31.849130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:52.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898160+0000","last_change":"2024-09-21T07:49:30.866983+0000","last_active":"2024-09-21T07:49:31.898160+0000","last_peered":"2024-09-21T07:49:31.898160+0000","last_clean":"2024-09-21T07:49:31.898160+0000","last_became_active":"2024-09-21T07:49:30.866405+0000","last_became_peered":"2024-09-21T07:49:30.866405+0000","last_unstale":"2024-09-21T07:49:31.898160+0000","last_undegraded":"2024-09-21T07:49:31.898160+0000","last_fullsized":"2024-09-21T07:49:31.898160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:33.824401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897940+0000","last_change":"2024-09-21T07:49:30.847189+0000","last_active":"2024-09-21T07:49:31.897940+0000","last_peered":"2024-09-21T07:49:31.897940+0000","last_clean":"2024-09-21T07:49:31.897940+0000","last_became_active":"2024-09-21T07:49:30.846811+0000","last_became_peered":"2024-09-21T07:49:30.846811+0000","last_unstale":"2024-09-21T07:49:31.897940+0000","last_undegraded":"2024-09-21T07:49:31.897940+0000","last_fullsized":"2024-09-21T07:49:31.897940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:33.055293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898197+0000","last_change":"2024-09-21T07:49:30.868732+0000","last_active":"2024-09-21T07:49:31.898197+0000","last_peered":"2024-09-21T07:49:31.898197+0000","last_clean":"2024-09-21T07:49:31.898197+0000","last_became_active":"2024-09-21T07:49:30.868358+0000","last_became_peered":"2024-09-21T07:49:30.868358+0000","last_unstale":"2024-09-21T07:49:31.898197+0000","last_undegraded":"2024-09-21T07:49:31.898197+0000","last_fullsized":"2024-09-21T07:49:31.898197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:09.597807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852995+0000","last_change":"2024-09-21T07:49:30.860588+0000","last_active":"2024-09-21T07:49:31.852995+0000","last_peered":"2024-09-21T07:49:31.852995+0000","last_clean":"2024-09-21T07:49:31.852995+0000","last_became_active":"2024-09-21T07:49:30.858226+0000","last_became_peered":"2024-09-21T07:49:30.858226+0000","last_unstale":"2024-09-21T07:49:31.852995+0000","last_undegraded":"2024-09-21T07:49:31.852995+0000","last_fullsized":"2024-09-21T07:49:31.852995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:16.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852846+0000","last_change":"2024-09-21T07:49:30.850556+0000","last_active":"2024-09-21T07:49:31.852846+0000","last_peered":"2024-09-21T07:49:31.852846+0000","last_clean":"2024-09-21T07:49:31.852846+0000","last_became_active":"2024-09-21T07:49:30.848534+0000","last_became_peered":"2024-09-21T07:49:30.848534+0000","last_unstale":"2024-09-21T07:49:31.852846+0000","last_undegraded":"2024-09-21T07:49:31.852846+0000","last_fullsized":"2024-09-21T07:49:31.852846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:16.784201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897404+0000","last_change":"2024-09-21T07:49:30.848469+0000","last_active":"2024-09-21T07:49:31.897404+0000","last_peered":"2024-09-21T07:49:31.897404+0000","last_clean":"2024-09-21T07:49:31.897404+0000","last_became_active":"2024-09-21T07:49:30.848130+0000","last_became_peered":"2024-09-21T07:49:30.848130+0000","last_unstale":"2024-09-21T07:49:31.897404+0000","last_undegraded":"2024-09-21T07:49:31.897404+0000","last_fullsized":"2024-09-21T07:49:31.897404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:34.319960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897737+0000","last_change":"2024-09-21T07:49:30.862236+0000","last_active":"2024-09-21T07:49:31.897737+0000","last_peered":"2024-09-21T07:49:31.897737+0000","last_clean":"2024-09-21T07:49:31.897737+0000","last_became_active":"2024-09-21T07:49:30.861285+0000","last_became_peered":"2024-09-21T07:49:30.861285+0000","last_unstale":"2024-09-21T07:49:31.897737+0000","last_undegraded":"2024-09-21T07:49:31.897737+0000","last_fullsized":"2024-09-21T07:49:31.897737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.866684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851367+0000","last_change":"2024-09-21T07:49:30.852919+0000","last_active":"2024-09-21T07:49:31.851367+0000","last_peered":"2024-09-21T07:49:31.851367+0000","last_clean":"2024-09-21T07:49:31.851367+0000","last_became_active":"2024-09-21T07:49:30.852244+0000","last_became_peered":"2024-09-21T07:49:30.852244+0000","last_unstale":"2024-09-21T07:49:31.851367+0000","last_undegraded":"2024-09-21T07:49:31.851367+0000","last_fullsized":"2024-09-21T07:49:31.851367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.682369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858896+0000","last_change":"2024-09-21T07:49:30.844684+0000","last_active":"2024-09-21T07:49:31.858896+0000","last_peered":"2024-09-21T07:49:31.858896+0000","last_clean":"2024-09-21T07:49:31.858896+0000","last_became_active":"2024-09-21T07:49:30.844376+0000","last_became_peered":"2024-09-21T07:49:30.844376+0000","last_unstale":"2024-09-21T07:49:31.858896+0000","last_undegraded":"2024-09-21T07:49:31.858896+0000","last_fullsized":"2024-09-21T07:49:31.858896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:57.962690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853432+0000","last_change":"2024-09-21T07:49:29.833079+0000","last_active":"2024-09-21T07:49:31.853432+0000","last_peered":"2024-09-21T07:49:31.853432+0000","last_clean":"2024-09-21T07:49:31.853432+0000","last_became_active":"2024-09-21T07:49:29.832745+0000","last_became_peered":"2024-09-21T07:49:29.832745+0000","last_unstale":"2024-09-21T07:49:31.853432+0000","last_undegraded":"2024-09-21T07:49:31.853432+0000","last_fullsized":"2024-09-21T07:49:31.853432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:25.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.859088+0000","last_change":"2024-09-21T07:49:30.845386+0000","last_active":"2024-09-21T07:49:31.859088+0000","last_peered":"2024-09-21T07:49:31.859088+0000","last_clean":"2024-09-21T07:49:31.859088+0000","last_became_active":"2024-09-21T07:49:30.845086+0000","last_became_peered":"2024-09-21T07:49:30.845086+0000","last_unstale":"2024-09-21T07:49:31.859088+0000","last_undegraded":"2024-09-21T07:49:31.859088+0000","last_fullsized":"2024-09-21T07:49:31.859088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.106407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898195+0000","last_change":"2024-09-21T07:49:29.844161+0000","last_active":"2024-09-21T07:49:31.898195+0000","last_peered":"2024-09-21T07:49:31.898195+0000","last_clean":"2024-09-21T07:49:31.898195+0000","last_became_active":"2024-09-21T07:49:29.843722+0000","last_became_peered":"2024-09-21T07:49:29.843722+0000","last_unstale":"2024-09-21T07:49:31.898195+0000","last_undegraded":"2024-09-21T07:49:31.898195+0000","last_fullsized":"2024-09-21T07:49:31.898195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:56.739749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853714+0000","last_change":"2024-09-21T07:49:30.864136+0000","last_active":"2024-09-21T07:49:31.853714+0000","last_peered":"2024-09-21T07:49:31.853714+0000","last_clean":"2024-09-21T07:49:31.853714+0000","last_became_active":"2024-09-21T07:49:30.863803+0000","last_became_peered":"2024-09-21T07:49:30.863803+0000","last_unstale":"2024-09-21T07:49:31.853714+0000","last_undegraded":"2024-09-21T07:49:31.853714+0000","last_fullsized":"2024-09-21T07:49:31.853714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:29.737578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858441+0000","last_change":"2024-09-21T07:49:29.841731+0000","last_active":"2024-09-21T07:49:31.858441+0000","last_peered":"2024-09-21T07:49:31.858441+0000","last_clean":"2024-09-21T07:49:31.858441+0000","last_became_active":"2024-09-21T07:49:29.841419+0000","last_became_peered":"2024-09-21T07:49:29.841419+0000","last_unstale":"2024-09-21T07:49:31.858441+0000","last_undegraded":"2024-09-21T07:49:31.858441+0000","last_fullsized":"2024-09-21T07:49:31.858441+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:45.852919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858495+0000","last_change":"2024-09-21T07:49:30.850320+0000","last_active":"2024-09-21T07:49:31.858495+0000","last_peered":"2024-09-21T07:49:31.858495+0000","last_clean":"2024-09-21T07:49:31.858495+0000","last_became_active":"2024-09-21T07:49:30.849791+0000","last_became_peered":"2024-09-21T07:49:30.849791+0000","last_unstale":"2024-09-21T07:49:31.858495+0000","last_undegraded":"2024-09-21T07:49:31.858495+0000","last_fullsized":"2024-09-21T07:49:31.858495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:20.450951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897679+0000","last_change":"2024-09-21T07:49:29.839504+0000","last_active":"2024-09-21T07:49:31.897679+0000","last_peered":"2024-09-21T07:49:31.897679+0000","last_clean":"2024-09-21T07:49:31.897679+0000","last_became_active":"2024-09-21T07:49:29.838264+0000","last_became_peered":"2024-09-21T07:49:29.838264+0000","last_unstale":"2024-09-21T07:49:31.897679+0000","last_undegraded":"2024-09-21T07:49:31.897679+0000","last_fullsized":"2024-09-21T07:49:31.897679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:02.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853346+0000","last_change":"2024-09-21T07:49:30.859157+0000","last_active":"2024-09-21T07:49:31.853346+0000","last_peered":"2024-09-21T07:49:31.853346+0000","last_clean":"2024-09-21T07:49:31.853346+0000","last_became_active":"2024-09-21T07:49:30.858858+0000","last_became_peered":"2024-09-21T07:49:30.858858+0000","last_unstale":"2024-09-21T07:49:31.853346+0000","last_undegraded":"2024-09-21T07:49:31.853346+0000","last_fullsized":"2024-09-21T07:49:31.853346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:49:49.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861569+0000","last_change":"2024-09-21T07:49:29.835030+0000","last_active":"2024-09-21T07:49:31.861569+0000","last_peered":"2024-09-21T07:49:31.861569+0000","last_clean":"2024-09-21T07:49:31.861569+0000","last_became_active":"2024-09-21T07:49:29.832816+0000","last_became_peered":"2024-09-21T07:49:29.832816+0000","last_unstale":"2024-09-21T07:49:31.861569+0000","last_undegraded":"2024-09-21T07:49:31.861569+0000","last_fullsized":"2024-09-21T07:49:31.861569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:12.631298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852420+0000","last_change":"2024-09-21T07:49:30.851949+0000","last_active":"2024-09-21T07:49:31.852420+0000","last_peered":"2024-09-21T07:49:31.852420+0000","last_clean":"2024-09-21T07:49:31.852420+0000","last_became_active":"2024-09-21T07:49:30.851637+0000","last_became_peered":"2024-09-21T07:49:30.851637+0000","last_unstale":"2024-09-21T07:49:31.852420+0000","last_undegraded":"2024-09-21T07:49:31.852420+0000","last_fullsized":"2024-09-21T07:49:31.852420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:25.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858446+0000","last_change":"2024-09-21T07:49:29.840419+0000","last_active":"2024-09-21T07:49:31.858446+0000","last_peered":"2024-09-21T07:49:31.858446+0000","last_clean":"2024-09-21T07:49:31.858446+0000","last_became_active":"2024-09-21T07:49:29.839959+0000","last_became_peered":"2024-09-21T07:49:29.839959+0000","last_unstale":"2024-09-21T07:49:31.858446+0000","last_undegraded":"2024-09-21T07:49:31.858446+0000","last_fullsized":"2024-09-21T07:49:31.858446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:42.438425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861316+0000","last_change":"2024-09-21T07:49:30.841186+0000","last_active":"2024-09-21T07:49:31.861316+0000","last_peered":"2024-09-21T07:49:31.861316+0000","last_clean":"2024-09-21T07:49:31.861316+0000","last_became_active":"2024-09-21T07:49:30.840647+0000","last_became_peered":"2024-09-21T07:49:30.840647+0000","last_unstale":"2024-09-21T07:49:31.861316+0000","last_undegraded":"2024-09-21T07:49:31.861316+0000","last_fullsized":"2024-09-21T07:49:31.861316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:15.627223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849194+0000","last_change":"2024-09-21T07:49:29.835362+0000","last_active":"2024-09-21T07:49:31.849194+0000","last_peered":"2024-09-21T07:49:31.849194+0000","last_clean":"2024-09-21T07:49:31.849194+0000","last_became_active":"2024-09-21T07:49:29.834971+0000","last_became_peered":"2024-09-21T07:49:29.834971+0000","last_unstale":"2024-09-21T07:49:31.849194+0000","last_undegraded":"2024-09-21T07:49:31.849194+0000","last_fullsized":"2024-09-21T07:49:31.849194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:32.215749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858179+0000","last_change":"2024-09-21T07:49:30.849385+0000","last_active":"2024-09-21T07:49:31.858179+0000","last_peered":"2024-09-21T07:49:31.858179+0000","last_clean":"2024-09-21T07:49:31.858179+0000","last_became_active":"2024-09-21T07:49:30.849031+0000","last_became_peered":"2024-09-21T07:49:30.849031+0000","last_unstale":"2024-09-21T07:49:31.858179+0000","last_undegraded":"2024-09-21T07:49:31.858179+0000","last_fullsized":"2024-09-21T07:49:31.858179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:29.406829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897090+0000","last_change":"2024-09-21T07:49:29.830256+0000","last_active":"2024-09-21T07:49:31.897090+0000","last_peered":"2024-09-21T07:49:31.897090+0000","last_clean":"2024-09-21T07:49:31.897090+0000","last_became_active":"2024-09-21T07:49:29.829961+0000","last_became_peered":"2024-09-21T07:49:29.829961+0000","last_unstale":"2024-09-21T07:49:31.897090+0000","last_undegraded":"2024-09-21T07:49:31.897090+0000","last_fullsized":"2024-09-21T07:49:31.897090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:05.746170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898021+0000","last_change":"2024-09-21T07:49:30.856701+0000","last_active":"2024-09-21T07:49:31.898021+0000","last_peered":"2024-09-21T07:49:31.898021+0000","last_clean":"2024-09-21T07:49:31.898021+0000","last_became_active":"2024-09-21T07:49:30.855565+0000","last_became_peered":"2024-09-21T07:49:30.855565+0000","last_unstale":"2024-09-21T07:49:31.898021+0000","last_undegraded":"2024-09-21T07:49:31.898021+0000","last_fullsized":"2024-09-21T07:49:31.898021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:57.962690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849696+0000","last_change":"2024-09-21T07:49:29.841680+0000","last_active":"2024-09-21T07:49:31.849696+0000","last_peered":"2024-09-21T07:49:31.849696+0000","last_clean":"2024-09-21T07:49:31.849696+0000","last_became_active":"2024-09-21T07:49:29.841344+0000","last_became_peered":"2024-09-21T07:49:29.841344+0000","last_unstale":"2024-09-21T07:49:31.849696+0000","last_undegraded":"2024-09-21T07:49:31.849696+0000","last_fullsized":"2024-09-21T07:49:31.849696+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.801003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861955+0000","last_change":"2024-09-21T07:49:30.856508+0000","last_active":"2024-09-21T07:49:31.861955+0000","last_peered":"2024-09-21T07:49:31.861955+0000","last_clean":"2024-09-21T07:49:31.861955+0000","last_became_active":"2024-09-21T07:49:30.856203+0000","last_became_peered":"2024-09-21T07:49:30.856203+0000","last_unstale":"2024-09-21T07:49:31.861955+0000","last_undegraded":"2024-09-21T07:49:31.861955+0000","last_fullsized":"2024-09-21T07:49:31.861955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:45.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853459+0000","last_change":"2024-09-21T07:49:29.834642+0000","last_active":"2024-09-21T07:49:31.853459+0000","last_peered":"2024-09-21T07:49:31.853459+0000","last_clean":"2024-09-21T07:49:31.853459+0000","last_became_active":"2024-09-21T07:49:29.833980+0000","last_became_peered":"2024-09-21T07:49:29.833980+0000","last_unstale":"2024-09-21T07:49:31.853459+0000","last_undegraded":"2024-09-21T07:49:31.853459+0000","last_fullsized":"2024-09-21T07:49:31.853459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:49.214019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853503+0000","last_change":"2024-09-21T07:49:30.863536+0000","last_active":"2024-09-21T07:49:31.853503+0000","last_peered":"2024-09-21T07:49:31.853503+0000","last_clean":"2024-09-21T07:49:31.853503+0000","last_became_active":"2024-09-21T07:49:30.863234+0000","last_became_peered":"2024-09-21T07:49:30.863234+0000","last_unstale":"2024-09-21T07:49:31.853503+0000","last_undegraded":"2024-09-21T07:49:31.853503+0000","last_fullsized":"2024-09-21T07:49:31.853503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:51.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897729+0000","last_change":"2024-09-21T07:49:29.842330+0000","last_active":"2024-09-21T07:49:31.897729+0000","last_peered":"2024-09-21T07:49:31.897729+0000","last_clean":"2024-09-21T07:49:31.897729+0000","last_became_active":"2024-09-21T07:49:29.842047+0000","last_became_peered":"2024-09-21T07:49:29.842047+0000","last_unstale":"2024-09-21T07:49:31.897729+0000","last_undegraded":"2024-09-21T07:49:31.897729+0000","last_fullsized":"2024-09-21T07:49:31.897729+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.490202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897565+0000","last_change":"2024-09-21T07:49:30.859923+0000","last_active":"2024-09-21T07:49:31.897565+0000","last_peered":"2024-09-21T07:49:31.897565+0000","last_clean":"2024-09-21T07:49:31.897565+0000","last_became_active":"2024-09-21T07:49:30.859557+0000","last_became_peered":"2024-09-21T07:49:30.859557+0000","last_unstale":"2024-09-21T07:49:31.897565+0000","last_undegraded":"2024-09-21T07:49:31.897565+0000","last_fullsized":"2024-09-21T07:49:31.897565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:57.514682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897920+0000","last_change":"2024-09-21T07:49:29.842473+0000","last_active":"2024-09-21T07:49:31.897920+0000","last_peered":"2024-09-21T07:49:31.897920+0000","last_clean":"2024-09-21T07:49:31.897920+0000","last_became_active":"2024-09-21T07:49:29.842149+0000","last_became_peered":"2024-09-21T07:49:29.842149+0000","last_unstale":"2024-09-21T07:49:31.897920+0000","last_undegraded":"2024-09-21T07:49:31.897920+0000","last_fullsized":"2024-09-21T07:49:31.897920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:19.977891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849328+0000","last_change":"2024-09-21T07:49:30.867765+0000","last_active":"2024-09-21T07:49:31.849328+0000","last_peered":"2024-09-21T07:49:31.849328+0000","last_clean":"2024-09-21T07:49:31.849328+0000","last_became_active":"2024-09-21T07:49:30.867424+0000","last_became_peered":"2024-09-21T07:49:30.867424+0000","last_unstale":"2024-09-21T07:49:31.849328+0000","last_undegraded":"2024-09-21T07:49:31.849328+0000","last_fullsized":"2024-09-21T07:49:31.849328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:32.624299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852930+0000","last_change":"2024-09-21T07:49:29.832526+0000","last_active":"2024-09-21T07:49:31.852930+0000","last_peered":"2024-09-21T07:49:31.852930+0000","last_clean":"2024-09-21T07:49:31.852930+0000","last_became_active":"2024-09-21T07:49:29.832191+0000","last_became_peered":"2024-09-21T07:49:29.832191+0000","last_unstale":"2024-09-21T07:49:31.852930+0000","last_undegraded":"2024-09-21T07:49:31.852930+0000","last_fullsized":"2024-09-21T07:49:31.852930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:40.367868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861439+0000","last_change":"2024-09-21T07:49:30.847902+0000","last_active":"2024-09-21T07:49:31.861439+0000","last_peered":"2024-09-21T07:49:31.861439+0000","last_clean":"2024-09-21T07:49:31.861439+0000","last_became_active":"2024-09-21T07:49:30.846155+0000","last_became_peered":"2024-09-21T07:49:30.846155+0000","last_unstale":"2024-09-21T07:49:31.861439+0000","last_undegraded":"2024-09-21T07:49:31.861439+0000","last_fullsized":"2024-09-21T07:49:31.861439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.363283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897581+0000","last_change":"2024-09-21T07:49:29.835475+0000","last_active":"2024-09-21T07:49:31.897581+0000","last_peered":"2024-09-21T07:49:31.897581+0000","last_clean":"2024-09-21T07:49:31.897581+0000","last_became_active":"2024-09-21T07:49:29.835130+0000","last_became_peered":"2024-09-21T07:49:29.835130+0000","last_unstale":"2024-09-21T07:49:31.897581+0000","last_undegraded":"2024-09-21T07:49:31.897581+0000","last_fullsized":"2024-09-21T07:49:31.897581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:54.479050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849197+0000","last_change":"2024-09-21T07:49:30.845676+0000","last_active":"2024-09-21T07:49:31.849197+0000","last_peered":"2024-09-21T07:49:31.849197+0000","last_clean":"2024-09-21T07:49:31.849197+0000","last_became_active":"2024-09-21T07:49:30.845360+0000","last_became_peered":"2024-09-21T07:49:30.845360+0000","last_unstale":"2024-09-21T07:49:31.849197+0000","last_undegraded":"2024-09-21T07:49:31.849197+0000","last_fullsized":"2024-09-21T07:49:31.849197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:13.860556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898147+0000","last_change":"2024-09-21T07:49:29.825131+0000","last_active":"2024-09-21T07:49:31.898147+0000","last_peered":"2024-09-21T07:49:31.898147+0000","last_clean":"2024-09-21T07:49:31.898147+0000","last_became_active":"2024-09-21T07:49:29.824617+0000","last_became_peered":"2024-09-21T07:49:29.824617+0000","last_unstale":"2024-09-21T07:49:31.898147+0000","last_undegraded":"2024-09-21T07:49:31.898147+0000","last_fullsized":"2024-09-21T07:49:31.898147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:29.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898004+0000","last_change":"2024-09-21T07:49:30.849916+0000","last_active":"2024-09-21T07:49:31.898004+0000","last_peered":"2024-09-21T07:49:31.898004+0000","last_clean":"2024-09-21T07:49:31.898004+0000","last_became_active":"2024-09-21T07:49:30.849262+0000","last_became_peered":"2024-09-21T07:49:30.849262+0000","last_unstale":"2024-09-21T07:49:31.898004+0000","last_undegraded":"2024-09-21T07:49:31.898004+0000","last_fullsized":"2024-09-21T07:49:31.898004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:37.033463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":23,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897243+0000","last_change":"2024-09-21T07:49:29.836030+0000","last_active":"2024-09-21T07:49:31.897243+0000","last_peered":"2024-09-21T07:49:31.897243+0000","last_clean":"2024-09-21T07:49:31.897243+0000","last_became_active":"2024-09-21T07:49:29.835703+0000","last_became_peered":"2024-09-21T07:49:29.835703+0000","last_unstale":"2024-09-21T07:49:31.897243+0000","last_undegraded":"2024-09-21T07:49:31.897243+0000","last_fullsized":"2024-09-21T07:49:31.897243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:06.109094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861729+0000","last_change":"2024-09-21T07:49:30.836520+0000","last_active":"2024-09-21T07:49:31.861729+0000","last_peered":"2024-09-21T07:49:31.861729+0000","last_clean":"2024-09-21T07:49:31.861729+0000","last_became_active":"2024-09-21T07:49:30.836216+0000","last_became_peered":"2024-09-21T07:49:30.836216+0000","last_unstale":"2024-09-21T07:49:31.861729+0000","last_undegraded":"2024-09-21T07:49:31.861729+0000","last_fullsized":"2024-09-21T07:49:31.861729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:25.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:29.843999+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:29.843591+0000","last_became_peered":"2024-09-21T07:49:29.843591+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:56.147012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861983+0000","last_change":"2024-09-21T07:49:30.865501+0000","last_active":"2024-09-21T07:49:31.861983+0000","last_peered":"2024-09-21T07:49:31.861983+0000","last_clean":"2024-09-21T07:49:31.861983+0000","last_became_active":"2024-09-21T07:49:30.865151+0000","last_became_peered":"2024-09-21T07:49:30.865151+0000","last_unstale":"2024-09-21T07:49:31.861983+0000","last_undegraded":"2024-09-21T07:49:31.861983+0000","last_fullsized":"2024-09-21T07:49:31.861983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:05.484654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853521+0000","last_change":"2024-09-21T07:49:29.834330+0000","last_active":"2024-09-21T07:49:31.853521+0000","last_peered":"2024-09-21T07:49:31.853521+0000","last_clean":"2024-09-21T07:49:31.853521+0000","last_became_active":"2024-09-21T07:49:29.833510+0000","last_became_peered":"2024-09-21T07:49:29.833510+0000","last_unstale":"2024-09-21T07:49:31.853521+0000","last_undegraded":"2024-09-21T07:49:31.853521+0000","last_fullsized":"2024-09-21T07:49:31.853521+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:56.955229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897841+0000","last_change":"2024-09-21T07:49:30.861727+0000","last_active":"2024-09-21T07:49:31.897841+0000","last_peered":"2024-09-21T07:49:31.897841+0000","last_clean":"2024-09-21T07:49:31.897841+0000","last_became_active":"2024-09-21T07:49:30.861401+0000","last_became_peered":"2024-09-21T07:49:30.861401+0000","last_unstale":"2024-09-21T07:49:31.897841+0000","last_undegraded":"2024-09-21T07:49:31.897841+0000","last_fullsized":"2024-09-21T07:49:31.897841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:59.043011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861775+0000","last_change":"2024-09-21T07:49:29.843565+0000","last_active":"2024-09-21T07:49:31.861775+0000","last_peered":"2024-09-21T07:49:31.861775+0000","last_clean":"2024-09-21T07:49:31.861775+0000","last_became_active":"2024-09-21T07:49:29.843265+0000","last_became_peered":"2024-09-21T07:49:29.843265+0000","last_unstale":"2024-09-21T07:49:31.861775+0000","last_undegraded":"2024-09-21T07:49:31.861775+0000","last_fullsized":"2024-09-21T07:49:31.861775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:28.399368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897733+0000","last_change":"2024-09-21T07:49:30.862803+0000","last_active":"2024-09-21T07:49:31.897733+0000","last_peered":"2024-09-21T07:49:31.897733+0000","last_clean":"2024-09-21T07:49:31.897733+0000","last_became_active":"2024-09-21T07:49:30.862519+0000","last_became_peered":"2024-09-21T07:49:30.862519+0000","last_unstale":"2024-09-21T07:49:31.897733+0000","last_undegraded":"2024-09-21T07:49:31.897733+0000","last_fullsized":"2024-09-21T07:49:31.897733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:47.572947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853592+0000","last_change":"2024-09-21T07:49:29.834999+0000","last_active":"2024-09-21T07:49:31.853592+0000","last_peered":"2024-09-21T07:49:31.853592+0000","last_clean":"2024-09-21T07:49:31.853592+0000","last_became_active":"2024-09-21T07:49:29.834675+0000","last_became_peered":"2024-09-21T07:49:29.834675+0000","last_unstale":"2024-09-21T07:49:31.853592+0000","last_undegraded":"2024-09-21T07:49:31.853592+0000","last_fullsized":"2024-09-21T07:49:31.853592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:25.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849458+0000","last_change":"2024-09-21T07:49:30.859030+0000","last_active":"2024-09-21T07:49:31.849458+0000","last_peered":"2024-09-21T07:49:31.849458+0000","last_clean":"2024-09-21T07:49:31.849458+0000","last_became_active":"2024-09-21T07:49:30.858447+0000","last_became_peered":"2024-09-21T07:49:30.858447+0000","last_unstale":"2024-09-21T07:49:31.849458+0000","last_undegraded":"2024-09-21T07:49:31.849458+0000","last_fullsized":"2024-09-21T07:49:31.849458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:35.606907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851978+0000","last_change":"2024-09-21T07:49:29.839355+0000","last_active":"2024-09-21T07:49:31.851978+0000","last_peered":"2024-09-21T07:49:31.851978+0000","last_clean":"2024-09-21T07:49:31.851978+0000","last_became_active":"2024-09-21T07:49:29.838434+0000","last_became_peered":"2024-09-21T07:49:29.838434+0000","last_unstale":"2024-09-21T07:49:31.851978+0000","last_undegraded":"2024-09-21T07:49:31.851978+0000","last_fullsized":"2024-09-21T07:49:31.851978+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:49.214019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852911+0000","last_change":"2024-09-21T07:49:30.853099+0000","last_active":"2024-09-21T07:49:31.852911+0000","last_peered":"2024-09-21T07:49:31.852911+0000","last_clean":"2024-09-21T07:49:31.852911+0000","last_became_active":"2024-09-21T07:49:30.852549+0000","last_became_peered":"2024-09-21T07:49:30.852549+0000","last_unstale":"2024-09-21T07:49:31.852911+0000","last_undegraded":"2024-09-21T07:49:31.852911+0000","last_fullsized":"2024-09-21T07:49:31.852911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:15.428162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852723+0000","last_change":"2024-09-21T07:49:29.831653+0000","last_active":"2024-09-21T07:49:31.852723+0000","last_peered":"2024-09-21T07:49:31.852723+0000","last_clean":"2024-09-21T07:49:31.852723+0000","last_became_active":"2024-09-21T07:49:29.831254+0000","last_became_peered":"2024-09-21T07:49:29.831254+0000","last_unstale":"2024-09-21T07:49:31.852723+0000","last_undegraded":"2024-09-21T07:49:31.852723+0000","last_fullsized":"2024-09-21T07:49:31.852723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:10.655160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852388+0000","last_change":"2024-09-21T07:49:30.853166+0000","last_active":"2024-09-21T07:49:31.852388+0000","last_peered":"2024-09-21T07:49:31.852388+0000","last_clean":"2024-09-21T07:49:31.852388+0000","last_became_active":"2024-09-21T07:49:30.852810+0000","last_became_peered":"2024-09-21T07:49:30.852810+0000","last_unstale":"2024-09-21T07:49:31.852388+0000","last_undegraded":"2024-09-21T07:49:31.852388+0000","last_fullsized":"2024-09-21T07:49:31.852388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:57.174205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849298+0000","last_change":"2024-09-21T07:49:29.832146+0000","last_active":"2024-09-21T07:49:31.849298+0000","last_peered":"2024-09-21T07:49:31.849298+0000","last_clean":"2024-09-21T07:49:31.849298+0000","last_became_active":"2024-09-21T07:49:29.831853+0000","last_became_peered":"2024-09-21T07:49:29.831853+0000","last_unstale":"2024-09-21T07:49:31.849298+0000","last_undegraded":"2024-09-21T07:49:31.849298+0000","last_fullsized":"2024-09-21T07:49:31.849298+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:04.356597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861915+0000","last_change":"2024-09-21T07:49:30.861233+0000","last_active":"2024-09-21T07:49:31.861915+0000","last_peered":"2024-09-21T07:49:31.861915+0000","last_clean":"2024-09-21T07:49:31.861915+0000","last_became_active":"2024-09-21T07:49:30.860929+0000","last_became_peered":"2024-09-21T07:49:30.860929+0000","last_unstale":"2024-09-21T07:49:31.861915+0000","last_undegraded":"2024-09-21T07:49:31.861915+0000","last_fullsized":"2024-09-21T07:49:31.861915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:45.970839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898152+0000","last_change":"2024-09-21T07:49:29.843093+0000","last_active":"2024-09-21T07:49:31.898152+0000","last_peered":"2024-09-21T07:49:31.898152+0000","last_clean":"2024-09-21T07:49:31.898152+0000","last_became_active":"2024-09-21T07:49:29.842766+0000","last_became_peered":"2024-09-21T07:49:29.842766+0000","last_unstale":"2024-09-21T07:49:31.898152+0000","last_undegraded":"2024-09-21T07:49:31.898152+0000","last_fullsized":"2024-09-21T07:49:31.898152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:50.990394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.862173+0000","last_change":"2024-09-21T07:49:30.850235+0000","last_active":"2024-09-21T07:49:31.862173+0000","last_peered":"2024-09-21T07:49:31.862173+0000","last_clean":"2024-09-21T07:49:31.862173+0000","last_became_active":"2024-09-21T07:49:30.849887+0000","last_became_peered":"2024-09-21T07:49:30.849887+0000","last_unstale":"2024-09-21T07:49:31.862173+0000","last_undegraded":"2024-09-21T07:49:31.862173+0000","last_fullsized":"2024-09-21T07:49:31.862173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:07.116555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858909+0000","last_change":"2024-09-21T07:49:29.839854+0000","last_active":"2024-09-21T07:49:31.858909+0000","last_peered":"2024-09-21T07:49:31.858909+0000","last_clean":"2024-09-21T07:49:31.858909+0000","last_became_active":"2024-09-21T07:49:29.839555+0000","last_became_peered":"2024-09-21T07:49:29.839555+0000","last_unstale":"2024-09-21T07:49:31.858909+0000","last_undegraded":"2024-09-21T07:49:31.858909+0000","last_fullsized":"2024-09-21T07:49:31.858909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:38.177514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:30.868811+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:30.868487+0000","last_became_peered":"2024-09-21T07:49:30.868487+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:14.218794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849802+0000","last_change":"2024-09-21T07:49:29.840533+0000","last_active":"2024-09-21T07:49:31.849802+0000","last_peered":"2024-09-21T07:49:31.849802+0000","last_clean":"2024-09-21T07:49:31.849802+0000","last_became_active":"2024-09-21T07:49:29.840191+0000","last_became_peered":"2024-09-21T07:49:29.840191+0000","last_unstale":"2024-09-21T07:49:31.849802+0000","last_undegraded":"2024-09-21T07:49:31.849802+0000","last_fullsized":"2024-09-21T07:49:31.849802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:33.764162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853460+0000","last_change":"2024-09-21T07:49:30.862140+0000","last_active":"2024-09-21T07:49:31.853460+0000","last_peered":"2024-09-21T07:49:31.853460+0000","last_clean":"2024-09-21T07:49:31.853460+0000","last_became_active":"2024-09-21T07:49:30.853963+0000","last_became_peered":"2024-09-21T07:49:30.853963+0000","last_unstale":"2024-09-21T07:49:31.853460+0000","last_undegraded":"2024-09-21T07:49:31.853460+0000","last_fullsized":"2024-09-21T07:49:31.853460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:14.505442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861465+0000","last_change":"2024-09-21T07:49:29.838408+0000","last_active":"2024-09-21T07:49:31.861465+0000","last_peered":"2024-09-21T07:49:31.861465+0000","last_clean":"2024-09-21T07:49:31.861465+0000","last_became_active":"2024-09-21T07:49:29.838107+0000","last_became_peered":"2024-09-21T07:49:29.838107+0000","last_unstale":"2024-09-21T07:49:31.861465+0000","last_undegraded":"2024-09-21T07:49:31.861465+0000","last_fullsized":"2024-09-21T07:49:31.861465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:39.016144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.859198+0000","last_change":"2024-09-21T07:49:30.851203+0000","last_active":"2024-09-21T07:49:31.859198+0000","last_peered":"2024-09-21T07:49:31.859198+0000","last_clean":"2024-09-21T07:49:31.859198+0000","last_became_active":"2024-09-21T07:49:30.850870+0000","last_became_peered":"2024-09-21T07:49:30.850870+0000","last_unstale":"2024-09-21T07:49:31.859198+0000","last_undegraded":"2024-09-21T07:49:31.859198+0000","last_fullsized":"2024-09-21T07:49:31.859198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:41.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897754+0000","last_change":"2024-09-21T07:49:29.842367+0000","last_active":"2024-09-21T07:49:31.897754+0000","last_peered":"2024-09-21T07:49:31.897754+0000","last_clean":"2024-09-21T07:49:31.897754+0000","last_became_active":"2024-09-21T07:49:29.842035+0000","last_became_peered":"2024-09-21T07:49:29.842035+0000","last_unstale":"2024-09-21T07:49:31.897754+0000","last_undegraded":"2024-09-21T07:49:31.897754+0000","last_fullsized":"2024-09-21T07:49:31.897754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:53.958285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897612+0000","last_change":"2024-09-21T07:49:30.855589+0000","last_active":"2024-09-21T07:49:31.897612+0000","last_peered":"2024-09-21T07:49:31.897612+0000","last_clean":"2024-09-21T07:49:31.897612+0000","last_became_active":"2024-09-21T07:49:30.854096+0000","last_became_peered":"2024-09-21T07:49:30.854096+0000","last_unstale":"2024-09-21T07:49:31.897612+0000","last_undegraded":"2024-09-21T07:49:31.897612+0000","last_fullsized":"2024-09-21T07:49:31.897612+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:11.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861355+0000","last_change":"2024-09-21T07:49:29.841930+0000","last_active":"2024-09-21T07:49:31.861355+0000","last_peered":"2024-09-21T07:49:31.861355+0000","last_clean":"2024-09-21T07:49:31.861355+0000","last_became_active":"2024-09-21T07:49:29.841523+0000","last_became_peered":"2024-09-21T07:49:29.841523+0000","last_unstale":"2024-09-21T07:49:31.861355+0000","last_undegraded":"2024-09-21T07:49:31.861355+0000","last_fullsized":"2024-09-21T07:49:31.861355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:24.795233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851666+0000","last_change":"2024-09-21T07:49:30.853544+0000","last_active":"2024-09-21T07:49:31.851666+0000","last_peered":"2024-09-21T07:49:31.851666+0000","last_clean":"2024-09-21T07:49:31.851666+0000","last_became_active":"2024-09-21T07:49:30.849986+0000","last_became_peered":"2024-09-21T07:49:30.849986+0000","last_unstale":"2024-09-21T07:49:31.851666+0000","last_undegraded":"2024-09-21T07:49:31.851666+0000","last_fullsized":"2024-09-21T07:49:31.851666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:04.039987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858078+0000","last_change":"2024-09-21T07:49:29.834315+0000","last_active":"2024-09-21T07:49:31.858078+0000","last_peered":"2024-09-21T07:49:31.858078+0000","last_clean":"2024-09-21T07:49:31.858078+0000","last_became_active":"2024-09-21T07:49:29.831497+0000","last_became_peered":"2024-09-21T07:49:29.831497+0000","last_unstale":"2024-09-21T07:49:31.858078+0000","last_undegraded":"2024-09-21T07:49:31.858078+0000","last_fullsized":"2024-09-21T07:49:31.858078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:30.000676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897800+0000","last_change":"2024-09-21T07:49:30.866291+0000","last_active":"2024-09-21T07:49:31.897800+0000","last_peered":"2024-09-21T07:49:31.897800+0000","last_clean":"2024-09-21T07:49:31.897800+0000","last_became_active":"2024-09-21T07:49:30.863674+0000","last_became_peered":"2024-09-21T07:49:30.863674+0000","last_unstale":"2024-09-21T07:49:31.897800+0000","last_undegraded":"2024-09-21T07:49:31.897800+0000","last_fullsized":"2024-09-21T07:49:31.897800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:02.165356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851800+0000","last_change":"2024-09-21T07:49:29.837438+0000","last_active":"2024-09-21T07:49:31.851800+0000","last_peered":"2024-09-21T07:49:31.851800+0000","last_clean":"2024-09-21T07:49:31.851800+0000","last_became_active":"2024-09-21T07:49:29.836913+0000","last_became_peered":"2024-09-21T07:49:29.836913+0000","last_unstale":"2024-09-21T07:49:31.851800+0000","last_undegraded":"2024-09-21T07:49:31.851800+0000","last_fullsized":"2024-09-21T07:49:31.851800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:38.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852109+0000","last_change":"2024-09-21T07:49:30.857265+0000","last_active":"2024-09-21T07:49:31.852109+0000","last_peered":"2024-09-21T07:49:31.852109+0000","last_clean":"2024-09-21T07:49:31.852109+0000","last_became_active":"2024-09-21T07:49:30.854815+0000","last_became_peered":"2024-09-21T07:49:30.854815+0000","last_unstale":"2024-09-21T07:49:31.852109+0000","last_undegraded":"2024-09-21T07:49:31.852109+0000","last_fullsized":"2024-09-21T07:49:31.852109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:58.762682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897355+0000","last_change":"2024-09-21T07:49:29.840701+0000","last_active":"2024-09-21T07:49:31.897355+0000","last_peered":"2024-09-21T07:49:31.897355+0000","last_clean":"2024-09-21T07:49:31.897355+0000","last_became_active":"2024-09-21T07:49:29.838599+0000","last_became_peered":"2024-09-21T07:49:29.838599+0000","last_unstale":"2024-09-21T07:49:31.897355+0000","last_undegraded":"2024-09-21T07:49:31.897355+0000","last_fullsized":"2024-09-21T07:49:31.897355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:38.001102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849394+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:49:31.849394+0000","last_peered":"2024-09-21T07:49:31.849394+0000","last_clean":"2024-09-21T07:49:31.849394+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:49:31.849394+0000","last_undegraded":"2024-09-21T07:49:31.849394+0000","last_fullsized":"2024-09-21T07:49:31.849394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":712704,"data_stored":293699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476786,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":295381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476783,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1306624,"data_stored":884559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":720896,"data_stored":295221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":293631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1306624,"data_stored":884011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":716800,"data_stored":294225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":293701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:36.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:49:36.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f010a000640 1 -- 172.21.15.88:0/830317962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00e8005150 msgr2=0x7f00e800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f010a000640 1 --2- 172.21.15.88:0/830317962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00e8005150 0x7f00e800b9c0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f00fc001d90 tx=0x7f00e8005830 comp rx=0 tx=0).stop 2024-09-21T07:49:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f0103e00640 1 -- 172.21.15.88:0/830317962 reap_dead start 2024-09-21T07:49:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f010a000640 1 -- 172.21.15.88:0/830317962 shutdown_connections 2024-09-21T07:49:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f010a000640 1 -- 172.21.15.88:0/830317962 >> 172.21.15.88:0/830317962 conn(0x7f010406d730 msgr2=0x7f01040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f010a000640 1 -- 172.21.15.88:0/830317962 shutdown_connections 2024-09-21T07:49:36.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.507+0000 7f010a000640 1 -- 172.21.15.88:0/830317962 wait complete. 2024-09-21T07:49:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.509+0000 7f010a000640 1 Processor -- start 2024-09-21T07:49:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.510+0000 7f010a000640 1 -- start start 2024-09-21T07:49:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f010a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e8005150 0x7f01041589d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f010a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01041e61d0 0x7f01041e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f010a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 0x7f010409c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f010a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f010406f220 con 0x7f010409c270 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f010a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f010406f0a0 con 0x7f00e8005150 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f010a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f010406f3a0 con 0x7f01041e61d0 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f0103e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 0x7f010409c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f0103e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 0x7f010409c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45688/0 (socket says 172.21.15.88:45688) 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f0103e00640 1 -- 172.21.15.88:0/1385728549 learned_addr learned my addr 172.21.15.88:0/1385728549 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f0103400640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e8005150 0x7f01041589d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.511+0000 7f00faa00640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01041e61d0 0x7f01041e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f0103e00640 1 -- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01041e61d0 msgr2=0x7f01041e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f0103e00640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01041e61d0 0x7f01041e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f0103e00640 1 -- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e8005150 msgr2=0x7f01041589d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f0103e00640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e8005150 0x7f01041589d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f0103e00640 1 -- 172.21.15.88:0/1385728549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010409cbc0 con 0x7f010409c270 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f0103e00640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 0x7f010409c680 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f00fc001eb0 tx=0x7f00fc001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc0060b0 con 0x7f010409c270 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00fc006250 con 0x7f010409c270 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01041570a0 con 0x7f010409c270 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.512+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00fc006660 con 0x7f010409c270 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.513+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0104157660 con 0x7f010409c270 2024-09-21T07:49:36.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.513+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01040b31e0 con 0x7f010409c270 2024-09-21T07:49:36.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.515+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f00fc033480 con 0x7f010409c270 2024-09-21T07:49:36.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.516+0000 7f00fbe00640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f00e0078d40 0x7f00e007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:36.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.516+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f00fc0baca0 con 0x7f010409c270 2024-09-21T07:49:36.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.516+0000 7f0103400640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f00e0078d40 0x7f00e007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:36.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.517+0000 7f0103400640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f00e0078d40 0x7f00e007b250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006cb0 tx=0x7f00f40067a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:36.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.520+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00fc082790 con 0x7f010409c270 2024-09-21T07:49:36.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.773+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7f01041e5ad0 con 0x7f010409c270 2024-09-21T07:49:36.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.773+0000 7f00fbe00640 1 -- 172.21.15.88:0/1385728549 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1055 (secure 0 0 0) 0x7f00fc0871c0 con 0x7f010409c270 2024-09-21T07:49:36.775 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:36.776 INFO:teuthology.orchestra.run.smithi088.stdout:{"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":238,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":60,"num_osds":8,"num_up_osds":8,"osd_up_since":1726904755,"num_in_osds":8,"osd_in_since":1726904744,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":97}],"num_pgs":97,"num_pools":3,"num_objects":24,"data_bytes":592778,"bytes_used":258195456,"bytes_avail":772835917824,"bytes_total":773094113280,"write_bytes_sec":2350,"read_op_per_sec":0,"write_op_per_sec":7},"fsmap":{"epoch":36,"btime":"2024-09-21T07:49:33:853295+0000","id":4,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":4,"rank":0,"name":"d","status":"up:active","gid":5763}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":8,"modified":"2024-09-21T07:48:45.898558+0000","services":{}},"progress_events":{}} 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.775+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f00e0078d40 msgr2=0x7f00e007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.775+0000 7f010a000640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f00e0078d40 0x7f00e007b250 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f00f4006cb0 tx=0x7f00f40067a0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.775+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 msgr2=0x7f010409c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.775+0000 7f010a000640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 0x7f010409c680 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f00fc001eb0 tx=0x7f00fc001af0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 shutdown_connections 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f00e0078d40 0x7f00e007b250 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01041e61d0 0x7f01041e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f010409c270 0x7f010409c680 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 --2- 172.21.15.88:0/1385728549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00e8005150 0x7f01041589d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:36.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 >> 172.21.15.88:0/1385728549 conn(0x7f010406d730 msgr2=0x7f01040b0810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:36.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 shutdown_connections 2024-09-21T07:49:36.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:36.776+0000 7f010a000640 1 -- 172.21.15.88:0/1385728549 wait complete. 2024-09-21T07:49:36.801 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:49:36.801 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:37.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 -- 172.21.15.88:0/1013475683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 msgr2=0x7f67e00d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 --2- 172.21.15.88:0/1013475683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00d1990 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f67e00d2050 tx=0x7f67d802a800 comp rx=0 tx=0).stop 2024-09-21T07:49:37.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 -- 172.21.15.88:0/1013475683 shutdown_connections 2024-09-21T07:49:37.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 --2- 172.21.15.88:0/1013475683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00d1990 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 --2- 172.21.15.88:0/1013475683 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67e00b9e00 0x7f67e0162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 -- 172.21.15.88:0/1013475683 >> 172.21.15.88:0/1013475683 conn(0x7f67e008bfe0 msgr2=0x7f67e009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:37.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 -- 172.21.15.88:0/1013475683 shutdown_connections 2024-09-21T07:49:37.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.020+0000 7f67df200640 1 -- 172.21.15.88:0/1013475683 wait complete. 2024-09-21T07:49:37.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.022+0000 7f67df200640 1 Processor -- start 2024-09-21T07:49:37.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.022+0000 7f67df200640 1 -- start start 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67df200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67e00b3160 0x7f67e015c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67df200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e00b9e00 0x7f67e015b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67df200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00a1c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67df200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67e00d4700 con 0x7f67e00d1580 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67df200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67e00d4580 con 0x7f67e00b9e00 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67df200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67e00d4880 con 0x7f67e00b3160 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00a1c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67dde00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67e00b3160 0x7f67e015c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00a1c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45714/0 (socket says 172.21.15.88:45714) 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 -- 172.21.15.88:0/3175775529 learned_addr learned my addr 172.21.15.88:0/3175775529 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67d5400640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e00b9e00 0x7f67e015b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 -- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67e00b3160 msgr2=0x7f67e015c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67e00b3160 0x7f67e015c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 -- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e00b9e00 msgr2=0x7f67e015b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e00b9e00 0x7f67e015b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.024+0000 7f67de800640 1 -- 172.21.15.88:0/3175775529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67e01f62e0 con 0x7f67e00d1580 2024-09-21T07:49:37.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.025+0000 7f67de800640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00a1c30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f67d8035040 tx=0x7f67d8033e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:37.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.025+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8042020 con 0x7f67e00d1580 2024-09-21T07:49:37.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.025+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67d8004330 con 0x7f67e00d1580 2024-09-21T07:49:37.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.025+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67e01f65c0 con 0x7f67e00d1580 2024-09-21T07:49:37.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.025+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67d8037040 con 0x7f67e00d1580 2024-09-21T07:49:37.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.025+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67e00a1710 con 0x7f67e00d1580 2024-09-21T07:49:37.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.026+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67e00d40f0 con 0x7f67e00d1580 2024-09-21T07:49:37.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.027+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f67d8045020 con 0x7f67e00d1580 2024-09-21T07:49:37.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.028+0000 7f67d6a00640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f67bc078d70 0x7f67bc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:37.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.028+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f67d80bc0d0 con 0x7f67e00d1580 2024-09-21T07:49:37.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.029+0000 7f67dde00640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f67bc078d70 0x7f67bc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:37.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.029+0000 7f67dde00640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f67bc078d70 0x7f67bc07b280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005fd0 tx=0x7f67d000a250 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:37.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.033+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f67e00d40f0 con 0x7f67e00d1580 2024-09-21T07:49:37.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.213+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f67e0162a50 con 0x7f67bc078d70 2024-09-21T07:49:37.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.218+0000 7f67d6a00640 1 -- 172.21.15.88:0/3175775529 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251947 (secure 0 0 0) 0x7f67e0162a50 con 0x7f67bc078d70 2024-09-21T07:49:37.223 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:37.224 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:49:37.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.222+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f67bc078d70 msgr2=0x7f67bc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.222+0000 7f67df200640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f67bc078d70 0x7f67bc07b280 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f67d0005fd0 tx=0x7f67d000a250 comp rx=0 tx=0).stop 2024-09-21T07:49:37.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 msgr2=0x7f67e00a1c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:37.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00a1c30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f67d8035040 tx=0x7f67d8033e80 comp rx=0 tx=0).stop 2024-09-21T07:49:37.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 shutdown_connections 2024-09-21T07:49:37.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f67bc078d70 0x7f67bc07b280 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67e00d1580 0x7f67e00a1c30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67e00b9e00 0x7f67e015b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 --2- 172.21.15.88:0/3175775529 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67e00b3160 0x7f67e015c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 >> 172.21.15.88:0/3175775529 conn(0x7f67e008bfe0 msgr2=0x7f67e00d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 shutdown_connections 2024-09-21T07:49:37.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:37.223+0000 7f67df200640 1 -- 172.21.15.88:0/3175775529 wait complete. 2024-09-21T07:49:37.247 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":167,"stamp":"2024-09-21T07:49:35.912601+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":167,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252144,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722576,"statfs":{"total":773094113280,"available":772835917824,"internally_reserved":0,"allocated":6922240,"data_stored":3534428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.098966"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858884+0000","last_change":"2024-09-21T07:49:30.865835+0000","last_active":"2024-09-21T07:49:31.858884+0000","last_peered":"2024-09-21T07:49:31.858884+0000","last_clean":"2024-09-21T07:49:31.858884+0000","last_became_active":"2024-09-21T07:49:30.863575+0000","last_became_peered":"2024-09-21T07:49:30.863575+0000","last_unstale":"2024-09-21T07:49:31.858884+0000","last_undegraded":"2024-09-21T07:49:31.858884+0000","last_fullsized":"2024-09-21T07:49:31.858884+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:26.732169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898139+0000","last_change":"2024-09-21T07:49:30.866746+0000","last_active":"2024-09-21T07:49:31.898139+0000","last_peered":"2024-09-21T07:49:31.898139+0000","last_clean":"2024-09-21T07:49:31.898139+0000","last_became_active":"2024-09-21T07:49:30.866386+0000","last_became_peered":"2024-09-21T07:49:30.866386+0000","last_unstale":"2024-09-21T07:49:31.898139+0000","last_undegraded":"2024-09-21T07:49:31.898139+0000","last_fullsized":"2024-09-21T07:49:31.898139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:41.251356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861624+0000","last_change":"2024-09-21T07:49:30.842935+0000","last_active":"2024-09-21T07:49:31.861624+0000","last_peered":"2024-09-21T07:49:31.861624+0000","last_clean":"2024-09-21T07:49:31.861624+0000","last_became_active":"2024-09-21T07:49:30.841630+0000","last_became_peered":"2024-09-21T07:49:30.841630+0000","last_unstale":"2024-09-21T07:49:31.861624+0000","last_undegraded":"2024-09-21T07:49:31.861624+0000","last_fullsized":"2024-09-21T07:49:31.861624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:22.601026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898018+0000","last_change":"2024-09-21T07:49:30.862748+0000","last_active":"2024-09-21T07:49:31.898018+0000","last_peered":"2024-09-21T07:49:31.898018+0000","last_clean":"2024-09-21T07:49:31.898018+0000","last_became_active":"2024-09-21T07:49:30.862527+0000","last_became_peered":"2024-09-21T07:49:30.862527+0000","last_unstale":"2024-09-21T07:49:31.898018+0000","last_undegraded":"2024-09-21T07:49:31.898018+0000","last_fullsized":"2024-09-21T07:49:31.898018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:17.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858588+0000","last_change":"2024-09-21T07:49:30.857817+0000","last_active":"2024-09-21T07:49:31.858588+0000","last_peered":"2024-09-21T07:49:31.858588+0000","last_clean":"2024-09-21T07:49:31.858588+0000","last_became_active":"2024-09-21T07:49:30.857065+0000","last_became_peered":"2024-09-21T07:49:30.857065+0000","last_unstale":"2024-09-21T07:49:31.858588+0000","last_undegraded":"2024-09-21T07:49:31.858588+0000","last_fullsized":"2024-09-21T07:49:31.858588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:39.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858182+0000","last_change":"2024-09-21T07:49:30.850469+0000","last_active":"2024-09-21T07:49:31.858182+0000","last_peered":"2024-09-21T07:49:31.858182+0000","last_clean":"2024-09-21T07:49:31.858182+0000","last_became_active":"2024-09-21T07:49:30.850119+0000","last_became_peered":"2024-09-21T07:49:30.850119+0000","last_unstale":"2024-09-21T07:49:31.858182+0000","last_undegraded":"2024-09-21T07:49:31.858182+0000","last_fullsized":"2024-09-21T07:49:31.858182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:23.015017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897597+0000","last_change":"2024-09-21T07:49:30.845351+0000","last_active":"2024-09-21T07:49:31.897597+0000","last_peered":"2024-09-21T07:49:31.897597+0000","last_clean":"2024-09-21T07:49:31.897597+0000","last_became_active":"2024-09-21T07:49:30.843920+0000","last_became_peered":"2024-09-21T07:49:30.843920+0000","last_unstale":"2024-09-21T07:49:31.897597+0000","last_undegraded":"2024-09-21T07:49:31.897597+0000","last_fullsized":"2024-09-21T07:49:31.897597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:25.494394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851362+0000","last_change":"2024-09-21T07:49:30.850184+0000","last_active":"2024-09-21T07:49:31.851362+0000","last_peered":"2024-09-21T07:49:31.851362+0000","last_clean":"2024-09-21T07:49:31.851362+0000","last_became_active":"2024-09-21T07:49:30.849571+0000","last_became_peered":"2024-09-21T07:49:30.849571+0000","last_unstale":"2024-09-21T07:49:31.851362+0000","last_undegraded":"2024-09-21T07:49:31.851362+0000","last_fullsized":"2024-09-21T07:49:31.851362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:25.897438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849745+0000","last_change":"2024-09-21T07:49:30.858444+0000","last_active":"2024-09-21T07:49:31.849745+0000","last_peered":"2024-09-21T07:49:31.849745+0000","last_clean":"2024-09-21T07:49:31.849745+0000","last_became_active":"2024-09-21T07:49:30.857207+0000","last_became_peered":"2024-09-21T07:49:30.857207+0000","last_unstale":"2024-09-21T07:49:31.849745+0000","last_undegraded":"2024-09-21T07:49:31.849745+0000","last_fullsized":"2024-09-21T07:49:31.849745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:43.445886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858632+0000","last_change":"2024-09-21T07:49:30.849692+0000","last_active":"2024-09-21T07:49:31.858632+0000","last_peered":"2024-09-21T07:49:31.858632+0000","last_clean":"2024-09-21T07:49:31.858632+0000","last_became_active":"2024-09-21T07:49:30.849329+0000","last_became_peered":"2024-09-21T07:49:30.849329+0000","last_unstale":"2024-09-21T07:49:31.858632+0000","last_undegraded":"2024-09-21T07:49:31.858632+0000","last_fullsized":"2024-09-21T07:49:31.858632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:50.221480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852174+0000","last_change":"2024-09-21T07:49:30.857819+0000","last_active":"2024-09-21T07:49:31.852174+0000","last_peered":"2024-09-21T07:49:31.852174+0000","last_clean":"2024-09-21T07:49:31.852174+0000","last_became_active":"2024-09-21T07:49:30.857491+0000","last_became_peered":"2024-09-21T07:49:30.857491+0000","last_unstale":"2024-09-21T07:49:31.852174+0000","last_undegraded":"2024-09-21T07:49:31.852174+0000","last_fullsized":"2024-09-21T07:49:31.852174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:22.601026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849627+0000","last_change":"2024-09-21T07:49:30.845589+0000","last_active":"2024-09-21T07:49:31.849627+0000","last_peered":"2024-09-21T07:49:31.849627+0000","last_clean":"2024-09-21T07:49:31.849627+0000","last_became_active":"2024-09-21T07:49:30.845259+0000","last_became_peered":"2024-09-21T07:49:30.845259+0000","last_unstale":"2024-09-21T07:49:31.849627+0000","last_undegraded":"2024-09-21T07:49:31.849627+0000","last_fullsized":"2024-09-21T07:49:31.849627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:18.427024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897362+0000","last_change":"2024-09-21T07:49:30.850524+0000","last_active":"2024-09-21T07:49:31.897362+0000","last_peered":"2024-09-21T07:49:31.897362+0000","last_clean":"2024-09-21T07:49:31.897362+0000","last_became_active":"2024-09-21T07:49:30.849502+0000","last_became_peered":"2024-09-21T07:49:30.849502+0000","last_unstale":"2024-09-21T07:49:31.897362+0000","last_undegraded":"2024-09-21T07:49:31.897362+0000","last_fullsized":"2024-09-21T07:49:31.897362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:19.251551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861401+0000","last_change":"2024-09-21T07:49:30.855844+0000","last_active":"2024-09-21T07:49:31.861401+0000","last_peered":"2024-09-21T07:49:31.861401+0000","last_clean":"2024-09-21T07:49:31.861401+0000","last_became_active":"2024-09-21T07:49:30.854339+0000","last_became_peered":"2024-09-21T07:49:30.854339+0000","last_unstale":"2024-09-21T07:49:31.861401+0000","last_undegraded":"2024-09-21T07:49:31.861401+0000","last_fullsized":"2024-09-21T07:49:31.861401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.682369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897940+0000","last_change":"2024-09-21T07:49:30.856897+0000","last_active":"2024-09-21T07:49:31.897940+0000","last_peered":"2024-09-21T07:49:31.897940+0000","last_clean":"2024-09-21T07:49:31.897940+0000","last_became_active":"2024-09-21T07:49:30.856435+0000","last_became_peered":"2024-09-21T07:49:30.856435+0000","last_unstale":"2024-09-21T07:49:31.897940+0000","last_undegraded":"2024-09-21T07:49:31.897940+0000","last_fullsized":"2024-09-21T07:49:31.897940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:50.221480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858315+0000","last_change":"2024-09-21T07:49:30.849113+0000","last_active":"2024-09-21T07:49:31.858315+0000","last_peered":"2024-09-21T07:49:31.858315+0000","last_clean":"2024-09-21T07:49:31.858315+0000","last_became_active":"2024-09-21T07:49:30.846401+0000","last_became_peered":"2024-09-21T07:49:30.846401+0000","last_unstale":"2024-09-21T07:49:31.858315+0000","last_undegraded":"2024-09-21T07:49:31.858315+0000","last_fullsized":"2024-09-21T07:49:31.858315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:19.251551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858976+0000","last_change":"2024-09-21T07:49:30.855205+0000","last_active":"2024-09-21T07:49:31.858976+0000","last_peered":"2024-09-21T07:49:31.858976+0000","last_clean":"2024-09-21T07:49:31.858976+0000","last_became_active":"2024-09-21T07:49:30.853927+0000","last_became_peered":"2024-09-21T07:49:30.853927+0000","last_unstale":"2024-09-21T07:49:31.858976+0000","last_undegraded":"2024-09-21T07:49:31.858976+0000","last_fullsized":"2024-09-21T07:49:31.858976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:54.485238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861697+0000","last_change":"2024-09-21T07:49:30.851466+0000","last_active":"2024-09-21T07:49:31.861697+0000","last_peered":"2024-09-21T07:49:31.861697+0000","last_clean":"2024-09-21T07:49:31.861697+0000","last_became_active":"2024-09-21T07:49:30.851064+0000","last_became_peered":"2024-09-21T07:49:30.851064+0000","last_unstale":"2024-09-21T07:49:31.861697+0000","last_undegraded":"2024-09-21T07:49:31.861697+0000","last_fullsized":"2024-09-21T07:49:31.861697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:40.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897965+0000","last_change":"2024-09-21T07:49:30.870745+0000","last_active":"2024-09-21T07:49:31.897965+0000","last_peered":"2024-09-21T07:49:31.897965+0000","last_clean":"2024-09-21T07:49:31.897965+0000","last_became_active":"2024-09-21T07:49:30.870361+0000","last_became_peered":"2024-09-21T07:49:30.870361+0000","last_unstale":"2024-09-21T07:49:31.897965+0000","last_undegraded":"2024-09-21T07:49:31.897965+0000","last_fullsized":"2024-09-21T07:49:31.897965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:18.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852286+0000","last_change":"2024-09-21T07:49:30.847507+0000","last_active":"2024-09-21T07:49:31.852286+0000","last_peered":"2024-09-21T07:49:31.852286+0000","last_clean":"2024-09-21T07:49:31.852286+0000","last_became_active":"2024-09-21T07:49:30.847122+0000","last_became_peered":"2024-09-21T07:49:30.847122+0000","last_unstale":"2024-09-21T07:49:31.852286+0000","last_undegraded":"2024-09-21T07:49:31.852286+0000","last_fullsized":"2024-09-21T07:49:31.852286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:40.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897475+0000","last_change":"2024-09-21T07:49:30.845594+0000","last_active":"2024-09-21T07:49:31.897475+0000","last_peered":"2024-09-21T07:49:31.897475+0000","last_clean":"2024-09-21T07:49:31.897475+0000","last_became_active":"2024-09-21T07:49:30.844794+0000","last_became_peered":"2024-09-21T07:49:30.844794+0000","last_unstale":"2024-09-21T07:49:31.897475+0000","last_undegraded":"2024-09-21T07:49:31.897475+0000","last_fullsized":"2024-09-21T07:49:31.897475+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:39.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851646+0000","last_change":"2024-09-21T07:49:30.862336+0000","last_active":"2024-09-21T07:49:31.851646+0000","last_peered":"2024-09-21T07:49:31.851646+0000","last_clean":"2024-09-21T07:49:31.851646+0000","last_became_active":"2024-09-21T07:49:30.858105+0000","last_became_peered":"2024-09-21T07:49:30.858105+0000","last_unstale":"2024-09-21T07:49:31.851646+0000","last_undegraded":"2024-09-21T07:49:31.851646+0000","last_fullsized":"2024-09-21T07:49:31.851646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.363283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897813+0000","last_change":"2024-09-21T07:49:30.852704+0000","last_active":"2024-09-21T07:49:31.897813+0000","last_peered":"2024-09-21T07:49:31.897813+0000","last_clean":"2024-09-21T07:49:31.897813+0000","last_became_active":"2024-09-21T07:49:30.851579+0000","last_became_peered":"2024-09-21T07:49:30.851579+0000","last_unstale":"2024-09-21T07:49:31.897813+0000","last_undegraded":"2024-09-21T07:49:31.897813+0000","last_fullsized":"2024-09-21T07:49:31.897813+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:29.279321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849130+0000","last_change":"2024-09-21T07:49:30.852598+0000","last_active":"2024-09-21T07:49:31.849130+0000","last_peered":"2024-09-21T07:49:31.849130+0000","last_clean":"2024-09-21T07:49:31.849130+0000","last_became_active":"2024-09-21T07:49:30.852201+0000","last_became_peered":"2024-09-21T07:49:30.852201+0000","last_unstale":"2024-09-21T07:49:31.849130+0000","last_undegraded":"2024-09-21T07:49:31.849130+0000","last_fullsized":"2024-09-21T07:49:31.849130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:52.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898160+0000","last_change":"2024-09-21T07:49:30.866983+0000","last_active":"2024-09-21T07:49:31.898160+0000","last_peered":"2024-09-21T07:49:31.898160+0000","last_clean":"2024-09-21T07:49:31.898160+0000","last_became_active":"2024-09-21T07:49:30.866405+0000","last_became_peered":"2024-09-21T07:49:30.866405+0000","last_unstale":"2024-09-21T07:49:31.898160+0000","last_undegraded":"2024-09-21T07:49:31.898160+0000","last_fullsized":"2024-09-21T07:49:31.898160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:33.824401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897940+0000","last_change":"2024-09-21T07:49:30.847189+0000","last_active":"2024-09-21T07:49:31.897940+0000","last_peered":"2024-09-21T07:49:31.897940+0000","last_clean":"2024-09-21T07:49:31.897940+0000","last_became_active":"2024-09-21T07:49:30.846811+0000","last_became_peered":"2024-09-21T07:49:30.846811+0000","last_unstale":"2024-09-21T07:49:31.897940+0000","last_undegraded":"2024-09-21T07:49:31.897940+0000","last_fullsized":"2024-09-21T07:49:31.897940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:33.055293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898197+0000","last_change":"2024-09-21T07:49:30.868732+0000","last_active":"2024-09-21T07:49:31.898197+0000","last_peered":"2024-09-21T07:49:31.898197+0000","last_clean":"2024-09-21T07:49:31.898197+0000","last_became_active":"2024-09-21T07:49:30.868358+0000","last_became_peered":"2024-09-21T07:49:30.868358+0000","last_unstale":"2024-09-21T07:49:31.898197+0000","last_undegraded":"2024-09-21T07:49:31.898197+0000","last_fullsized":"2024-09-21T07:49:31.898197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:09.597807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852995+0000","last_change":"2024-09-21T07:49:30.860588+0000","last_active":"2024-09-21T07:49:31.852995+0000","last_peered":"2024-09-21T07:49:31.852995+0000","last_clean":"2024-09-21T07:49:31.852995+0000","last_became_active":"2024-09-21T07:49:30.858226+0000","last_became_peered":"2024-09-21T07:49:30.858226+0000","last_unstale":"2024-09-21T07:49:31.852995+0000","last_undegraded":"2024-09-21T07:49:31.852995+0000","last_fullsized":"2024-09-21T07:49:31.852995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:16.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852846+0000","last_change":"2024-09-21T07:49:30.850556+0000","last_active":"2024-09-21T07:49:31.852846+0000","last_peered":"2024-09-21T07:49:31.852846+0000","last_clean":"2024-09-21T07:49:31.852846+0000","last_became_active":"2024-09-21T07:49:30.848534+0000","last_became_peered":"2024-09-21T07:49:30.848534+0000","last_unstale":"2024-09-21T07:49:31.852846+0000","last_undegraded":"2024-09-21T07:49:31.852846+0000","last_fullsized":"2024-09-21T07:49:31.852846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:16.784201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897404+0000","last_change":"2024-09-21T07:49:30.848469+0000","last_active":"2024-09-21T07:49:31.897404+0000","last_peered":"2024-09-21T07:49:31.897404+0000","last_clean":"2024-09-21T07:49:31.897404+0000","last_became_active":"2024-09-21T07:49:30.848130+0000","last_became_peered":"2024-09-21T07:49:30.848130+0000","last_unstale":"2024-09-21T07:49:31.897404+0000","last_undegraded":"2024-09-21T07:49:31.897404+0000","last_fullsized":"2024-09-21T07:49:31.897404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:34.319960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897737+0000","last_change":"2024-09-21T07:49:30.862236+0000","last_active":"2024-09-21T07:49:31.897737+0000","last_peered":"2024-09-21T07:49:31.897737+0000","last_clean":"2024-09-21T07:49:31.897737+0000","last_became_active":"2024-09-21T07:49:30.861285+0000","last_became_peered":"2024-09-21T07:49:30.861285+0000","last_unstale":"2024-09-21T07:49:31.897737+0000","last_undegraded":"2024-09-21T07:49:31.897737+0000","last_fullsized":"2024-09-21T07:49:31.897737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.866684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851367+0000","last_change":"2024-09-21T07:49:30.852919+0000","last_active":"2024-09-21T07:49:31.851367+0000","last_peered":"2024-09-21T07:49:31.851367+0000","last_clean":"2024-09-21T07:49:31.851367+0000","last_became_active":"2024-09-21T07:49:30.852244+0000","last_became_peered":"2024-09-21T07:49:30.852244+0000","last_unstale":"2024-09-21T07:49:31.851367+0000","last_undegraded":"2024-09-21T07:49:31.851367+0000","last_fullsized":"2024-09-21T07:49:31.851367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.682369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858896+0000","last_change":"2024-09-21T07:49:30.844684+0000","last_active":"2024-09-21T07:49:31.858896+0000","last_peered":"2024-09-21T07:49:31.858896+0000","last_clean":"2024-09-21T07:49:31.858896+0000","last_became_active":"2024-09-21T07:49:30.844376+0000","last_became_peered":"2024-09-21T07:49:30.844376+0000","last_unstale":"2024-09-21T07:49:31.858896+0000","last_undegraded":"2024-09-21T07:49:31.858896+0000","last_fullsized":"2024-09-21T07:49:31.858896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:57.962690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853432+0000","last_change":"2024-09-21T07:49:29.833079+0000","last_active":"2024-09-21T07:49:31.853432+0000","last_peered":"2024-09-21T07:49:31.853432+0000","last_clean":"2024-09-21T07:49:31.853432+0000","last_became_active":"2024-09-21T07:49:29.832745+0000","last_became_peered":"2024-09-21T07:49:29.832745+0000","last_unstale":"2024-09-21T07:49:31.853432+0000","last_undegraded":"2024-09-21T07:49:31.853432+0000","last_fullsized":"2024-09-21T07:49:31.853432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:25.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.859088+0000","last_change":"2024-09-21T07:49:30.845386+0000","last_active":"2024-09-21T07:49:31.859088+0000","last_peered":"2024-09-21T07:49:31.859088+0000","last_clean":"2024-09-21T07:49:31.859088+0000","last_became_active":"2024-09-21T07:49:30.845086+0000","last_became_peered":"2024-09-21T07:49:30.845086+0000","last_unstale":"2024-09-21T07:49:31.859088+0000","last_undegraded":"2024-09-21T07:49:31.859088+0000","last_fullsized":"2024-09-21T07:49:31.859088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.106407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898195+0000","last_change":"2024-09-21T07:49:29.844161+0000","last_active":"2024-09-21T07:49:31.898195+0000","last_peered":"2024-09-21T07:49:31.898195+0000","last_clean":"2024-09-21T07:49:31.898195+0000","last_became_active":"2024-09-21T07:49:29.843722+0000","last_became_peered":"2024-09-21T07:49:29.843722+0000","last_unstale":"2024-09-21T07:49:31.898195+0000","last_undegraded":"2024-09-21T07:49:31.898195+0000","last_fullsized":"2024-09-21T07:49:31.898195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:56.739749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853714+0000","last_change":"2024-09-21T07:49:30.864136+0000","last_active":"2024-09-21T07:49:31.853714+0000","last_peered":"2024-09-21T07:49:31.853714+0000","last_clean":"2024-09-21T07:49:31.853714+0000","last_became_active":"2024-09-21T07:49:30.863803+0000","last_became_peered":"2024-09-21T07:49:30.863803+0000","last_unstale":"2024-09-21T07:49:31.853714+0000","last_undegraded":"2024-09-21T07:49:31.853714+0000","last_fullsized":"2024-09-21T07:49:31.853714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:29.737578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858441+0000","last_change":"2024-09-21T07:49:29.841731+0000","last_active":"2024-09-21T07:49:31.858441+0000","last_peered":"2024-09-21T07:49:31.858441+0000","last_clean":"2024-09-21T07:49:31.858441+0000","last_became_active":"2024-09-21T07:49:29.841419+0000","last_became_peered":"2024-09-21T07:49:29.841419+0000","last_unstale":"2024-09-21T07:49:31.858441+0000","last_undegraded":"2024-09-21T07:49:31.858441+0000","last_fullsized":"2024-09-21T07:49:31.858441+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:45.852919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858495+0000","last_change":"2024-09-21T07:49:30.850320+0000","last_active":"2024-09-21T07:49:31.858495+0000","last_peered":"2024-09-21T07:49:31.858495+0000","last_clean":"2024-09-21T07:49:31.858495+0000","last_became_active":"2024-09-21T07:49:30.849791+0000","last_became_peered":"2024-09-21T07:49:30.849791+0000","last_unstale":"2024-09-21T07:49:31.858495+0000","last_undegraded":"2024-09-21T07:49:31.858495+0000","last_fullsized":"2024-09-21T07:49:31.858495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:20.450951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897679+0000","last_change":"2024-09-21T07:49:29.839504+0000","last_active":"2024-09-21T07:49:31.897679+0000","last_peered":"2024-09-21T07:49:31.897679+0000","last_clean":"2024-09-21T07:49:31.897679+0000","last_became_active":"2024-09-21T07:49:29.838264+0000","last_became_peered":"2024-09-21T07:49:29.838264+0000","last_unstale":"2024-09-21T07:49:31.897679+0000","last_undegraded":"2024-09-21T07:49:31.897679+0000","last_fullsized":"2024-09-21T07:49:31.897679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:02.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853346+0000","last_change":"2024-09-21T07:49:30.859157+0000","last_active":"2024-09-21T07:49:31.853346+0000","last_peered":"2024-09-21T07:49:31.853346+0000","last_clean":"2024-09-21T07:49:31.853346+0000","last_became_active":"2024-09-21T07:49:30.858858+0000","last_became_peered":"2024-09-21T07:49:30.858858+0000","last_unstale":"2024-09-21T07:49:31.853346+0000","last_undegraded":"2024-09-21T07:49:31.853346+0000","last_fullsized":"2024-09-21T07:49:31.853346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:49:49.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861569+0000","last_change":"2024-09-21T07:49:29.835030+0000","last_active":"2024-09-21T07:49:31.861569+0000","last_peered":"2024-09-21T07:49:31.861569+0000","last_clean":"2024-09-21T07:49:31.861569+0000","last_became_active":"2024-09-21T07:49:29.832816+0000","last_became_peered":"2024-09-21T07:49:29.832816+0000","last_unstale":"2024-09-21T07:49:31.861569+0000","last_undegraded":"2024-09-21T07:49:31.861569+0000","last_fullsized":"2024-09-21T07:49:31.861569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:12.631298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852420+0000","last_change":"2024-09-21T07:49:30.851949+0000","last_active":"2024-09-21T07:49:31.852420+0000","last_peered":"2024-09-21T07:49:31.852420+0000","last_clean":"2024-09-21T07:49:31.852420+0000","last_became_active":"2024-09-21T07:49:30.851637+0000","last_became_peered":"2024-09-21T07:49:30.851637+0000","last_unstale":"2024-09-21T07:49:31.852420+0000","last_undegraded":"2024-09-21T07:49:31.852420+0000","last_fullsized":"2024-09-21T07:49:31.852420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:25.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858446+0000","last_change":"2024-09-21T07:49:29.840419+0000","last_active":"2024-09-21T07:49:31.858446+0000","last_peered":"2024-09-21T07:49:31.858446+0000","last_clean":"2024-09-21T07:49:31.858446+0000","last_became_active":"2024-09-21T07:49:29.839959+0000","last_became_peered":"2024-09-21T07:49:29.839959+0000","last_unstale":"2024-09-21T07:49:31.858446+0000","last_undegraded":"2024-09-21T07:49:31.858446+0000","last_fullsized":"2024-09-21T07:49:31.858446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:42.438425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861316+0000","last_change":"2024-09-21T07:49:30.841186+0000","last_active":"2024-09-21T07:49:31.861316+0000","last_peered":"2024-09-21T07:49:31.861316+0000","last_clean":"2024-09-21T07:49:31.861316+0000","last_became_active":"2024-09-21T07:49:30.840647+0000","last_became_peered":"2024-09-21T07:49:30.840647+0000","last_unstale":"2024-09-21T07:49:31.861316+0000","last_undegraded":"2024-09-21T07:49:31.861316+0000","last_fullsized":"2024-09-21T07:49:31.861316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:15.627223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849194+0000","last_change":"2024-09-21T07:49:29.835362+0000","last_active":"2024-09-21T07:49:31.849194+0000","last_peered":"2024-09-21T07:49:31.849194+0000","last_clean":"2024-09-21T07:49:31.849194+0000","last_became_active":"2024-09-21T07:49:29.834971+0000","last_became_peered":"2024-09-21T07:49:29.834971+0000","last_unstale":"2024-09-21T07:49:31.849194+0000","last_undegraded":"2024-09-21T07:49:31.849194+0000","last_fullsized":"2024-09-21T07:49:31.849194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:32.215749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858179+0000","last_change":"2024-09-21T07:49:30.849385+0000","last_active":"2024-09-21T07:49:31.858179+0000","last_peered":"2024-09-21T07:49:31.858179+0000","last_clean":"2024-09-21T07:49:31.858179+0000","last_became_active":"2024-09-21T07:49:30.849031+0000","last_became_peered":"2024-09-21T07:49:30.849031+0000","last_unstale":"2024-09-21T07:49:31.858179+0000","last_undegraded":"2024-09-21T07:49:31.858179+0000","last_fullsized":"2024-09-21T07:49:31.858179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:29.406829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897090+0000","last_change":"2024-09-21T07:49:29.830256+0000","last_active":"2024-09-21T07:49:31.897090+0000","last_peered":"2024-09-21T07:49:31.897090+0000","last_clean":"2024-09-21T07:49:31.897090+0000","last_became_active":"2024-09-21T07:49:29.829961+0000","last_became_peered":"2024-09-21T07:49:29.829961+0000","last_unstale":"2024-09-21T07:49:31.897090+0000","last_undegraded":"2024-09-21T07:49:31.897090+0000","last_fullsized":"2024-09-21T07:49:31.897090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:05.746170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898021+0000","last_change":"2024-09-21T07:49:30.856701+0000","last_active":"2024-09-21T07:49:31.898021+0000","last_peered":"2024-09-21T07:49:31.898021+0000","last_clean":"2024-09-21T07:49:31.898021+0000","last_became_active":"2024-09-21T07:49:30.855565+0000","last_became_peered":"2024-09-21T07:49:30.855565+0000","last_unstale":"2024-09-21T07:49:31.898021+0000","last_undegraded":"2024-09-21T07:49:31.898021+0000","last_fullsized":"2024-09-21T07:49:31.898021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:57.962690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849696+0000","last_change":"2024-09-21T07:49:29.841680+0000","last_active":"2024-09-21T07:49:31.849696+0000","last_peered":"2024-09-21T07:49:31.849696+0000","last_clean":"2024-09-21T07:49:31.849696+0000","last_became_active":"2024-09-21T07:49:29.841344+0000","last_became_peered":"2024-09-21T07:49:29.841344+0000","last_unstale":"2024-09-21T07:49:31.849696+0000","last_undegraded":"2024-09-21T07:49:31.849696+0000","last_fullsized":"2024-09-21T07:49:31.849696+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.801003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861955+0000","last_change":"2024-09-21T07:49:30.856508+0000","last_active":"2024-09-21T07:49:31.861955+0000","last_peered":"2024-09-21T07:49:31.861955+0000","last_clean":"2024-09-21T07:49:31.861955+0000","last_became_active":"2024-09-21T07:49:30.856203+0000","last_became_peered":"2024-09-21T07:49:30.856203+0000","last_unstale":"2024-09-21T07:49:31.861955+0000","last_undegraded":"2024-09-21T07:49:31.861955+0000","last_fullsized":"2024-09-21T07:49:31.861955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:45.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853459+0000","last_change":"2024-09-21T07:49:29.834642+0000","last_active":"2024-09-21T07:49:31.853459+0000","last_peered":"2024-09-21T07:49:31.853459+0000","last_clean":"2024-09-21T07:49:31.853459+0000","last_became_active":"2024-09-21T07:49:29.833980+0000","last_became_peered":"2024-09-21T07:49:29.833980+0000","last_unstale":"2024-09-21T07:49:31.853459+0000","last_undegraded":"2024-09-21T07:49:31.853459+0000","last_fullsized":"2024-09-21T07:49:31.853459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:49.214019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853503+0000","last_change":"2024-09-21T07:49:30.863536+0000","last_active":"2024-09-21T07:49:31.853503+0000","last_peered":"2024-09-21T07:49:31.853503+0000","last_clean":"2024-09-21T07:49:31.853503+0000","last_became_active":"2024-09-21T07:49:30.863234+0000","last_became_peered":"2024-09-21T07:49:30.863234+0000","last_unstale":"2024-09-21T07:49:31.853503+0000","last_undegraded":"2024-09-21T07:49:31.853503+0000","last_fullsized":"2024-09-21T07:49:31.853503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:51.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897729+0000","last_change":"2024-09-21T07:49:29.842330+0000","last_active":"2024-09-21T07:49:31.897729+0000","last_peered":"2024-09-21T07:49:31.897729+0000","last_clean":"2024-09-21T07:49:31.897729+0000","last_became_active":"2024-09-21T07:49:29.842047+0000","last_became_peered":"2024-09-21T07:49:29.842047+0000","last_unstale":"2024-09-21T07:49:31.897729+0000","last_undegraded":"2024-09-21T07:49:31.897729+0000","last_fullsized":"2024-09-21T07:49:31.897729+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.490202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897565+0000","last_change":"2024-09-21T07:49:30.859923+0000","last_active":"2024-09-21T07:49:31.897565+0000","last_peered":"2024-09-21T07:49:31.897565+0000","last_clean":"2024-09-21T07:49:31.897565+0000","last_became_active":"2024-09-21T07:49:30.859557+0000","last_became_peered":"2024-09-21T07:49:30.859557+0000","last_unstale":"2024-09-21T07:49:31.897565+0000","last_undegraded":"2024-09-21T07:49:31.897565+0000","last_fullsized":"2024-09-21T07:49:31.897565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:57.514682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897920+0000","last_change":"2024-09-21T07:49:29.842473+0000","last_active":"2024-09-21T07:49:31.897920+0000","last_peered":"2024-09-21T07:49:31.897920+0000","last_clean":"2024-09-21T07:49:31.897920+0000","last_became_active":"2024-09-21T07:49:29.842149+0000","last_became_peered":"2024-09-21T07:49:29.842149+0000","last_unstale":"2024-09-21T07:49:31.897920+0000","last_undegraded":"2024-09-21T07:49:31.897920+0000","last_fullsized":"2024-09-21T07:49:31.897920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:19.977891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849328+0000","last_change":"2024-09-21T07:49:30.867765+0000","last_active":"2024-09-21T07:49:31.849328+0000","last_peered":"2024-09-21T07:49:31.849328+0000","last_clean":"2024-09-21T07:49:31.849328+0000","last_became_active":"2024-09-21T07:49:30.867424+0000","last_became_peered":"2024-09-21T07:49:30.867424+0000","last_unstale":"2024-09-21T07:49:31.849328+0000","last_undegraded":"2024-09-21T07:49:31.849328+0000","last_fullsized":"2024-09-21T07:49:31.849328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:32.624299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852930+0000","last_change":"2024-09-21T07:49:29.832526+0000","last_active":"2024-09-21T07:49:31.852930+0000","last_peered":"2024-09-21T07:49:31.852930+0000","last_clean":"2024-09-21T07:49:31.852930+0000","last_became_active":"2024-09-21T07:49:29.832191+0000","last_became_peered":"2024-09-21T07:49:29.832191+0000","last_unstale":"2024-09-21T07:49:31.852930+0000","last_undegraded":"2024-09-21T07:49:31.852930+0000","last_fullsized":"2024-09-21T07:49:31.852930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:40.367868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861439+0000","last_change":"2024-09-21T07:49:30.847902+0000","last_active":"2024-09-21T07:49:31.861439+0000","last_peered":"2024-09-21T07:49:31.861439+0000","last_clean":"2024-09-21T07:49:31.861439+0000","last_became_active":"2024-09-21T07:49:30.846155+0000","last_became_peered":"2024-09-21T07:49:30.846155+0000","last_unstale":"2024-09-21T07:49:31.861439+0000","last_undegraded":"2024-09-21T07:49:31.861439+0000","last_fullsized":"2024-09-21T07:49:31.861439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.363283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897581+0000","last_change":"2024-09-21T07:49:29.835475+0000","last_active":"2024-09-21T07:49:31.897581+0000","last_peered":"2024-09-21T07:49:31.897581+0000","last_clean":"2024-09-21T07:49:31.897581+0000","last_became_active":"2024-09-21T07:49:29.835130+0000","last_became_peered":"2024-09-21T07:49:29.835130+0000","last_unstale":"2024-09-21T07:49:31.897581+0000","last_undegraded":"2024-09-21T07:49:31.897581+0000","last_fullsized":"2024-09-21T07:49:31.897581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:54.479050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849197+0000","last_change":"2024-09-21T07:49:30.845676+0000","last_active":"2024-09-21T07:49:31.849197+0000","last_peered":"2024-09-21T07:49:31.849197+0000","last_clean":"2024-09-21T07:49:31.849197+0000","last_became_active":"2024-09-21T07:49:30.845360+0000","last_became_peered":"2024-09-21T07:49:30.845360+0000","last_unstale":"2024-09-21T07:49:31.849197+0000","last_undegraded":"2024-09-21T07:49:31.849197+0000","last_fullsized":"2024-09-21T07:49:31.849197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:13.860556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898147+0000","last_change":"2024-09-21T07:49:29.825131+0000","last_active":"2024-09-21T07:49:31.898147+0000","last_peered":"2024-09-21T07:49:31.898147+0000","last_clean":"2024-09-21T07:49:31.898147+0000","last_became_active":"2024-09-21T07:49:29.824617+0000","last_became_peered":"2024-09-21T07:49:29.824617+0000","last_unstale":"2024-09-21T07:49:31.898147+0000","last_undegraded":"2024-09-21T07:49:31.898147+0000","last_fullsized":"2024-09-21T07:49:31.898147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:29.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898004+0000","last_change":"2024-09-21T07:49:30.849916+0000","last_active":"2024-09-21T07:49:31.898004+0000","last_peered":"2024-09-21T07:49:31.898004+0000","last_clean":"2024-09-21T07:49:31.898004+0000","last_became_active":"2024-09-21T07:49:30.849262+0000","last_became_peered":"2024-09-21T07:49:30.849262+0000","last_unstale":"2024-09-21T07:49:31.898004+0000","last_undegraded":"2024-09-21T07:49:31.898004+0000","last_fullsized":"2024-09-21T07:49:31.898004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:37.033463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":23,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897243+0000","last_change":"2024-09-21T07:49:29.836030+0000","last_active":"2024-09-21T07:49:31.897243+0000","last_peered":"2024-09-21T07:49:31.897243+0000","last_clean":"2024-09-21T07:49:31.897243+0000","last_became_active":"2024-09-21T07:49:29.835703+0000","last_became_peered":"2024-09-21T07:49:29.835703+0000","last_unstale":"2024-09-21T07:49:31.897243+0000","last_undegraded":"2024-09-21T07:49:31.897243+0000","last_fullsized":"2024-09-21T07:49:31.897243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:06.109094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861729+0000","last_change":"2024-09-21T07:49:30.836520+0000","last_active":"2024-09-21T07:49:31.861729+0000","last_peered":"2024-09-21T07:49:31.861729+0000","last_clean":"2024-09-21T07:49:31.861729+0000","last_became_active":"2024-09-21T07:49:30.836216+0000","last_became_peered":"2024-09-21T07:49:30.836216+0000","last_unstale":"2024-09-21T07:49:31.861729+0000","last_undegraded":"2024-09-21T07:49:31.861729+0000","last_fullsized":"2024-09-21T07:49:31.861729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:25.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:29.843999+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:29.843591+0000","last_became_peered":"2024-09-21T07:49:29.843591+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:56.147012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861983+0000","last_change":"2024-09-21T07:49:30.865501+0000","last_active":"2024-09-21T07:49:31.861983+0000","last_peered":"2024-09-21T07:49:31.861983+0000","last_clean":"2024-09-21T07:49:31.861983+0000","last_became_active":"2024-09-21T07:49:30.865151+0000","last_became_peered":"2024-09-21T07:49:30.865151+0000","last_unstale":"2024-09-21T07:49:31.861983+0000","last_undegraded":"2024-09-21T07:49:31.861983+0000","last_fullsized":"2024-09-21T07:49:31.861983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:05.484654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853521+0000","last_change":"2024-09-21T07:49:29.834330+0000","last_active":"2024-09-21T07:49:31.853521+0000","last_peered":"2024-09-21T07:49:31.853521+0000","last_clean":"2024-09-21T07:49:31.853521+0000","last_became_active":"2024-09-21T07:49:29.833510+0000","last_became_peered":"2024-09-21T07:49:29.833510+0000","last_unstale":"2024-09-21T07:49:31.853521+0000","last_undegraded":"2024-09-21T07:49:31.853521+0000","last_fullsized":"2024-09-21T07:49:31.853521+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:56.955229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897841+0000","last_change":"2024-09-21T07:49:30.861727+0000","last_active":"2024-09-21T07:49:31.897841+0000","last_peered":"2024-09-21T07:49:31.897841+0000","last_clean":"2024-09-21T07:49:31.897841+0000","last_became_active":"2024-09-21T07:49:30.861401+0000","last_became_peered":"2024-09-21T07:49:30.861401+0000","last_unstale":"2024-09-21T07:49:31.897841+0000","last_undegraded":"2024-09-21T07:49:31.897841+0000","last_fullsized":"2024-09-21T07:49:31.897841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:59.043011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861775+0000","last_change":"2024-09-21T07:49:29.843565+0000","last_active":"2024-09-21T07:49:31.861775+0000","last_peered":"2024-09-21T07:49:31.861775+0000","last_clean":"2024-09-21T07:49:31.861775+0000","last_became_active":"2024-09-21T07:49:29.843265+0000","last_became_peered":"2024-09-21T07:49:29.843265+0000","last_unstale":"2024-09-21T07:49:31.861775+0000","last_undegraded":"2024-09-21T07:49:31.861775+0000","last_fullsized":"2024-09-21T07:49:31.861775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:28.399368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897733+0000","last_change":"2024-09-21T07:49:30.862803+0000","last_active":"2024-09-21T07:49:31.897733+0000","last_peered":"2024-09-21T07:49:31.897733+0000","last_clean":"2024-09-21T07:49:31.897733+0000","last_became_active":"2024-09-21T07:49:30.862519+0000","last_became_peered":"2024-09-21T07:49:30.862519+0000","last_unstale":"2024-09-21T07:49:31.897733+0000","last_undegraded":"2024-09-21T07:49:31.897733+0000","last_fullsized":"2024-09-21T07:49:31.897733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:47.572947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853592+0000","last_change":"2024-09-21T07:49:29.834999+0000","last_active":"2024-09-21T07:49:31.853592+0000","last_peered":"2024-09-21T07:49:31.853592+0000","last_clean":"2024-09-21T07:49:31.853592+0000","last_became_active":"2024-09-21T07:49:29.834675+0000","last_became_peered":"2024-09-21T07:49:29.834675+0000","last_unstale":"2024-09-21T07:49:31.853592+0000","last_undegraded":"2024-09-21T07:49:31.853592+0000","last_fullsized":"2024-09-21T07:49:31.853592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:25.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849458+0000","last_change":"2024-09-21T07:49:30.859030+0000","last_active":"2024-09-21T07:49:31.849458+0000","last_peered":"2024-09-21T07:49:31.849458+0000","last_clean":"2024-09-21T07:49:31.849458+0000","last_became_active":"2024-09-21T07:49:30.858447+0000","last_became_peered":"2024-09-21T07:49:30.858447+0000","last_unstale":"2024-09-21T07:49:31.849458+0000","last_undegraded":"2024-09-21T07:49:31.849458+0000","last_fullsized":"2024-09-21T07:49:31.849458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:35.606907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851978+0000","last_change":"2024-09-21T07:49:29.839355+0000","last_active":"2024-09-21T07:49:31.851978+0000","last_peered":"2024-09-21T07:49:31.851978+0000","last_clean":"2024-09-21T07:49:31.851978+0000","last_became_active":"2024-09-21T07:49:29.838434+0000","last_became_peered":"2024-09-21T07:49:29.838434+0000","last_unstale":"2024-09-21T07:49:31.851978+0000","last_undegraded":"2024-09-21T07:49:31.851978+0000","last_fullsized":"2024-09-21T07:49:31.851978+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:49.214019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852911+0000","last_change":"2024-09-21T07:49:30.853099+0000","last_active":"2024-09-21T07:49:31.852911+0000","last_peered":"2024-09-21T07:49:31.852911+0000","last_clean":"2024-09-21T07:49:31.852911+0000","last_became_active":"2024-09-21T07:49:30.852549+0000","last_became_peered":"2024-09-21T07:49:30.852549+0000","last_unstale":"2024-09-21T07:49:31.852911+0000","last_undegraded":"2024-09-21T07:49:31.852911+0000","last_fullsized":"2024-09-21T07:49:31.852911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:15.428162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852723+0000","last_change":"2024-09-21T07:49:29.831653+0000","last_active":"2024-09-21T07:49:31.852723+0000","last_peered":"2024-09-21T07:49:31.852723+0000","last_clean":"2024-09-21T07:49:31.852723+0000","last_became_active":"2024-09-21T07:49:29.831254+0000","last_became_peered":"2024-09-21T07:49:29.831254+0000","last_unstale":"2024-09-21T07:49:31.852723+0000","last_undegraded":"2024-09-21T07:49:31.852723+0000","last_fullsized":"2024-09-21T07:49:31.852723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:10.655160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852388+0000","last_change":"2024-09-21T07:49:30.853166+0000","last_active":"2024-09-21T07:49:31.852388+0000","last_peered":"2024-09-21T07:49:31.852388+0000","last_clean":"2024-09-21T07:49:31.852388+0000","last_became_active":"2024-09-21T07:49:30.852810+0000","last_became_peered":"2024-09-21T07:49:30.852810+0000","last_unstale":"2024-09-21T07:49:31.852388+0000","last_undegraded":"2024-09-21T07:49:31.852388+0000","last_fullsized":"2024-09-21T07:49:31.852388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:57.174205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849298+0000","last_change":"2024-09-21T07:49:29.832146+0000","last_active":"2024-09-21T07:49:31.849298+0000","last_peered":"2024-09-21T07:49:31.849298+0000","last_clean":"2024-09-21T07:49:31.849298+0000","last_became_active":"2024-09-21T07:49:29.831853+0000","last_became_peered":"2024-09-21T07:49:29.831853+0000","last_unstale":"2024-09-21T07:49:31.849298+0000","last_undegraded":"2024-09-21T07:49:31.849298+0000","last_fullsized":"2024-09-21T07:49:31.849298+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:04.356597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861915+0000","last_change":"2024-09-21T07:49:30.861233+0000","last_active":"2024-09-21T07:49:31.861915+0000","last_peered":"2024-09-21T07:49:31.861915+0000","last_clean":"2024-09-21T07:49:31.861915+0000","last_became_active":"2024-09-21T07:49:30.860929+0000","last_became_peered":"2024-09-21T07:49:30.860929+0000","last_unstale":"2024-09-21T07:49:31.861915+0000","last_undegraded":"2024-09-21T07:49:31.861915+0000","last_fullsized":"2024-09-21T07:49:31.861915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:45.970839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898152+0000","last_change":"2024-09-21T07:49:29.843093+0000","last_active":"2024-09-21T07:49:31.898152+0000","last_peered":"2024-09-21T07:49:31.898152+0000","last_clean":"2024-09-21T07:49:31.898152+0000","last_became_active":"2024-09-21T07:49:29.842766+0000","last_became_peered":"2024-09-21T07:49:29.842766+0000","last_unstale":"2024-09-21T07:49:31.898152+0000","last_undegraded":"2024-09-21T07:49:31.898152+0000","last_fullsized":"2024-09-21T07:49:31.898152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:50.990394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.862173+0000","last_change":"2024-09-21T07:49:30.850235+0000","last_active":"2024-09-21T07:49:31.862173+0000","last_peered":"2024-09-21T07:49:31.862173+0000","last_clean":"2024-09-21T07:49:31.862173+0000","last_became_active":"2024-09-21T07:49:30.849887+0000","last_became_peered":"2024-09-21T07:49:30.849887+0000","last_unstale":"2024-09-21T07:49:31.862173+0000","last_undegraded":"2024-09-21T07:49:31.862173+0000","last_fullsized":"2024-09-21T07:49:31.862173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:07.116555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858909+0000","last_change":"2024-09-21T07:49:29.839854+0000","last_active":"2024-09-21T07:49:31.858909+0000","last_peered":"2024-09-21T07:49:31.858909+0000","last_clean":"2024-09-21T07:49:31.858909+0000","last_became_active":"2024-09-21T07:49:29.839555+0000","last_became_peered":"2024-09-21T07:49:29.839555+0000","last_unstale":"2024-09-21T07:49:31.858909+0000","last_undegraded":"2024-09-21T07:49:31.858909+0000","last_fullsized":"2024-09-21T07:49:31.858909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:38.177514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:30.868811+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:30.868487+0000","last_became_peered":"2024-09-21T07:49:30.868487+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:14.218794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849802+0000","last_change":"2024-09-21T07:49:29.840533+0000","last_active":"2024-09-21T07:49:31.849802+0000","last_peered":"2024-09-21T07:49:31.849802+0000","last_clean":"2024-09-21T07:49:31.849802+0000","last_became_active":"2024-09-21T07:49:29.840191+0000","last_became_peered":"2024-09-21T07:49:29.840191+0000","last_unstale":"2024-09-21T07:49:31.849802+0000","last_undegraded":"2024-09-21T07:49:31.849802+0000","last_fullsized":"2024-09-21T07:49:31.849802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:33.764162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853460+0000","last_change":"2024-09-21T07:49:30.862140+0000","last_active":"2024-09-21T07:49:31.853460+0000","last_peered":"2024-09-21T07:49:31.853460+0000","last_clean":"2024-09-21T07:49:31.853460+0000","last_became_active":"2024-09-21T07:49:30.853963+0000","last_became_peered":"2024-09-21T07:49:30.853963+0000","last_unstale":"2024-09-21T07:49:31.853460+0000","last_undegraded":"2024-09-21T07:49:31.853460+0000","last_fullsized":"2024-09-21T07:49:31.853460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:14.505442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861465+0000","last_change":"2024-09-21T07:49:29.838408+0000","last_active":"2024-09-21T07:49:31.861465+0000","last_peered":"2024-09-21T07:49:31.861465+0000","last_clean":"2024-09-21T07:49:31.861465+0000","last_became_active":"2024-09-21T07:49:29.838107+0000","last_became_peered":"2024-09-21T07:49:29.838107+0000","last_unstale":"2024-09-21T07:49:31.861465+0000","last_undegraded":"2024-09-21T07:49:31.861465+0000","last_fullsized":"2024-09-21T07:49:31.861465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:39.016144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.859198+0000","last_change":"2024-09-21T07:49:30.851203+0000","last_active":"2024-09-21T07:49:31.859198+0000","last_peered":"2024-09-21T07:49:31.859198+0000","last_clean":"2024-09-21T07:49:31.859198+0000","last_became_active":"2024-09-21T07:49:30.850870+0000","last_became_peered":"2024-09-21T07:49:30.850870+0000","last_unstale":"2024-09-21T07:49:31.859198+0000","last_undegraded":"2024-09-21T07:49:31.859198+0000","last_fullsized":"2024-09-21T07:49:31.859198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:41.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897754+0000","last_change":"2024-09-21T07:49:29.842367+0000","last_active":"2024-09-21T07:49:31.897754+0000","last_peered":"2024-09-21T07:49:31.897754+0000","last_clean":"2024-09-21T07:49:31.897754+0000","last_became_active":"2024-09-21T07:49:29.842035+0000","last_became_peered":"2024-09-21T07:49:29.842035+0000","last_unstale":"2024-09-21T07:49:31.897754+0000","last_undegraded":"2024-09-21T07:49:31.897754+0000","last_fullsized":"2024-09-21T07:49:31.897754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:53.958285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897612+0000","last_change":"2024-09-21T07:49:30.855589+0000","last_active":"2024-09-21T07:49:31.897612+0000","last_peered":"2024-09-21T07:49:31.897612+0000","last_clean":"2024-09-21T07:49:31.897612+0000","last_became_active":"2024-09-21T07:49:30.854096+0000","last_became_peered":"2024-09-21T07:49:30.854096+0000","last_unstale":"2024-09-21T07:49:31.897612+0000","last_undegraded":"2024-09-21T07:49:31.897612+0000","last_fullsized":"2024-09-21T07:49:31.897612+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:11.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861355+0000","last_change":"2024-09-21T07:49:29.841930+0000","last_active":"2024-09-21T07:49:31.861355+0000","last_peered":"2024-09-21T07:49:31.861355+0000","last_clean":"2024-09-21T07:49:31.861355+0000","last_became_active":"2024-09-21T07:49:29.841523+0000","last_became_peered":"2024-09-21T07:49:29.841523+0000","last_unstale":"2024-09-21T07:49:31.861355+0000","last_undegraded":"2024-09-21T07:49:31.861355+0000","last_fullsized":"2024-09-21T07:49:31.861355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:24.795233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851666+0000","last_change":"2024-09-21T07:49:30.853544+0000","last_active":"2024-09-21T07:49:31.851666+0000","last_peered":"2024-09-21T07:49:31.851666+0000","last_clean":"2024-09-21T07:49:31.851666+0000","last_became_active":"2024-09-21T07:49:30.849986+0000","last_became_peered":"2024-09-21T07:49:30.849986+0000","last_unstale":"2024-09-21T07:49:31.851666+0000","last_undegraded":"2024-09-21T07:49:31.851666+0000","last_fullsized":"2024-09-21T07:49:31.851666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:04.039987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858078+0000","last_change":"2024-09-21T07:49:29.834315+0000","last_active":"2024-09-21T07:49:31.858078+0000","last_peered":"2024-09-21T07:49:31.858078+0000","last_clean":"2024-09-21T07:49:31.858078+0000","last_became_active":"2024-09-21T07:49:29.831497+0000","last_became_peered":"2024-09-21T07:49:29.831497+0000","last_unstale":"2024-09-21T07:49:31.858078+0000","last_undegraded":"2024-09-21T07:49:31.858078+0000","last_fullsized":"2024-09-21T07:49:31.858078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:30.000676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897800+0000","last_change":"2024-09-21T07:49:30.866291+0000","last_active":"2024-09-21T07:49:31.897800+0000","last_peered":"2024-09-21T07:49:31.897800+0000","last_clean":"2024-09-21T07:49:31.897800+0000","last_became_active":"2024-09-21T07:49:30.863674+0000","last_became_peered":"2024-09-21T07:49:30.863674+0000","last_unstale":"2024-09-21T07:49:31.897800+0000","last_undegraded":"2024-09-21T07:49:31.897800+0000","last_fullsized":"2024-09-21T07:49:31.897800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:02.165356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851800+0000","last_change":"2024-09-21T07:49:29.837438+0000","last_active":"2024-09-21T07:49:31.851800+0000","last_peered":"2024-09-21T07:49:31.851800+0000","last_clean":"2024-09-21T07:49:31.851800+0000","last_became_active":"2024-09-21T07:49:29.836913+0000","last_became_peered":"2024-09-21T07:49:29.836913+0000","last_unstale":"2024-09-21T07:49:31.851800+0000","last_undegraded":"2024-09-21T07:49:31.851800+0000","last_fullsized":"2024-09-21T07:49:31.851800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:38.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852109+0000","last_change":"2024-09-21T07:49:30.857265+0000","last_active":"2024-09-21T07:49:31.852109+0000","last_peered":"2024-09-21T07:49:31.852109+0000","last_clean":"2024-09-21T07:49:31.852109+0000","last_became_active":"2024-09-21T07:49:30.854815+0000","last_became_peered":"2024-09-21T07:49:30.854815+0000","last_unstale":"2024-09-21T07:49:31.852109+0000","last_undegraded":"2024-09-21T07:49:31.852109+0000","last_fullsized":"2024-09-21T07:49:31.852109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:58.762682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897355+0000","last_change":"2024-09-21T07:49:29.840701+0000","last_active":"2024-09-21T07:49:31.897355+0000","last_peered":"2024-09-21T07:49:31.897355+0000","last_clean":"2024-09-21T07:49:31.897355+0000","last_became_active":"2024-09-21T07:49:29.838599+0000","last_became_peered":"2024-09-21T07:49:29.838599+0000","last_unstale":"2024-09-21T07:49:31.897355+0000","last_undegraded":"2024-09-21T07:49:31.897355+0000","last_fullsized":"2024-09-21T07:49:31.897355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:38.001102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849394+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:49:31.849394+0000","last_peered":"2024-09-21T07:49:31.849394+0000","last_clean":"2024-09-21T07:49:31.849394+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:49:31.849394+0000","last_undegraded":"2024-09-21T07:49:31.849394+0000","last_fullsized":"2024-09-21T07:49:31.849394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":712704,"data_stored":293699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476786,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":295381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476783,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1306624,"data_stored":884559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":720896,"data_stored":295221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509486,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":293631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509487,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1306624,"data_stored":884011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509487,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":716800,"data_stored":294225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542191,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":293701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:40.287 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:49:40.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 -- 172.21.15.88:0/2114925304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80af170 msgr2=0x7f05c80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 --2- 172.21.15.88:0/2114925304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80af170 0x7f05c80b2f60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f05c80cfe20 tx=0x7f05b002a410 comp rx=0 tx=0).stop 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 -- 172.21.15.88:0/2114925304 shutdown_connections 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 --2- 172.21.15.88:0/2114925304 >> v2:172.21.15.175:3300/0 conn(0x7f05c815a0f0 0x7f05c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 --2- 172.21.15.88:0/2114925304 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80af170 0x7f05c80b2f60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 -- 172.21.15.88:0/2114925304 >> 172.21.15.88:0/2114925304 conn(0x7f05c80b1e40 msgr2=0x7f05c80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 -- 172.21.15.88:0/2114925304 shutdown_connections 2024-09-21T07:49:40.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.515+0000 7f05c7200640 1 -- 172.21.15.88:0/2114925304 wait complete. 2024-09-21T07:49:40.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.517+0000 7f05c7200640 1 Processor -- start 2024-09-21T07:49:40.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.517+0000 7f05c7200640 1 -- start start 2024-09-21T07:49:40.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 0x7f05c8145930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05c80af170 0x7f05c81e2a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05c815a0f0 0x7f05c81f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05c806f290 con 0x7f05c80ad9c0 2024-09-21T07:49:40.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f05c8070c80 con 0x7f05c815a0f0 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f05c806f410 con 0x7f05c80af170 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 0x7f05c8145930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 0x7f05c8145930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45720/0 (socket says 172.21.15.88:45720) 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 -- 172.21.15.88:0/3083120915 learned_addr learned my addr 172.21.15.88:0/3083120915 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 -- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05c80af170 msgr2=0x7f05c81e2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05c80af170 0x7f05c81e2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 -- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05c815a0f0 msgr2=0x7f05c81f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c6800640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05c815a0f0 0x7f05c81f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.519+0000 7f05c5e00640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05c815a0f0 0x7f05c81f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05c5e00640 1 -- 172.21.15.88:0/3083120915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05c81e2f70 con 0x7f05c80ad9c0 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05c6800640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05c815a0f0 0x7f05c81f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05c5e00640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 0x7f05c8145930 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f05b800c3f0 tx=0x7f05b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:40.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8017070 con 0x7f05c80ad9c0 2024-09-21T07:49:40.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f05b80035f0 con 0x7f05c80ad9c0 2024-09-21T07:49:40.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05b8003a40 con 0x7f05c80ad9c0 2024-09-21T07:49:40.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05c81578e0 con 0x7f05c80ad9c0 2024-09-21T07:49:40.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.520+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05c8148c60 con 0x7f05c80ad9c0 2024-09-21T07:49:40.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.521+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05c806f930 con 0x7f05c80ad9c0 2024-09-21T07:49:40.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.522+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f05b801a070 con 0x7f05c80ad9c0 2024-09-21T07:49:40.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.524+0000 7f05bea00640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f05a4078d70 0x7f05a407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:40.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.524+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f05b809a510 con 0x7f05c80ad9c0 2024-09-21T07:49:40.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.525+0000 7f05bd400640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f05a4078d70 0x7f05a407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:40.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.526+0000 7f05bd400640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f05a4078d70 0x7f05a407b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f05b0034000 tx=0x7f05b00046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:40.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.529+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f05c81578e0 con 0x7f05c80ad9c0 2024-09-21T07:49:40.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.709+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f05c8145e70 con 0x7f05a4078d70 2024-09-21T07:49:40.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.714+0000 7f05bea00640 1 -- 172.21.15.88:0/3083120915 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251900 (secure 0 0 0) 0x7f05c8145e70 con 0x7f05a4078d70 2024-09-21T07:49:40.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:40.720 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.718+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f05a4078d70 msgr2=0x7f05a407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.718+0000 7f05c7200640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f05a4078d70 0x7f05a407b280 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f05b0034000 tx=0x7f05b00046c0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.718+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 msgr2=0x7f05c8145930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.718+0000 7f05c7200640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 0x7f05c8145930 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f05b800c3f0 tx=0x7f05b800c8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 shutdown_connections 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f05a4078d70 0x7f05a407b280 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f05c80af170 0x7f05c81e2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f05c815a0f0 0x7f05c81f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 --2- 172.21.15.88:0/3083120915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05c80ad9c0 0x7f05c8145930 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:40.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 >> 172.21.15.88:0/3083120915 conn(0x7f05c80b1e40 msgr2=0x7f05c80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:40.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 shutdown_connections 2024-09-21T07:49:40.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:40.719+0000 7f05c7200640 1 -- 172.21.15.88:0/3083120915 wait complete. 2024-09-21T07:49:40.746 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":169,"stamp":"2024-09-21T07:49:39.913764+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":146,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":111,"ondisk_log_size":111,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":252144,"kb_used_data":6760,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":754722576,"statfs":{"total":773094113280,"available":772835917824,"internally_reserved":0,"allocated":6922240,"data_stored":3534428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":251055665},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":12,"apply_latency_ms":12,"commit_latency_ns":12000000,"apply_latency_ns":12000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.084527"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858884+0000","last_change":"2024-09-21T07:49:30.865835+0000","last_active":"2024-09-21T07:49:31.858884+0000","last_peered":"2024-09-21T07:49:31.858884+0000","last_clean":"2024-09-21T07:49:31.858884+0000","last_became_active":"2024-09-21T07:49:30.863575+0000","last_became_peered":"2024-09-21T07:49:30.863575+0000","last_unstale":"2024-09-21T07:49:31.858884+0000","last_undegraded":"2024-09-21T07:49:31.858884+0000","last_fullsized":"2024-09-21T07:49:31.858884+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:26.732169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898139+0000","last_change":"2024-09-21T07:49:30.866746+0000","last_active":"2024-09-21T07:49:31.898139+0000","last_peered":"2024-09-21T07:49:31.898139+0000","last_clean":"2024-09-21T07:49:31.898139+0000","last_became_active":"2024-09-21T07:49:30.866386+0000","last_became_peered":"2024-09-21T07:49:30.866386+0000","last_unstale":"2024-09-21T07:49:31.898139+0000","last_undegraded":"2024-09-21T07:49:31.898139+0000","last_fullsized":"2024-09-21T07:49:31.898139+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:41.251356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861624+0000","last_change":"2024-09-21T07:49:30.842935+0000","last_active":"2024-09-21T07:49:31.861624+0000","last_peered":"2024-09-21T07:49:31.861624+0000","last_clean":"2024-09-21T07:49:31.861624+0000","last_became_active":"2024-09-21T07:49:30.841630+0000","last_became_peered":"2024-09-21T07:49:30.841630+0000","last_unstale":"2024-09-21T07:49:31.861624+0000","last_undegraded":"2024-09-21T07:49:31.861624+0000","last_fullsized":"2024-09-21T07:49:31.861624+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:22.601026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898018+0000","last_change":"2024-09-21T07:49:30.862748+0000","last_active":"2024-09-21T07:49:31.898018+0000","last_peered":"2024-09-21T07:49:31.898018+0000","last_clean":"2024-09-21T07:49:31.898018+0000","last_became_active":"2024-09-21T07:49:30.862527+0000","last_became_peered":"2024-09-21T07:49:30.862527+0000","last_unstale":"2024-09-21T07:49:31.898018+0000","last_undegraded":"2024-09-21T07:49:31.898018+0000","last_fullsized":"2024-09-21T07:49:31.898018+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:17.326924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858588+0000","last_change":"2024-09-21T07:49:30.857817+0000","last_active":"2024-09-21T07:49:31.858588+0000","last_peered":"2024-09-21T07:49:31.858588+0000","last_clean":"2024-09-21T07:49:31.858588+0000","last_became_active":"2024-09-21T07:49:30.857065+0000","last_became_peered":"2024-09-21T07:49:30.857065+0000","last_unstale":"2024-09-21T07:49:31.858588+0000","last_undegraded":"2024-09-21T07:49:31.858588+0000","last_fullsized":"2024-09-21T07:49:31.858588+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:39.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858182+0000","last_change":"2024-09-21T07:49:30.850469+0000","last_active":"2024-09-21T07:49:31.858182+0000","last_peered":"2024-09-21T07:49:31.858182+0000","last_clean":"2024-09-21T07:49:31.858182+0000","last_became_active":"2024-09-21T07:49:30.850119+0000","last_became_peered":"2024-09-21T07:49:30.850119+0000","last_unstale":"2024-09-21T07:49:31.858182+0000","last_undegraded":"2024-09-21T07:49:31.858182+0000","last_fullsized":"2024-09-21T07:49:31.858182+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:23.015017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897597+0000","last_change":"2024-09-21T07:49:30.845351+0000","last_active":"2024-09-21T07:49:31.897597+0000","last_peered":"2024-09-21T07:49:31.897597+0000","last_clean":"2024-09-21T07:49:31.897597+0000","last_became_active":"2024-09-21T07:49:30.843920+0000","last_became_peered":"2024-09-21T07:49:30.843920+0000","last_unstale":"2024-09-21T07:49:31.897597+0000","last_undegraded":"2024-09-21T07:49:31.897597+0000","last_fullsized":"2024-09-21T07:49:31.897597+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:19:25.494394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851362+0000","last_change":"2024-09-21T07:49:30.850184+0000","last_active":"2024-09-21T07:49:31.851362+0000","last_peered":"2024-09-21T07:49:31.851362+0000","last_clean":"2024-09-21T07:49:31.851362+0000","last_became_active":"2024-09-21T07:49:30.849571+0000","last_became_peered":"2024-09-21T07:49:30.849571+0000","last_unstale":"2024-09-21T07:49:31.851362+0000","last_undegraded":"2024-09-21T07:49:31.851362+0000","last_fullsized":"2024-09-21T07:49:31.851362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:46:25.897438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849745+0000","last_change":"2024-09-21T07:49:30.858444+0000","last_active":"2024-09-21T07:49:31.849745+0000","last_peered":"2024-09-21T07:49:31.849745+0000","last_clean":"2024-09-21T07:49:31.849745+0000","last_became_active":"2024-09-21T07:49:30.857207+0000","last_became_peered":"2024-09-21T07:49:30.857207+0000","last_unstale":"2024-09-21T07:49:31.849745+0000","last_undegraded":"2024-09-21T07:49:31.849745+0000","last_fullsized":"2024-09-21T07:49:31.849745+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:43.445886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858632+0000","last_change":"2024-09-21T07:49:30.849692+0000","last_active":"2024-09-21T07:49:31.858632+0000","last_peered":"2024-09-21T07:49:31.858632+0000","last_clean":"2024-09-21T07:49:31.858632+0000","last_became_active":"2024-09-21T07:49:30.849329+0000","last_became_peered":"2024-09-21T07:49:30.849329+0000","last_unstale":"2024-09-21T07:49:31.858632+0000","last_undegraded":"2024-09-21T07:49:31.858632+0000","last_fullsized":"2024-09-21T07:49:31.858632+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:50.221480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852174+0000","last_change":"2024-09-21T07:49:30.857819+0000","last_active":"2024-09-21T07:49:31.852174+0000","last_peered":"2024-09-21T07:49:31.852174+0000","last_clean":"2024-09-21T07:49:31.852174+0000","last_became_active":"2024-09-21T07:49:30.857491+0000","last_became_peered":"2024-09-21T07:49:30.857491+0000","last_unstale":"2024-09-21T07:49:31.852174+0000","last_undegraded":"2024-09-21T07:49:31.852174+0000","last_fullsized":"2024-09-21T07:49:31.852174+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:34:22.601026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849627+0000","last_change":"2024-09-21T07:49:30.845589+0000","last_active":"2024-09-21T07:49:31.849627+0000","last_peered":"2024-09-21T07:49:31.849627+0000","last_clean":"2024-09-21T07:49:31.849627+0000","last_became_active":"2024-09-21T07:49:30.845259+0000","last_became_peered":"2024-09-21T07:49:30.845259+0000","last_unstale":"2024-09-21T07:49:31.849627+0000","last_undegraded":"2024-09-21T07:49:31.849627+0000","last_fullsized":"2024-09-21T07:49:31.849627+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:18.427024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897362+0000","last_change":"2024-09-21T07:49:30.850524+0000","last_active":"2024-09-21T07:49:31.897362+0000","last_peered":"2024-09-21T07:49:31.897362+0000","last_clean":"2024-09-21T07:49:31.897362+0000","last_became_active":"2024-09-21T07:49:30.849502+0000","last_became_peered":"2024-09-21T07:49:30.849502+0000","last_unstale":"2024-09-21T07:49:31.897362+0000","last_undegraded":"2024-09-21T07:49:31.897362+0000","last_fullsized":"2024-09-21T07:49:31.897362+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:19.251551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861401+0000","last_change":"2024-09-21T07:49:30.855844+0000","last_active":"2024-09-21T07:49:31.861401+0000","last_peered":"2024-09-21T07:49:31.861401+0000","last_clean":"2024-09-21T07:49:31.861401+0000","last_became_active":"2024-09-21T07:49:30.854339+0000","last_became_peered":"2024-09-21T07:49:30.854339+0000","last_unstale":"2024-09-21T07:49:31.861401+0000","last_undegraded":"2024-09-21T07:49:31.861401+0000","last_fullsized":"2024-09-21T07:49:31.861401+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.682369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897940+0000","last_change":"2024-09-21T07:49:30.856897+0000","last_active":"2024-09-21T07:49:31.897940+0000","last_peered":"2024-09-21T07:49:31.897940+0000","last_clean":"2024-09-21T07:49:31.897940+0000","last_became_active":"2024-09-21T07:49:30.856435+0000","last_became_peered":"2024-09-21T07:49:30.856435+0000","last_unstale":"2024-09-21T07:49:31.897940+0000","last_undegraded":"2024-09-21T07:49:31.897940+0000","last_fullsized":"2024-09-21T07:49:31.897940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:50.221480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858315+0000","last_change":"2024-09-21T07:49:30.849113+0000","last_active":"2024-09-21T07:49:31.858315+0000","last_peered":"2024-09-21T07:49:31.858315+0000","last_clean":"2024-09-21T07:49:31.858315+0000","last_became_active":"2024-09-21T07:49:30.846401+0000","last_became_peered":"2024-09-21T07:49:30.846401+0000","last_unstale":"2024-09-21T07:49:31.858315+0000","last_undegraded":"2024-09-21T07:49:31.858315+0000","last_fullsized":"2024-09-21T07:49:31.858315+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:27:19.251551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858976+0000","last_change":"2024-09-21T07:49:30.855205+0000","last_active":"2024-09-21T07:49:31.858976+0000","last_peered":"2024-09-21T07:49:31.858976+0000","last_clean":"2024-09-21T07:49:31.858976+0000","last_became_active":"2024-09-21T07:49:30.853927+0000","last_became_peered":"2024-09-21T07:49:30.853927+0000","last_unstale":"2024-09-21T07:49:31.858976+0000","last_undegraded":"2024-09-21T07:49:31.858976+0000","last_fullsized":"2024-09-21T07:49:31.858976+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:54.485238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861697+0000","last_change":"2024-09-21T07:49:30.851466+0000","last_active":"2024-09-21T07:49:31.861697+0000","last_peered":"2024-09-21T07:49:31.861697+0000","last_clean":"2024-09-21T07:49:31.861697+0000","last_became_active":"2024-09-21T07:49:30.851064+0000","last_became_peered":"2024-09-21T07:49:30.851064+0000","last_unstale":"2024-09-21T07:49:31.861697+0000","last_undegraded":"2024-09-21T07:49:31.861697+0000","last_fullsized":"2024-09-21T07:49:31.861697+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:40.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897965+0000","last_change":"2024-09-21T07:49:30.870745+0000","last_active":"2024-09-21T07:49:31.897965+0000","last_peered":"2024-09-21T07:49:31.897965+0000","last_clean":"2024-09-21T07:49:31.897965+0000","last_became_active":"2024-09-21T07:49:30.870361+0000","last_became_peered":"2024-09-21T07:49:30.870361+0000","last_unstale":"2024-09-21T07:49:31.897965+0000","last_undegraded":"2024-09-21T07:49:31.897965+0000","last_fullsized":"2024-09-21T07:49:31.897965+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:18.361591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852286+0000","last_change":"2024-09-21T07:49:30.847507+0000","last_active":"2024-09-21T07:49:31.852286+0000","last_peered":"2024-09-21T07:49:31.852286+0000","last_clean":"2024-09-21T07:49:31.852286+0000","last_became_active":"2024-09-21T07:49:30.847122+0000","last_became_peered":"2024-09-21T07:49:30.847122+0000","last_unstale":"2024-09-21T07:49:31.852286+0000","last_undegraded":"2024-09-21T07:49:31.852286+0000","last_fullsized":"2024-09-21T07:49:31.852286+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:40.325593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897475+0000","last_change":"2024-09-21T07:49:30.845594+0000","last_active":"2024-09-21T07:49:31.897475+0000","last_peered":"2024-09-21T07:49:31.897475+0000","last_clean":"2024-09-21T07:49:31.897475+0000","last_became_active":"2024-09-21T07:49:30.844794+0000","last_became_peered":"2024-09-21T07:49:30.844794+0000","last_unstale":"2024-09-21T07:49:31.897475+0000","last_undegraded":"2024-09-21T07:49:31.897475+0000","last_fullsized":"2024-09-21T07:49:31.897475+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:39.114613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851646+0000","last_change":"2024-09-21T07:49:30.862336+0000","last_active":"2024-09-21T07:49:31.851646+0000","last_peered":"2024-09-21T07:49:31.851646+0000","last_clean":"2024-09-21T07:49:31.851646+0000","last_became_active":"2024-09-21T07:49:30.858105+0000","last_became_peered":"2024-09-21T07:49:30.858105+0000","last_unstale":"2024-09-21T07:49:31.851646+0000","last_undegraded":"2024-09-21T07:49:31.851646+0000","last_fullsized":"2024-09-21T07:49:31.851646+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.363283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897813+0000","last_change":"2024-09-21T07:49:30.852704+0000","last_active":"2024-09-21T07:49:31.897813+0000","last_peered":"2024-09-21T07:49:31.897813+0000","last_clean":"2024-09-21T07:49:31.897813+0000","last_became_active":"2024-09-21T07:49:30.851579+0000","last_became_peered":"2024-09-21T07:49:30.851579+0000","last_unstale":"2024-09-21T07:49:31.897813+0000","last_undegraded":"2024-09-21T07:49:31.897813+0000","last_fullsized":"2024-09-21T07:49:31.897813+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:40:29.279321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849130+0000","last_change":"2024-09-21T07:49:30.852598+0000","last_active":"2024-09-21T07:49:31.849130+0000","last_peered":"2024-09-21T07:49:31.849130+0000","last_clean":"2024-09-21T07:49:31.849130+0000","last_became_active":"2024-09-21T07:49:30.852201+0000","last_became_peered":"2024-09-21T07:49:30.852201+0000","last_unstale":"2024-09-21T07:49:31.849130+0000","last_undegraded":"2024-09-21T07:49:31.849130+0000","last_fullsized":"2024-09-21T07:49:31.849130+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:52.297054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898160+0000","last_change":"2024-09-21T07:49:30.866983+0000","last_active":"2024-09-21T07:49:31.898160+0000","last_peered":"2024-09-21T07:49:31.898160+0000","last_clean":"2024-09-21T07:49:31.898160+0000","last_became_active":"2024-09-21T07:49:30.866405+0000","last_became_peered":"2024-09-21T07:49:30.866405+0000","last_unstale":"2024-09-21T07:49:31.898160+0000","last_undegraded":"2024-09-21T07:49:31.898160+0000","last_fullsized":"2024-09-21T07:49:31.898160+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:57:33.824401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897940+0000","last_change":"2024-09-21T07:49:30.847189+0000","last_active":"2024-09-21T07:49:31.897940+0000","last_peered":"2024-09-21T07:49:31.897940+0000","last_clean":"2024-09-21T07:49:31.897940+0000","last_became_active":"2024-09-21T07:49:30.846811+0000","last_became_peered":"2024-09-21T07:49:30.846811+0000","last_unstale":"2024-09-21T07:49:31.897940+0000","last_undegraded":"2024-09-21T07:49:31.897940+0000","last_fullsized":"2024-09-21T07:49:31.897940+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:33.055293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898197+0000","last_change":"2024-09-21T07:49:30.868732+0000","last_active":"2024-09-21T07:49:31.898197+0000","last_peered":"2024-09-21T07:49:31.898197+0000","last_clean":"2024-09-21T07:49:31.898197+0000","last_became_active":"2024-09-21T07:49:30.868358+0000","last_became_peered":"2024-09-21T07:49:30.868358+0000","last_unstale":"2024-09-21T07:49:31.898197+0000","last_undegraded":"2024-09-21T07:49:31.898197+0000","last_fullsized":"2024-09-21T07:49:31.898197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:09.597807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852995+0000","last_change":"2024-09-21T07:49:30.860588+0000","last_active":"2024-09-21T07:49:31.852995+0000","last_peered":"2024-09-21T07:49:31.852995+0000","last_clean":"2024-09-21T07:49:31.852995+0000","last_became_active":"2024-09-21T07:49:30.858226+0000","last_became_peered":"2024-09-21T07:49:30.858226+0000","last_unstale":"2024-09-21T07:49:31.852995+0000","last_undegraded":"2024-09-21T07:49:31.852995+0000","last_fullsized":"2024-09-21T07:49:31.852995+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:16.869080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852846+0000","last_change":"2024-09-21T07:49:30.850556+0000","last_active":"2024-09-21T07:49:31.852846+0000","last_peered":"2024-09-21T07:49:31.852846+0000","last_clean":"2024-09-21T07:49:31.852846+0000","last_became_active":"2024-09-21T07:49:30.848534+0000","last_became_peered":"2024-09-21T07:49:30.848534+0000","last_unstale":"2024-09-21T07:49:31.852846+0000","last_undegraded":"2024-09-21T07:49:31.852846+0000","last_fullsized":"2024-09-21T07:49:31.852846+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:16.784201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897404+0000","last_change":"2024-09-21T07:49:30.848469+0000","last_active":"2024-09-21T07:49:31.897404+0000","last_peered":"2024-09-21T07:49:31.897404+0000","last_clean":"2024-09-21T07:49:31.897404+0000","last_became_active":"2024-09-21T07:49:30.848130+0000","last_became_peered":"2024-09-21T07:49:30.848130+0000","last_unstale":"2024-09-21T07:49:31.897404+0000","last_undegraded":"2024-09-21T07:49:31.897404+0000","last_fullsized":"2024-09-21T07:49:31.897404+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:34.319960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897737+0000","last_change":"2024-09-21T07:49:30.862236+0000","last_active":"2024-09-21T07:49:31.897737+0000","last_peered":"2024-09-21T07:49:31.897737+0000","last_clean":"2024-09-21T07:49:31.897737+0000","last_became_active":"2024-09-21T07:49:30.861285+0000","last_became_peered":"2024-09-21T07:49:30.861285+0000","last_unstale":"2024-09-21T07:49:31.897737+0000","last_undegraded":"2024-09-21T07:49:31.897737+0000","last_fullsized":"2024-09-21T07:49:31.897737+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:39.866684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851367+0000","last_change":"2024-09-21T07:49:30.852919+0000","last_active":"2024-09-21T07:49:31.851367+0000","last_peered":"2024-09-21T07:49:31.851367+0000","last_clean":"2024-09-21T07:49:31.851367+0000","last_became_active":"2024-09-21T07:49:30.852244+0000","last_became_peered":"2024-09-21T07:49:30.852244+0000","last_unstale":"2024-09-21T07:49:31.851367+0000","last_undegraded":"2024-09-21T07:49:31.851367+0000","last_fullsized":"2024-09-21T07:49:31.851367+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:49.682369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858896+0000","last_change":"2024-09-21T07:49:30.844684+0000","last_active":"2024-09-21T07:49:31.858896+0000","last_peered":"2024-09-21T07:49:31.858896+0000","last_clean":"2024-09-21T07:49:31.858896+0000","last_became_active":"2024-09-21T07:49:30.844376+0000","last_became_peered":"2024-09-21T07:49:30.844376+0000","last_unstale":"2024-09-21T07:49:31.858896+0000","last_undegraded":"2024-09-21T07:49:31.858896+0000","last_fullsized":"2024-09-21T07:49:31.858896+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:57.962690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1f","version":"59'4","reported_seq":19,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853432+0000","last_change":"2024-09-21T07:49:29.833079+0000","last_active":"2024-09-21T07:49:31.853432+0000","last_peered":"2024-09-21T07:49:31.853432+0000","last_clean":"2024-09-21T07:49:31.853432+0000","last_became_active":"2024-09-21T07:49:29.832745+0000","last_became_peered":"2024-09-21T07:49:29.832745+0000","last_unstale":"2024-09-21T07:49:31.853432+0000","last_undegraded":"2024-09-21T07:49:31.853432+0000","last_fullsized":"2024-09-21T07:49:31.853432+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:25.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.859088+0000","last_change":"2024-09-21T07:49:30.845386+0000","last_active":"2024-09-21T07:49:31.859088+0000","last_peered":"2024-09-21T07:49:31.859088+0000","last_clean":"2024-09-21T07:49:31.859088+0000","last_became_active":"2024-09-21T07:49:30.845086+0000","last_became_peered":"2024-09-21T07:49:30.845086+0000","last_unstale":"2024-09-21T07:49:31.859088+0000","last_undegraded":"2024-09-21T07:49:31.859088+0000","last_fullsized":"2024-09-21T07:49:31.859088+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:05:51.106407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1e","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898195+0000","last_change":"2024-09-21T07:49:29.844161+0000","last_active":"2024-09-21T07:49:31.898195+0000","last_peered":"2024-09-21T07:49:31.898195+0000","last_clean":"2024-09-21T07:49:31.898195+0000","last_became_active":"2024-09-21T07:49:29.843722+0000","last_became_peered":"2024-09-21T07:49:29.843722+0000","last_unstale":"2024-09-21T07:49:31.898195+0000","last_undegraded":"2024-09-21T07:49:31.898195+0000","last_fullsized":"2024-09-21T07:49:31.898195+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:52:56.739749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853714+0000","last_change":"2024-09-21T07:49:30.864136+0000","last_active":"2024-09-21T07:49:31.853714+0000","last_peered":"2024-09-21T07:49:31.853714+0000","last_clean":"2024-09-21T07:49:31.853714+0000","last_became_active":"2024-09-21T07:49:30.863803+0000","last_became_peered":"2024-09-21T07:49:30.863803+0000","last_unstale":"2024-09-21T07:49:31.853714+0000","last_undegraded":"2024-09-21T07:49:31.853714+0000","last_fullsized":"2024-09-21T07:49:31.853714+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:29.737578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1d","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858441+0000","last_change":"2024-09-21T07:49:29.841731+0000","last_active":"2024-09-21T07:49:31.858441+0000","last_peered":"2024-09-21T07:49:31.858441+0000","last_clean":"2024-09-21T07:49:31.858441+0000","last_became_active":"2024-09-21T07:49:29.841419+0000","last_became_peered":"2024-09-21T07:49:29.841419+0000","last_unstale":"2024-09-21T07:49:31.858441+0000","last_undegraded":"2024-09-21T07:49:31.858441+0000","last_fullsized":"2024-09-21T07:49:31.858441+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:04:45.852919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858495+0000","last_change":"2024-09-21T07:49:30.850320+0000","last_active":"2024-09-21T07:49:31.858495+0000","last_peered":"2024-09-21T07:49:31.858495+0000","last_clean":"2024-09-21T07:49:31.858495+0000","last_became_active":"2024-09-21T07:49:30.849791+0000","last_became_peered":"2024-09-21T07:49:30.849791+0000","last_unstale":"2024-09-21T07:49:31.858495+0000","last_undegraded":"2024-09-21T07:49:31.858495+0000","last_fullsized":"2024-09-21T07:49:31.858495+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:18:20.450951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.1c","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897679+0000","last_change":"2024-09-21T07:49:29.839504+0000","last_active":"2024-09-21T07:49:31.897679+0000","last_peered":"2024-09-21T07:49:31.897679+0000","last_clean":"2024-09-21T07:49:31.897679+0000","last_became_active":"2024-09-21T07:49:29.838264+0000","last_became_peered":"2024-09-21T07:49:29.838264+0000","last_unstale":"2024-09-21T07:49:31.897679+0000","last_undegraded":"2024-09-21T07:49:31.897679+0000","last_fullsized":"2024-09-21T07:49:31.897679+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:02.854089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853346+0000","last_change":"2024-09-21T07:49:30.859157+0000","last_active":"2024-09-21T07:49:31.853346+0000","last_peered":"2024-09-21T07:49:31.853346+0000","last_clean":"2024-09-21T07:49:31.853346+0000","last_became_active":"2024-09-21T07:49:30.858858+0000","last_became_peered":"2024-09-21T07:49:30.858858+0000","last_unstale":"2024-09-21T07:49:31.853346+0000","last_undegraded":"2024-09-21T07:49:31.853346+0000","last_fullsized":"2024-09-21T07:49:31.853346+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:49:49.132180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1b","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861569+0000","last_change":"2024-09-21T07:49:29.835030+0000","last_active":"2024-09-21T07:49:31.861569+0000","last_peered":"2024-09-21T07:49:31.861569+0000","last_clean":"2024-09-21T07:49:31.861569+0000","last_became_active":"2024-09-21T07:49:29.832816+0000","last_became_peered":"2024-09-21T07:49:29.832816+0000","last_unstale":"2024-09-21T07:49:31.861569+0000","last_undegraded":"2024-09-21T07:49:31.861569+0000","last_fullsized":"2024-09-21T07:49:31.861569+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:12.631298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852420+0000","last_change":"2024-09-21T07:49:30.851949+0000","last_active":"2024-09-21T07:49:31.852420+0000","last_peered":"2024-09-21T07:49:31.852420+0000","last_clean":"2024-09-21T07:49:31.852420+0000","last_became_active":"2024-09-21T07:49:30.851637+0000","last_became_peered":"2024-09-21T07:49:30.851637+0000","last_unstale":"2024-09-21T07:49:31.852420+0000","last_undegraded":"2024-09-21T07:49:31.852420+0000","last_fullsized":"2024-09-21T07:49:31.852420+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:25.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.1a","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858446+0000","last_change":"2024-09-21T07:49:29.840419+0000","last_active":"2024-09-21T07:49:31.858446+0000","last_peered":"2024-09-21T07:49:31.858446+0000","last_clean":"2024-09-21T07:49:31.858446+0000","last_became_active":"2024-09-21T07:49:29.839959+0000","last_became_peered":"2024-09-21T07:49:29.839959+0000","last_unstale":"2024-09-21T07:49:31.858446+0000","last_undegraded":"2024-09-21T07:49:31.858446+0000","last_fullsized":"2024-09-21T07:49:31.858446+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:42.438425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861316+0000","last_change":"2024-09-21T07:49:30.841186+0000","last_active":"2024-09-21T07:49:31.861316+0000","last_peered":"2024-09-21T07:49:31.861316+0000","last_clean":"2024-09-21T07:49:31.861316+0000","last_became_active":"2024-09-21T07:49:30.840647+0000","last_became_peered":"2024-09-21T07:49:30.840647+0000","last_unstale":"2024-09-21T07:49:31.861316+0000","last_undegraded":"2024-09-21T07:49:31.861316+0000","last_fullsized":"2024-09-21T07:49:31.861316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:15.627223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849194+0000","last_change":"2024-09-21T07:49:29.835362+0000","last_active":"2024-09-21T07:49:31.849194+0000","last_peered":"2024-09-21T07:49:31.849194+0000","last_clean":"2024-09-21T07:49:31.849194+0000","last_became_active":"2024-09-21T07:49:29.834971+0000","last_became_peered":"2024-09-21T07:49:29.834971+0000","last_unstale":"2024-09-21T07:49:31.849194+0000","last_undegraded":"2024-09-21T07:49:31.849194+0000","last_fullsized":"2024-09-21T07:49:31.849194+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:32.215749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858179+0000","last_change":"2024-09-21T07:49:30.849385+0000","last_active":"2024-09-21T07:49:31.858179+0000","last_peered":"2024-09-21T07:49:31.858179+0000","last_clean":"2024-09-21T07:49:31.858179+0000","last_became_active":"2024-09-21T07:49:30.849031+0000","last_became_peered":"2024-09-21T07:49:30.849031+0000","last_unstale":"2024-09-21T07:49:31.858179+0000","last_undegraded":"2024-09-21T07:49:31.858179+0000","last_fullsized":"2024-09-21T07:49:31.858179+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:29.406829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897090+0000","last_change":"2024-09-21T07:49:29.830256+0000","last_active":"2024-09-21T07:49:31.897090+0000","last_peered":"2024-09-21T07:49:31.897090+0000","last_clean":"2024-09-21T07:49:31.897090+0000","last_became_active":"2024-09-21T07:49:29.829961+0000","last_became_peered":"2024-09-21T07:49:29.829961+0000","last_unstale":"2024-09-21T07:49:31.897090+0000","last_undegraded":"2024-09-21T07:49:31.897090+0000","last_fullsized":"2024-09-21T07:49:31.897090+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:05.746170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898021+0000","last_change":"2024-09-21T07:49:30.856701+0000","last_active":"2024-09-21T07:49:31.898021+0000","last_peered":"2024-09-21T07:49:31.898021+0000","last_clean":"2024-09-21T07:49:31.898021+0000","last_became_active":"2024-09-21T07:49:30.855565+0000","last_became_peered":"2024-09-21T07:49:30.855565+0000","last_unstale":"2024-09-21T07:49:31.898021+0000","last_undegraded":"2024-09-21T07:49:31.898021+0000","last_fullsized":"2024-09-21T07:49:31.898021+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:57.962690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849696+0000","last_change":"2024-09-21T07:49:29.841680+0000","last_active":"2024-09-21T07:49:31.849696+0000","last_peered":"2024-09-21T07:49:31.849696+0000","last_clean":"2024-09-21T07:49:31.849696+0000","last_became_active":"2024-09-21T07:49:29.841344+0000","last_became_peered":"2024-09-21T07:49:29.841344+0000","last_unstale":"2024-09-21T07:49:31.849696+0000","last_undegraded":"2024-09-21T07:49:31.849696+0000","last_fullsized":"2024-09-21T07:49:31.849696+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:18:45.801003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861955+0000","last_change":"2024-09-21T07:49:30.856508+0000","last_active":"2024-09-21T07:49:31.861955+0000","last_peered":"2024-09-21T07:49:31.861955+0000","last_clean":"2024-09-21T07:49:31.861955+0000","last_became_active":"2024-09-21T07:49:30.856203+0000","last_became_peered":"2024-09-21T07:49:30.856203+0000","last_unstale":"2024-09-21T07:49:31.861955+0000","last_undegraded":"2024-09-21T07:49:31.861955+0000","last_fullsized":"2024-09-21T07:49:31.861955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:45.508268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853459+0000","last_change":"2024-09-21T07:49:29.834642+0000","last_active":"2024-09-21T07:49:31.853459+0000","last_peered":"2024-09-21T07:49:31.853459+0000","last_clean":"2024-09-21T07:49:31.853459+0000","last_became_active":"2024-09-21T07:49:29.833980+0000","last_became_peered":"2024-09-21T07:49:29.833980+0000","last_unstale":"2024-09-21T07:49:31.853459+0000","last_undegraded":"2024-09-21T07:49:31.853459+0000","last_fullsized":"2024-09-21T07:49:31.853459+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:49.214019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853503+0000","last_change":"2024-09-21T07:49:30.863536+0000","last_active":"2024-09-21T07:49:31.853503+0000","last_peered":"2024-09-21T07:49:31.853503+0000","last_clean":"2024-09-21T07:49:31.853503+0000","last_became_active":"2024-09-21T07:49:30.863234+0000","last_became_peered":"2024-09-21T07:49:30.863234+0000","last_unstale":"2024-09-21T07:49:31.853503+0000","last_undegraded":"2024-09-21T07:49:31.853503+0000","last_fullsized":"2024-09-21T07:49:31.853503+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:16:51.473873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.15","version":"59'3","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897729+0000","last_change":"2024-09-21T07:49:29.842330+0000","last_active":"2024-09-21T07:49:31.897729+0000","last_peered":"2024-09-21T07:49:31.897729+0000","last_clean":"2024-09-21T07:49:31.897729+0000","last_became_active":"2024-09-21T07:49:29.842047+0000","last_became_peered":"2024-09-21T07:49:29.842047+0000","last_unstale":"2024-09-21T07:49:31.897729+0000","last_undegraded":"2024-09-21T07:49:31.897729+0000","last_fullsized":"2024-09-21T07:49:31.897729+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:31.490202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897565+0000","last_change":"2024-09-21T07:49:30.859923+0000","last_active":"2024-09-21T07:49:31.897565+0000","last_peered":"2024-09-21T07:49:31.897565+0000","last_clean":"2024-09-21T07:49:31.897565+0000","last_became_active":"2024-09-21T07:49:30.859557+0000","last_became_peered":"2024-09-21T07:49:30.859557+0000","last_unstale":"2024-09-21T07:49:31.897565+0000","last_undegraded":"2024-09-21T07:49:31.897565+0000","last_fullsized":"2024-09-21T07:49:31.897565+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:32:57.514682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.14","version":"59'6","reported_seq":21,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897920+0000","last_change":"2024-09-21T07:49:29.842473+0000","last_active":"2024-09-21T07:49:31.897920+0000","last_peered":"2024-09-21T07:49:31.897920+0000","last_clean":"2024-09-21T07:49:31.897920+0000","last_became_active":"2024-09-21T07:49:29.842149+0000","last_became_peered":"2024-09-21T07:49:29.842149+0000","last_unstale":"2024-09-21T07:49:31.897920+0000","last_undegraded":"2024-09-21T07:49:31.897920+0000","last_fullsized":"2024-09-21T07:49:31.897920+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:19.977891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849328+0000","last_change":"2024-09-21T07:49:30.867765+0000","last_active":"2024-09-21T07:49:31.849328+0000","last_peered":"2024-09-21T07:49:31.849328+0000","last_clean":"2024-09-21T07:49:31.849328+0000","last_became_active":"2024-09-21T07:49:30.867424+0000","last_became_peered":"2024-09-21T07:49:30.867424+0000","last_unstale":"2024-09-21T07:49:31.849328+0000","last_undegraded":"2024-09-21T07:49:31.849328+0000","last_fullsized":"2024-09-21T07:49:31.849328+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:32.624299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.13","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852930+0000","last_change":"2024-09-21T07:49:29.832526+0000","last_active":"2024-09-21T07:49:31.852930+0000","last_peered":"2024-09-21T07:49:31.852930+0000","last_clean":"2024-09-21T07:49:31.852930+0000","last_became_active":"2024-09-21T07:49:29.832191+0000","last_became_peered":"2024-09-21T07:49:29.832191+0000","last_unstale":"2024-09-21T07:49:31.852930+0000","last_undegraded":"2024-09-21T07:49:31.852930+0000","last_fullsized":"2024-09-21T07:49:31.852930+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:40.367868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861439+0000","last_change":"2024-09-21T07:49:30.847902+0000","last_active":"2024-09-21T07:49:31.861439+0000","last_peered":"2024-09-21T07:49:31.861439+0000","last_clean":"2024-09-21T07:49:31.861439+0000","last_became_active":"2024-09-21T07:49:30.846155+0000","last_became_peered":"2024-09-21T07:49:30.846155+0000","last_unstale":"2024-09-21T07:49:31.861439+0000","last_undegraded":"2024-09-21T07:49:31.861439+0000","last_fullsized":"2024-09-21T07:49:31.861439+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.363283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897581+0000","last_change":"2024-09-21T07:49:29.835475+0000","last_active":"2024-09-21T07:49:31.897581+0000","last_peered":"2024-09-21T07:49:31.897581+0000","last_clean":"2024-09-21T07:49:31.897581+0000","last_became_active":"2024-09-21T07:49:29.835130+0000","last_became_peered":"2024-09-21T07:49:29.835130+0000","last_unstale":"2024-09-21T07:49:31.897581+0000","last_undegraded":"2024-09-21T07:49:31.897581+0000","last_fullsized":"2024-09-21T07:49:31.897581+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:54.479050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849197+0000","last_change":"2024-09-21T07:49:30.845676+0000","last_active":"2024-09-21T07:49:31.849197+0000","last_peered":"2024-09-21T07:49:31.849197+0000","last_clean":"2024-09-21T07:49:31.849197+0000","last_became_active":"2024-09-21T07:49:30.845360+0000","last_became_peered":"2024-09-21T07:49:30.845360+0000","last_unstale":"2024-09-21T07:49:31.849197+0000","last_undegraded":"2024-09-21T07:49:31.849197+0000","last_fullsized":"2024-09-21T07:49:31.849197+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:13.860556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898147+0000","last_change":"2024-09-21T07:49:29.825131+0000","last_active":"2024-09-21T07:49:31.898147+0000","last_peered":"2024-09-21T07:49:31.898147+0000","last_clean":"2024-09-21T07:49:31.898147+0000","last_became_active":"2024-09-21T07:49:29.824617+0000","last_became_peered":"2024-09-21T07:49:29.824617+0000","last_unstale":"2024-09-21T07:49:31.898147+0000","last_undegraded":"2024-09-21T07:49:31.898147+0000","last_fullsized":"2024-09-21T07:49:31.898147+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:29.601267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898004+0000","last_change":"2024-09-21T07:49:30.849916+0000","last_active":"2024-09-21T07:49:31.898004+0000","last_peered":"2024-09-21T07:49:31.898004+0000","last_clean":"2024-09-21T07:49:31.898004+0000","last_became_active":"2024-09-21T07:49:30.849262+0000","last_became_peered":"2024-09-21T07:49:30.849262+0000","last_unstale":"2024-09-21T07:49:31.898004+0000","last_undegraded":"2024-09-21T07:49:31.898004+0000","last_fullsized":"2024-09-21T07:49:31.898004+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:37.033463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.10","version":"59'8","reported_seq":23,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897243+0000","last_change":"2024-09-21T07:49:29.836030+0000","last_active":"2024-09-21T07:49:31.897243+0000","last_peered":"2024-09-21T07:49:31.897243+0000","last_clean":"2024-09-21T07:49:31.897243+0000","last_became_active":"2024-09-21T07:49:29.835703+0000","last_became_peered":"2024-09-21T07:49:29.835703+0000","last_unstale":"2024-09-21T07:49:31.897243+0000","last_undegraded":"2024-09-21T07:49:31.897243+0000","last_fullsized":"2024-09-21T07:49:31.897243+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:06.109094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861729+0000","last_change":"2024-09-21T07:49:30.836520+0000","last_active":"2024-09-21T07:49:31.861729+0000","last_peered":"2024-09-21T07:49:31.861729+0000","last_clean":"2024-09-21T07:49:31.861729+0000","last_became_active":"2024-09-21T07:49:30.836216+0000","last_became_peered":"2024-09-21T07:49:30.836216+0000","last_unstale":"2024-09-21T07:49:31.861729+0000","last_undegraded":"2024-09-21T07:49:31.861729+0000","last_fullsized":"2024-09-21T07:49:31.861729+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:53:25.399625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.f","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:29.843999+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:29.843591+0000","last_became_peered":"2024-09-21T07:49:29.843591+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:56.147012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861983+0000","last_change":"2024-09-21T07:49:30.865501+0000","last_active":"2024-09-21T07:49:31.861983+0000","last_peered":"2024-09-21T07:49:31.861983+0000","last_clean":"2024-09-21T07:49:31.861983+0000","last_became_active":"2024-09-21T07:49:30.865151+0000","last_became_peered":"2024-09-21T07:49:30.865151+0000","last_unstale":"2024-09-21T07:49:31.861983+0000","last_undegraded":"2024-09-21T07:49:31.861983+0000","last_fullsized":"2024-09-21T07:49:31.861983+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:05.484654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853521+0000","last_change":"2024-09-21T07:49:29.834330+0000","last_active":"2024-09-21T07:49:31.853521+0000","last_peered":"2024-09-21T07:49:31.853521+0000","last_clean":"2024-09-21T07:49:31.853521+0000","last_became_active":"2024-09-21T07:49:29.833510+0000","last_became_peered":"2024-09-21T07:49:29.833510+0000","last_unstale":"2024-09-21T07:49:31.853521+0000","last_undegraded":"2024-09-21T07:49:31.853521+0000","last_fullsized":"2024-09-21T07:49:31.853521+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:56.955229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897841+0000","last_change":"2024-09-21T07:49:30.861727+0000","last_active":"2024-09-21T07:49:31.897841+0000","last_peered":"2024-09-21T07:49:31.897841+0000","last_clean":"2024-09-21T07:49:31.897841+0000","last_became_active":"2024-09-21T07:49:30.861401+0000","last_became_peered":"2024-09-21T07:49:30.861401+0000","last_unstale":"2024-09-21T07:49:31.897841+0000","last_undegraded":"2024-09-21T07:49:31.897841+0000","last_fullsized":"2024-09-21T07:49:31.897841+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:59.043011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.d","version":"59'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861775+0000","last_change":"2024-09-21T07:49:29.843565+0000","last_active":"2024-09-21T07:49:31.861775+0000","last_peered":"2024-09-21T07:49:31.861775+0000","last_clean":"2024-09-21T07:49:31.861775+0000","last_became_active":"2024-09-21T07:49:29.843265+0000","last_became_peered":"2024-09-21T07:49:29.843265+0000","last_unstale":"2024-09-21T07:49:31.861775+0000","last_undegraded":"2024-09-21T07:49:31.861775+0000","last_fullsized":"2024-09-21T07:49:31.861775+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:28.399368+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897733+0000","last_change":"2024-09-21T07:49:30.862803+0000","last_active":"2024-09-21T07:49:31.897733+0000","last_peered":"2024-09-21T07:49:31.897733+0000","last_clean":"2024-09-21T07:49:31.897733+0000","last_became_active":"2024-09-21T07:49:30.862519+0000","last_became_peered":"2024-09-21T07:49:30.862519+0000","last_unstale":"2024-09-21T07:49:31.897733+0000","last_undegraded":"2024-09-21T07:49:31.897733+0000","last_fullsized":"2024-09-21T07:49:31.897733+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:47.572947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853592+0000","last_change":"2024-09-21T07:49:29.834999+0000","last_active":"2024-09-21T07:49:31.853592+0000","last_peered":"2024-09-21T07:49:31.853592+0000","last_clean":"2024-09-21T07:49:31.853592+0000","last_became_active":"2024-09-21T07:49:29.834675+0000","last_became_peered":"2024-09-21T07:49:29.834675+0000","last_unstale":"2024-09-21T07:49:31.853592+0000","last_undegraded":"2024-09-21T07:49:31.853592+0000","last_fullsized":"2024-09-21T07:49:31.853592+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:34:25.724708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849458+0000","last_change":"2024-09-21T07:49:30.859030+0000","last_active":"2024-09-21T07:49:31.849458+0000","last_peered":"2024-09-21T07:49:31.849458+0000","last_clean":"2024-09-21T07:49:31.849458+0000","last_became_active":"2024-09-21T07:49:30.858447+0000","last_became_peered":"2024-09-21T07:49:30.858447+0000","last_unstale":"2024-09-21T07:49:31.849458+0000","last_undegraded":"2024-09-21T07:49:31.849458+0000","last_fullsized":"2024-09-21T07:49:31.849458+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:35.606907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.b","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851978+0000","last_change":"2024-09-21T07:49:29.839355+0000","last_active":"2024-09-21T07:49:31.851978+0000","last_peered":"2024-09-21T07:49:31.851978+0000","last_clean":"2024-09-21T07:49:31.851978+0000","last_became_active":"2024-09-21T07:49:29.838434+0000","last_became_peered":"2024-09-21T07:49:29.838434+0000","last_unstale":"2024-09-21T07:49:31.851978+0000","last_undegraded":"2024-09-21T07:49:31.851978+0000","last_fullsized":"2024-09-21T07:49:31.851978+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:49.214019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852911+0000","last_change":"2024-09-21T07:49:30.853099+0000","last_active":"2024-09-21T07:49:31.852911+0000","last_peered":"2024-09-21T07:49:31.852911+0000","last_clean":"2024-09-21T07:49:31.852911+0000","last_became_active":"2024-09-21T07:49:30.852549+0000","last_became_peered":"2024-09-21T07:49:30.852549+0000","last_unstale":"2024-09-21T07:49:31.852911+0000","last_undegraded":"2024-09-21T07:49:31.852911+0000","last_fullsized":"2024-09-21T07:49:31.852911+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:15.428162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852723+0000","last_change":"2024-09-21T07:49:29.831653+0000","last_active":"2024-09-21T07:49:31.852723+0000","last_peered":"2024-09-21T07:49:31.852723+0000","last_clean":"2024-09-21T07:49:31.852723+0000","last_became_active":"2024-09-21T07:49:29.831254+0000","last_became_peered":"2024-09-21T07:49:29.831254+0000","last_unstale":"2024-09-21T07:49:31.852723+0000","last_undegraded":"2024-09-21T07:49:31.852723+0000","last_fullsized":"2024-09-21T07:49:31.852723+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:10.655160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852388+0000","last_change":"2024-09-21T07:49:30.853166+0000","last_active":"2024-09-21T07:49:31.852388+0000","last_peered":"2024-09-21T07:49:31.852388+0000","last_clean":"2024-09-21T07:49:31.852388+0000","last_became_active":"2024-09-21T07:49:30.852810+0000","last_became_peered":"2024-09-21T07:49:30.852810+0000","last_unstale":"2024-09-21T07:49:31.852388+0000","last_undegraded":"2024-09-21T07:49:31.852388+0000","last_fullsized":"2024-09-21T07:49:31.852388+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:52:57.174205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849298+0000","last_change":"2024-09-21T07:49:29.832146+0000","last_active":"2024-09-21T07:49:31.849298+0000","last_peered":"2024-09-21T07:49:31.849298+0000","last_clean":"2024-09-21T07:49:31.849298+0000","last_became_active":"2024-09-21T07:49:29.831853+0000","last_became_peered":"2024-09-21T07:49:29.831853+0000","last_unstale":"2024-09-21T07:49:31.849298+0000","last_undegraded":"2024-09-21T07:49:31.849298+0000","last_fullsized":"2024-09-21T07:49:31.849298+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:04.356597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861915+0000","last_change":"2024-09-21T07:49:30.861233+0000","last_active":"2024-09-21T07:49:31.861915+0000","last_peered":"2024-09-21T07:49:31.861915+0000","last_clean":"2024-09-21T07:49:31.861915+0000","last_became_active":"2024-09-21T07:49:30.860929+0000","last_became_peered":"2024-09-21T07:49:30.860929+0000","last_unstale":"2024-09-21T07:49:31.861915+0000","last_undegraded":"2024-09-21T07:49:31.861915+0000","last_fullsized":"2024-09-21T07:49:31.861915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:45.970839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.7","version":"59'2","reported_seq":17,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898152+0000","last_change":"2024-09-21T07:49:29.843093+0000","last_active":"2024-09-21T07:49:31.898152+0000","last_peered":"2024-09-21T07:49:31.898152+0000","last_clean":"2024-09-21T07:49:31.898152+0000","last_became_active":"2024-09-21T07:49:29.842766+0000","last_became_peered":"2024-09-21T07:49:29.842766+0000","last_unstale":"2024-09-21T07:49:31.898152+0000","last_undegraded":"2024-09-21T07:49:31.898152+0000","last_fullsized":"2024-09-21T07:49:31.898152+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:36:50.990394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.862173+0000","last_change":"2024-09-21T07:49:30.850235+0000","last_active":"2024-09-21T07:49:31.862173+0000","last_peered":"2024-09-21T07:49:31.862173+0000","last_clean":"2024-09-21T07:49:31.862173+0000","last_became_active":"2024-09-21T07:49:30.849887+0000","last_became_peered":"2024-09-21T07:49:30.849887+0000","last_unstale":"2024-09-21T07:49:31.862173+0000","last_undegraded":"2024-09-21T07:49:31.862173+0000","last_fullsized":"2024-09-21T07:49:31.862173+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:07.116555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.6","version":"60'1","reported_seq":16,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858909+0000","last_change":"2024-09-21T07:49:29.839854+0000","last_active":"2024-09-21T07:49:31.858909+0000","last_peered":"2024-09-21T07:49:31.858909+0000","last_clean":"2024-09-21T07:49:31.858909+0000","last_became_active":"2024-09-21T07:49:29.839555+0000","last_became_peered":"2024-09-21T07:49:29.839555+0000","last_unstale":"2024-09-21T07:49:31.858909+0000","last_undegraded":"2024-09-21T07:49:31.858909+0000","last_fullsized":"2024-09-21T07:49:31.858909+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:38.177514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.898056+0000","last_change":"2024-09-21T07:49:30.868811+0000","last_active":"2024-09-21T07:49:31.898056+0000","last_peered":"2024-09-21T07:49:31.898056+0000","last_clean":"2024-09-21T07:49:31.898056+0000","last_became_active":"2024-09-21T07:49:30.868487+0000","last_became_peered":"2024-09-21T07:49:30.868487+0000","last_unstale":"2024-09-21T07:49:31.898056+0000","last_undegraded":"2024-09-21T07:49:31.898056+0000","last_fullsized":"2024-09-21T07:49:31.898056+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:14.218794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849802+0000","last_change":"2024-09-21T07:49:29.840533+0000","last_active":"2024-09-21T07:49:31.849802+0000","last_peered":"2024-09-21T07:49:31.849802+0000","last_clean":"2024-09-21T07:49:31.849802+0000","last_became_active":"2024-09-21T07:49:29.840191+0000","last_became_peered":"2024-09-21T07:49:29.840191+0000","last_unstale":"2024-09-21T07:49:31.849802+0000","last_undegraded":"2024-09-21T07:49:31.849802+0000","last_fullsized":"2024-09-21T07:49:31.849802+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:33.764162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.853460+0000","last_change":"2024-09-21T07:49:30.862140+0000","last_active":"2024-09-21T07:49:31.853460+0000","last_peered":"2024-09-21T07:49:31.853460+0000","last_clean":"2024-09-21T07:49:31.853460+0000","last_became_active":"2024-09-21T07:49:30.853963+0000","last_became_peered":"2024-09-21T07:49:30.853963+0000","last_unstale":"2024-09-21T07:49:31.853460+0000","last_undegraded":"2024-09-21T07:49:31.853460+0000","last_fullsized":"2024-09-21T07:49:31.853460+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:14.505442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861465+0000","last_change":"2024-09-21T07:49:29.838408+0000","last_active":"2024-09-21T07:49:31.861465+0000","last_peered":"2024-09-21T07:49:31.861465+0000","last_clean":"2024-09-21T07:49:31.861465+0000","last_became_active":"2024-09-21T07:49:29.838107+0000","last_became_peered":"2024-09-21T07:49:29.838107+0000","last_unstale":"2024-09-21T07:49:31.861465+0000","last_undegraded":"2024-09-21T07:49:31.861465+0000","last_fullsized":"2024-09-21T07:49:31.861465+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:13:39.016144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.859198+0000","last_change":"2024-09-21T07:49:30.851203+0000","last_active":"2024-09-21T07:49:31.859198+0000","last_peered":"2024-09-21T07:49:31.859198+0000","last_clean":"2024-09-21T07:49:31.859198+0000","last_became_active":"2024-09-21T07:49:30.850870+0000","last_became_peered":"2024-09-21T07:49:30.850870+0000","last_unstale":"2024-09-21T07:49:31.859198+0000","last_undegraded":"2024-09-21T07:49:31.859198+0000","last_fullsized":"2024-09-21T07:49:31.859198+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:41.375329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897754+0000","last_change":"2024-09-21T07:49:29.842367+0000","last_active":"2024-09-21T07:49:31.897754+0000","last_peered":"2024-09-21T07:49:31.897754+0000","last_clean":"2024-09-21T07:49:31.897754+0000","last_became_active":"2024-09-21T07:49:29.842035+0000","last_became_peered":"2024-09-21T07:49:29.842035+0000","last_unstale":"2024-09-21T07:49:31.897754+0000","last_undegraded":"2024-09-21T07:49:31.897754+0000","last_fullsized":"2024-09-21T07:49:31.897754+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:46:53.958285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897612+0000","last_change":"2024-09-21T07:49:30.855589+0000","last_active":"2024-09-21T07:49:31.897612+0000","last_peered":"2024-09-21T07:49:31.897612+0000","last_clean":"2024-09-21T07:49:31.897612+0000","last_became_active":"2024-09-21T07:49:30.854096+0000","last_became_peered":"2024-09-21T07:49:30.854096+0000","last_unstale":"2024-09-21T07:49:31.897612+0000","last_undegraded":"2024-09-21T07:49:31.897612+0000","last_fullsized":"2024-09-21T07:49:31.897612+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:11.662622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.861355+0000","last_change":"2024-09-21T07:49:29.841930+0000","last_active":"2024-09-21T07:49:31.861355+0000","last_peered":"2024-09-21T07:49:31.861355+0000","last_clean":"2024-09-21T07:49:31.861355+0000","last_became_active":"2024-09-21T07:49:29.841523+0000","last_became_peered":"2024-09-21T07:49:29.841523+0000","last_unstale":"2024-09-21T07:49:31.861355+0000","last_undegraded":"2024-09-21T07:49:31.861355+0000","last_fullsized":"2024-09-21T07:49:31.861355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:02:24.795233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851666+0000","last_change":"2024-09-21T07:49:30.853544+0000","last_active":"2024-09-21T07:49:31.851666+0000","last_peered":"2024-09-21T07:49:31.851666+0000","last_clean":"2024-09-21T07:49:31.851666+0000","last_became_active":"2024-09-21T07:49:30.849986+0000","last_became_peered":"2024-09-21T07:49:30.849986+0000","last_unstale":"2024-09-21T07:49:31.851666+0000","last_undegraded":"2024-09-21T07:49:31.851666+0000","last_fullsized":"2024-09-21T07:49:31.851666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:04.039987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.858078+0000","last_change":"2024-09-21T07:49:29.834315+0000","last_active":"2024-09-21T07:49:31.858078+0000","last_peered":"2024-09-21T07:49:31.858078+0000","last_clean":"2024-09-21T07:49:31.858078+0000","last_became_active":"2024-09-21T07:49:29.831497+0000","last_became_peered":"2024-09-21T07:49:29.831497+0000","last_unstale":"2024-09-21T07:49:31.858078+0000","last_undegraded":"2024-09-21T07:49:31.858078+0000","last_fullsized":"2024-09-21T07:49:31.858078+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:40:30.000676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897800+0000","last_change":"2024-09-21T07:49:30.866291+0000","last_active":"2024-09-21T07:49:31.897800+0000","last_peered":"2024-09-21T07:49:31.897800+0000","last_clean":"2024-09-21T07:49:31.897800+0000","last_became_active":"2024-09-21T07:49:30.863674+0000","last_became_peered":"2024-09-21T07:49:30.863674+0000","last_unstale":"2024-09-21T07:49:31.897800+0000","last_undegraded":"2024-09-21T07:49:31.897800+0000","last_fullsized":"2024-09-21T07:49:31.897800+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:02.165356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.851800+0000","last_change":"2024-09-21T07:49:29.837438+0000","last_active":"2024-09-21T07:49:31.851800+0000","last_peered":"2024-09-21T07:49:31.851800+0000","last_clean":"2024-09-21T07:49:31.851800+0000","last_became_active":"2024-09-21T07:49:29.836913+0000","last_became_peered":"2024-09-21T07:49:29.836913+0000","last_unstale":"2024-09-21T07:49:31.851800+0000","last_undegraded":"2024-09-21T07:49:31.851800+0000","last_fullsized":"2024-09-21T07:49:31.851800+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:54:38.107152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.852109+0000","last_change":"2024-09-21T07:49:30.857265+0000","last_active":"2024-09-21T07:49:31.852109+0000","last_peered":"2024-09-21T07:49:31.852109+0000","last_clean":"2024-09-21T07:49:31.852109+0000","last_became_active":"2024-09-21T07:49:30.854815+0000","last_became_peered":"2024-09-21T07:49:30.854815+0000","last_unstale":"2024-09-21T07:49:31.852109+0000","last_undegraded":"2024-09-21T07:49:31.852109+0000","last_fullsized":"2024-09-21T07:49:31.852109+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:29.808717+0000","last_clean_scrub_stamp":"2024-09-21T07:49:29.808717+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:58.762682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":15,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.897355+0000","last_change":"2024-09-21T07:49:29.840701+0000","last_active":"2024-09-21T07:49:31.897355+0000","last_peered":"2024-09-21T07:49:31.897355+0000","last_clean":"2024-09-21T07:49:31.897355+0000","last_became_active":"2024-09-21T07:49:29.838599+0000","last_became_peered":"2024-09-21T07:49:29.838599+0000","last_unstale":"2024-09-21T07:49:31.897355+0000","last_undegraded":"2024-09-21T07:49:31.897355+0000","last_fullsized":"2024-09-21T07:49:31.897355+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:49:28.801255+0000","last_clean_scrub_stamp":"2024-09-21T07:49:28.801255+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:11:38.001102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":182,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-21T07:49:31.849394+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:49:31.849394+0000","last_peered":"2024-09-21T07:49:31.849394+0000","last_clean":"2024-09-21T07:49:31.849394+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:49:31.849394+0000","last_undegraded":"2024-09-21T07:49:31.849394+0000","last_fullsized":"2024-09-21T07:49:31.849394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476786,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31892,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339948,"statfs":{"total":96636764160,"available":96604106752,"internally_reserved":0,"allocated":712704,"data_stored":293699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476786,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31908,"kb_used_data":712,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339932,"statfs":{"total":96636764160,"available":96604090368,"internally_reserved":0,"allocated":729088,"data_stored":295381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476783,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1306624,"data_stored":884559,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476783,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27732,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344108,"statfs":{"total":96636764160,"available":96608366592,"internally_reserved":0,"allocated":720896,"data_stored":295221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509487,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31884,"kb_used_data":696,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339956,"statfs":{"total":96636764160,"available":96604114944,"internally_reserved":0,"allocated":712704,"data_stored":293631,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509488,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1276,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1306624,"data_stored":884011,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509488,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31896,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339944,"statfs":{"total":96636764160,"available":96604102656,"internally_reserved":0,"allocated":716800,"data_stored":294225,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542192,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31888,"kb_used_data":700,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339952,"statfs":{"total":96636764160,"available":96604110848,"internally_reserved":0,"allocated":716800,"data_stored":293701,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:49:40.750 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:49:40.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:49:41.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.015+0000 7f713ea00640 1 -- 172.21.15.88:0/3941469664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 msgr2=0x7f713815d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.015+0000 7f713ea00640 1 --2- 172.21.15.88:0/3941469664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f713815d790 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f7138072e60 tx=0x7f712c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 -- 172.21.15.88:0/3941469664 shutdown_connections 2024-09-21T07:49:41.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 --2- 172.21.15.88:0/3941469664 >> v2:172.21.15.175:3300/0 conn(0x7f7138160780 0x7f71380cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 --2- 172.21.15.88:0/3941469664 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160370 0x7f71380ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 --2- 172.21.15.88:0/3941469664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f713815d790 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 -- 172.21.15.88:0/3941469664 >> 172.21.15.88:0/3941469664 conn(0x7f71380b1dd0 msgr2=0x7f71380b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 -- 172.21.15.88:0/3941469664 shutdown_connections 2024-09-21T07:49:41.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.016+0000 7f713ea00640 1 -- 172.21.15.88:0/3941469664 wait complete. 2024-09-21T07:49:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.018+0000 7f713ea00640 1 Processor -- start 2024-09-21T07:49:41.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.018+0000 7f713ea00640 1 -- start start 2024-09-21T07:49:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.019+0000 7f713ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f71381464e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.019+0000 7f713ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7138160370 0x7f7138148e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.019+0000 7f713ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160780 0x7f7138149370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.019+0000 7f713ea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71380b4140 con 0x7f71380af4e0 2024-09-21T07:49:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.019+0000 7f713ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71380b3fc0 con 0x7f7138160370 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.019+0000 7f713ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71380b42c0 con 0x7f7138160780 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f71381464e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f71381464e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45742/0 (socket says 172.21.15.88:45742) 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 -- 172.21.15.88:0/1739880188 learned_addr learned my addr 172.21.15.88:0/1739880188 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713cc00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7138160370 0x7f7138148e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713e000640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160780 0x7f7138149370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 -- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160780 msgr2=0x7f7138149370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160780 0x7f7138149370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 -- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7138160370 msgr2=0x7f7138148e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7138160370 0x7f7138148e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 -- 172.21.15.88:0/1739880188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f713815b8a0 con 0x7f71380af4e0 2024-09-21T07:49:41.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713e000640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160780 0x7f7138149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.020+0000 7f713d600640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f71381464e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f712c034040 tx=0x7f712c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.021+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c03e030 con 0x7f71380af4e0 2024-09-21T07:49:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.021+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f712c003e00 con 0x7f71380af4e0 2024-09-21T07:49:41.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.021+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f712c035040 con 0x7f71380af4e0 2024-09-21T07:49:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.021+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f713815bd40 con 0x7f71380af4e0 2024-09-21T07:49:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.021+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f713815c300 con 0x7f71380af4e0 2024-09-21T07:49:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.022+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f713815d790 con 0x7f71380af4e0 2024-09-21T07:49:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.023+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f712c041030 con 0x7f71380af4e0 2024-09-21T07:49:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.024+0000 7f7135600640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7118078d70 0x7f711807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.024+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f712c0bb900 con 0x7f71380af4e0 2024-09-21T07:49:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.025+0000 7f713cc00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7118078d70 0x7f711807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.025+0000 7f713cc00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7118078d70 0x7f711807b280 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f71380aebf0 tx=0x7f712400b260 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.029+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f712c083350 con 0x7f71380af4e0 2024-09-21T07:49:41.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.283+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f71380ccda0 con 0x7f71380af4e0 2024-09-21T07:49:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.288+0000 7f7135600640 1 -- 172.21.15.88:0/1739880188 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v11) ==== 190+0+147 (secure 0 0 0) 0x7f712c0bb020 con 0x7f71380af4e0 2024-09-21T07:49:41.290 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:49:41.290 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:49:41.290 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:49:41.291 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:49:41.291 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:49:41.291 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.290+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7118078d70 msgr2=0x7f711807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.290+0000 7f713ea00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7118078d70 0x7f711807b280 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f71380aebf0 tx=0x7f712400b260 comp rx=0 tx=0).stop 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.290+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 msgr2=0x7f71381464e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.290+0000 7f713ea00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f71381464e0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f712c034040 tx=0x7f712c003910 comp rx=0 tx=0).stop 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 shutdown_connections 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7118078d70 0x7f711807b280 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7138160780 0x7f7138149370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7138160370 0x7f7138148e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 --2- 172.21.15.88:0/1739880188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71380af4e0 0x7f71381464e0 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 >> 172.21.15.88:0/1739880188 conn(0x7f71380b1dd0 msgr2=0x7f71380b0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 shutdown_connections 2024-09-21T07:49:41.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.291+0000 7f713ea00640 1 -- 172.21.15.88:0/1739880188 wait complete. 2024-09-21T07:49:41.319 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 -- 172.21.15.88:0/1667561277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 msgr2=0x7f317015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 --2- 172.21.15.88:0/1667561277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015f3d0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f3170072810 tx=0x7f315c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 -- 172.21.15.88:0/1667561277 shutdown_connections 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 --2- 172.21.15.88:0/1667561277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f317015f910 0x7f317015bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 --2- 172.21.15.88:0/1667561277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015f3d0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 --2- 172.21.15.88:0/1667561277 >> v2:172.21.15.175:3300/0 conn(0x7f31700aebc0 0x7f3170152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.535+0000 7f3178400640 1 -- 172.21.15.88:0/1667561277 >> 172.21.15.88:0/1667561277 conn(0x7f317006d730 msgr2=0x7f31700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.536+0000 7f3178400640 1 -- 172.21.15.88:0/1667561277 shutdown_connections 2024-09-21T07:49:41.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.536+0000 7f3178400640 1 -- 172.21.15.88:0/1667561277 wait complete. 2024-09-21T07:49:41.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.538+0000 7f3178400640 1 Processor -- start 2024-09-21T07:49:41.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.538+0000 7f3178400640 1 -- start start 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.539+0000 7f3178400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31700aebc0 0x7f3170074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.539+0000 7f3178400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.539+0000 7f3178400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f317015f910 0x7f317015dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.539+0000 7f3178400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f317015c660 con 0x7f31700b2750 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.539+0000 7f3178400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f317015c4e0 con 0x7f317015f910 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.539+0000 7f3178400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f317015c7e0 con 0x7f31700aebc0 2024-09-21T07:49:41.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45764/0 (socket says 172.21.15.88:45764) 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 -- 172.21.15.88:0/3550791680 learned_addr learned my addr 172.21.15.88:0/3550791680 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3176400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f317015f910 0x7f317015dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175a00640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31700aebc0 0x7f3170074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 -- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31700aebc0 msgr2=0x7f3170074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31700aebc0 0x7f3170074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 -- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f317015f910 msgr2=0x7f317015dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f317015f910 0x7f317015dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 -- 172.21.15.88:0/3550791680 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3170074d50 con 0x7f31700b2750 2024-09-21T07:49:41.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175a00640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31700aebc0 0x7f3170074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:41.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.540+0000 7f3175000640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015d7d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f315c034040 tx=0x7f315c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.541+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c03e030 con 0x7f31700b2750 2024-09-21T07:49:41.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.541+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f315c003ca0 con 0x7f31700b2750 2024-09-21T07:49:41.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.541+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f315c035040 con 0x7f31700b2750 2024-09-21T07:49:41.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.541+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31700cdb00 con 0x7f31700b2750 2024-09-21T07:49:41.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.541+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31700ce0c0 con 0x7f31700b2750 2024-09-21T07:49:41.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.542+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f317015f3d0 con 0x7f31700b2750 2024-09-21T07:49:41.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.543+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f315c041030 con 0x7f31700b2750 2024-09-21T07:49:41.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.544+0000 7f316d600640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3150078d40 0x7f315007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:41.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.544+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f315c0bb730 con 0x7f31700b2750 2024-09-21T07:49:41.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.545+0000 7f3175a00640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3150078d40 0x7f315007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:41.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.546+0000 7f3175a00640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3150078d40 0x7f315007b250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3164009b00 tx=0x7f3164005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:41.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.549+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f315c083180 con 0x7f31700b2750 2024-09-21T07:49:41.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.771+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f317015a7a0 con 0x7f31700b2750 2024-09-21T07:49:41.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.771+0000 7f316d600640 1 -- 172.21.15.88:0/3550791680 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+5226 (secure 0 0 0) 0x7f315c087bb0 con 0x7f31700b2750 2024-09-21T07:49:41.774 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:41.774 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":36,"btime":"2024-09-21T07:49:33:853295+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:30.312143+0000","modified":"2024-09-21T07:49:33.340468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2365988221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2365988221},{"type":"v1","addr":"172.21.15.175:6837","nonce":2365988221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5763,"qdb_cluster":[5763]},"id":4}]} 2024-09-21T07:49:41.774 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 36 2024-09-21T07:49:41.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.773+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3150078d40 msgr2=0x7f315007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.773+0000 7f3178400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3150078d40 0x7f315007b250 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f3164009b00 tx=0x7f3164005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:41.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.773+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 msgr2=0x7f317015d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.773+0000 7f3178400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015d7d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f315c034040 tx=0x7f315c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 shutdown_connections 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3150078d40 0x7f315007b250 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f317015f910 0x7f317015dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31700b2750 0x7f317015d7d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 --2- 172.21.15.88:0/3550791680 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31700aebc0 0x7f3170074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 >> 172.21.15.88:0/3550791680 conn(0x7f317006d730 msgr2=0x7f317015bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 shutdown_connections 2024-09-21T07:49:41.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:41.774+0000 7f3178400640 1 -- 172.21.15.88:0/3550791680 wait complete. 2024-09-21T07:49:42.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 36, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:49:30.312143+0000', 'modified': '2024-09-21T07:49:33.340468+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5763}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5763': {'gid': 5763, 'name': 'd', 'rank': 0, 'incarnation': 32, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6837/2365988221', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6835', 'nonce': 2365988221}, {'type': 'v1', 'addr': '172.21.15.175:6837', 'nonce': 2365988221}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 5763, 'qdb_cluster': [5763]} 2024-09-21T07:49:42.205 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:49:42.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2024-09-21T07:49:42.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8009800640 1 -- 172.21.15.88:0/3122769417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fe8003a00 msgr2=0x7f7fe8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8009800640 1 --2- 172.21.15.88:0/3122769417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fe8003a00 0x7f7fe8003e70 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7ff400bbb0 tx=0x7f7fe8005830 comp rx=0 tx=0).stop 2024-09-21T07:49:42.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8003400640 1 -- 172.21.15.88:0/3122769417 reap_dead start 2024-09-21T07:49:42.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8009800640 1 -- 172.21.15.88:0/3122769417 shutdown_connections 2024-09-21T07:49:42.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8009800640 1 -- 172.21.15.88:0/3122769417 >> 172.21.15.88:0/3122769417 conn(0x7f800406d3d0 msgr2=0x7f80040b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8009800640 1 -- 172.21.15.88:0/3122769417 shutdown_connections 2024-09-21T07:49:42.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.427+0000 7f8009800640 1 -- 172.21.15.88:0/3122769417 wait complete. 2024-09-21T07:49:42.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.429+0000 7f8009800640 1 Processor -- start 2024-09-21T07:49:42.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.429+0000 7f8009800640 1 -- start start 2024-09-21T07:49:42.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8009800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fe8003a00 0x7f80041f9380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8009800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 0x7f80040ae620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8009800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80040afe90 0x7f80040b02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8009800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8004070b40 con 0x7f80040ae210 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8009800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80040709c0 con 0x7f80040afe90 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8009800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8004070cc0 con 0x7f7fe8003a00 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 0x7f80040ae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 0x7f80040ae620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45784/0 (socket says 172.21.15.88:45784) 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 -- 172.21.15.88:0/4164935319 learned_addr learned my addr 172.21.15.88:0/4164935319 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8003400640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80040afe90 0x7f80040b02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002a00640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fe8003a00 0x7f80041f9380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 -- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fe8003a00 msgr2=0x7f80041f9380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fe8003a00 0x7f80041f9380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 -- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80040afe90 msgr2=0x7f80040b02a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.431+0000 7f8002000640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80040afe90 0x7f80040b02a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f8002000640 1 -- 172.21.15.88:0/4164935319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80040aeb60 con 0x7f80040ae210 2024-09-21T07:49:42.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f8003400640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80040afe90 0x7f80040b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:42.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f8002000640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 0x7f80040ae620 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000fa00 tx=0x7f7ff000fec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f7ffaa00640 1 -- 172.21.15.88:0/4164935319 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff000bcb0 con 0x7f80040ae210 2024-09-21T07:49:42.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f7ffaa00640 1 -- 172.21.15.88:0/4164935319 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff0011440 con 0x7f80040ae210 2024-09-21T07:49:42.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f7ffaa00640 1 -- 172.21.15.88:0/4164935319 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0011870 con 0x7f80040ae210 2024-09-21T07:49:42.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80041f5760 con 0x7f80040ae210 2024-09-21T07:49:42.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.432+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80041f5d20 con 0x7f80040ae210 2024-09-21T07:49:42.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.435+0000 7f7ffaa00640 1 -- 172.21.15.88:0/4164935319 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f7ff0019020 con 0x7f80040ae210 2024-09-21T07:49:42.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.435+0000 7f7ffaa00640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7fe4078d40 0x7f7fe407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.436+0000 7f7ffaa00640 1 -- 172.21.15.88:0/4164935319 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff009bc80 con 0x7f80040ae210 2024-09-21T07:49:42.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.436+0000 7f801085a740 1 Processor -- start 2024-09-21T07:49:42.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.436+0000 7f801085a740 1 -- start start 2024-09-21T07:49:42.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.436+0000 7f8002a00640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7fe4078d40 0x7f7fe407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.437+0000 7f8002a00640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7fe4078d40 0x7f7fe407b250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4035000 tx=0x7f7ff40046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c637970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f801085a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 0x56210c637eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c246590 0x56210c607f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f801085a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56210c591430 con 0x56210c23e3b0 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56210c5815d0 con 0x56210c267270 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56210c570a00 con 0x56210c246590 2024-09-21T07:49:42.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 0x56210c637eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 0x56210c637eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45800/0 (socket says 172.21.15.88:45800) 2024-09-21T07:49:42.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 -- 172.21.15.88:0/2682805169 learned_addr learned my addr 172.21.15.88:0/2682805169 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:42.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002a00640 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c637970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8003400640 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c246590 0x56210c607f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 -- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c246590 msgr2=0x56210c607f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c246590 0x56210c607f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 -- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 msgr2=0x56210c637970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c637970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8002000640 1 -- 172.21.15.88:0/2682805169 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210c539520 con 0x56210c23e3b0 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.438+0000 7f8003400640 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c246590 0x56210c607f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.439+0000 7f8002000640 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 0x56210c637eb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0063310 tx=0x7f7ff0063020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ee0711a66c4437 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.439+0000 7f7fef400640 1 -- 172.21.15.88:0/2682805169 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff009a2f0 con 0x56210c23e3b0 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.439+0000 7f7fef400640 1 -- 172.21.15.88:0/2682805169 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff0063610 con 0x56210c23e3b0 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.439+0000 7f7fef400640 1 -- 172.21.15.88:0/2682805169 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff006c280 con 0x56210c23e3b0 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 -- 172.21.15.88:0/2682805169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 msgr2=0x56210c637eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 0x56210c637eb0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0063310 tx=0x7f7ff0063020 comp rx=0 tx=0).stop 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 -- 172.21.15.88:0/2682805169 shutdown_connections 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c246590 0x56210c607f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c23e3b0 0x56210c637eb0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 --2- 172.21.15.88:0/2682805169 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c637970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 -- 172.21.15.88:0/2682805169 >> 172.21.15.88:0/2682805169 conn(0x56210c25b3c0 msgr2=0x56210c60d970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 -- 172.21.15.88:0/2682805169 shutdown_connections 2024-09-21T07:49:42.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.440+0000 7f801085a740 1 -- 172.21.15.88:0/2682805169 wait complete. 2024-09-21T07:49:42.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.441+0000 7f801085a740 1 -- start start 2024-09-21T07:49:42.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.441+0000 7f801085a740 1 Processor -- start 2024-09-21T07:49:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c64c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c246590 0x56210c631c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 0x56210c6627b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56210c517330 con 0x56210c267270 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56210c574810 con 0x56210c246590 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56210c2e1620 con 0x56210c23e3b0 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f801085a740 10 client.0 _ll_register_callbacks cb 0x56210c51c420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 0x56210c6627b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 0x56210c6627b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45804/0 (socket says 172.21.15.88:45804) 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 -- 172.21.15.88:0/89081290 learned_addr learned my addr 172.21.15.88:0/89081290 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8002a00640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c64c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8002000640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c246590 0x56210c631c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 -- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 msgr2=0x56210c64c1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c64c1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 -- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c246590 msgr2=0x56210c631c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c246590 0x56210c631c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.443+0000 7f8003400640 1 -- 172.21.15.88:0/89081290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210c2d2850 con 0x56210c267270 2024-09-21T07:49:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f8003400640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 0x56210c6627b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f80040ce990 tx=0x7f7ffc003e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=980694aecffeef12 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f7fe3e00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:49:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f7fe3e00640 1 -- 172.21.15.88:0/89081290 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ffc012440 con 0x56210c267270 2024-09-21T07:49:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f7fe3e00640 1 -- 172.21.15.88:0/89081290 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f7ffc018930 con 0x56210c267270 2024-09-21T07:49:42.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff0063610 con 0x56210c267270 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f7fe3e00640 1 -- 172.21.15.88:0/89081290 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ffc018b30 con 0x56210c267270 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f801085a740 10 client.5910 fetch_fsmap learned FSMap version 36 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x56210c3496b0 con 0x56210c267270 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.444+0000 7f7fe3e00640 1 -- 172.21.15.88:0/89081290 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff0063610 con 0x56210c267270 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f7fe3e00640 1 -- 172.21.15.88:0/89081290 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f7ffc025070 con 0x56210c267270 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f801085a740 10 client.5910 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f801085a740 10 client.5910 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f801085a740 20 client.5910 populate_metadata read hostname 'smithi088' 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f801085a740 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c225e40 0x56210c6655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f801085a740 4 client.5910 mds_command: new command op to 5763 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 --> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56210c583520 con 0x56210c225e40 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.445+0000 7f8002a00640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c225e40 0x56210c6655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.446+0000 7f8002a00640 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c225e40 0x56210c6655e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe2fc2ecea917660 server_cookie=10aba5bb276cebce in_seq=0 out_seq=0 2024-09-21T07:49:42.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.446+0000 7f7fe3e00640 10 client.5910 ms_handle_connect on v2:172.21.15.175:6835/2365988221 2024-09-21T07:49:42.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f7fe3e00640 1 -- 172.21.15.88:0/89081290 <== mds.0 v2:172.21.15.175:6835/2365988221 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56210c583520 con 0x56210c225e40 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f7fe3e00640 10 client.5910 handle_command_reply: tid=0 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 client.5910 shutdown 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 msgr2=0x56210c6627b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 0x56210c6627b0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f80040ce990 tx=0x7f7ffc003e20 comp rx=0 tx=0).stop 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 shutdown_connections 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c225e40 0x56210c6655e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c267270 0x56210c6627b0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c246590 0x56210c631c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 --2- 172.21.15.88:0/89081290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c64c1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 >> 172.21.15.88:0/89081290 conn(0x56210c25b3c0 msgr2=0x56210c6553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.447+0000 7f7fe3e00640 0 client.5910 ms_handle_reset on v2:172.21.15.175:6835/2365988221 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.448+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 shutdown_connections 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.448+0000 7f801085a740 1 -- 172.21.15.88:0/89081290 wait complete. 2024-09-21T07:49:42.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.448+0000 7f801085a740 20 client.5910 trim_cache size 0 max 16384 2024-09-21T07:49:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.467+0000 7f801085a740 1 Processor -- start 2024-09-21T07:49:42.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.467+0000 7f801085a740 1 -- start start 2024-09-21T07:49:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.468+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c6625d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.468+0000 7f801085a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.468+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.468+0000 7f801085a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8004201830 con 0x56210c246590 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.468+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8004158b50 con 0x56210c267270 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.468+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f80042011e0 con 0x56210c23e3b0 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45806/0 (socket says 172.21.15.88:45806) 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 -- 172.21.15.88:0/2691923765 learned_addr learned my addr 172.21.15.88:0/2691923765 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002a00640 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c6625d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8003400640 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 -- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 msgr2=0x56210c6625d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c6625d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 -- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 msgr2=0x56210c631b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 -- 172.21.15.88:0/2691923765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56210c60bd60 con 0x56210c246590 2024-09-21T07:49:42.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f8002000640 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0063310 tx=0x7f7ff0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=673d060811bb8874 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.469+0000 7f7feea00640 1 -- 172.21.15.88:0/2691923765 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff009a2f0 con 0x56210c246590 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f7feea00640 1 -- 172.21.15.88:0/2691923765 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff00680f0 con 0x56210c246590 2024-09-21T07:49:42.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f7feea00640 1 -- 172.21.15.88:0/2691923765 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff007ddc0 con 0x56210c246590 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 -- 172.21.15.88:0/2691923765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 msgr2=0x56210c6315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f7ff0063310 tx=0x7f7ff0000bc0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 -- 172.21.15.88:0/2691923765 shutdown_connections 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 --2- 172.21.15.88:0/2691923765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c23e3b0 0x56210c6625d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 -- 172.21.15.88:0/2691923765 >> 172.21.15.88:0/2691923765 conn(0x56210c25b3c0 msgr2=0x56210c60d940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 -- 172.21.15.88:0/2691923765 shutdown_connections 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.470+0000 7f801085a740 1 -- 172.21.15.88:0/2691923765 wait complete. 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.471+0000 7f801085a740 1 -- start start 2024-09-21T07:49:42.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.471+0000 7f801085a740 1 Processor -- start 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c23e3b0 0x56210c6cc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56210c64b090 con 0x56210c246590 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56210c64aad0 con 0x56210c23e3b0 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56210c630130 con 0x56210c267270 2024-09-21T07:49:42.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f801085a740 10 client.0 _ll_register_callbacks cb 0x7f7ffc006ca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f8002000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45812/0 (socket says 172.21.15.88:45812) 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f8002000640 1 -- 172.21.15.88:0/3672257264 learned_addr learned my addr 172.21.15.88:0/3672257264 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f8002a00640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c23e3b0 0x56210c6cc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.473+0000 7f8003400640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f8002000640 1 -- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c267270 msgr2=0x56210c631b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f8002000640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f8002000640 1 -- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c23e3b0 msgr2=0x56210c6cc5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f8002000640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c23e3b0 0x56210c6cc5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f8002000640 1 -- 172.21.15.88:0/3672257264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56210c66be80 con 0x56210c246590 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f8002000640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000f9d0 tx=0x7f7ff009a320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483c9833e367ebba server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f7fe3e00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:49:42.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f7fe3e00640 1 -- 172.21.15.88:0/3672257264 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ff0082060 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f7fe3e00640 1 -- 172.21.15.88:0/3672257264 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f7ff007db60 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ff00680f0 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f7fe3e00640 1 -- 172.21.15.88:0/3672257264 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7ff0099e10 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f801085a740 10 client.5916 fetch_fsmap learned FSMap version 36 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.474+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=2+}) -- 0x56210c6365a0 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f7fe3e00640 1 -- 172.21.15.88:0/3672257264 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ff00680f0 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f7fe3e00640 1 -- 172.21.15.88:0/3672257264 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 36) ==== 3502+0+0 (secure 0 0 0) 0x7f7ff0023030 con 0x56210c246590 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f801085a740 10 client.5916 fetch_fsmap finished waiting for FSMap version 36 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f801085a740 10 client.5916 resolve_mds: resolved name 'd' to daemon mds.d 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f801085a740 20 client.5916 populate_metadata read hostname 'smithi088' 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f801085a740 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c635910 0x56210c635d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f801085a740 4 client.5916 mds_command: new command op to 5763 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 --> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] -- command(tid 0: {"prefix": "status"}) -- 0x56210c57efd0 con 0x56210c635910 2024-09-21T07:49:42.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.475+0000 7f8002a00640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c635910 0x56210c635d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.476+0000 7f7fe3e00640 10 client.5916 ms_handle_connect on v2:172.21.15.175:6835/2365988221 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.476+0000 7f8002a00640 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c635910 0x56210c635d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c248a95757c46e3 server_cookie=8a7a9e21c65f33c5 in_seq=0 out_seq=0 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.476+0000 7f7fe3e00640 1 -- 172.21.15.88:0/3672257264 <== mds.0 v2:172.21.15.175:6835/2365988221 1 ==== command_reply(tid 0: 0 ) ==== 8+0+325 (crc 0 0 0) 0x56210c57efd0 con 0x56210c635910 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.476+0000 7f7fe3e00640 10 client.5916 handle_command_reply: tid=0 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.476+0000 7f801085a740 1 client.5916 shutdown 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 msgr2=0x56210c6315d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000f9d0 tx=0x7f7ff009a320 comp rx=0 tx=0).stop 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 shutdown_connections 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56210c267270 0x56210c631b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:6835/2365988221,v1:172.21.15.175:6837/2365988221] conn(0x56210c635910 0x56210c635d20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56210c246590 0x56210c6315d0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 --2- 172.21.15.88:0/3672257264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56210c23e3b0 0x56210c6cc5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 >> 172.21.15.88:0/3672257264 conn(0x56210c25b3c0 msgr2=0x56210c6326f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f7fe3e00640 0 client.5916 ms_handle_reset on v2:172.21.15.175:6835/2365988221 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 shutdown_connections 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 1 -- 172.21.15.88:0/3672257264 wait complete. 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f801085a740 20 client.5916 trim_cache size 0 max 16384 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 5763, 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:49:42.482 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:49:42.483 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 11.639091146, 2024-09-21T07:49:42.483 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 36, 2024-09-21T07:49:42.483 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 60, 2024-09-21T07:49:42.483 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 60, 2024-09-21T07:49:42.483 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 16.661824231000001 2024-09-21T07:49:42.483 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7fe4078d40 msgr2=0x7f7fe407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f8009800640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7fe4078d40 0x7f7fe407b250 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f7ff4035000 tx=0x7f7ff40046c0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 msgr2=0x7f80040ae620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.477+0000 7f8009800640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 0x7f80040ae620 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7ff000fa00 tx=0x7f7ff000fec0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 shutdown_connections 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7fe4078d40 0x7f7fe407b250 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80040afe90 0x7f80040b02a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80040ae210 0x7f80040ae620 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 --2- 172.21.15.88:0/4164935319 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fe8003a00 0x7f80041f9380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 >> 172.21.15.88:0/4164935319 conn(0x7f800406d3d0 msgr2=0x7f800415c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 shutdown_connections 2024-09-21T07:49:42.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.478+0000 7f8009800640 1 -- 172.21.15.88:0/4164935319 wait complete. 2024-09-21T07:49:42.503 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 -- 172.21.15.88:0/1292977189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6958004800 msgr2=0x7f6958004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 --2- 172.21.15.88:0/1292977189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6958004800 0x7f6958004cb0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f695c006fb0 tx=0x7f6958005830 comp rx=0 tx=0).stop 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 -- 172.21.15.88:0/1292977189 shutdown_connections 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 --2- 172.21.15.88:0/1292977189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6958004800 0x7f6958004cb0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 -- 172.21.15.88:0/1292977189 >> 172.21.15.88:0/1292977189 conn(0x7f69740b1e40 msgr2=0x7f69740b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 -- 172.21.15.88:0/1292977189 shutdown_connections 2024-09-21T07:49:42.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.721+0000 7f6979800640 1 -- 172.21.15.88:0/1292977189 wait complete. 2024-09-21T07:49:42.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.724+0000 7f6979800640 1 Processor -- start 2024-09-21T07:49:42.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.724+0000 7f6979800640 1 -- start start 2024-09-21T07:49:42.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6979800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6958004800 0x7f6974149ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6979800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697414fc10 0x7f6974154430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6979800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 0x7f697409aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6979800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f697415a0f0 con 0x7f6974154970 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6979800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69740b3440 con 0x7f6958004800 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6979800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f697415a270 con 0x7f697414fc10 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.725+0000 7f6978e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 0x7f697409aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 0x7f697409aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45826/0 (socket says 172.21.15.88:45826) 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 -- 172.21.15.88:0/1422287412 learned_addr learned my addr 172.21.15.88:0/1422287412 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6973e00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6958004800 0x7f6974149ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f696b400640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697414fc10 0x7f6974154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 -- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697414fc10 msgr2=0x7f6974154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697414fc10 0x7f6974154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 -- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6958004800 msgr2=0x7f6974149ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6958004800 0x7f6974149ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 -- 172.21.15.88:0/1422287412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6974154d80 con 0x7f6974154970 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6978e00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 0x7f697409aa50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f697415cc30 tx=0x7f696c004700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.726+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c004b20 con 0x7f6974154970 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.727+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f696c004cc0 con 0x7f6974154970 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.727+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69741f6bf0 con 0x7f6974154970 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.727+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f696c010d10 con 0x7f6974154970 2024-09-21T07:49:42.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.727+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f697409ccf0 con 0x7f6974154970 2024-09-21T07:49:42.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.728+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f697415d6c0 con 0x7f6974154970 2024-09-21T07:49:42.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.729+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f696c01c020 con 0x7f6974154970 2024-09-21T07:49:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.730+0000 7f6970c00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6950078d40 0x7f695007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:42.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.731+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f696c09b8c0 con 0x7f6974154970 2024-09-21T07:49:42.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.731+0000 7f6973e00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6950078d40 0x7f695007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:42.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.731+0000 7f6973e00640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6950078d40 0x7f695007b250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6974052860 tx=0x7f6964006b10 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:42.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.735+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6974154d80 con 0x7f6974154970 2024-09-21T07:49:42.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.958+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6974150380 con 0x7f6974154970 2024-09-21T07:49:42.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.959+0000 7f6970c00640 1 -- 172.21.15.88:0/1422287412 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f696c0638e0 con 0x7f6974154970 2024-09-21T07:49:42.961 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:49:42.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6950078d40 msgr2=0x7f695007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6950078d40 0x7f695007b250 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f6974052860 tx=0x7f6964006b10 comp rx=0 tx=0).stop 2024-09-21T07:49:42.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 msgr2=0x7f697409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:42.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 0x7f697409aa50 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f697415cc30 tx=0x7f696c004700 comp rx=0 tx=0).stop 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 shutdown_connections 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6950078d40 0x7f695007b250 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f697414fc10 0x7f6974154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6974154970 0x7f697409aa50 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 --2- 172.21.15.88:0/1422287412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6958004800 0x7f6974149ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 >> 172.21.15.88:0/1422287412 conn(0x7f69740b1e40 msgr2=0x7f69740afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 shutdown_connections 2024-09-21T07:49:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:42.961+0000 7f6979800640 1 -- 172.21.15.88:0/1422287412 wait complete. 2024-09-21T07:49:42.987 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:49:42.987 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:49:42.987 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:49:42.987 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:49:42.987 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:42.987 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:49:42.988 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:49:42.988 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:49:42.988 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:42.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:42.988 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:49:43.023 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:49:43.023 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:49:43.024 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42511sec preferred_lft 42511sec 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:49:43.025 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:43.026 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.026 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:49:43.117 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:49:43.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.120 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:49:43.155 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:49:43.155 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:49:43.156 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:49:43.156 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:49:43.156 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:49:43.156 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:49:43.156 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:49:43.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:49:43.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:49:43.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:49:43.156 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:49:43.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:49:43.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:43.351 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.351 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:49:43.386 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:49:43.386 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:49:43.387 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:43.387 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.387 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:49:43.482 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:43.482 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:43.517 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:49:43.517 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:43.517 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:43.591 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:49:43.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.591 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:49:43.785 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:49:43.786 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:49:43.786 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:49:43.806 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:49:43.806 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:49:43.806 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:49:43.806 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:49:43.806 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:49:43.837 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:49:43.837 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:49:43.838 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:49:43.838 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:49:43.838 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:49:43.838 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:49:43.838 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:49:43.838 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:49:43.839 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:49:43.839 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:49:43.839 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:49:43.839 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:49:43.839 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:49:43.842 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:49:43.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:43.842 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:49:43.898 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:49:43.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:43 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:49:43.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:43.997 INFO:teuthology.orchestra.run.smithi088.stdout:client5925 2024-09-21T07:49:43.997 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:49:43.997 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:44.033 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:49:44.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 -- 172.21.15.88:0/125106478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5398003a50 msgr2=0x7f5398003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:44.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 --2- 172.21.15.88:0/125106478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5398003a50 0x7f5398003ec0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009b50 tx=0x7f5398005830 comp rx=0 tx=0).stop 2024-09-21T07:49:44.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 -- 172.21.15.88:0/125106478 shutdown_connections 2024-09-21T07:49:44.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 --2- 172.21.15.88:0/125106478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5398003a50 0x7f5398003ec0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 -- 172.21.15.88:0/125106478 >> 172.21.15.88:0/125106478 conn(0x7f53b40b1e40 msgr2=0x7f53b40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:44.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 -- 172.21.15.88:0/125106478 shutdown_connections 2024-09-21T07:49:44.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.293+0000 7f53ba000640 1 -- 172.21.15.88:0/125106478 wait complete. 2024-09-21T07:49:44.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.295+0000 7f53ba000640 1 Processor -- start 2024-09-21T07:49:44.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.296+0000 7f53ba000640 1 -- start start 2024-09-21T07:49:44.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53ba000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5398003a50 0x7f53b414b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53ba000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53b409dac0 0x7f53b4150650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53ba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 0x7f53b414c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53ba000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4073600 con 0x7f53b413f9e0 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53ba000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4073480 con 0x7f5398003a50 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53ba000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53b4070000 con 0x7f53b409dac0 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53b9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 0x7f53b414c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53b9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 0x7f53b414c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45846/0 (socket says 172.21.15.88:45846) 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53b8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5398003a50 0x7f53b414b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.297+0000 7f53b9600640 1 -- 172.21.15.88:0/3902896789 learned_addr learned my addr 172.21.15.88:0/3902896789 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:44.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53abe00640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53b409dac0 0x7f53b4150650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:44.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53b9600640 1 -- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53b409dac0 msgr2=0x7f53b4150650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:44.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53b9600640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53b409dac0 0x7f53b4150650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53b9600640 1 -- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5398003a50 msgr2=0x7f53b414b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:44.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53b9600640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5398003a50 0x7f53b414b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53b9600640 1 -- 172.21.15.88:0/3902896789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b4150e80 con 0x7f53b413f9e0 2024-09-21T07:49:44.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.298+0000 7f53b9600640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 0x7f53b414c7b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0012e0 tx=0x7f53ac00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:44.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.300+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac00ca20 con 0x7f53b413f9e0 2024-09-21T07:49:44.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.300+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53ac018070 con 0x7f53b413f9e0 2024-09-21T07:49:44.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.300+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac011460 con 0x7f53b413f9e0 2024-09-21T07:49:44.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.300+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b414ade0 con 0x7f53b413f9e0 2024-09-21T07:49:44.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.300+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b414b3a0 con 0x7f53b413f9e0 2024-09-21T07:49:44.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.301+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53b415bb90 con 0x7f53b413f9e0 2024-09-21T07:49:44.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.302+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f53ac011600 con 0x7f53b413f9e0 2024-09-21T07:49:44.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.303+0000 7f53b1600640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5390078d40 0x7f539007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:44.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.303+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f53ac09b420 con 0x7f53b413f9e0 2024-09-21T07:49:44.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.303+0000 7f53b8c00640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5390078d40 0x7f539007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:44.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.304+0000 7f53b8c00640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5390078d40 0x7f539007b250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53a4035000 tx=0x7f53a40046c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:44.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.308+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53ac062e70 con 0x7f53b413f9e0 2024-09-21T07:49:44.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.529+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f53b4150c10 con 0x7f53b413f9e0 2024-09-21T07:49:44.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.530+0000 7f53b1600640 1 -- 172.21.15.88:0/3902896789 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v36) ==== 53+0+73 (secure 0 0 0) 0x7f53ac0678a0 con 0x7f53b413f9e0 2024-09-21T07:49:44.532 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5390078d40 msgr2=0x7f539007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5390078d40 0x7f539007b250 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f53a4035000 tx=0x7f53a40046c0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 msgr2=0x7f53b414c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 0x7f53b414c7b0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f53ac0012e0 tx=0x7f53ac00ed10 comp rx=0 tx=0).stop 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 shutdown_connections 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5390078d40 0x7f539007b250 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53b409dac0 0x7f53b4150650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b413f9e0 0x7f53b414c7b0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.532+0000 7f53ba000640 1 --2- 172.21.15.88:0/3902896789 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5398003a50 0x7f53b414b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:44.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.533+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 >> 172.21.15.88:0/3902896789 conn(0x7f53b40b1e40 msgr2=0x7f53b4152200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:44.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.533+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 shutdown_connections 2024-09-21T07:49:44.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:44.533+0000 7f53ba000640 1 -- 172.21.15.88:0/3902896789 wait complete. 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:49:44.556 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:49:44.557 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:49:44.557 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:49:44.557 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:44.557 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:49:44.596 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:49:44.597 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42460sec preferred_lft 42460sec 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:49:44.598 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:49:44.599 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:49:44.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:44.600 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:49:44.699 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:49:44.702 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:44.702 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:49:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:49:44.740 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:49:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:49:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:49:44.741 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:49:44.741 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:49:44.741 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:49:44.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:49:44.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:49:44.741 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:49:44.741 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:49:44.859 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:49:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:49:44.930 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:49:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:44.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:44.938 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:49:44.974 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:49:44.974 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:49:44.974 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:49:44.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:44.975 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:49:45.070 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:45.071 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:49:45.105 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:49:45.105 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:49:45.105 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:49:45.180 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:49:45.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:45.181 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:49:45.376 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:49:45.376 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:49:45.376 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:49:45.396 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:49:45.396 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:49:45.396 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:49:45.396 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:49:45.396 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:49:45.426 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:49:45.428 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:49:45.428 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:49:45.428 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:49:45.428 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:49:45.428 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:49:45.431 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:49:45.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:45.431 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:49:45.486 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:49:45.563 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:49:45 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:49:45.566 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:49:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:49:45.574 INFO:teuthology.orchestra.run.smithi175.stdout:client5399 2024-09-21T07:49:45.574 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:49:45.575 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:49:45.610 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:49:45.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 -- 172.21.15.88:0/14606920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c8162190 msgr2=0x7f41c80d23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 --2- 172.21.15.88:0/14606920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c8162190 0x7f41c80d23d0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f41c80cb580 tx=0x7f41c002aa30 comp rx=0 tx=0).stop 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 -- 172.21.15.88:0/14606920 shutdown_connections 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 --2- 172.21.15.88:0/14606920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c8162190 0x7f41c80d23d0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 --2- 172.21.15.88:0/14606920 >> v2:172.21.15.175:3300/0 conn(0x7f41c80b59f0 0x7f41c80b3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 -- 172.21.15.88:0/14606920 >> 172.21.15.88:0/14606920 conn(0x7f41c809d7e0 msgr2=0x7f41c808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 -- 172.21.15.88:0/14606920 shutdown_connections 2024-09-21T07:49:45.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.844+0000 7f41cd000640 1 -- 172.21.15.88:0/14606920 wait complete. 2024-09-21T07:49:45.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.846+0000 7f41cd000640 1 Processor -- start 2024-09-21T07:49:45.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.846+0000 7f41cd000640 1 -- start start 2024-09-21T07:49:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41cd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c80b59f0 0x7f41c8154980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41cd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8161d80 0x7f41c81516a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41cd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 0x7f41c80d3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41cd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80d2a90 con 0x7f41c80cb060 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41cd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f41c80d2910 con 0x7f41c8161d80 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41cd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f41c80d2c10 con 0x7f41c80b59f0 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 0x7f41c80d3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 0x7f41c80d3480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56058/0 (socket says 172.21.15.88:56058) 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41c7e00640 1 -- 172.21.15.88:0/3634463979 learned_addr learned my addr 172.21.15.88:0/3634463979 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41c6a00640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8161d80 0x7f41c81516a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.848+0000 7f41c7400640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c80b59f0 0x7f41c8154980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41c7e00640 1 -- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c80b59f0 msgr2=0x7f41c8154980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41c7e00640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c80b59f0 0x7f41c8154980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41c7e00640 1 -- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8161d80 msgr2=0x7f41c81516a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:45.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41c7e00640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8161d80 0x7f41c81516a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41c7e00640 1 -- 172.21.15.88:0/3634463979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41c8154ec0 con 0x7f41c80cb060 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41c7e00640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 0x7f41c80d3480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f41c002b020 tx=0x7f41c0003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0033050 con 0x7f41c80cb060 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f41c0004450 con 0x7f41c80cb060 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41c8151be0 con 0x7f41c80cb060 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.849+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41c0038450 con 0x7f41c80cb060 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.850+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41c80ca580 con 0x7f41c80cb060 2024-09-21T07:49:45.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.851+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41c80d30c0 con 0x7f41c80cb060 2024-09-21T07:49:45.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.852+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f41c0036070 con 0x7f41c80cb060 2024-09-21T07:49:45.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.853+0000 7f41bf400640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f41a8078d70 0x7f41a807b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:45.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.853+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(60..60 src has 1..60) ==== 5709+0+0 (secure 0 0 0) 0x7f41c00bc6c0 con 0x7f41c80cb060 2024-09-21T07:49:45.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.853+0000 7f41c7400640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f41a8078d70 0x7f41a807b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:45.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.854+0000 7f41c7400640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f41a8078d70 0x7f41a807b280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009b30 tx=0x7f41b8005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:45.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:45.858+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f41c8154ec0 con 0x7f41c80cb060 2024-09-21T07:49:46.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.082+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f41c80b3e80 con 0x7f41c80cb060 2024-09-21T07:49:46.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.304+0000 7f41bf400640 1 -- 172.21.15.88:0/3634463979 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v37) ==== 136+0+0 (secure 0 0 0) 0x7f41c0084120 con 0x7f41c80cb060 2024-09-21T07:49:46.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f41a8078d70 msgr2=0x7f41a807b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f41a8078d70 0x7f41a807b280 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f41b8009b30 tx=0x7f41b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 msgr2=0x7f41c80d3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 0x7f41c80d3480 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f41c002b020 tx=0x7f41c0003f90 comp rx=0 tx=0).stop 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 shutdown_connections 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f41a8078d70 0x7f41a807b280 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41c80cb060 0x7f41c80d3480 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f41c8161d80 0x7f41c81516a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 --2- 172.21.15.88:0/3634463979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f41c80b59f0 0x7f41c8154980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 >> 172.21.15.88:0/3634463979 conn(0x7f41c809d7e0 msgr2=0x7f41c80d0b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.308+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 shutdown_connections 2024-09-21T07:49:46.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.309+0000 7f41cd000640 1 -- 172.21.15.88:0/3634463979 wait complete. 2024-09-21T07:49:46.337 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:49:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.553+0000 7f6cd0600640 1 -- 172.21.15.88:0/1448886338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 msgr2=0x7f6cc80cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.553+0000 7f6cd0600640 1 --2- 172.21.15.88:0/1448886338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc80cfab0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f6cc8075570 tx=0x7f6cc002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.553+0000 7f6cd0600640 1 -- 172.21.15.88:0/1448886338 shutdown_connections 2024-09-21T07:49:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.553+0000 7f6cd0600640 1 --2- 172.21.15.88:0/1448886338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc80cfab0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.553+0000 7f6cd0600640 1 --2- 172.21.15.88:0/1448886338 >> v2:172.21.15.175:3300/0 conn(0x7f6cc80cd000 0x7f6cc80cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.553+0000 7f6cd0600640 1 -- 172.21.15.88:0/1448886338 >> 172.21.15.88:0/1448886338 conn(0x7f6cc806d3d0 msgr2=0x7f6cc80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.554+0000 7f6cd0600640 1 -- 172.21.15.88:0/1448886338 shutdown_connections 2024-09-21T07:49:46.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.554+0000 7f6cd0600640 1 -- 172.21.15.88:0/1448886338 wait complete. 2024-09-21T07:49:46.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.556+0000 7f6cd0600640 1 Processor -- start 2024-09-21T07:49:46.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.556+0000 7f6cd0600640 1 -- start start 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.557+0000 7f6cd0600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc80b3d60 0x7f6cc8157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.557+0000 7f6cd0600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80cd000 0x7f6cc80aeb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.557+0000 7f6cd0600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc815c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.557+0000 7f6cd0600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc80b4380 con 0x7f6cc80cd410 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.557+0000 7f6cd0600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cc80711c0 con 0x7f6cc80b3d60 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.557+0000 7f6cd0600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6cc8071340 con 0x7f6cc80cd000 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc815c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc815c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56068/0 (socket says 172.21.15.88:56068) 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 -- 172.21.15.88:0/3634441258 learned_addr learned my addr 172.21.15.88:0/3634441258 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6ccdc00640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc80b3d60 0x7f6cc8157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6ccd200640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80cd000 0x7f6cc80aeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:46.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 -- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80cd000 msgr2=0x7f6cc80aeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80cd000 0x7f6cc80aeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 -- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc80b3d60 msgr2=0x7f6cc8157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc80b3d60 0x7f6cc8157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 -- 172.21.15.88:0/3634441258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cc80add70 con 0x7f6cc80cd410 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6ccdc00640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc80b3d60 0x7f6cc8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6ccd200640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80cd000 0x7f6cc80aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.558+0000 7f6cce600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc815c4f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0034040 tx=0x7f6cc002f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.559+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0041030 con 0x7f6cc80cd410 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.559+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cc0037040 con 0x7f6cc80cd410 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.559+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cc80ae050 con 0x7f6cc80cd410 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.559+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc0036070 con 0x7f6cc80cd410 2024-09-21T07:49:46.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.559+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cc8157fc0 con 0x7f6cc80cd410 2024-09-21T07:49:46.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.560+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cc81581f0 con 0x7f6cc80cd410 2024-09-21T07:49:46.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.561+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7f6cc0044070 con 0x7f6cc80cd410 2024-09-21T07:49:46.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.562+0000 7f6cc5600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ca8078d40 0x7f6ca807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:46.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.562+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7f6cc00bbfe0 con 0x7f6cc80cd410 2024-09-21T07:49:46.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.562+0000 7f6ccdc00640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ca8078d40 0x7f6ca807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:46.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.563+0000 7f6ccdc00640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ca8078d40 0x7f6ca807b250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000f60 tx=0x7f6cbc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:46.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.566+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cc004b070 con 0x7f6cc80cd410 2024-09-21T07:49:46.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.775+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f6cc80b5260 con 0x7f6ca8078d40 2024-09-21T07:49:46.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.777+0000 7f6cc5600640 1 -- 172.21.15.88:0/3634441258 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f6cbc001c50 con 0x7f6ca8078d40 2024-09-21T07:49:46.779 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:49:46.779 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:46.779 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:49:46.779 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:46.780 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.779+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ca8078d40 msgr2=0x7f6ca807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.779+0000 7f6cd0600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ca8078d40 0x7f6ca807b250 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc000f60 tx=0x7f6cbc009040 comp rx=0 tx=0).stop 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.779+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 msgr2=0x7f6cc815c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.779+0000 7f6cd0600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc815c4f0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6cc0034040 tx=0x7f6cc002f610 comp rx=0 tx=0).stop 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.779+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 shutdown_connections 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ca8078d40 0x7f6ca807b250 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cc80cd000 0x7f6cc80aeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cc80cd410 0x7f6cc815c4f0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 --2- 172.21.15.88:0/3634441258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cc80b3d60 0x7f6cc8157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 >> 172.21.15.88:0/3634441258 conn(0x7f6cc806d3d0 msgr2=0x7f6cc81e8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 shutdown_connections 2024-09-21T07:49:46.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:46.780+0000 7f6cd0600640 1 -- 172.21.15.88:0/3634441258 wait complete. 2024-09-21T07:49:46.805 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3359063678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44073c50 msgr2=0x7fbf440cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3359063678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44073c50 0x7fbf440cd610 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbf44072bd0 tx=0x7fbf3c02ab40 comp rx=0 tx=0).stop 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3359063678 shutdown_connections 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3359063678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44073c50 0x7fbf440cd610 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3359063678 >> v2:172.21.15.175:3300/0 conn(0x7fbf44145550 0x7fbf4406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3359063678 >> 172.21.15.88:0/3359063678 conn(0x7fbf4406d1c0 msgr2=0x7fbf4406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3359063678 shutdown_connections 2024-09-21T07:49:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.030+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3359063678 wait complete. 2024-09-21T07:49:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.033+0000 7fbf4aa00640 1 Processor -- start 2024-09-21T07:49:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.033+0000 7fbf4aa00640 1 -- start start 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf4aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 0x7fbf440b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf4aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf44145a40 0x7fbf440aee10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf4aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf440adbf0 0x7fbf440ae000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf4aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440ce0e0 con 0x7fbf44145550 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf4aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbf440cdf60 con 0x7fbf440adbf0 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf4aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbf440ce260 con 0x7fbf44145a40 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf43e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 0x7fbf440b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf43e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 0x7fbf440b4ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56098/0 (socket says 172.21.15.88:56098) 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.034+0000 7fbf43e00640 1 -- 172.21.15.88:0/3052258525 learned_addr learned my addr 172.21.15.88:0/3052258525 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf48a00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf440adbf0 0x7fbf440ae000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf3b400640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf44145a40 0x7fbf440aee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf43e00640 1 -- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf44145a40 msgr2=0x7fbf440aee10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf43e00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf44145a40 0x7fbf440aee10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf43e00640 1 -- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf440adbf0 msgr2=0x7fbf440ae000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf43e00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf440adbf0 0x7fbf440ae000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf43e00640 1 -- 172.21.15.88:0/3052258525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf440b3e00 con 0x7fbf44145550 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf43e00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 0x7fbf440b4ea0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d780 tx=0x7fbf3400dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.035+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf34017070 con 0x7fbf44145550 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.036+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbf3400de90 con 0x7fbf44145550 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.036+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf440b40e0 con 0x7fbf44145550 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.036+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf340099d0 con 0x7fbf44145550 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.036+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf440b46a0 con 0x7fbf44145550 2024-09-21T07:49:47.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.037+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf440ce780 con 0x7fbf44145550 2024-09-21T07:49:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.038+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fbf34009b70 con 0x7fbf44145550 2024-09-21T07:49:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.039+0000 7fbf40c00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbf20078d40 0x7fbf2007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.039+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 5767+0+0 (secure 0 0 0) 0x7fbf3409b520 con 0x7fbf44145550 2024-09-21T07:49:47.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.039+0000 7fbf3b400640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbf20078d40 0x7fbf2007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.040+0000 7fbf3b400640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbf20078d40 0x7fbf2007b250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf4413c2c0 tx=0x7fbf2c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:47.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.044+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbf34020020 con 0x7fbf44145550 2024-09-21T07:49:47.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.230+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fbf440ccab0 con 0x7fbf44145550 2024-09-21T07:49:47.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.230+0000 7fbf40c00640 1 -- 172.21.15.88:0/3052258525 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v36)=0 v36) ==== 122+0+0 (secure 0 0 0) 0x7fbf34062ea0 con 0x7fbf44145550 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.232+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbf20078d40 msgr2=0x7fbf2007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.232+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbf20078d40 0x7fbf2007b250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf4413c2c0 tx=0x7fbf2c009040 comp rx=0 tx=0).stop 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.232+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 msgr2=0x7fbf440b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.232+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 0x7fbf440b4ea0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbf3400d780 tx=0x7fbf3400dc40 comp rx=0 tx=0).stop 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 shutdown_connections 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fbf20078d40 0x7fbf2007b250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbf44145a40 0x7fbf440aee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbf440adbf0 0x7fbf440ae000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 --2- 172.21.15.88:0/3052258525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf44145550 0x7fbf440b4ea0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 >> 172.21.15.88:0/3052258525 conn(0x7fbf4406d1c0 msgr2=0x7fbf4413f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 shutdown_connections 2024-09-21T07:49:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.233+0000 7fbf4aa00640 1 -- 172.21.15.88:0/3052258525 wait complete. 2024-09-21T07:49:47.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:47.257 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol30470) 2024-09-21T07:49:47.324 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:47.324 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:49:47.361 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:49:47.361 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:47.361 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/_nogroup/subvol30470) 2024-09-21T07:49:47.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol97741 2024-09-21T07:49:47.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce3a800640 1 -- 172.21.15.88:0/3845308354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 msgr2=0x7fce1c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce3a800640 1 --2- 172.21.15.88:0/3845308354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce1c003ea0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fce2800c310 tx=0x7fce1c005830 comp rx=0 tx=0).stop 2024-09-21T07:49:47.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce39e00640 1 -- 172.21.15.88:0/3845308354 reap_dead start 2024-09-21T07:49:47.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce3a800640 1 -- 172.21.15.88:0/3845308354 shutdown_connections 2024-09-21T07:49:47.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce3a800640 1 -- 172.21.15.88:0/3845308354 >> 172.21.15.88:0/3845308354 conn(0x7fce3406d620 msgr2=0x7fce340b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:47.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce3a800640 1 -- 172.21.15.88:0/3845308354 shutdown_connections 2024-09-21T07:49:47.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.686+0000 7fce3a800640 1 -- 172.21.15.88:0/3845308354 wait complete. 2024-09-21T07:49:47.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.688+0000 7fce3a800640 1 Processor -- start 2024-09-21T07:49:47.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.688+0000 7fce3a800640 1 -- start start 2024-09-21T07:49:47.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce3a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce34156f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce3a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce341f6440 0x7fce3406ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce3a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3406f450 0x7fce341f8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce3a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fce341624b0 con 0x7fce1c003a30 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce3a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce340cfc20 con 0x7fce3406f450 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce3a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce34162630 con 0x7fce341f6440 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce39400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce34156f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce39400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce34156f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56114/0 (socket says 172.21.15.88:56114) 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce39400640 1 -- 172.21.15.88:0/3738772462 learned_addr learned my addr 172.21.15.88:0/3738772462 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce39400640 1 -- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce341f6440 msgr2=0x7fce3406ef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce38a00640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce341f6440 0x7fce3406ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.690+0000 7fce39e00640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3406f450 0x7fce341f8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce39400640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce341f6440 0x7fce3406ef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce39400640 1 -- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3406f450 msgr2=0x7fce341f8f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce39400640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3406f450 0x7fce341f8f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce39400640 1 -- 172.21.15.88:0/3738772462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3406f860 con 0x7fce1c003a30 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce38a00640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce341f6440 0x7fce3406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:47.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce39e00640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3406f450 0x7fce341f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce39400640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce34156f30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fce2802b180 tx=0x7fce28001db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28031040 con 0x7fce1c003a30 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce280065e0 con 0x7fce1c003a30 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28003a60 con 0x7fce1c003a30 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce34144ce0 con 0x7fce1c003a30 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.691+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce341452a0 con 0x7fce1c003a30 2024-09-21T07:49:47.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.692+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3415f6a0 con 0x7fce1c003a30 2024-09-21T07:49:47.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.694+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 9) ==== 101594+0+0 (secure 0 0 0) 0x7fce2803e030 con 0x7fce1c003a30 2024-09-21T07:49:47.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.695+0000 7fce31600640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fce14078d40 0x7fce1407b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:47.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.695+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7fce280baf50 con 0x7fce1c003a30 2024-09-21T07:49:47.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.695+0000 7fce38a00640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fce14078d40 0x7fce1407b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:47.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.696+0000 7fce38a00640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fce14078d40 0x7fce1407b250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce20000c00 tx=0x7fce20006ae0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:47.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.699+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce3406f860 con 0x7fce1c003a30 2024-09-21T07:49:47.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.914+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol97741", "target": ["mon-mgr", ""]}) -- 0x7fce340b2ab0 con 0x7fce14078d40 2024-09-21T07:49:47.923 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:47.921+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:47.923 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:47.921+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:47.923 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:47.921+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:47.923 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:47.921+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:47.923 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:47.921+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:47.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.966+0000 7fce31600640 1 -- 172.21.15.88:0/3738772462 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fce340b2ab0 con 0x7fce14078d40 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fce14078d40 msgr2=0x7fce1407b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fce14078d40 0x7fce1407b250 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fce20000c00 tx=0x7fce20006ae0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 msgr2=0x7fce34156f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce34156f30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fce2802b180 tx=0x7fce28001db0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 shutdown_connections 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fce14078d40 0x7fce1407b250 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce341f6440 0x7fce3406ef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3406f450 0x7fce341f8f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 --2- 172.21.15.88:0/3738772462 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce1c003a30 0x7fce34156f30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:47.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 >> 172.21.15.88:0/3738772462 conn(0x7fce3406d620 msgr2=0x7fce34161bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:47.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 shutdown_connections 2024-09-21T07:49:47.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:47.968+0000 7fce3a800640 1 -- 172.21.15.88:0/3738772462 wait complete. 2024-09-21T07:49:47.996 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:47.997 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cp ./volumes/_nogroup/subvol97741/.meta ./volumes/_nogroup/subvol30470) 2024-09-21T07:49:50.837 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol30470 2024-09-21T07:49:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.068+0000 7fd715400640 1 -- 172.21.15.88:0/2045958551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710074440 msgr2=0x7fd71006ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.068+0000 7fd715400640 1 --2- 172.21.15.88:0/2045958551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710074440 0x7fd71006ddb0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fd700005fd0 tx=0x7fd70002a540 comp rx=0 tx=0).stop 2024-09-21T07:49:51.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.069+0000 7fd715400640 1 -- 172.21.15.88:0/2045958551 shutdown_connections 2024-09-21T07:49:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.069+0000 7fd715400640 1 --2- 172.21.15.88:0/2045958551 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd710070a70 0x7fd71006e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.069+0000 7fd715400640 1 --2- 172.21.15.88:0/2045958551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710074440 0x7fd71006ddb0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.069+0000 7fd715400640 1 -- 172.21.15.88:0/2045958551 >> 172.21.15.88:0/2045958551 conn(0x7fd710073920 msgr2=0x7fd710073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.069+0000 7fd715400640 1 -- 172.21.15.88:0/2045958551 shutdown_connections 2024-09-21T07:49:51.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.069+0000 7fd715400640 1 -- 172.21.15.88:0/2045958551 wait complete. 2024-09-21T07:49:51.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.071+0000 7fd715400640 1 Processor -- start 2024-09-21T07:49:51.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.071+0000 7fd715400640 1 -- start start 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.072+0000 7fd715400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 0x7fd7101440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.072+0000 7fd715400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd710074440 0x7fd710144b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.072+0000 7fd715400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71015b920 0x7fd7101450a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.072+0000 7fd715400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100b41b0 con 0x7fd710070a70 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.072+0000 7fd715400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7100d0590 con 0x7fd71015b920 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.072+0000 7fd715400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7100b4330 con 0x7fd710074440 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 0x7fd7101440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 0x7fd7101440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56136/0 (socket says 172.21.15.88:56136) 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 -- 172.21.15.88:0/3323367987 learned_addr learned my addr 172.21.15.88:0/3323367987 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd705e00640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd710074440 0x7fd710144b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70f200640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71015b920 0x7fd7101450a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 -- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd710074440 msgr2=0x7fd710144b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd710074440 0x7fd710144b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 -- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71015b920 msgr2=0x7fd7101450a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71015b920 0x7fd7101450a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 -- 172.21.15.88:0/3323367987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7100cb880 con 0x7fd710070a70 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.073+0000 7fd70e800640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 0x7fd7101440f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd700003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.074+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd70003e030 con 0x7fd710070a70 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.074+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd700003e00 con 0x7fd710070a70 2024-09-21T07:49:51.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.074+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd700035040 con 0x7fd710070a70 2024-09-21T07:49:51.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.074+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd710156730 con 0x7fd710070a70 2024-09-21T07:49:51.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.074+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7100cbe20 con 0x7fd710070a70 2024-09-21T07:49:51.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.075+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7100b4880 con 0x7fd710070a70 2024-09-21T07:49:51.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.076+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7fd700041030 con 0x7fd710070a70 2024-09-21T07:49:51.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.078+0000 7fd707400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd6ec078dd0 0x7fd6ec07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.078+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7fd7000bb960 con 0x7fd710070a70 2024-09-21T07:49:51.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.078+0000 7fd705e00640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd6ec078dd0 0x7fd6ec07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.079+0000 7fd705e00640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd6ec078dd0 0x7fd6ec07b2e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd71015e9c0 tx=0x7fd6f800b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.083+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd7000832d0 con 0x7fd710070a70 2024-09-21T07:49:51.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.294+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol30470", "target": ["mon-mgr", ""]}) -- 0x7fd7100705c0 con 0x7fd6ec078dd0 2024-09-21T07:49:51.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.337+0000 7fd707400640 1 -- 172.21.15.88:0/3323367987 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fd7100705c0 con 0x7fd6ec078dd0 2024-09-21T07:49:51.338 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol30470 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd6ec078dd0 msgr2=0x7fd6ec07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd6ec078dd0 0x7fd6ec07b2e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fd71015e9c0 tx=0x7fd6f800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 msgr2=0x7fd7101440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 0x7fd7101440f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fd700000c00 tx=0x7fd700003910 comp rx=0 tx=0).stop 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 shutdown_connections 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd6ec078dd0 0x7fd6ec07b2e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd710074440 0x7fd710144b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd71015b920 0x7fd7101450a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 --2- 172.21.15.88:0/3323367987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd710070a70 0x7fd7101440f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.339+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 >> 172.21.15.88:0/3323367987 conn(0x7fd710073920 msgr2=0x7fd710159840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.340+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 shutdown_connections 2024-09-21T07:49:51.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.340+0000 7fd715400640 1 -- 172.21.15.88:0/3323367987 wait complete. 2024-09-21T07:49:51.364 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol30470 2024-09-21T07:49:51.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 -- 172.21.15.88:0/3261705894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 msgr2=0x7fa6a41624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/3261705894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41624b0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fa690009830 tx=0x7fa69002a480 comp rx=0 tx=0).stop 2024-09-21T07:49:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 -- 172.21.15.88:0/3261705894 shutdown_connections 2024-09-21T07:49:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/3261705894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 0x7fa6a40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/3261705894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41624b0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 -- 172.21.15.88:0/3261705894 >> 172.21.15.88:0/3261705894 conn(0x7fa6a406d620 msgr2=0x7fa6a40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 -- 172.21.15.88:0/3261705894 shutdown_connections 2024-09-21T07:49:51.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.585+0000 7fa6a9a00640 1 -- 172.21.15.88:0/3261705894 wait complete. 2024-09-21T07:49:51.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.587+0000 7fa6a9a00640 1 Processor -- start 2024-09-21T07:49:51.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.587+0000 7fa6a9a00640 1 -- start start 2024-09-21T07:49:51.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6a40742b0 0x7fa6a40998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 0x7fa6a40ae0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a4073240 con 0x7fa6a40b42a0 2024-09-21T07:49:51.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a40730c0 con 0x7fa6a40742b0 2024-09-21T07:49:51.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6a40733c0 con 0x7fa6a40cf2c0 2024-09-21T07:49:51.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56164/0 (socket says 172.21.15.88:56164) 2024-09-21T07:49:51.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a3400640 1 -- 172.21.15.88:0/1925146762 learned_addr learned my addr 172.21.15.88:0/1925146762 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:51.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6a40742b0 0x7fa6a40998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.589+0000 7fa6a9000640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 0x7fa6a40ae0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3400640 1 -- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 msgr2=0x7fa6a40ae0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3400640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 0x7fa6a40ae0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3400640 1 -- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6a40742b0 msgr2=0x7fa6a40998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3400640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6a40742b0 0x7fa6a40998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3400640 1 -- 172.21.15.88:0/1925146762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a41f9e30 con 0x7fa6a40b42a0 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6a40742b0 0x7fa6a40998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a9000640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 0x7fa6a40ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a3400640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41f9100 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa690009950 tx=0x7fa690033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:51.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa690005c90 con 0x7fa6a40b42a0 2024-09-21T07:49:51.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa690031e50 con 0x7fa6a40b42a0 2024-09-21T07:49:51.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6900332b0 con 0x7fa6a40b42a0 2024-09-21T07:49:51.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a4154490 con 0x7fa6a40b42a0 2024-09-21T07:49:51.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.590+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a4099de0 con 0x7fa6a40b42a0 2024-09-21T07:49:51.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.591+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6a41f9b80 con 0x7fa6a40b42a0 2024-09-21T07:49:51.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.593+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7fa690033450 con 0x7fa6a40b42a0 2024-09-21T07:49:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.594+0000 7fa69be00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa684078dd0 0x7fa68407b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.594+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7fa6900bc6a0 con 0x7fa6a40b42a0 2024-09-21T07:49:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.595+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa684078dd0 0x7fa68407b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:51.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.595+0000 7fa6a3e00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa684078dd0 0x7fa68407b2e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6940016a0 tx=0x7fa694009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.599+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa69003f030 con 0x7fa6a40b42a0 2024-09-21T07:49:51.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.810+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol30470", "target": ["mon-mgr", ""]}) -- 0x7fa6a41542d0 con 0x7fa684078dd0 2024-09-21T07:49:51.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.817+0000 7fa69be00640 1 -- 172.21.15.88:0/1925146762 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+30 (secure 0 0 0) 0x7fa6940012e0 con 0x7fa684078dd0 2024-09-21T07:49:51.818 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol30470 2024-09-21T07:49:51.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa684078dd0 msgr2=0x7fa68407b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa684078dd0 0x7fa68407b2e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fa6940016a0 tx=0x7fa694009040 comp rx=0 tx=0).stop 2024-09-21T07:49:51.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 msgr2=0x7fa6a41f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:51.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41f9100 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fa690009950 tx=0x7fa690033c90 comp rx=0 tx=0).stop 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 shutdown_connections 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa684078dd0 0x7fa68407b2e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6a40cf2c0 0x7fa6a40ae0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a40b42a0 0x7fa6a41f9100 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 --2- 172.21.15.88:0/1925146762 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6a40742b0 0x7fa6a40998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 >> 172.21.15.88:0/1925146762 conn(0x7fa6a406d620 msgr2=0x7fa6a4161b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 shutdown_connections 2024-09-21T07:49:51.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:51.819+0000 7fa6a9a00640 1 -- 172.21.15.88:0/1925146762 wait complete. 2024-09-21T07:49:51.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:49:51.843 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/beea02b56bfa14cb66ffa27dec9fe787.meta) 2024-09-21T07:49:51.903 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:49:51.903 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:49:51.903 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:49:51.903 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol30470 2024-09-21T07:49:51.903 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:49:51.903 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:51.906 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume authorize cephfs subvol30470 alice 2024-09-21T07:49:52.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.134+0000 7f7514000640 1 -- 172.21.15.88:0/786608137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f4004d40 msgr2=0x7f74f4005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.134+0000 7f7514000640 1 --2- 172.21.15.88:0/786608137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74f4004d40 0x7f74f4005150 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f7504005fd0 tx=0x7f74f4005830 comp rx=0 tx=0).stop 2024-09-21T07:49:52.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.134+0000 7f7512000640 1 -- 172.21.15.88:0/786608137 reap_dead start 2024-09-21T07:49:52.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.134+0000 7f7514000640 1 -- 172.21.15.88:0/786608137 shutdown_connections 2024-09-21T07:49:52.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.134+0000 7f7514000640 1 -- 172.21.15.88:0/786608137 >> 172.21.15.88:0/786608137 conn(0x7f750c0b2540 msgr2=0x7f750c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:52.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.134+0000 7f7514000640 1 -- 172.21.15.88:0/786608137 shutdown_connections 2024-09-21T07:49:52.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.135+0000 7f7514000640 1 -- 172.21.15.88:0/786608137 wait complete. 2024-09-21T07:49:52.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.137+0000 7f7514000640 1 Processor -- start 2024-09-21T07:49:52.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.137+0000 7f7514000640 1 -- start start 2024-09-21T07:49:52.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.138+0000 7f7514000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f4004d40 0x7f750c1f89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.138+0000 7f7514000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 0x7f750c0b1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.138+0000 7f7514000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f750c0b1570 0x7f750c0cea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.138+0000 7f7514000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0b3120 con 0x7f750c099820 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.138+0000 7f7514000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f750c0afb30 con 0x7f74f4004d40 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.138+0000 7f7514000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f750c0b32a0 con 0x7f750c0b1570 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 0x7f750c0b1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 0x7f750c0b1030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56180/0 (socket says 172.21.15.88:56180) 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 -- 172.21.15.88:0/3031359886 learned_addr learned my addr 172.21.15.88:0/3031359886 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7511600640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f4004d40 0x7f750c1f89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7512000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f750c0b1570 0x7f750c0cea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 -- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f750c0b1570 msgr2=0x7f750c0cea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f750c0b1570 0x7f750c0cea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 -- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f4004d40 msgr2=0x7f750c1f89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f4004d40 0x7f750c1f89e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 -- 172.21.15.88:0/3031359886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f750c0b1980 con 0x7f750c099820 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.139+0000 7f7510c00640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 0x7f750c0b1030 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f74f8011b10 tx=0x7f74f8009eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.140+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f801e070 con 0x7f750c099820 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.140+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f74f8004050 con 0x7f750c099820 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.140+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74f800fea0 con 0x7f750c099820 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.140+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f750c1e19f0 con 0x7f750c099820 2024-09-21T07:49:52.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.140+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f750c155530 con 0x7f750c099820 2024-09-21T07:49:52.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.141+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f750c0d1030 con 0x7f750c099820 2024-09-21T07:49:52.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.142+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f74f8021070 con 0x7f750c099820 2024-09-21T07:49:52.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.143+0000 7f7509600640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f74ec078e00 0x7f74ec07b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.143+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f74f809bd50 con 0x7f750c099820 2024-09-21T07:49:52.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.144+0000 7f7511600640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f74ec078e00 0x7f74ec07b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.145+0000 7f7511600640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f74ec078e00 0x7f74ec07b310 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7500001860 tx=0x7f7500001630 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:52.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.147+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f750c0b1980 con 0x7f750c099820 2024-09-21T07:49:52.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.359+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume authorize", "vol_name": "cephfs", "sub_name": "subvol30470", "auth_id": "alice", "target": ["mon-mgr", ""]}) -- 0x7f750c15bf10 con 0x7f74ec078e00 2024-09-21T07:49:52.371 INFO:tasks.ceph.mgr.x.smithi175.stderr:Creating meta for ID alice with tenant None 2024-09-21T07:49:52.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.413+0000 7f7509600640 1 -- 172.21.15.88:0/3031359886 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f750c15bf10 con 0x7f74ec078e00 2024-09-21T07:49:52.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f74ec078e00 msgr2=0x7f74ec07b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f74ec078e00 0x7f74ec07b310 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f7500001860 tx=0x7f7500001630 comp rx=0 tx=0).stop 2024-09-21T07:49:52.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 msgr2=0x7f750c0b1030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 0x7f750c0b1030 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f74f8011b10 tx=0x7f74f8009eb0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 shutdown_connections 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f74ec078e00 0x7f74ec07b310 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f750c0b1570 0x7f750c0cea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f750c099820 0x7f750c0b1030 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 --2- 172.21.15.88:0/3031359886 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74f4004d40 0x7f750c1f89e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 >> 172.21.15.88:0/3031359886 conn(0x7f750c0b2540 msgr2=0x7f750c0c9f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 shutdown_connections 2024-09-21T07:49:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.415+0000 7f7514000640 1 -- 172.21.15.88:0/3031359886 wait complete. 2024-09-21T07:49:52.438 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth get client.alice --format=json-pretty 2024-09-21T07:49:52.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.657+0000 7f2abf200640 1 -- 172.21.15.88:0/6847670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00d2fb0 msgr2=0x7f2ac006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.657+0000 7f2abf200640 1 --2- 172.21.15.88:0/6847670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00d2fb0 0x7f2ac006e8a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8005fd0 tx=0x7f2aa802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.657+0000 7f2abf200640 1 -- 172.21.15.88:0/6847670 shutdown_connections 2024-09-21T07:49:52.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.657+0000 7f2abf200640 1 --2- 172.21.15.88:0/6847670 >> v2:172.21.15.175:3300/0 conn(0x7f2ac006ede0 0x7f2ac006f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.657+0000 7f2abf200640 1 --2- 172.21.15.88:0/6847670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00d2fb0 0x7f2ac006e8a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.658+0000 7f2abf200640 1 -- 172.21.15.88:0/6847670 >> 172.21.15.88:0/6847670 conn(0x7f2ac009d880 msgr2=0x7f2ac008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:52.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.658+0000 7f2abf200640 1 -- 172.21.15.88:0/6847670 shutdown_connections 2024-09-21T07:49:52.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.658+0000 7f2abf200640 1 -- 172.21.15.88:0/6847670 wait complete. 2024-09-21T07:49:52.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.660+0000 7f2abf200640 1 Processor -- start 2024-09-21T07:49:52.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.660+0000 7f2abf200640 1 -- start start 2024-09-21T07:49:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.661+0000 7f2abf200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac006ede0 0x7f2ac015b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.661+0000 7f2abf200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 0x7f2ac00b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.661+0000 7f2abf200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac015bff0 0x7f2ac00b2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.661+0000 7f2abf200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac00b9330 con 0x7f2ac00b9570 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.661+0000 7f2abf200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac00b91b0 con 0x7f2ac006ede0 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.661+0000 7f2abf200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac006f730 con 0x7f2ac015bff0 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 0x7f2ac00b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 0x7f2ac00b2940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56182/0 (socket says 172.21.15.88:56182) 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 -- 172.21.15.88:0/2797389742 learned_addr learned my addr 172.21.15.88:0/2797389742 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2abe800640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac015bff0 0x7f2ac00b2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2abde00640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac006ede0 0x7f2ac015b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 -- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac015bff0 msgr2=0x7f2ac00b2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac015bff0 0x7f2ac00b2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 -- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac006ede0 msgr2=0x7f2ac015b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac006ede0 0x7f2ac015b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 -- 172.21.15.88:0/2797389742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac00d1ae0 con 0x7f2ac00b9570 2024-09-21T07:49:52.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2abde00640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac006ede0 0x7f2ac015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:52.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.662+0000 7f2ab5400640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 0x7f2ac00b2940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8006010 tx=0x7f2aa8005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.663+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa803e030 con 0x7f2ac00b9570 2024-09-21T07:49:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.663+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aa80039f0 con 0x7f2ac00b9570 2024-09-21T07:49:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.663+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac01f5360 con 0x7f2ac00b9570 2024-09-21T07:49:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.663+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8035040 con 0x7f2ac00b9570 2024-09-21T07:49:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.663+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac01f5920 con 0x7f2ac00b9570 2024-09-21T07:49:52.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.664+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac00d1da0 con 0x7f2ac00b9570 2024-09-21T07:49:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.665+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f2aa8041030 con 0x7f2ac00b9570 2024-09-21T07:49:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.666+0000 7f2ab6a00640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2a9c078dd0 0x7f2a9c07b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.666+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f2aa80bb6b0 con 0x7f2ac00b9570 2024-09-21T07:49:52.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.667+0000 7f2abde00640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2a9c078dd0 0x7f2a9c07b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.667+0000 7f2abde00640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2a9c078dd0 0x7f2a9c07b2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ba230 tx=0x7f2ab0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:52.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.671+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2aa8003e20 con 0x7f2ac00b9570 2024-09-21T07:49:52.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.924+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"} v 0) -- 0x7f2ac006cda0 con 0x7f2ac00b9570 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.925+0000 7f2ab6a00640 1 -- 172.21.15.88:0/2797389742 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth get", "entity": "client.alice", "format": "json-pretty"}]=0 v12) ==== 107+0+280 (secure 0 0 0) 0x7f2aa8082fa0 con 0x7f2ac00b9570 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.alice", 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCgeu5mvQn5FhAAVOJsrbSENHWL0gKr4vrAiQ==", 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol30470", 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-21T07:49:52.926 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:52.927 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:49:52.927 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:49:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2a9c078dd0 msgr2=0x7f2a9c07b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2a9c078dd0 0x7f2a9c07b2e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f2ac00ba230 tx=0x7f2ab0009040 comp rx=0 tx=0).stop 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 msgr2=0x7f2ac00b2940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 0x7f2ac00b2940 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8006010 tx=0x7f2aa8005e80 comp rx=0 tx=0).stop 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 shutdown_connections 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2a9c078dd0 0x7f2a9c07b2e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac015bff0 0x7f2ac00b2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac00b9570 0x7f2ac00b2940 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 --2- 172.21.15.88:0/2797389742 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac006ede0 0x7f2ac015b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 >> 172.21.15.88:0/2797389742 conn(0x7f2ac009d880 msgr2=0x7f2ac0161300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 shutdown_connections 2024-09-21T07:49:52.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:52.927+0000 7f2abf200640 1 -- 172.21.15.88:0/2797389742 wait complete. 2024-09-21T07:49:52.952 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol30470 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.173+0000 7f2ac6400640 1 -- 172.21.15.88:0/4238583703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0073c50 msgr2=0x7f2ac01674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.173+0000 7f2ac6400640 1 --2- 172.21.15.88:0/4238583703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0073c50 0x7f2ac01674a0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f2ab8005fd0 tx=0x7f2ab802a9d0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.173+0000 7f2ac6400640 1 -- 172.21.15.88:0/4238583703 shutdown_connections 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.173+0000 7f2ac6400640 1 --2- 172.21.15.88:0/4238583703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0073c50 0x7f2ac01674a0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.173+0000 7f2ac6400640 1 -- 172.21.15.88:0/4238583703 >> 172.21.15.88:0/4238583703 conn(0x7f2ac006d1c0 msgr2=0x7f2ac006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.174+0000 7f2ac6400640 1 -- 172.21.15.88:0/4238583703 shutdown_connections 2024-09-21T07:49:53.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.174+0000 7f2ac6400640 1 -- 172.21.15.88:0/4238583703 wait complete. 2024-09-21T07:49:53.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.176+0000 7f2ac6400640 1 Processor -- start 2024-09-21T07:49:53.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.176+0000 7f2ac6400640 1 -- start start 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.177+0000 7f2ac6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac0145550 0x7f2ac0158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.177+0000 7f2ac6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 0x7f2ac00b4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.177+0000 7f2ac6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00b4b80 0x7f2ac013d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.177+0000 7f2ac6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0167d70 con 0x7f2ac0145a40 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.177+0000 7f2ac6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2ac0167bf0 con 0x7f2ac0145550 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.177+0000 7f2ac6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2ac0167ef0 con 0x7f2ac00b4b80 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 0x7f2ac00b4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 0x7f2ac00b4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56188/0 (socket says 172.21.15.88:56188) 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 -- 172.21.15.88:0/1097556230 learned_addr learned my addr 172.21.15.88:0/1097556230 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abf400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac0145550 0x7f2ac0158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 -- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00b4b80 msgr2=0x7f2ac013d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abfe00640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00b4b80 0x7f2ac013d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00b4b80 0x7f2ac013d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 -- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac0145550 msgr2=0x7f2ac0158b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac0145550 0x7f2ac0158b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 -- 172.21.15.88:0/1097556230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ac0159080 con 0x7f2ac0145a40 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abf400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac0145550 0x7f2ac0158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abfe00640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00b4b80 0x7f2ac013d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.178+0000 7f2abea00640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 0x7f2ac00b4640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c400 tx=0x7f2aac00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.179+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac017070 con 0x7f2ac0145a40 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.179+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2aac0053c0 con 0x7f2ac0145a40 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.179+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aac003830 con 0x7f2ac0145a40 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.179+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ac00b3790 con 0x7f2ac0145a40 2024-09-21T07:49:53.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.179+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ac00b3d50 con 0x7f2ac0145a40 2024-09-21T07:49:53.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.180+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2ac00b5120 con 0x7f2ac0145a40 2024-09-21T07:49:53.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.181+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f2aac01a070 con 0x7f2ac0145a40 2024-09-21T07:49:53.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.182+0000 7f2ab7400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2aa0078e00 0x7f2aa007b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.182+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f2aac09a4f0 con 0x7f2ac0145a40 2024-09-21T07:49:53.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.183+0000 7f2abf400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2aa0078e00 0x7f2aa007b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.183+0000 7f2abf400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2aa0078e00 0x7f2aa007b310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000f60 tx=0x7f2ab0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:53.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.186+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2aac0625e0 con 0x7f2ac0145a40 2024-09-21T07:49:53.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.396+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol30470", "target": ["mon-mgr", ""]}) -- 0x7f2ac006fb40 con 0x7f2aa0078e00 2024-09-21T07:49:53.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.427+0000 7f2ab7400640 1 -- 172.21.15.88:0/1097556230 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2ac006fb40 con 0x7f2aa0078e00 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2aa0078e00 msgr2=0x7f2aa007b310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2aa0078e00 0x7f2aa007b310 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2ab0000f60 tx=0x7f2ab0009040 comp rx=0 tx=0).stop 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 msgr2=0x7f2ac00b4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 0x7f2ac00b4640 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00c400 tx=0x7f2aac00c8c0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 shutdown_connections 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2aa0078e00 0x7f2aa007b310 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2ac00b4b80 0x7f2ac013d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ac0145a40 0x7f2ac00b4640 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 --2- 172.21.15.88:0/1097556230 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2ac0145550 0x7f2ac0158b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 >> 172.21.15.88:0/1097556230 conn(0x7f2ac006d1c0 msgr2=0x7f2ac0141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:53.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.429+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 shutdown_connections 2024-09-21T07:49:53.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.430+0000 7f2ac6400640 1 -- 172.21.15.88:0/1097556230 wait complete. 2024-09-21T07:49:53.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.431+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.431+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.431+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.431+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.433 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.431+0000 7ff29f400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.452 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.451+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.452 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.451+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.453 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.451+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.453 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.451+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.453 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:49:53.451+0000 7ff29fe00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:49:53.454 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol97741 2024-09-21T07:49:53.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 -- 172.21.15.88:0/146656021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f08004800 msgr2=0x7f6f08004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 --2- 172.21.15.88:0/146656021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f08004800 0x7f6f08004c10 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6f0c006fa0 tx=0x7f6f08005830 comp rx=0 tx=0).stop 2024-09-21T07:49:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 -- 172.21.15.88:0/146656021 shutdown_connections 2024-09-21T07:49:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 --2- 172.21.15.88:0/146656021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f08004800 0x7f6f08004c10 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 -- 172.21.15.88:0/146656021 >> 172.21.15.88:0/146656021 conn(0x7f6f2006d1c0 msgr2=0x7f6f2006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:49:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 -- 172.21.15.88:0/146656021 shutdown_connections 2024-09-21T07:49:53.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.678+0000 7f6f27e00640 1 -- 172.21.15.88:0/146656021 wait complete. 2024-09-21T07:49:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.680+0000 7f6f27e00640 1 Processor -- start 2024-09-21T07:49:53.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.680+0000 7f6f27e00640 1 -- start start 2024-09-21T07:49:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f27e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f08004800 0x7f6f2013fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f27e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f201f6220 0x7f6f201f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 0x7f6f2014de70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f27e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20073e90 con 0x7f6f201f92d0 2024-09-21T07:49:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f27e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20073d10 con 0x7f6f201f6220 2024-09-21T07:49:53.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f27e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f2006f180 con 0x7f6f08004800 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f25e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 0x7f6f2014de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f25e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 0x7f6f2014de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56206/0 (socket says 172.21.15.88:56206) 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f25e00640 1 -- 172.21.15.88:0/3196177044 learned_addr learned my addr 172.21.15.88:0/3196177044 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f24a00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f201f6220 0x7f6f201f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.682+0000 7f6f25400640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f08004800 0x7f6f2013fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25e00640 1 -- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f08004800 msgr2=0x7f6f2013fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f08004800 0x7f6f2013fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25e00640 1 -- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f201f6220 msgr2=0x7f6f201f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f201f6220 0x7f6f201f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25e00640 1 -- 172.21.15.88:0/3196177044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f2013d530 con 0x7f6f201f92d0 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25400640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f08004800 0x7f6f2013fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f24a00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f201f6220 0x7f6f201f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:49:53.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f25e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 0x7f6f2014de70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6f200b5ad0 tx=0x7f6f18006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18011900 con 0x7f6f201f92d0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f18018070 con 0x7f6f201f92d0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18011d40 con 0x7f6f201f92d0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f20140150 con 0x7f6f201f92d0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.683+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f200b4740 con 0x7f6f201f92d0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.684+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f2006f6e0 con 0x7f6f201f92d0 2024-09-21T07:49:53.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.686+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 10) ==== 101645+0+0 (secure 0 0 0) 0x7f6f1801d460 con 0x7f6f201f92d0 2024-09-21T07:49:53.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.687+0000 7f6f1d600640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f00078dd0 0x7f6f0007b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:49:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.687+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f6f1809bbf0 con 0x7f6f201f92d0 2024-09-21T07:49:53.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.687+0000 7f6f25400640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f00078dd0 0x7f6f0007b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:49:53.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.688+0000 7f6f25400640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f00078dd0 0x7f6f0007b2e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009cd0 tx=0x7f6f14005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:49:53.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.691+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f18063c50 con 0x7f6f201f92d0 2024-09-21T07:49:53.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:53.907+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol97741", "target": ["mon-mgr", ""]}) -- 0x7f6f20161080 con 0x7f6f00078dd0 2024-09-21T07:49:54.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:49:54.027+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f6f1805f7c0 con 0x7f6f201f92d0 2024-09-21T07:49:56.544 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:49:56.549 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:00.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.834+0000 7f6f1d600640 1 -- 172.21.15.88:0/3196177044 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6f20161080 con 0x7f6f00078dd0 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.837+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f00078dd0 msgr2=0x7f6f0007b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.837+0000 7f6f27e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f00078dd0 0x7f6f0007b2e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f6f14009cd0 tx=0x7f6f14005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 msgr2=0x7f6f2014de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 0x7f6f2014de70 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f6f200b5ad0 tx=0x7f6f18006430 comp rx=0 tx=0).stop 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 shutdown_connections 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f00078dd0 0x7f6f0007b2e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:00.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201f92d0 0x7f6f2014de70 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f201f6220 0x7f6f201f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 --2- 172.21.15.88:0/3196177044 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f08004800 0x7f6f2013fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 >> 172.21.15.88:0/3196177044 conn(0x7f6f2006d1c0 msgr2=0x7f6f2013d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 shutdown_connections 2024-09-21T07:50:00.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00.838+0000 7f6f27e00640 1 -- 172.21.15.88:0/3196177044 wait complete. 2024-09-21T07:50:00.869 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:50:00.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:50:00.956 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:50:00.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:50:00.965 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:50:00.966 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:01.057 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:01.142 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:50:01.142 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:01.142 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:01.143 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:01.178 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:01.178 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:01.253 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 -- 172.21.15.88:0/2667559952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 msgr2=0x7f28700b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 --2- 172.21.15.88:0/2667559952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f28700b2f60 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f285c009830 tx=0x7f285c02a480 comp rx=0 tx=0).stop 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 -- 172.21.15.88:0/2667559952 shutdown_connections 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 --2- 172.21.15.88:0/2667559952 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287015a0f0 0x7f287015d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 --2- 172.21.15.88:0/2667559952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f28700b2f60 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 -- 172.21.15.88:0/2667559952 >> 172.21.15.88:0/2667559952 conn(0x7f28700b1e40 msgr2=0x7f28700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 -- 172.21.15.88:0/2667559952 shutdown_connections 2024-09-21T07:50:01.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.512+0000 7f2876e00640 1 -- 172.21.15.88:0/2667559952 wait complete. 2024-09-21T07:50:01.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.515+0000 7f2876e00640 1 Processor -- start 2024-09-21T07:50:01.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.515+0000 7f2876e00640 1 -- start start 2024-09-21T07:50:01.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2876e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28700ad9c0 0x7f2870157000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:01.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2876e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f2870158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:01.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2876e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287015a0f0 0x7f2870155580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2876e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f287006f290 con 0x7f28700af170 2024-09-21T07:50:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2876e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f287015a5e0 con 0x7f28700ad9c0 2024-09-21T07:50:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2876e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f287006f410 con 0x7f287015a0f0 2024-09-21T07:50:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2875000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f2870158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2875000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f2870158340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40978/0 (socket says 172.21.15.88:40978) 2024-09-21T07:50:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.516+0000 7f2875000640 1 -- 172.21.15.88:0/3160871388 learned_addr learned my addr 172.21.15.88:0/3160871388 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2876400640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287015a0f0 0x7f2870155580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875a00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28700ad9c0 0x7f2870157000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875000640 1 -- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287015a0f0 msgr2=0x7f2870155580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875000640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287015a0f0 0x7f2870155580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875000640 1 -- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28700ad9c0 msgr2=0x7f2870157000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875000640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28700ad9c0 0x7f2870157000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875000640 1 -- 172.21.15.88:0/3160871388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28701f6110 con 0x7f28700af170 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f2875000640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f2870158340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f285c02aa70 tx=0x7f285c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.517+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c031d60 con 0x7f28700af170 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.518+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f285c033e10 con 0x7f28700af170 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.518+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285c005260 con 0x7f28700af170 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.518+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28700cdc40 con 0x7f28700af170 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.518+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28700ce200 con 0x7f28700af170 2024-09-21T07:50:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.519+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f287006f970 con 0x7f28700af170 2024-09-21T07:50:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.520+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f285c005400 con 0x7f28700af170 2024-09-21T07:50:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.521+0000 7f286d600640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2850078ef0 0x7f285007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.521+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f285c0bc7b0 con 0x7f28700af170 2024-09-21T07:50:01.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.522+0000 7f2875a00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2850078ef0 0x7f285007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:01.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.522+0000 7f2875a00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2850078ef0 0x7f285007b400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2864007fd0 tx=0x7f2864009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:01.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.525+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f285c03f030 con 0x7f28700af170 2024-09-21T07:50:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.720+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f287006de60 con 0x7f28700af170 2024-09-21T07:50:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.720+0000 7f286d600640 1 -- 172.21.15.88:0/3160871388 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) ==== 80+0+0 (secure 0 0 0) 0x7f285c084600 con 0x7f28700af170 2024-09-21T07:50:01.721 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.722+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2850078ef0 msgr2=0x7f285007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2850078ef0 0x7f285007b400 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f2864007fd0 tx=0x7f2864009040 comp rx=0 tx=0).stop 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 msgr2=0x7f2870158340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f2870158340 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f285c02aa70 tx=0x7f285c033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 shutdown_connections 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2850078ef0 0x7f285007b400 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f287015a0f0 0x7f2870155580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28700af170 0x7f2870158340 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 --2- 172.21.15.88:0/3160871388 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28700ad9c0 0x7f2870157000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:01.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 >> 172.21.15.88:0/3160871388 conn(0x7f28700b1e40 msgr2=0x7f28700b4060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:01.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 shutdown_connections 2024-09-21T07:50:01.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:01.723+0000 7f2876e00640 1 -- 172.21.15.88:0/3160871388 wait complete. 2024-09-21T07:50:01.746 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:50:01.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:01.746 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:50:01.836 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:50:01.837 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:01.837 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:50:01.884 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:01.885 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:01.885 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:01.922 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:01.922 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:01.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 -- 172.21.15.88:0/1262696860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c0705b0 msgr2=0x7ff84c0709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 --2- 172.21.15.88:0/1262696860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c0705b0 0x7ff84c0709c0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7ff838005fd0 tx=0x7ff83802a540 comp rx=0 tx=0).stop 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 -- 172.21.15.88:0/1262696860 shutdown_connections 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 --2- 172.21.15.88:0/1262696860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c0705b0 0x7ff84c0709c0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 --2- 172.21.15.88:0/1262696860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff84c0cfc80 0x7ff84c156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 -- 172.21.15.88:0/1262696860 >> 172.21.15.88:0/1262696860 conn(0x7ff84c06d3d0 msgr2=0x7ff84c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 -- 172.21.15.88:0/1262696860 shutdown_connections 2024-09-21T07:50:02.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.219+0000 7ff853e00640 1 -- 172.21.15.88:0/1262696860 wait complete. 2024-09-21T07:50:02.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.221+0000 7ff853e00640 1 Processor -- start 2024-09-21T07:50:02.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.222+0000 7ff853e00640 1 -- start start 2024-09-21T07:50:02.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff853e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c0705b0 0x7ff84c1f31b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff853e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff84c0cfc80 0x7ff84c0c9550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff853e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 0x7ff84c0c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff853e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c0cced0 con 0x7ff84c15ce30 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff853e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff84c0d0170 con 0x7ff84c0705b0 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff853e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff84c0cd050 con 0x7ff84c0cfc80 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff851e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 0x7ff84c0c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff851e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 0x7ff84c0c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40998/0 (socket says 172.21.15.88:40998) 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.223+0000 7ff851e00640 1 -- 172.21.15.88:0/2579825438 learned_addr learned my addr 172.21.15.88:0/2579825438 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff850a00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff84c0cfc80 0x7ff84c0c9550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851400640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c0705b0 0x7ff84c1f31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851e00640 1 -- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff84c0cfc80 msgr2=0x7ff84c0c9550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff84c0cfc80 0x7ff84c0c9550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851e00640 1 -- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c0705b0 msgr2=0x7ff84c1f31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c0705b0 0x7ff84c1f31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851e00640 1 -- 172.21.15.88:0/2579825438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff84c0741b0 con 0x7ff84c15ce30 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff851e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 0x7ff84c0c9a90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff84400c730 tx=0x7ff84400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844003b70 con 0x7ff84c15ce30 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff844003d10 con 0x7ff84c15ce30 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff84c074490 con 0x7ff84c15ce30 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.224+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff844005330 con 0x7ff84c15ce30 2024-09-21T07:50:02.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.225+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff84c0c3720 con 0x7ff84c15ce30 2024-09-21T07:50:02.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.226+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff84c073f20 con 0x7ff84c15ce30 2024-09-21T07:50:02.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.227+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ff8440054d0 con 0x7ff84c15ce30 2024-09-21T07:50:02.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.228+0000 7ff849600640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff82c078ef0 0x7ff82c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.228+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7ff84409b0e0 con 0x7ff84c15ce30 2024-09-21T07:50:02.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.228+0000 7ff851400640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff82c078ef0 0x7ff82c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.229+0000 7ff851400640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff82c078ef0 0x7ff82c07b400 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff84c09e510 tx=0x7ff840005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:02.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.232+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff84c070f00 con 0x7ff84c15ce30 2024-09-21T07:50:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.428+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff84c0cd570 con 0x7ff84c15ce30 2024-09-21T07:50:02.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.428+0000 7ff849600640 1 -- 172.21.15.88:0/2579825438 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v62) ==== 80+0+0 (secure 0 0 0) 0x7ff8440628c0 con 0x7ff84c15ce30 2024-09-21T07:50:02.430 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.430+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff82c078ef0 msgr2=0x7ff82c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.430+0000 7ff853e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff82c078ef0 0x7ff82c07b400 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7ff84c09e510 tx=0x7ff840005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.430+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 msgr2=0x7ff84c0c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.430+0000 7ff853e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 0x7ff84c0c9a90 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7ff84400c730 tx=0x7ff84400cb30 comp rx=0 tx=0).stop 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 shutdown_connections 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff82c078ef0 0x7ff82c07b400 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff84c0cfc80 0x7ff84c0c9550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff84c15ce30 0x7ff84c0c9a90 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 --2- 172.21.15.88:0/2579825438 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff84c0705b0 0x7ff84c1f31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 >> 172.21.15.88:0/2579825438 conn(0x7ff84c06d3d0 msgr2=0x7ff84c0d0a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 shutdown_connections 2024-09-21T07:50:02.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.431+0000 7ff853e00640 1 -- 172.21.15.88:0/2579825438 wait complete. 2024-09-21T07:50:02.455 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:50:02.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:02.456 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:50:02.520 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:50:02.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:02.521 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:50:02.557 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:02.557 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:02.557 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:02.593 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:02.593 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:02.593 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:02.593 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:02.633 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:02.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 -- 172.21.15.88:0/2656363219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 msgr2=0x7fb404162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 --2- 172.21.15.88:0/2656363219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 0x7fb404162a50 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb4040d2050 tx=0x7fb3f402a480 comp rx=0 tx=0).stop 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 -- 172.21.15.88:0/2656363219 shutdown_connections 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 --2- 172.21.15.88:0/2656363219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040d1580 0x7fb4040d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 --2- 172.21.15.88:0/2656363219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 0x7fb404162a50 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 -- 172.21.15.88:0/2656363219 >> 172.21.15.88:0/2656363219 conn(0x7fb40408bfe0 msgr2=0x7fb40409d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 -- 172.21.15.88:0/2656363219 shutdown_connections 2024-09-21T07:50:02.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.856+0000 7fb403e00640 1 -- 172.21.15.88:0/2656363219 wait complete. 2024-09-21T07:50:02.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.859+0000 7fb403e00640 1 Processor -- start 2024-09-21T07:50:02.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.859+0000 7fb403e00640 1 -- start start 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.860+0000 7fb403e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 0x7fb40414d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.860+0000 7fb403e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 0x7fb4041eb440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.860+0000 7fb403e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4040d1580 0x7fb4040a1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.860+0000 7fb403e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040d44b0 con 0x7fb4040d1580 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.860+0000 7fb403e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb4040d4330 con 0x7fb4040b9e00 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.860+0000 7fb403e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb4040d4630 con 0x7fb4040b3160 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 0x7fb40414d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb403400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4040d1580 0x7fb4040a1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 0x7fb40414d9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:44090/0 (socket says 172.21.15.88:44090) 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 -- 172.21.15.88:0/3062094093 learned_addr learned my addr 172.21.15.88:0/3062094093 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:02.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402000640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 0x7fb4041eb440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 -- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 msgr2=0x7fb4041eb440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 0x7fb4041eb440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 -- 172.21.15.88:0/3062094093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4040d1580 msgr2=0x7fb4040a1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4040d1580 0x7fb4040a1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 -- 172.21.15.88:0/3062094093 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4040a02a0 con 0x7fb4040b3160 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb403400640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4040d1580 0x7fb4040a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.861+0000 7fb402a00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 0x7fb40414d9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009780 tx=0x7fb3f4033c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.862+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4031d60 con 0x7fb4040b3160 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.862+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3f4033e10 con 0x7fb4040b3160 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.862+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3f4005260 con 0x7fb4040b3160 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.862+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4040a0580 con 0x7fb4040b3160 2024-09-21T07:50:02.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.862+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4040a0b40 con 0x7fb4040b3160 2024-09-21T07:50:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.863+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb4040d4b50 con 0x7fb4040b3160 2024-09-21T07:50:02.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.865+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fb3f4005400 con 0x7fb4040b3160 2024-09-21T07:50:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.865+0000 7fb3faa00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb3e4078f20 0x7fb3e407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:02.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.866+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7fb3f40bca10 con 0x7fb4040b3160 2024-09-21T07:50:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.866+0000 7fb402000640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb3e4078f20 0x7fb3e407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:02.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.866+0000 7fb402000640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb3e4078f20 0x7fb3e407b430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0007fd0 tx=0x7fb3f0001370 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:02.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:02.870+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb3f403f030 con 0x7fb4040b3160 2024-09-21T07:50:03.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.096+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb4040b4620 con 0x7fb4040b3160 2024-09-21T07:50:03.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.097+0000 7fb3faa00640 1 -- 172.21.15.88:0/3062094093 <== mon.2 v2:172.21.15.175:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7fb3f40841e0 con 0x7fb4040b3160 2024-09-21T07:50:03.098 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:03.099 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":37,"btime":"2024-09-21T07:49:46:300801+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:30.312143+0000","modified":"2024-09-21T07:49:33.340468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2365988221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2365988221},{"type":"v1","addr":"172.21.15.175:6837","nonce":2365988221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5763,"qdb_cluster":[5763]},"id":4}]} 2024-09-21T07:50:03.099 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 37 2024-09-21T07:50:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb3e4078f20 msgr2=0x7fb3e407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb3e4078f20 0x7fb3e407b430 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0007fd0 tx=0x7fb3f0001370 comp rx=0 tx=0).stop 2024-09-21T07:50:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 msgr2=0x7fb40414d9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 0x7fb40414d9d0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fb3f4009780 tx=0x7fb3f4033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:03.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 shutdown_connections 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb3e4078f20 0x7fb3e407b430 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb4040d1580 0x7fb4040a1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb4040b9e00 0x7fb4041eb440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 --2- 172.21.15.88:0/3062094093 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb4040b3160 0x7fb40414d9d0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 >> 172.21.15.88:0/3062094093 conn(0x7fb40408bfe0 msgr2=0x7fb4040d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 shutdown_connections 2024-09-21T07:50:03.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.100+0000 7fb403e00640 1 -- 172.21.15.88:0/3062094093 wait complete. 2024-09-21T07:50:03.124 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.343+0000 7f0c2f400640 1 -- 172.21.15.88:0/2715385596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0039e0 msgr2=0x7f0c0c003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.343+0000 7f0c2f400640 1 --2- 172.21.15.88:0/2715385596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c0c0039e0 0x7f0c0c003e10 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f0c1800c530 tx=0x7f0c0c005830 comp rx=0 tx=0).stop 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.343+0000 7f0c2d400640 1 -- 172.21.15.88:0/2715385596 reap_dead start 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.343+0000 7f0c2f400640 1 -- 172.21.15.88:0/2715385596 shutdown_connections 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.343+0000 7f0c2f400640 1 -- 172.21.15.88:0/2715385596 >> 172.21.15.88:0/2715385596 conn(0x7f0c2806d730 msgr2=0x7f0c280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.344+0000 7f0c2f400640 1 -- 172.21.15.88:0/2715385596 shutdown_connections 2024-09-21T07:50:03.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.344+0000 7f0c2f400640 1 -- 172.21.15.88:0/2715385596 wait complete. 2024-09-21T07:50:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.346+0000 7f0c2f400640 1 Processor -- start 2024-09-21T07:50:03.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.346+0000 7f0c2f400640 1 -- start start 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.347+0000 7f0c2f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c0039e0 0x7f0c2814c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.347+0000 7f0c2f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b1240 0x7f0c281426d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.347+0000 7f0c2f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 0x7f0c2815ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.347+0000 7f0c2f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0c280b4110 con 0x7f0c281487b0 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.347+0000 7f0c2f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0c280ae6e0 con 0x7f0c0c0039e0 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.347+0000 7f0c2f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0c280b4290 con 0x7f0c280b1240 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 0x7f0c2815ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 0x7f0c2815ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41016/0 (socket says 172.21.15.88:41016) 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 -- 172.21.15.88:0/3898225059 learned_addr learned my addr 172.21.15.88:0/3898225059 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2ca00640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c0039e0 0x7f0c2814c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c27e00640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b1240 0x7f0c281426d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 -- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b1240 msgr2=0x7f0c281426d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b1240 0x7f0c281426d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 -- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c0039e0 msgr2=0x7f0c2814c7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c0039e0 0x7f0c2814c7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 -- 172.21.15.88:0/3898225059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c281f87d0 con 0x7f0c281487b0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c27e00640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b1240 0x7f0c281426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.348+0000 7f0c2d400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 0x7f0c2815ce10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c004780 tx=0x7f0c20001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.349+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011430 con 0x7f0c281487b0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.349+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c20016020 con 0x7f0c281487b0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.349+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c20011870 con 0x7f0c281487b0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.349+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c281f8ab0 con 0x7f0c281487b0 2024-09-21T07:50:03.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.349+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c281f9070 con 0x7f0c281487b0 2024-09-21T07:50:03.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.350+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c281f4f00 con 0x7f0c281487b0 2024-09-21T07:50:03.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.351+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0c20011a10 con 0x7f0c281487b0 2024-09-21T07:50:03.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.352+0000 7f0c24c00640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0c08078f20 0x7f0c0807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.352+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f0c2009bd30 con 0x7f0c281487b0 2024-09-21T07:50:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.353+0000 7f0c2ca00640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0c08078f20 0x7f0c0807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.354+0000 7f0c2ca00640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0c08078f20 0x7f0c0807b430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0c18033000 tx=0x7f0c18001cb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:03.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.356+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c281f4f00 con 0x7f0c281487b0 2024-09-21T07:50:03.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.579+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0c2815b090 con 0x7f0c281487b0 2024-09-21T07:50:03.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.580+0000 7f0c24c00640 1 -- 172.21.15.88:0/3898225059 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7f0c20020020 con 0x7f0c281487b0 2024-09-21T07:50:03.581 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:03.581 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":37,"btime":"2024-09-21T07:49:46:300801+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:30.312143+0000","modified":"2024-09-21T07:49:33.340468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2365988221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2365988221},{"type":"v1","addr":"172.21.15.175:6837","nonce":2365988221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5763,"qdb_cluster":[5763]},"id":4}]} 2024-09-21T07:50:03.581 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 37 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0c08078f20 msgr2=0x7f0c0807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0c08078f20 0x7f0c0807b430 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0c18033000 tx=0x7f0c18001cb0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 msgr2=0x7f0c2815ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 0x7f0c2815ce10 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c004780 tx=0x7f0c20001710 comp rx=0 tx=0).stop 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 shutdown_connections 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0c08078f20 0x7f0c0807b430 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0c280b1240 0x7f0c281426d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c281487b0 0x7f0c2815ce10 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 --2- 172.21.15.88:0/3898225059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0c0c0039e0 0x7f0c2814c7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.582+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 >> 172.21.15.88:0/3898225059 conn(0x7f0c2806d730 msgr2=0x7f0c2815b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.583+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 shutdown_connections 2024-09-21T07:50:03.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.583+0000 7f0c2f400640 1 -- 172.21.15.88:0/3898225059 wait complete. 2024-09-21T07:50:03.608 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.828+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1861248244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc0ad0e0 msgr2=0x7fd5bc0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.828+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1861248244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc0ad0e0 0x7fd5bc0ad4f0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc075250 tx=0x7fd5a802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1861248244 shutdown_connections 2024-09-21T07:50:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1861248244 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc160310 0x7fd5bc160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1861248244 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc0ad0e0 0x7fd5bc0ad4f0 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1861248244 >> v2:172.21.15.175:3300/0 conn(0x7fd5bc15d760 0x7fd5bc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1861248244 >> 172.21.15.88:0/1861248244 conn(0x7fd5bc06d730 msgr2=0x7fd5bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1861248244 shutdown_connections 2024-09-21T07:50:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.829+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1861248244 wait complete. 2024-09-21T07:50:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.831+0000 7fd5c1c00640 1 Processor -- start 2024-09-21T07:50:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.831+0000 7fd5c1c00640 1 -- start start 2024-09-21T07:50:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5c1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0ad0e0 0x7fd5bc156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5c1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 0x7fd5bc1ea5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5c1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc160310 0x7fd5bc1f4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5c1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b4da0 con 0x7fd5bc15d760 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5c1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5bc0b4c20 con 0x7fd5bc0ad0e0 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5c1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd5bc160c60 con 0x7fd5bc160310 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 0x7fd5bc1ea5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 0x7fd5bc1ea5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41038/0 (socket says 172.21.15.88:41038) 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 -- 172.21.15.88:0/1152139148 learned_addr learned my addr 172.21.15.88:0/1152139148 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5baa00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0ad0e0 0x7fd5bc156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5bb400640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc160310 0x7fd5bc1f4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 -- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc160310 msgr2=0x7fd5bc1f4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc160310 0x7fd5bc1f4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 -- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0ad0e0 msgr2=0x7fd5bc156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0ad0e0 0x7fd5bc156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.833+0000 7fd5ba000640 1 -- 172.21.15.88:0/1152139148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5bc155280 con 0x7fd5bc15d760 2024-09-21T07:50:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.834+0000 7fd5ba000640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 0x7fd5bc1ea5b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8034040 tx=0x7fd5a8003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.834+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a803e030 con 0x7fd5bc15d760 2024-09-21T07:50:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.834+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd5a8003e00 con 0x7fd5bc15d760 2024-09-21T07:50:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.834+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd5a8035040 con 0x7fd5bc15d760 2024-09-21T07:50:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.834+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5bc155560 con 0x7fd5bc15d760 2024-09-21T07:50:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.834+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5bc155b20 con 0x7fd5bc15d760 2024-09-21T07:50:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.835+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5bc1535c0 con 0x7fd5bc15d760 2024-09-21T07:50:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.836+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fd5a8041030 con 0x7fd5bc15d760 2024-09-21T07:50:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.837+0000 7fd5b2a00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd59c078ef0 0x7fd59c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.837+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7fd5a80bbb20 con 0x7fd5bc15d760 2024-09-21T07:50:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.838+0000 7fd5baa00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd59c078ef0 0x7fd59c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.838+0000 7fd5baa00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd59c078ef0 0x7fd59c07b400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005d70 tx=0x7fd5ac005d00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:03.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:03.842+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd5a8083ae0 con 0x7fd5bc15d760 2024-09-21T07:50:04.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.028+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd5bc15cd90 con 0x7fd5bc15d760 2024-09-21T07:50:04.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.029+0000 7fd5b2a00640 1 -- 172.21.15.88:0/1152139148 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16833 (secure 0 0 0) 0x7fd5a808c020 con 0x7fd5bc15d760 2024-09-21T07:50:04.031 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:04.031 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":62,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:49:47.306599+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:28.221593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:29.283162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.27","mappings":[{"from":6,"to":1}]},{"pgid":"9.37","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd59c078ef0 msgr2=0x7fd59c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd59c078ef0 0x7fd59c07b400 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fd5ac005d70 tx=0x7fd5ac005d00 comp rx=0 tx=0).stop 2024-09-21T07:50:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 msgr2=0x7fd5bc1ea5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 0x7fd5bc1ea5b0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fd5a8034040 tx=0x7fd5a8003910 comp rx=0 tx=0).stop 2024-09-21T07:50:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 shutdown_connections 2024-09-21T07:50:04.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd59c078ef0 0x7fd59c07b400 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5bc160310 0x7fd5bc1f4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd5bc15d760 0x7fd5bc1ea5b0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 --2- 172.21.15.88:0/1152139148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd5bc0ad0e0 0x7fd5bc156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 >> 172.21.15.88:0/1152139148 conn(0x7fd5bc06d730 msgr2=0x7fd5bc0b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 shutdown_connections 2024-09-21T07:50:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.032+0000 7fd5c1c00640 1 -- 172.21.15.88:0/1152139148 wait complete. 2024-09-21T07:50:04.056 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:50:04.057 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 -- 172.21.15.88:0/1344402307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 msgr2=0x7fda7c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 --2- 172.21.15.88:0/1344402307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 0x7fda7c0b3960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fda74005fd0 tx=0x7fda7402aa10 comp rx=0 tx=0).stop 2024-09-21T07:50:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 -- 172.21.15.88:0/1344402307 shutdown_connections 2024-09-21T07:50:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 --2- 172.21.15.88:0/1344402307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 0x7fda7c0b3960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 --2- 172.21.15.88:0/1344402307 >> v2:172.21.15.175:3300/0 conn(0x7fda7c15d760 0x7fda7c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 -- 172.21.15.88:0/1344402307 >> 172.21.15.88:0/1344402307 conn(0x7fda7c06d730 msgr2=0x7fda7c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 -- 172.21.15.88:0/1344402307 shutdown_connections 2024-09-21T07:50:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.276+0000 7fda83600640 1 -- 172.21.15.88:0/1344402307 wait complete. 2024-09-21T07:50:04.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.278+0000 7fda83600640 1 Processor -- start 2024-09-21T07:50:04.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.278+0000 7fda83600640 1 -- start start 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda83600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 0x7fda7c073bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda83600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 0x7fda7c1f7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda83600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda7c15d760 0x7fda7c1fa3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda83600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c06f240 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda83600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda7c0b4e30 con 0x7fda7c15d760 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda83600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda7c0b3ea0 con 0x7fda7c0b3550 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda80c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 0x7fda7c073bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda80c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 0x7fda7c073bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41042/0 (socket says 172.21.15.88:41042) 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda80c00640 1 -- 172.21.15.88:0/2318054953 learned_addr learned my addr 172.21.15.88:0/2318054953 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda7be00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 0x7fda7c1f7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.280+0000 7fda81600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda7c15d760 0x7fda7c1fa3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda80c00640 1 -- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 msgr2=0x7fda7c1f7790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda80c00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 0x7fda7c1f7790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda80c00640 1 -- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda7c15d760 msgr2=0x7fda7c1fa3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda80c00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda7c15d760 0x7fda7c1fa3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda80c00640 1 -- 172.21.15.88:0/2318054953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda7c1fb290 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda80c00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 0x7fda7c073bc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e790 tx=0x7fda7000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda7000eea0 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fda7000cdb0 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda7c099ce0 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.281+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda70010690 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.282+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda7c0730e0 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.283+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda7c0b42f0 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.284+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fda70010830 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.285+0000 7fda78c00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fda5c078ef0 0x7fda5c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.285+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7fda7009bdf0 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.285+0000 7fda7be00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fda5c078ef0 0x7fda5c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.286+0000 7fda7be00640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fda5c078ef0 0x7fda5c07b400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0cf740 tx=0x7fda68009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.289+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda70063640 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.508+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fda7c160f00 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.509+0000 7fda78c00640 1 -- 172.21.15.88:0/2318054953 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7fda70068070 con 0x7fda7c0ad0e0 2024-09-21T07:50:04.510 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:04.510 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":37,"btime":"2024-09-21T07:49:46:300801+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:30.312143+0000","modified":"2024-09-21T07:49:33.340468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2365988221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2365988221},{"type":"v1","addr":"172.21.15.175:6837","nonce":2365988221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5763,"qdb_cluster":[5763]},"id":4}]} 2024-09-21T07:50:04.511 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 37 2024-09-21T07:50:04.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fda5c078ef0 msgr2=0x7fda5c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fda5c078ef0 0x7fda5c07b400 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fda7c0cf740 tx=0x7fda68009040 comp rx=0 tx=0).stop 2024-09-21T07:50:04.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 msgr2=0x7fda7c073bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 0x7fda7c073bc0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fda7000e790 tx=0x7fda7000ec50 comp rx=0 tx=0).stop 2024-09-21T07:50:04.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 shutdown_connections 2024-09-21T07:50:04.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fda5c078ef0 0x7fda5c07b400 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda7c0b3550 0x7fda7c1f7790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda7c15d760 0x7fda7c1fa3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 --2- 172.21.15.88:0/2318054953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda7c0ad0e0 0x7fda7c073bc0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 >> 172.21.15.88:0/2318054953 conn(0x7fda7c06d730 msgr2=0x7fda7c15dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:04.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 shutdown_connections 2024-09-21T07:50:04.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.512+0000 7fda83600640 1 -- 172.21.15.88:0/2318054953 wait complete. 2024-09-21T07:50:04.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f01a6200640 1 -- 172.21.15.88:0/2690705398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01840039e0 msgr2=0x7f0184003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f01a6200640 1 --2- 172.21.15.88:0/2690705398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01840039e0 0x7f0184003e10 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f0190009b50 tx=0x7f0184005830 comp rx=0 tx=0).stop 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f019fe00640 1 -- 172.21.15.88:0/2690705398 reap_dead start 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f01a6200640 1 -- 172.21.15.88:0/2690705398 shutdown_connections 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f01a6200640 1 -- 172.21.15.88:0/2690705398 >> 172.21.15.88:0/2690705398 conn(0x7f01a006d730 msgr2=0x7f01a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f01a6200640 1 -- 172.21.15.88:0/2690705398 shutdown_connections 2024-09-21T07:50:04.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.754+0000 7f01a6200640 1 -- 172.21.15.88:0/2690705398 wait complete. 2024-09-21T07:50:04.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.757+0000 7f01a6200640 1 Processor -- start 2024-09-21T07:50:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.757+0000 7f01a6200640 1 -- start start 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.758+0000 7f01a6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01840039e0 0x7f01a00cef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.758+0000 7f01a6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a01586c0 0x7f01a0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.758+0000 7f01a6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 0x7f01a01f8890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.758+0000 7f01a6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00b3fe0 con 0x7f01a0073580 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.758+0000 7f01a6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f01a00b3e60 con 0x7f01a01586c0 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.758+0000 7f01a6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f01a00b4160 con 0x7f01840039e0 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 0x7f01a01f8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 0x7f01a01f8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41060/0 (socket says 172.21.15.88:41060) 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 -- 172.21.15.88:0/1356462114 learned_addr learned my addr 172.21.15.88:0/1356462114 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019f400640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01840039e0 0x7f01a00cef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019ea00640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a01586c0 0x7f01a0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 -- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01840039e0 msgr2=0x7f01a00cef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01840039e0 0x7f01a00cef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 -- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a01586c0 msgr2=0x7f01a0073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a01586c0 0x7f01a0073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 -- 172.21.15.88:0/1356462114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01a0157620 con 0x7f01a0073580 2024-09-21T07:50:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f019fe00640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 0x7f01a01f8890 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0198000f80 tx=0x7f019800ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.759+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019800c910 con 0x7f01a0073580 2024-09-21T07:50:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.760+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0198017070 con 0x7f01a0073580 2024-09-21T07:50:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.760+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01a0157900 con 0x7f01a0073580 2024-09-21T07:50:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.760+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019800ce80 con 0x7f01a0073580 2024-09-21T07:50:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.760+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01a0157ec0 con 0x7f01a0073580 2024-09-21T07:50:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.761+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01a00b46f0 con 0x7f01a0073580 2024-09-21T07:50:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.762+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f019801c460 con 0x7f01a0073580 2024-09-21T07:50:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.763+0000 7f0197400640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0180078ef0 0x7f018007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.763+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f019809b580 con 0x7f01a0073580 2024-09-21T07:50:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.763+0000 7f019f400640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0180078ef0 0x7f018007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.764+0000 7f019f400640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0180078ef0 0x7f018007b400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0190035000 tx=0x7f019002d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:04.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.767+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0198062e60 con 0x7f01a0073580 2024-09-21T07:50:04.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.992+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f01a00b21d0 con 0x7f01a0073580 2024-09-21T07:50:04.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.992+0000 7f0197400640 1 -- 172.21.15.88:0/1356462114 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 37 v37) ==== 94+0+5226 (secure 0 0 0) 0x7f0198067890 con 0x7f01a0073580 2024-09-21T07:50:04.994 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:04.994 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":37,"btime":"2024-09-21T07:49:46:300801+0000","default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[{"mdsmap":{"epoch":36,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:49:30.312143+0000","modified":"2024-09-21T07:49:33.340468+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":5763},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5763":{"gid":5763,"name":"d","rank":0,"incarnation":32,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6837/2365988221","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2365988221},{"type":"v1","addr":"172.21.15.175:6837","nonce":2365988221}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":5763,"qdb_cluster":[5763]},"id":4}]} 2024-09-21T07:50:04.994 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 37 2024-09-21T07:50:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0180078ef0 msgr2=0x7f018007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0180078ef0 0x7f018007b400 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0190035000 tx=0x7f019002d040 comp rx=0 tx=0).stop 2024-09-21T07:50:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 msgr2=0x7f01a01f8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:04.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 0x7f01a01f8890 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f0198000f80 tx=0x7f019800ea90 comp rx=0 tx=0).stop 2024-09-21T07:50:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 shutdown_connections 2024-09-21T07:50:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0180078ef0 0x7f018007b400 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f01a0073580 0x7f01a01f8890 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f01a01586c0 0x7f01a0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 --2- 172.21.15.88:0/1356462114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f01840039e0 0x7f01a00cef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:04.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 >> 172.21.15.88:0/1356462114 conn(0x7f01a006d730 msgr2=0x7f01a015cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:04.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 shutdown_connections 2024-09-21T07:50:04.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:04.995+0000 7f01a6200640 1 -- 172.21.15.88:0/1356462114 wait complete. 2024-09-21T07:50:05.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325f200640 1 -- 172.21.15.88:0/541170783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3244004d90 msgr2=0x7f3244005240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325f200640 1 --2- 172.21.15.88:0/541170783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3244004d90 0x7f3244005240 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f325800c690 tx=0x7f3244005830 comp rx=0 tx=0).stop 2024-09-21T07:50:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325e800640 1 -- 172.21.15.88:0/541170783 reap_dead start 2024-09-21T07:50:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325f200640 1 -- 172.21.15.88:0/541170783 shutdown_connections 2024-09-21T07:50:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325f200640 1 -- 172.21.15.88:0/541170783 >> 172.21.15.88:0/541170783 conn(0x7f32600d44a0 msgr2=0x7f32600d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325f200640 1 -- 172.21.15.88:0/541170783 shutdown_connections 2024-09-21T07:50:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.241+0000 7f325f200640 1 -- 172.21.15.88:0/541170783 wait complete. 2024-09-21T07:50:05.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.243+0000 7f325f200640 1 Processor -- start 2024-09-21T07:50:05.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.243+0000 7f325f200640 1 -- start start 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004d90 0x7f32601501f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32601508b0 0x7f3260150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 0x7f32600ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f32600d26a0 con 0x7f32600ca4c0 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f32600d2520 con 0x7f3244004d90 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f32600d0d10 con 0x7f32601508b0 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 0x7f32600ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 0x7f32600ca8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41082/0 (socket says 172.21.15.88:41082) 2024-09-21T07:50:05.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 -- 172.21.15.88:0/2733271880 learned_addr learned my addr 172.21.15.88:0/2733271880 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:05.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325de00640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004d90 0x7f32601501f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f3255400640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32601508b0 0x7f3260150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 -- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32601508b0 msgr2=0x7f3260150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32601508b0 0x7f3260150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 -- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004d90 msgr2=0x7f32601501f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004d90 0x7f32601501f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.245+0000 7f325e800640 1 -- 172.21.15.88:0/2733271880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32600cae10 con 0x7f32600ca4c0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.246+0000 7f325e800640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 0x7f32600ca8d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f325802b400 tx=0x7f3258004bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.246+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f325802bb20 con 0x7f32600ca4c0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.246+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f325802bdb0 con 0x7f32600ca4c0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.246+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3260152640 con 0x7f32600ca4c0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.246+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32580062b0 con 0x7f32600ca4c0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.246+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3260151880 con 0x7f32600ca4c0 2024-09-21T07:50:05.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.247+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3260151310 con 0x7f32600ca4c0 2024-09-21T07:50:05.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.248+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f3258006450 con 0x7f32600ca4c0 2024-09-21T07:50:05.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.250+0000 7f3256a00640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f323c078ef0 0x7f323c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.250+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f32580bb6a0 con 0x7f32600ca4c0 2024-09-21T07:50:05.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.250+0000 7f325de00640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f323c078ef0 0x7f323c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.251+0000 7f325de00640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f323c078ef0 0x7f323c07b400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3250000c00 tx=0x7f3250006b60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:05.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.254+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f32600cae10 con 0x7f32600ca4c0 2024-09-21T07:50:05.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.441+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3260162230 con 0x7f32600ca4c0 2024-09-21T07:50:05.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.442+0000 7f3256a00640 1 -- 172.21.15.88:0/2733271880 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v62) ==== 74+0+16833 (secure 0 0 0) 0x7f3260162230 con 0x7f32600ca4c0 2024-09-21T07:50:05.443 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:05.443 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":62,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:49:47.306599+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:49:28.221593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2024-09-21T07:49:29.283162+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"9.27","mappings":[{"from":6,"to":1}]},{"pgid":"9.37","mappings":[{"from":6,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f323c078ef0 msgr2=0x7f323c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f323c078ef0 0x7f323c07b400 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f3250000c00 tx=0x7f3250006b60 comp rx=0 tx=0).stop 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 msgr2=0x7f32600ca8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 0x7f32600ca8d0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f325802b400 tx=0x7f3258004bd0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 shutdown_connections 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f323c078ef0 0x7f323c07b400 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f32601508b0 0x7f3260150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32600ca4c0 0x7f32600ca8d0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 --2- 172.21.15.88:0/2733271880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3244004d90 0x7f32601501f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 >> 172.21.15.88:0/2733271880 conn(0x7f32600d44a0 msgr2=0x7f326006de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.444+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 shutdown_connections 2024-09-21T07:50:05.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.445+0000 7f325f200640 1 -- 172.21.15.88:0/2733271880 wait complete. 2024-09-21T07:50:05.468 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:50:05.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 -- 172.21.15.88:0/1869453337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2838005320 msgr2=0x7f283800b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 --2- 172.21.15.88:0/1869453337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2838005320 0x7f283800b9e0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f28540cf0d0 tx=0x7f2838005830 comp rx=0 tx=0).stop 2024-09-21T07:50:05.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 -- 172.21.15.88:0/1869453337 shutdown_connections 2024-09-21T07:50:05.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 --2- 172.21.15.88:0/1869453337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2838005320 0x7f283800b9e0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 -- 172.21.15.88:0/1869453337 >> 172.21.15.88:0/1869453337 conn(0x7f285406d3d0 msgr2=0x7f28540b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:05.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 -- 172.21.15.88:0/1869453337 shutdown_connections 2024-09-21T07:50:05.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.688+0000 7f2859800640 1 -- 172.21.15.88:0/1869453337 wait complete. 2024-09-21T07:50:05.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.691+0000 7f2859800640 1 Processor -- start 2024-09-21T07:50:05.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.691+0000 7f2859800640 1 -- start start 2024-09-21T07:50:05.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2859800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2838005320 0x7f2854159630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2859800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28540aeb90 0x7f28540aefa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2859800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 0x7f285415e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2859800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28540709c0 con 0x7f285415de40 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2859800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2854157320 con 0x7f2838005320 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2859800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2854070b40 con 0x7f28540aeb90 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.692+0000 7f2853400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 0x7f285415e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 0x7f285415e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41096/0 (socket says 172.21.15.88:41096) 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 -- 172.21.15.88:0/1071235928 learned_addr learned my addr 172.21.15.88:0/1071235928 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2852a00640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2838005320 0x7f2854159630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f284be00640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28540aeb90 0x7f28540aefa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 -- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28540aeb90 msgr2=0x7f28540aefa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28540aeb90 0x7f28540aefa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 -- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2838005320 msgr2=0x7f2854159630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2838005320 0x7f2854159630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 -- 172.21.15.88:0/1071235928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f285415e790 con 0x7f285415de40 2024-09-21T07:50:05.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f2853400640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 0x7f285415e250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c650 tx=0x7f284c001c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.693+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c02b700 con 0x7f285415de40 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.694+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f284c02b990 con 0x7f285415de40 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.694+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f284c02bde0 con 0x7f285415de40 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.694+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28540ad5a0 con 0x7f285415de40 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.694+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28540adb60 con 0x7f285415de40 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.695+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28540ad310 con 0x7f285415de40 2024-09-21T07:50:05.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.696+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f284c006070 con 0x7f285415de40 2024-09-21T07:50:05.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.697+0000 7f284aa00640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2830078ef0 0x7f283007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:05.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.697+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5767+0+0 (secure 0 0 0) 0x7f284c0bb420 con 0x7f285415de40 2024-09-21T07:50:05.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.698+0000 7f2852a00640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2830078ef0 0x7f283007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:05.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.698+0000 7f2852a00640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2830078ef0 0x7f283007b400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f285409e510 tx=0x7f2844005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:05.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.702+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f285415e790 con 0x7f285415de40 2024-09-21T07:50:05.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.928+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f285415d240 con 0x7f285415de40 2024-09-21T07:50:05.930 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:50:05.929+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:50:05.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.931+0000 7f284aa00640 1 -- 172.21.15.88:0/1071235928 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v38) ==== 161+0+0 (secure 0 0 0) 0x7f284c083380 con 0x7f285415de40 2024-09-21T07:50:05.932 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:50:05.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2830078ef0 msgr2=0x7f283007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2830078ef0 0x7f283007b400 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f285409e510 tx=0x7f2844005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 msgr2=0x7f285415e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 0x7f285415e250 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f284c00c650 tx=0x7f284c001c40 comp rx=0 tx=0).stop 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 shutdown_connections 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2830078ef0 0x7f283007b400 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28540aeb90 0x7f28540aefa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f285415de40 0x7f285415e250 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 --2- 172.21.15.88:0/1071235928 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2838005320 0x7f2854159630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.934+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 >> 172.21.15.88:0/1071235928 conn(0x7f285406d3d0 msgr2=0x7f285415f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.935+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 shutdown_connections 2024-09-21T07:50:05.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:05.935+0000 7f2859800640 1 -- 172.21.15.88:0/1071235928 wait complete. 2024-09-21T07:50:05.957 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:50:06.076 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.179+0000 7f6b9a000640 1 -- 172.21.15.88:0/1674726135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940ce800 msgr2=0x7f6b9409aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.179+0000 7f6b9a000640 1 --2- 172.21.15.88:0/1674726135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940ce800 0x7f6b9409aa50 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f6b84005fd0 tx=0x7f6b8402a540 comp rx=0 tx=0).stop 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.179+0000 7f6b9a000640 1 -- 172.21.15.88:0/1674726135 shutdown_connections 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.179+0000 7f6b9a000640 1 --2- 172.21.15.88:0/1674726135 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b9409af90 0x7f6b940ccce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.179+0000 7f6b9a000640 1 --2- 172.21.15.88:0/1674726135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940ce800 0x7f6b9409aa50 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.179+0000 7f6b9a000640 1 -- 172.21.15.88:0/1674726135 >> 172.21.15.88:0/1674726135 conn(0x7f6b940d2550 msgr2=0x7f6b940d2960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.180+0000 7f6b9a000640 1 -- 172.21.15.88:0/1674726135 shutdown_connections 2024-09-21T07:50:06.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.180+0000 7f6b9a000640 1 -- 172.21.15.88:0/1674726135 wait complete. 2024-09-21T07:50:06.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.182+0000 7f6b9a000640 1 Processor -- start 2024-09-21T07:50:06.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.182+0000 7f6b9a000640 1 -- start start 2024-09-21T07:50:06.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.183+0000 7f6b9a000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b9409af90 0x7f6b941f9990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.183+0000 7f6b9a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 0x7f6b940c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.183+0000 7f6b9a000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b940ce800 0x7f6b940c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.183+0000 7f6b9a000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b940b6450 con 0x7f6b940cd220 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.183+0000 7f6b9a000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b940cee20 con 0x7f6b940ce800 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.183+0000 7f6b9a000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b940b65d0 con 0x7f6b9409af90 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 0x7f6b940c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 0x7f6b940c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36874/0 (socket says 172.21.15.88:36874) 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 -- 172.21.15.88:0/625960955 learned_addr learned my addr 172.21.15.88:0/625960955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b93400640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b9409af90 0x7f6b941f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b93e00640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b940ce800 0x7f6b940c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:06.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 -- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b9409af90 msgr2=0x7f6b941f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b9409af90 0x7f6b941f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 -- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b940ce800 msgr2=0x7f6b940c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b940ce800 0x7f6b940c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 -- 172.21.15.88:0/625960955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b94162860 con 0x7f6b940cd220 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b92a00640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 0x7f6b940c8540 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009fd0 tx=0x7f6b8000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.184+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000ccc0 con 0x7f6b940cd220 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.185+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b8000ee70 con 0x7f6b940cd220 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.185+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b94164560 con 0x7f6b940cd220 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.185+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b8000ce60 con 0x7f6b940cd220 2024-09-21T07:50:06.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.185+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b941f9190 con 0x7f6b940cd220 2024-09-21T07:50:06.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.186+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b941642d0 con 0x7f6b940cd220 2024-09-21T07:50:06.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.187+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f6b80005260 con 0x7f6b940cd220 2024-09-21T07:50:06.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.188+0000 7f6b8b400640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6b74078f20 0x7f6b7407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:06.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.188+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5853+0+0 (secure 0 0 0) 0x7f6b8009bc90 con 0x7f6b940cd220 2024-09-21T07:50:06.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.188+0000 7f6b93400640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6b74078f20 0x7f6b7407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:06.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.189+0000 7f6b93400640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6b74078f20 0x7f6b7407b430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b84033000 tx=0x7f6b84004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:06.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.192+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b80063420 con 0x7f6b940cd220 2024-09-21T07:50:06.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.419+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f6b940ccce0 con 0x7f6b940cd220 2024-09-21T07:50:06.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.933+0000 7f6b8b400640 1 -- 172.21.15.88:0/625960955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v39) ==== 104+0+0 (secure 0 0 0) 0x7f6b80067e50 con 0x7f6b940cd220 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.935+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6b74078f20 msgr2=0x7f6b7407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.935+0000 7f6b9a000640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6b74078f20 0x7f6b7407b430 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6b84033000 tx=0x7f6b84004730 comp rx=0 tx=0).stop 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.935+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 msgr2=0x7f6b940c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.935+0000 7f6b9a000640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 0x7f6b940c8540 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f6b80009fd0 tx=0x7f6b8000ec20 comp rx=0 tx=0).stop 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 shutdown_connections 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6b74078f20 0x7f6b7407b430 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b940ce800 0x7f6b940c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b940cd220 0x7f6b940c8540 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 --2- 172.21.15.88:0/625960955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b9409af90 0x7f6b941f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:06.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 >> 172.21.15.88:0/625960955 conn(0x7f6b940d2550 msgr2=0x7f6b940c52b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:06.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 shutdown_connections 2024-09-21T07:50:06.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:06.936+0000 7f6b9a000640 1 -- 172.21.15.88:0/625960955 wait complete. 2024-09-21T07:50:06.961 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:50:07.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 -- 172.21.15.88:0/3877194408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c06d4b0 msgr2=0x7fa38c0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:07.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 --2- 172.21.15.88:0/3877194408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c06d4b0 0x7fa38c0b4620 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fa38c071040 tx=0x7fa37402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 -- 172.21.15.88:0/3877194408 shutdown_connections 2024-09-21T07:50:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 --2- 172.21.15.88:0/3877194408 >> v2:172.21.15.175:3300/0 conn(0x7fa38c0af100 0x7fa38c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 --2- 172.21.15.88:0/3877194408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c06d4b0 0x7fa38c0b4620 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 -- 172.21.15.88:0/3877194408 >> 172.21.15.88:0/3877194408 conn(0x7fa38c0b2540 msgr2=0x7fa38c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 -- 172.21.15.88:0/3877194408 shutdown_connections 2024-09-21T07:50:07.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.187+0000 7fa392600640 1 -- 172.21.15.88:0/3877194408 wait complete. 2024-09-21T07:50:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.189+0000 7fa392600640 1 Processor -- start 2024-09-21T07:50:07.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.190+0000 7fa392600640 1 -- start start 2024-09-21T07:50:07.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa392600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 0x7fa38c153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:07.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa392600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa38c0cc830 0x7fa38c098980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:07.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa392600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa38c09e020 0x7fa38c098ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:07.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa392600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c0b3620 con 0x7fa38c0af100 2024-09-21T07:50:07.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa392600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa38c0b34a0 con 0x7fa38c0cc830 2024-09-21T07:50:07.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa392600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa38c0b37a0 con 0x7fa38c09e020 2024-09-21T07:50:07.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa38b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 0x7fa38c153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:07.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa38b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 0x7fa38c153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36888/0 (socket says 172.21.15.88:36888) 2024-09-21T07:50:07.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.191+0000 7fa38b400640 1 -- 172.21.15.88:0/3393848499 learned_addr learned my addr 172.21.15.88:0/3393848499 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa38b400640 1 -- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa38c09e020 msgr2=0x7fa38c098ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa38b400640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa38c09e020 0x7fa38c098ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa382a00640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa38c0cc830 0x7fa38c098980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa38b400640 1 -- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa38c0cc830 msgr2=0x7fa38c098980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa38b400640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa38c0cc830 0x7fa38c098980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa38b400640 1 -- 172.21.15.88:0/3393848499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa38c154a30 con 0x7fa38c0af100 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa382a00640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa38c0cc830 0x7fa38c098980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa38b400640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 0x7fa38c153f90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c3a0 tx=0x7fa37c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c017070 con 0x7fa38c0af100 2024-09-21T07:50:07.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa37c0053c0 con 0x7fa38c0af100 2024-09-21T07:50:07.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c003850 con 0x7fa38c0af100 2024-09-21T07:50:07.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa38c1f5ae0 con 0x7fa38c0af100 2024-09-21T07:50:07.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.192+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa38c1f60a0 con 0x7fa38c0af100 2024-09-21T07:50:07.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.193+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa38c0b3cc0 con 0x7fa38c0af100 2024-09-21T07:50:07.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.195+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa37c01a070 con 0x7fa38c0af100 2024-09-21T07:50:07.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.196+0000 7fa383e00640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa368078ef0 0x7fa36807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:07.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.196+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(63..63 src has 1..63) ==== 5853+0+0 (secure 0 0 0) 0x7fa37c09a7e0 con 0x7fa38c0af100 2024-09-21T07:50:07.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.196+0000 7fa382a00640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa368078ef0 0x7fa36807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:07.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.197+0000 7fa382a00640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa368078ef0 0x7fa36807b400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa374033000 tx=0x7fa374004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:07.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.200+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa37c062650 con 0x7fa38c0af100 2024-09-21T07:50:07.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.385+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa38c0d1030 con 0x7fa38c0af100 2024-09-21T07:50:07.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.935+0000 7fa383e00640 1 -- 172.21.15.88:0/3393848499 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v64) ==== 181+0+0 (secure 0 0 0) 0x7fa37c061f50 con 0x7fa38c0af100 2024-09-21T07:50:07.936 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa368078ef0 msgr2=0x7fa36807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa368078ef0 0x7fa36807b400 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa374033000 tx=0x7fa374004730 comp rx=0 tx=0).stop 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 msgr2=0x7fa38c153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 0x7fa38c153f90 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00c3a0 tx=0x7fa37c00c860 comp rx=0 tx=0).stop 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 shutdown_connections 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa368078ef0 0x7fa36807b400 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa38c09e020 0x7fa38c098ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa38c0cc830 0x7fa38c098980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 --2- 172.21.15.88:0/3393848499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa38c0af100 0x7fa38c153f90 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.939+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 >> 172.21.15.88:0/3393848499 conn(0x7fa38c0b2540 msgr2=0x7fa38c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:07.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.940+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 shutdown_connections 2024-09-21T07:50:07.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:07.940+0000 7fa392600640 1 -- 172.21.15.88:0/3393848499 wait complete. 2024-09-21T07:50:07.966 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:50:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1898776873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc0af100 msgr2=0x7f7edc0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1898776873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc0af100 0x7f7edc0af510 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7edc0d01c0 tx=0x7f7ed402aa10 comp rx=0 tx=0).stop 2024-09-21T07:50:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1898776873 shutdown_connections 2024-09-21T07:50:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1898776873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc0af100 0x7f7edc0af510 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1898776873 >> v2:172.21.15.175:3300/0 conn(0x7f7edc0cc830 0x7f7edc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1898776873 >> 172.21.15.88:0/1898776873 conn(0x7f7edc0b2540 msgr2=0x7f7edc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:08.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1898776873 shutdown_connections 2024-09-21T07:50:08.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.189+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1898776873 wait complete. 2024-09-21T07:50:08.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.191+0000 7f7ee3a00640 1 Processor -- start 2024-09-21T07:50:08.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.191+0000 7f7ee3a00640 1 -- start start 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee3a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 0x7f7edc1531d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee3a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7edc0af100 0x7f7edc0c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee3a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7edc0cc830 0x7f7edc0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee3a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0b3620 con 0x7f7edc06cda0 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee3a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7edc0b34a0 con 0x7f7edc0cc830 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee3a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7edc0b37a0 con 0x7f7edc0af100 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 0x7f7edc1531d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 0x7f7edc1531d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36896/0 (socket says 172.21.15.88:36896) 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 -- 172.21.15.88:0/1310835697 learned_addr learned my addr 172.21.15.88:0/1310835697 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7edc0cc830 0x7f7edc0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ed3e00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7edc0af100 0x7f7edc0c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 -- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7edc0af100 msgr2=0x7f7edc0c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7edc0af100 0x7f7edc0c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 -- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7edc0cc830 msgr2=0x7f7edc0c9300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7edc0cc830 0x7f7edc0c9300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ee1000640 1 -- 172.21.15.88:0/1310835697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7edc1f73b0 con 0x7f7edc06cda0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.193+0000 7f7ed3e00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7edc0af100 0x7f7edc0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.194+0000 7f7ee1000640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 0x7f7edc1531d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c970 tx=0x7f7ecc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.194+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc007bb0 con 0x7f7edc06cda0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.194+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ecc007e40 con 0x7f7edc06cda0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.194+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7edc1f7690 con 0x7f7edc06cda0 2024-09-21T07:50:08.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.194+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00f690 con 0x7f7edc06cda0 2024-09-21T07:50:08.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.194+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7edc1f7c50 con 0x7f7edc06cda0 2024-09-21T07:50:08.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.195+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7edc0b3ce0 con 0x7f7edc06cda0 2024-09-21T07:50:08.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.196+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f7ecc00f830 con 0x7f7edc06cda0 2024-09-21T07:50:08.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.198+0000 7f7ed9600640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7eb8078f20 0x7f7eb807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:08.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.198+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(64..64 src has 1..64) ==== 5415+0+0 (secure 0 0 0) 0x7f7ecc09bb60 con 0x7f7edc06cda0 2024-09-21T07:50:08.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.198+0000 7f7ed3e00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7eb8078f20 0x7f7eb807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:08.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.199+0000 7f7ed3e00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7eb8078f20 0x7f7eb807b430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:08.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.202+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ecc014030 con 0x7f7edc06cda0 2024-09-21T07:50:08.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.385+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7edc0b1be0 con 0x7f7edc06cda0 2024-09-21T07:50:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.936+0000 7f7ed9600640 1 -- 172.21.15.88:0/1310835697 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v65) ==== 169+0+0 (secure 0 0 0) 0x7f7ecc063a60 con 0x7f7edc06cda0 2024-09-21T07:50:08.938 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:50:08.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.938+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7eb8078f20 msgr2=0x7f7eb807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.938+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7eb8078f20 0x7f7eb807b430 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4006fd0 tx=0x7f7ec4008040 comp rx=0 tx=0).stop 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.938+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 msgr2=0x7f7edc1531d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.938+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 0x7f7edc1531d0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00c970 tx=0x7f7ecc00ce30 comp rx=0 tx=0).stop 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.939+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 shutdown_connections 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.939+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7eb8078f20 0x7f7eb807b430 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.939+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7edc0af100 0x7f7edc0c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.939+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7edc0cc830 0x7f7edc0c9300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.939+0000 7f7ee3a00640 1 --2- 172.21.15.88:0/1310835697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7edc06cda0 0x7f7edc1531d0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:08.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.939+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 >> 172.21.15.88:0/1310835697 conn(0x7f7edc0b2540 msgr2=0x7f7edc0744a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:08.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.940+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 shutdown_connections 2024-09-21T07:50:08.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:08.941+0000 7f7ee3a00640 1 -- 172.21.15.88:0/1310835697 wait complete. 2024-09-21T07:50:08.972 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 -- 172.21.15.88:0/2313296252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400cfc80 msgr2=0x7f9a40156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 --2- 172.21.15.88:0/2313296252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400cfc80 0x7f9a40156e40 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f9a30005fd0 tx=0x7f9a3002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 -- 172.21.15.88:0/2313296252 shutdown_connections 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 --2- 172.21.15.88:0/2313296252 >> v2:172.21.15.175:3300/0 conn(0x7f9a400705b0 0x7f9a400709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 --2- 172.21.15.88:0/2313296252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400cfc80 0x7f9a40156e40 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 -- 172.21.15.88:0/2313296252 >> 172.21.15.88:0/2313296252 conn(0x7f9a4006d3d0 msgr2=0x7f9a400b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 -- 172.21.15.88:0/2313296252 shutdown_connections 2024-09-21T07:50:09.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.200+0000 7f9a44a00640 1 -- 172.21.15.88:0/2313296252 wait complete. 2024-09-21T07:50:09.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.202+0000 7f9a44a00640 1 Processor -- start 2024-09-21T07:50:09.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.203+0000 7f9a44a00640 1 -- start start 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 0x7f9a401f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a400cfc80 0x7f9a40156610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a44a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a4015ce30 0x7f9a401f38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a44a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400cced0 con 0x7f9a400705b0 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a44a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a400d0170 con 0x7f9a4015ce30 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a44a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a400cd050 con 0x7f9a400cfc80 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a3de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 0x7f9a401f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a3de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 0x7f9a401f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36910/0 (socket says 172.21.15.88:36910) 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.204+0000 7f9a3de00640 1 -- 172.21.15.88:0/3201775595 learned_addr learned my addr 172.21.15.88:0/3201775595 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3d400640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a400cfc80 0x7f9a40156610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3e800640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a4015ce30 0x7f9a401f38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3de00640 1 -- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a400cfc80 msgr2=0x7f9a40156610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3de00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a400cfc80 0x7f9a40156610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3de00640 1 -- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a4015ce30 msgr2=0x7f9a401f38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3de00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a4015ce30 0x7f9a401f38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3de00640 1 -- 172.21.15.88:0/3201775595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a401f7750 con 0x7f9a400705b0 2024-09-21T07:50:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a3de00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 0x7f9a401f6e50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a30034040 tx=0x7f9a300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3003e030 con 0x7f9a400705b0 2024-09-21T07:50:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a30003d90 con 0x7f9a400705b0 2024-09-21T07:50:09.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a401f7a30 con 0x7f9a400705b0 2024-09-21T07:50:09.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.205+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a30035040 con 0x7f9a400705b0 2024-09-21T07:50:09.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.206+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401f7ff0 con 0x7f9a400705b0 2024-09-21T07:50:09.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.207+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a400cd5b0 con 0x7f9a400705b0 2024-09-21T07:50:09.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.208+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f9a30041030 con 0x7f9a400705b0 2024-09-21T07:50:09.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.209+0000 7f9a36000640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a20078ef0 0x7f9a2007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.209+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f9a300bb8a0 con 0x7f9a400705b0 2024-09-21T07:50:09.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.209+0000 7f9a3d400640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a20078ef0 0x7f9a2007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.210+0000 7f9a3d400640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a20078ef0 0x7f9a2007b400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005fd0 tx=0x7f9a2c005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:09.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.213+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9a30083a80 con 0x7f9a400705b0 2024-09-21T07:50:09.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.401+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f9a4015f170 con 0x7f9a400705b0 2024-09-21T07:50:09.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.405+0000 7f9a36000640 1 -- 172.21.15.88:0/3201775595 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v37) ==== 71+0+0 (secure 0 0 0) 0x7f9a30083380 con 0x7f9a400705b0 2024-09-21T07:50:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a20078ef0 msgr2=0x7f9a2007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a20078ef0 0x7f9a2007b400 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c005fd0 tx=0x7f9a2c005850 comp rx=0 tx=0).stop 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 msgr2=0x7f9a401f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 0x7f9a401f6e50 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f9a30034040 tx=0x7f9a300038a0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 shutdown_connections 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f9a20078ef0 0x7f9a2007b400 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a400cfc80 0x7f9a40156610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a4015ce30 0x7f9a401f38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 --2- 172.21.15.88:0/3201775595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a400705b0 0x7f9a401f6e50 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 >> 172.21.15.88:0/3201775595 conn(0x7f9a4006d3d0 msgr2=0x7f9a400b2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 shutdown_connections 2024-09-21T07:50:09.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.408+0000 7f9a44a00640 1 -- 172.21.15.88:0/3201775595 wait complete. 2024-09-21T07:50:09.435 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade' 2024-09-21T07:50:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.652+0000 7ff307800640 1 -- 172.21.15.88:0/874398544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3000b3550 msgr2=0x7ff3000b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.652+0000 7ff307800640 1 --2- 172.21.15.88:0/874398544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3000b3550 0x7ff3000b3960 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8005fd0 tx=0x7ff2f802aa10 comp rx=0 tx=0).stop 2024-09-21T07:50:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.653+0000 7ff307800640 1 -- 172.21.15.88:0/874398544 shutdown_connections 2024-09-21T07:50:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.653+0000 7ff307800640 1 --2- 172.21.15.88:0/874398544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3000b3550 0x7ff3000b3960 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.653+0000 7ff307800640 1 --2- 172.21.15.88:0/874398544 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff30015d760 0x7ff3001535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.653+0000 7ff307800640 1 -- 172.21.15.88:0/874398544 >> 172.21.15.88:0/874398544 conn(0x7ff30006d730 msgr2=0x7ff3000b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.653+0000 7ff307800640 1 -- 172.21.15.88:0/874398544 shutdown_connections 2024-09-21T07:50:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.653+0000 7ff307800640 1 -- 172.21.15.88:0/874398544 wait complete. 2024-09-21T07:50:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.655+0000 7ff307800640 1 Processor -- start 2024-09-21T07:50:09.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.655+0000 7ff307800640 1 -- start start 2024-09-21T07:50:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.656+0000 7ff307800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3000ad0e0 0x7ff3001f6560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.656+0000 7ff307800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3000b3550 0x7ff300158600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.656+0000 7ff307800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 0x7ff300073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.656+0000 7ff307800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000b4da0 con 0x7ff30015d760 2024-09-21T07:50:09.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.656+0000 7ff307800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff3000b4c20 con 0x7ff3000ad0e0 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.656+0000 7ff307800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff3000b3ea0 con 0x7ff3000b3550 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 0x7ff300073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 0x7ff300073040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36916/0 (socket says 172.21.15.88:36916) 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 -- 172.21.15.88:0/2741801227 learned_addr learned my addr 172.21.15.88:0/2741801227 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff304e00640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3000ad0e0 0x7ff3001f6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff2ffe00640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3000b3550 0x7ff300158600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 -- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3000b3550 msgr2=0x7ff300158600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3000b3550 0x7ff300158600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 -- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3000ad0e0 msgr2=0x7ff3001f6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3000ad0e0 0x7ff3001f6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 -- 172.21.15.88:0/2741801227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3001591e0 con 0x7ff30015d760 2024-09-21T07:50:09.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff2ffe00640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3000b3550 0x7ff300158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:09.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.657+0000 7ff305800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 0x7ff300073040 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006010 tx=0x7ff2f8003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:09.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.658+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8046070 con 0x7ff30015d760 2024-09-21T07:50:09.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.658+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2f8004540 con 0x7ff30015d760 2024-09-21T07:50:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.658+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f8033070 con 0x7ff30015d760 2024-09-21T07:50:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.658+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3001594c0 con 0x7ff30015d760 2024-09-21T07:50:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.658+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3001f5d60 con 0x7ff30015d760 2024-09-21T07:50:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.659+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff300158f50 con 0x7ff30015d760 2024-09-21T07:50:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.660+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ff2f8049070 con 0x7ff30015d760 2024-09-21T07:50:09.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.661+0000 7ff2fcc00640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff2e0078ef0 0x7ff2e007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:09.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.661+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7ff2f80bc190 con 0x7ff30015d760 2024-09-21T07:50:09.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.662+0000 7ff304e00640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff2e0078ef0 0x7ff2e007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:09.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.662+0000 7ff304e00640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff2e0078ef0 0x7ff2e007b400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400bb50 tx=0x7ff2f400b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:09.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.666+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2f803e070 con 0x7ff30015d760 2024-09-21T07:50:09.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.925+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7ff3000b4330 con 0x7ff30015d760 2024-09-21T07:50:09.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.948+0000 7ff2fcc00640 1 -- 172.21.15.88:0/2741801227 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_malicious_metafile_on_legacy_to_v1_upgrade"]}]=0 v230) ==== 161+0+0 (secure 0 0 0) 0x7ff2f8083db0 con 0x7ff30015d760 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff2e0078ef0 msgr2=0x7ff2e007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff2e0078ef0 0x7ff2e007b400 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7ff2f400bb50 tx=0x7ff2f400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 msgr2=0x7ff300073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 0x7ff300073040 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7ff2f8006010 tx=0x7ff2f8003f10 comp rx=0 tx=0).stop 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 shutdown_connections 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff2e0078ef0 0x7ff2e007b400 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff3000b3550 0x7ff300158600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff30015d760 0x7ff300073040 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 --2- 172.21.15.88:0/2741801227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff3000ad0e0 0x7ff3001f6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:09.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.950+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 >> 172.21.15.88:0/2741801227 conn(0x7ff30006d730 msgr2=0x7ff3000cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:09.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.951+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 shutdown_connections 2024-09-21T07:50:09.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:09.951+0000 7ff307800640 1 -- 172.21.15.88:0/2741801227 wait complete. 2024-09-21T07:50:09.975 INFO:tasks.cephfs_test_runner:Validate handcrafted .meta file on legacy subvol root doesn't break the system ... ok 2024-09-21T07:50:09.975 INFO:tasks.cephfs_test_runner:Starting test: test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:50:09.976 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.193+0000 7f695ee00640 1 -- 172.21.15.88:0/2134792405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f695815fae0 msgr2=0x7f69580b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.193+0000 7f695ee00640 1 --2- 172.21.15.88:0/2134792405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f695815fae0 0x7f69580b3750 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f6958072810 tx=0x7f695002aa30 comp rx=0 tx=0).stop 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.193+0000 7f695ee00640 1 -- 172.21.15.88:0/2134792405 shutdown_connections 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.193+0000 7f695ee00640 1 --2- 172.21.15.88:0/2134792405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f695815fae0 0x7f69580b3750 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.193+0000 7f695ee00640 1 --2- 172.21.15.88:0/2134792405 >> v2:172.21.15.175:3300/0 conn(0x7f69580aebc0 0x7f6958152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.193+0000 7f695ee00640 1 -- 172.21.15.88:0/2134792405 >> 172.21.15.88:0/2134792405 conn(0x7f695806d730 msgr2=0x7f69580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:10.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.194+0000 7f695ee00640 1 -- 172.21.15.88:0/2134792405 shutdown_connections 2024-09-21T07:50:10.197 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.194+0000 7f695ee00640 1 -- 172.21.15.88:0/2134792405 wait complete. 2024-09-21T07:50:10.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.196+0000 7f695ee00640 1 Processor -- start 2024-09-21T07:50:10.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.196+0000 7f695ee00640 1 -- start start 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f695ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 0x7f6958151190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f695ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69580b2750 0x7f6958158830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f695ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69581f5fe0 0x7f6958158d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f695ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69580b40d0 con 0x7f69580aebc0 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f695ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69580b3f50 con 0x7f69581f5fe0 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f695ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69580b4250 con 0x7f69580b2750 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f6957e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 0x7f6958151190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f6957e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 0x7f6958151190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36934/0 (socket says 172.21.15.88:36934) 2024-09-21T07:50:10.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.197+0000 7f6957e00640 1 -- 172.21.15.88:0/1429053803 learned_addr learned my addr 172.21.15.88:0/1429053803 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957400640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69580b2750 0x7f6958158830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f695ce00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69581f5fe0 0x7f6958158d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957e00640 1 -- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69580b2750 msgr2=0x7f6958158830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957e00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69580b2750 0x7f6958158830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957e00640 1 -- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69581f5fe0 msgr2=0x7f6958158d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957e00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69581f5fe0 0x7f6958158d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957e00640 1 -- 172.21.15.88:0/1429053803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69581f5280 con 0x7f69580aebc0 2024-09-21T07:50:10.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f695ce00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69581f5fe0 0x7f6958158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:10.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f6957e00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 0x7f6958151190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f694800d7a0 tx=0x7f694800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:10.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.198+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6948017070 con 0x7f69580aebc0 2024-09-21T07:50:10.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.199+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6948009cc0 con 0x7f69580aebc0 2024-09-21T07:50:10.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.199+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69580b0ae0 con 0x7f69580aebc0 2024-09-21T07:50:10.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.199+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69480106c0 con 0x7f69580aebc0 2024-09-21T07:50:10.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.199+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69581f57e0 con 0x7f69580aebc0 2024-09-21T07:50:10.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.200+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69580b4770 con 0x7f69580aebc0 2024-09-21T07:50:10.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.201+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f694801a070 con 0x7f69580aebc0 2024-09-21T07:50:10.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.202+0000 7f694fe00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6938078ef0 0x7f693807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.202+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f694809b1a0 con 0x7f69580aebc0 2024-09-21T07:50:10.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.203+0000 7f6957400640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6938078ef0 0x7f693807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.203+0000 7f6957400640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6938078ef0 0x7f693807b400 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6958070840 tx=0x7f6944009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:10.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.206+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f69581f5280 con 0x7f69580aebc0 2024-09-21T07:50:10.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.396+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69580b2150 con 0x7f69580aebc0 2024-09-21T07:50:10.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.397+0000 7f694fe00640 1 -- 172.21.15.88:0/1429053803 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v65) ==== 74+0+13046 (secure 0 0 0) 0x7f69480633a0 con 0x7f69580aebc0 2024-09-21T07:50:10.399 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:10.399 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":65,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:08.935590+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":61,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6837/2365988221":"2024-09-22T07:50:05.929720+0000","172.21.15.175:6835/2365988221":"2024-09-22T07:50:05.929720+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:10.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6938078ef0 msgr2=0x7f693807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6938078ef0 0x7f693807b400 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f6958070840 tx=0x7f6944009040 comp rx=0 tx=0).stop 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 msgr2=0x7f6958151190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 0x7f6958151190 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f694800d7a0 tx=0x7f694800dc60 comp rx=0 tx=0).stop 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 shutdown_connections 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6938078ef0 0x7f693807b400 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69580b2750 0x7f6958158830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f69581f5fe0 0x7f6958158d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 --2- 172.21.15.88:0/1429053803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69580aebc0 0x7f6958151190 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 >> 172.21.15.88:0/1429053803 conn(0x7f695806d730 msgr2=0x7f695806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 shutdown_connections 2024-09-21T07:50:10.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.399+0000 7f695ee00640 1 -- 172.21.15.88:0/1429053803 wait complete. 2024-09-21T07:50:10.423 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:50:10.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 -- 172.21.15.88:0/2504208317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240cf080 msgr2=0x7f2624156df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 --2- 172.21.15.88:0/2504208317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240cf080 0x7f2624156df0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f2614005fd0 tx=0x7f261402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 -- 172.21.15.88:0/2504208317 shutdown_connections 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 --2- 172.21.15.88:0/2504208317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2624073030 0x7f262406f310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 --2- 172.21.15.88:0/2504208317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26240cf080 0x7f2624156df0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 -- 172.21.15.88:0/2504208317 >> 172.21.15.88:0/2504208317 conn(0x7f262406cf80 msgr2=0x7f262406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 -- 172.21.15.88:0/2504208317 shutdown_connections 2024-09-21T07:50:10.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.643+0000 7f2623200640 1 -- 172.21.15.88:0/2504208317 wait complete. 2024-09-21T07:50:10.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.646+0000 7f2623200640 1 Processor -- start 2024-09-21T07:50:10.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.646+0000 7f2623200640 1 -- start start 2024-09-21T07:50:10.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2623200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 0x7f262414cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2623200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2624073440 0x7f2624154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2623200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26240cf080 0x7f262415b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2623200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2624160530 con 0x7f2624073030 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2623200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26240d0d80 con 0x7f2624073440 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2623200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26241606b0 con 0x7f26240cf080 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2621e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 0x7f262414cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2621e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 0x7f262414cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36958/0 (socket says 172.21.15.88:36958) 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.647+0000 7f2621e00640 1 -- 172.21.15.88:0/357260399 learned_addr learned my addr 172.21.15.88:0/357260399 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2619400640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2624073440 0x7f2624154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2622800640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26240cf080 0x7f262415b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2621e00640 1 -- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26240cf080 msgr2=0x7f262415b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2621e00640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26240cf080 0x7f262415b090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2621e00640 1 -- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2624073440 msgr2=0x7f2624154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2621e00640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2624073440 0x7f2624154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2621e00640 1 -- 172.21.15.88:0/357260399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26240cb5d0 con 0x7f2624073030 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f2621e00640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 0x7f262414cfd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2614034040 tx=0x7f26140038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f261403e030 con 0x7f2624073030 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.648+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2614003d90 con 0x7f2624073030 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.649+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26240cb8b0 con 0x7f2624073030 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.649+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2614035040 con 0x7f2624073030 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.649+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26240cbe70 con 0x7f2624073030 2024-09-21T07:50:10.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.650+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2624160b60 con 0x7f2624073030 2024-09-21T07:50:10.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.651+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f2614041030 con 0x7f2624073030 2024-09-21T07:50:10.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.652+0000 7f261aa00640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2600078ef0 0x7f260007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:10.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.652+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f26140bbc10 con 0x7f2624073030 2024-09-21T07:50:10.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.652+0000 7f2619400640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2600078ef0 0x7f260007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:10.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.653+0000 7f2619400640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2600078ef0 0x7f260007b400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f262406d500 tx=0x7f260c00b2e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:10.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.655+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f26140838e0 con 0x7f2624073030 2024-09-21T07:50:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.842+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f262414d510 con 0x7f2624073030 2024-09-21T07:50:10.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.843+0000 7f261aa00640 1 -- 172.21.15.88:0/357260399 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) ==== 93+0+10 (secure 0 0 0) 0x7f2614088310 con 0x7f2624073030 2024-09-21T07:50:10.845 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2600078ef0 msgr2=0x7f260007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2600078ef0 0x7f260007b400 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f262406d500 tx=0x7f260c00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 msgr2=0x7f262414cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 0x7f262414cfd0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f2614034040 tx=0x7f26140038a0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 shutdown_connections 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2600078ef0 0x7f260007b400 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26240cf080 0x7f262415b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2624073440 0x7f2624154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 --2- 172.21.15.88:0/357260399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2624073030 0x7f262414cfd0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.845+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 >> 172.21.15.88:0/357260399 conn(0x7f262406cf80 msgr2=0x7f2624156000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.846+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 shutdown_connections 2024-09-21T07:50:10.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:10.846+0000 7f2623200640 1 -- 172.21.15.88:0/357260399 wait complete. 2024-09-21T07:50:10.871 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 -- 172.21.15.88:0/1202032326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 msgr2=0x7fb8940ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 --2- 172.21.15.88:0/1202032326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb8940ad4f0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fb880005fd0 tx=0x7fb88002a540 comp rx=0 tx=0).stop 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 -- 172.21.15.88:0/1202032326 shutdown_connections 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 --2- 172.21.15.88:0/1202032326 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8940b3550 0x7fb8940b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 --2- 172.21.15.88:0/1202032326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb8940ad4f0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 -- 172.21.15.88:0/1202032326 >> 172.21.15.88:0/1202032326 conn(0x7fb89406d730 msgr2=0x7fb8940b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:11.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 -- 172.21.15.88:0/1202032326 shutdown_connections 2024-09-21T07:50:11.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.089+0000 7fb89ba00640 1 -- 172.21.15.88:0/1202032326 wait complete. 2024-09-21T07:50:11.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.092+0000 7fb89ba00640 1 Processor -- start 2024-09-21T07:50:11.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.092+0000 7fb89ba00640 1 -- start start 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb89ba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb89414f430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb89ba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8940b3550 0x7fb894155990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb89ba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb89415d760 0x7fb89406f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb89ba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940b4da0 con 0x7fb8940ad0e0 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb89ba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8940b4c20 con 0x7fb8940b3550 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb89ba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8940b3ea0 con 0x7fb89415d760 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.093+0000 7fb899000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb89414f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb89414f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36980/0 (socket says 172.21.15.88:36980) 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 -- 172.21.15.88:0/3049474018 learned_addr learned my addr 172.21.15.88:0/3049474018 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:11.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899a00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb89415d760 0x7fb89406f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb893e00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8940b3550 0x7fb894155990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 -- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb89415d760 msgr2=0x7fb89406f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb89415d760 0x7fb89406f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 -- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8940b3550 msgr2=0x7fb894155990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8940b3550 0x7fb894155990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 -- 172.21.15.88:0/3049474018 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb894155ed0 con 0x7fb8940ad0e0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb893e00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8940b3550 0x7fb894155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb899000640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb89414f430 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c3d0 tx=0x7fb88800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.094+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888017070 con 0x7fb8940ad0e0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.095+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8880053c0 con 0x7fb8940ad0e0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.095+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb894153fe0 con 0x7fb8940ad0e0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.095+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb888003850 con 0x7fb8940ad0e0 2024-09-21T07:50:11.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.095+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8941545a0 con 0x7fb8940ad0e0 2024-09-21T07:50:11.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.096+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb894156150 con 0x7fb8940ad0e0 2024-09-21T07:50:11.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.097+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fb88801a070 con 0x7fb8940ad0e0 2024-09-21T07:50:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.098+0000 7fb890c00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb874078ef0 0x7fb87407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.098+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7fb88809a390 con 0x7fb8940ad0e0 2024-09-21T07:50:11.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.099+0000 7fb893e00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb874078ef0 0x7fb87407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:11.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.099+0000 7fb893e00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb874078ef0 0x7fb87407b400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb880033000 tx=0x7fb880004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:11.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.102+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb8940ada30 con 0x7fb8940ad0e0 2024-09-21T07:50:11.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.362+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7fb8940b4370 con 0x7fb8940ad0e0 2024-09-21T07:50:11.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.956+0000 7fb890c00640 1 -- 172.21.15.88:0/3049474018 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v232) ==== 134+0+0 (secure 0 0 0) 0x7fb888062620 con 0x7fb8940ad0e0 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb874078ef0 msgr2=0x7fb87407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb874078ef0 0x7fb87407b400 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb880033000 tx=0x7fb880004730 comp rx=0 tx=0).stop 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 msgr2=0x7fb89414f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb89414f430 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb88800c3d0 tx=0x7fb88800c890 comp rx=0 tx=0).stop 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 shutdown_connections 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fb874078ef0 0x7fb87407b400 secure :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fb880033000 tx=0x7fb880004730 comp rx=0 tx=0).stop 2024-09-21T07:50:11.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb89415d760 0x7fb89406f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8940b3550 0x7fb894155990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 --2- 172.21.15.88:0/3049474018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8940ad0e0 0x7fb89414f430 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 >> 172.21.15.88:0/3049474018 conn(0x7fb89406d730 msgr2=0x7fb8940cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 shutdown_connections 2024-09-21T07:50:11.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:11.959+0000 7fb89ba00640 1 -- 172.21.15.88:0/3049474018 wait complete. 2024-09-21T07:50:11.984 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 -- 172.21.15.88:0/2392366277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 msgr2=0x7fa2600ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 --2- 172.21.15.88:0/2392366277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa2600ad4f0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fa24c005fd0 tx=0x7fa24c02a4a0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 -- 172.21.15.88:0/2392366277 shutdown_connections 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 --2- 172.21.15.88:0/2392366277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2600b3550 0x7fa2600b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 --2- 172.21.15.88:0/2392366277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa2600ad4f0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 -- 172.21.15.88:0/2392366277 >> 172.21.15.88:0/2392366277 conn(0x7fa26006d730 msgr2=0x7fa2600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 -- 172.21.15.88:0/2392366277 shutdown_connections 2024-09-21T07:50:12.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.200+0000 7fa265a00640 1 -- 172.21.15.88:0/2392366277 wait complete. 2024-09-21T07:50:12.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.203+0000 7fa265a00640 1 Processor -- start 2024-09-21T07:50:12.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.203+0000 7fa265a00640 1 -- start start 2024-09-21T07:50:12.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa265a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa260148780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa265a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2600b3550 0x7fa260151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa265a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa26015d760 0x7fa2601515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa265a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600b4da0 con 0x7fa2600ad0e0 2024-09-21T07:50:12.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa265a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa2600b4c20 con 0x7fa26015d760 2024-09-21T07:50:12.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa265a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa2600b3ea0 con 0x7fa2600b3550 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa25ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa260148780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa25ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa260148780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37006/0 (socket says 172.21.15.88:37006) 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.204+0000 7fa25ea00640 1 -- 172.21.15.88:0/589057228 learned_addr learned my addr 172.21.15.88:0/589057228 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25f400640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa26015d760 0x7fa2601515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25e000640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2600b3550 0x7fa260151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25ea00640 1 -- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2600b3550 msgr2=0x7fa260151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25ea00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2600b3550 0x7fa260151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25ea00640 1 -- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa26015d760 msgr2=0x7fa2601515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25ea00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa26015d760 0x7fa2601515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25ea00640 1 -- 172.21.15.88:0/589057228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa260148cc0 con 0x7fa2600ad0e0 2024-09-21T07:50:12.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa25ea00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa260148780 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b3f0 tx=0x7fa25000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:12.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.205+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250015020 con 0x7fa2600ad0e0 2024-09-21T07:50:12.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.206+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa250011920 con 0x7fa2600ad0e0 2024-09-21T07:50:12.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.206+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa250011d70 con 0x7fa2600ad0e0 2024-09-21T07:50:12.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.206+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2601542d0 con 0x7fa2600ad0e0 2024-09-21T07:50:12.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.206+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2600b0280 con 0x7fa2600ad0e0 2024-09-21T07:50:12.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.207+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2600b4370 con 0x7fa2600ad0e0 2024-09-21T07:50:12.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.208+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa250018020 con 0x7fa2600ad0e0 2024-09-21T07:50:12.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.209+0000 7fa256a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa240078ef0 0x7fa24007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.209+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7fa25009acd0 con 0x7fa2600ad0e0 2024-09-21T07:50:12.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.209+0000 7fa25e000640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa240078ef0 0x7fa24007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.210+0000 7fa25e000640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa240078ef0 0x7fa24007b400 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa24c033000 tx=0x7fa24c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:12.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.213+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa25001f030 con 0x7fa2600ad0e0 2024-09-21T07:50:12.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.399+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fa2601526a0 con 0x7fa2600ad0e0 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.400+0000 7fa256a00640 1 -- 172.21.15.88:0/589057228 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+2294 (secure 0 0 0) 0x7fa250062f40 con 0x7fa2600ad0e0 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:50:12.402 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:50:12.403 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:50:12.404 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.402+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa240078ef0 msgr2=0x7fa24007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.402+0000 7fa265a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa240078ef0 0x7fa24007b400 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa24c033000 tx=0x7fa24c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.402+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 msgr2=0x7fa260148780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.402+0000 7fa265a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa260148780 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa25000b3f0 tx=0x7fa25000b8b0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 shutdown_connections 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa240078ef0 0x7fa24007b400 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa2600b3550 0x7fa260151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa26015d760 0x7fa2601515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 --2- 172.21.15.88:0/589057228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa2600ad0e0 0x7fa260148780 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 >> 172.21.15.88:0/589057228 conn(0x7fa26006d730 msgr2=0x7fa2600cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 shutdown_connections 2024-09-21T07:50:12.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.403+0000 7fa265a00640 1 -- 172.21.15.88:0/589057228 wait complete. 2024-09-21T07:50:12.426 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:50:12.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.645+0000 7f4b1ee00640 1 -- 172.21.15.88:0/1321814992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145550 msgr2=0x7f4b1806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.645+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/1321814992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145550 0x7f4b1806fb40 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f4b18160280 tx=0x7f4b0802a680 comp rx=0 tx=0).stop 2024-09-21T07:50:12.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.645+0000 7f4b1ee00640 1 -- 172.21.15.88:0/1321814992 shutdown_connections 2024-09-21T07:50:12.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.645+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/1321814992 >> v2:172.21.15.175:3300/0 conn(0x7f4b18145a40 0x7f4b18073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.645+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/1321814992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145550 0x7f4b1806fb40 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.646+0000 7f4b1ee00640 1 -- 172.21.15.88:0/1321814992 >> 172.21.15.88:0/1321814992 conn(0x7f4b1806d1c0 msgr2=0x7f4b1806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.646+0000 7f4b1ee00640 1 -- 172.21.15.88:0/1321814992 shutdown_connections 2024-09-21T07:50:12.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.646+0000 7f4b1ee00640 1 -- 172.21.15.88:0/1321814992 wait complete. 2024-09-21T07:50:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.648+0000 7f4b1ee00640 1 Processor -- start 2024-09-21T07:50:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.648+0000 7f4b1ee00640 1 -- start start 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.649+0000 7f4b1ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18073a80 0x7f4b1813aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.649+0000 7f4b1ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18145550 0x7f4b1813bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.649+0000 7f4b1ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 0x7f4b1813c070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.649+0000 7f4b1ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b18160a30 con 0x7f4b18145a40 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.649+0000 7f4b1ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b180ce870 con 0x7f4b18145550 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.649+0000 7f4b1ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b18160bb0 con 0x7f4b18073a80 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 0x7f4b1813c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 0x7f4b1813c070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37018/0 (socket says 172.21.15.88:37018) 2024-09-21T07:50:12.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 -- 172.21.15.88:0/861515945 learned_addr learned my addr 172.21.15.88:0/861515945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b17400640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18145550 0x7f4b1813bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b17e00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18073a80 0x7f4b1813aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 -- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18073a80 msgr2=0x7f4b1813aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18073a80 0x7f4b1813aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 -- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18145550 msgr2=0x7f4b1813bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18145550 0x7f4b1813bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 -- 172.21.15.88:0/861515945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b1815c8b0 con 0x7f4b18145a40 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b17400640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18145550 0x7f4b1813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:12.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.650+0000 7f4b1ce00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 0x7f4b1813c070 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c730 tx=0x7f4b1000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:12.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.651+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b10017070 con 0x7f4b18145a40 2024-09-21T07:50:12.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.651+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b10003c60 con 0x7f4b18145a40 2024-09-21T07:50:12.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.651+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b1815d410 con 0x7f4b18145a40 2024-09-21T07:50:12.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.651+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b100052d0 con 0x7f4b18145a40 2024-09-21T07:50:12.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.651+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b1815d9d0 con 0x7f4b18145a40 2024-09-21T07:50:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.652+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b181e64b0 con 0x7f4b18145a40 2024-09-21T07:50:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.653+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f4b1001a070 con 0x7f4b18145a40 2024-09-21T07:50:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.654+0000 7f4b0fe00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078ef0 0x7f4af807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:12.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.654+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f4b1009a850 con 0x7f4b18145a40 2024-09-21T07:50:12.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.654+0000 7f4b17e00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078ef0 0x7f4af807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:12.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.655+0000 7f4b17e00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078ef0 0x7f4af807b400 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b08033000 tx=0x7f4b08004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:12.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.658+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b18073e90 con 0x7f4b18145a40 2024-09-21T07:50:12.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.845+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f4b181610f0 con 0x7f4b18145a40 2024-09-21T07:50:12.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.850+0000 7f4b0fe00640 1 -- 172.21.15.88:0/861515945 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f4b100623a0 con 0x7f4b18145a40 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078ef0 msgr2=0x7f4af807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078ef0 0x7f4af807b400 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f4b08033000 tx=0x7f4b08004730 comp rx=0 tx=0).stop 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 msgr2=0x7f4b1813c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 0x7f4b1813c070 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f4b1000c730 tx=0x7f4b1000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 shutdown_connections 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078ef0 0x7f4af807b400 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b18145a40 0x7f4b1813c070 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b18145550 0x7f4b1813bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 --2- 172.21.15.88:0/861515945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b18073a80 0x7f4b1813aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 >> 172.21.15.88:0/861515945 conn(0x7f4b1806d1c0 msgr2=0x7f4b18068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:12.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 shutdown_connections 2024-09-21T07:50:12.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:12.852+0000 7f4b1ee00640 1 -- 172.21.15.88:0/861515945 wait complete. 2024-09-21T07:50:12.877 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:50:12.877 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:12.963 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:13.065 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:50:13.065 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:13.065 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:13.100 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:13.101 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:13.101 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:13.138 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:13.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:13.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:13.176 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:13.176 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:13.176 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:13.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:13.215 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:50:13.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 -- 172.21.15.88:0/2148550809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80cf120 msgr2=0x7fcaf80ccbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 --2- 172.21.15.88:0/2148550809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80cf120 0x7fcaf80ccbf0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fcaf0009830 tx=0x7fcaf002a510 comp rx=0 tx=0).stop 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 -- 172.21.15.88:0/2148550809 shutdown_connections 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 --2- 172.21.15.88:0/2148550809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80cf120 0x7fcaf80ccbf0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 --2- 172.21.15.88:0/2148550809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaf80d0350 0x7fcaf815e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 -- 172.21.15.88:0/2148550809 >> 172.21.15.88:0/2148550809 conn(0x7fcaf8071070 msgr2=0x7fcaf8071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 -- 172.21.15.88:0/2148550809 shutdown_connections 2024-09-21T07:50:13.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.432+0000 7fcafce00640 1 -- 172.21.15.88:0/2148550809 wait complete. 2024-09-21T07:50:13.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.435+0000 7fcafce00640 1 Processor -- start 2024-09-21T07:50:13.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.435+0000 7fcafce00640 1 -- start start 2024-09-21T07:50:13.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcafce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaf80d0350 0x7fcaf814be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcafce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 0x7fcaf81e3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcafce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf814c370 0x7fcaf814e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcafce00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf80cd980 con 0x7fcaf80d08c0 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcafce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcaf80cd800 con 0x7fcaf814c370 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcafce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcaf80cdb00 con 0x7fcaf80d0350 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 0x7fcaf81e3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcaeea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 0x7fcaf81e3c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37036/0 (socket says 172.21.15.88:37036) 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcaeea00640 1 -- 172.21.15.88:0/3764994895 learned_addr learned my addr 172.21.15.88:0/3764994895 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.436+0000 7fcaf7400640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaf80d0350 0x7fcaf814be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaf7e00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf814c370 0x7fcaf814e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaeea00640 1 -- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaf80d0350 msgr2=0x7fcaf814be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaeea00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaf80d0350 0x7fcaf814be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaeea00640 1 -- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf814c370 msgr2=0x7fcaf814e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaeea00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf814c370 0x7fcaf814e870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaeea00640 1 -- 172.21.15.88:0/3764994895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaf81e4a00 con 0x7fcaf80d08c0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaeea00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 0x7fcaf81e3c90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c3a0 tx=0x7fcae000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0011790 con 0x7fcaf80d08c0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcae0017070 con 0x7fcaf80d08c0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaf81e4ce0 con 0x7fcaf80d08c0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcae0011d20 con 0x7fcaf80d08c0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.437+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaf81e52a0 con 0x7fcaf80d08c0 2024-09-21T07:50:13.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.439+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcaf80ccbf0 con 0x7fcaf80d08c0 2024-09-21T07:50:13.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.440+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fcae001c460 con 0x7fcaf80d08c0 2024-09-21T07:50:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.441+0000 7fcaefe00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcad4078ef0 0x7fcad407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.441+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7fcae009adb0 con 0x7fcaf80d08c0 2024-09-21T07:50:13.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.442+0000 7fcaf7400640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcad4078ef0 0x7fcad407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:13.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.442+0000 7fcaf7400640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcad4078ef0 0x7fcad407b400 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005fd0 tx=0x7fcae80096c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:13.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.445+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcae0062980 con 0x7fcaf80d08c0 2024-09-21T07:50:13.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.667+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcaf80b6610 con 0x7fcaf80d08c0 2024-09-21T07:50:13.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.668+0000 7fcaefe00640 1 -- 172.21.15.88:0/3764994895 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+3914 (secure 0 0 0) 0x7fcae00673b0 con 0x7fcaf80d08c0 2024-09-21T07:50:13.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:13.671 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":39,"btime":"2024-09-21T07:50:06:931901+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5276,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/926372333","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":926372333},{"type":"v1","addr":"172.21.15.88:6835","nonce":926372333}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5504,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/2244378096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":2244378096},{"type":"v1","addr":"172.21.15.175:6837","nonce":2244378096}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":39},{"gid":5671,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2855370745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2855370745},{"type":"v1","addr":"172.21.15.175:6836","nonce":2855370745}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":31},{"gid":5748,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2109449416","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2109449416},{"type":"v1","addr":"172.21.15.88:6834","nonce":2109449416}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":30}],"filesystems":[]} 2024-09-21T07:50:13.671 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 39 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcad4078ef0 msgr2=0x7fcad407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcad4078ef0 0x7fcad407b400 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fcae8005fd0 tx=0x7fcae80096c0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 msgr2=0x7fcaf81e3c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 0x7fcaf81e3c90 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fcae000c3a0 tx=0x7fcae000c860 comp rx=0 tx=0).stop 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 shutdown_connections 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fcad4078ef0 0x7fcad407b400 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcaf814c370 0x7fcaf814e870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaf80d08c0 0x7fcaf81e3c90 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.670+0000 7fcafce00640 1 --2- 172.21.15.88:0/3764994895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcaf80d0350 0x7fcaf814be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:13.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.671+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 >> 172.21.15.88:0/3764994895 conn(0x7fcaf8071070 msgr2=0x7fcaf80b1200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:13.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.671+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 shutdown_connections 2024-09-21T07:50:13.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:13.671+0000 7fcafce00640 1 -- 172.21.15.88:0/3764994895 wait complete. 2024-09-21T07:50:13.698 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:50:13.698 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:50:13.699 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:50:13.699 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:13.699 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:50:13.699 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:50:13.699 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:50:13.699 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:13.699 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:50:13.699 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:50:13.700 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:50:13.700 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:13.700 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:50:13.700 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:50:13.700 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:50:13.700 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:13.723 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:50:13.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:50:13.724 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:50:13.724 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:50:13.725 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:50:13.725 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:50:13.726 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:50:13.726 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:50:13.769 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:50:13.770 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:13.771 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:50:13.771 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:13.771 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:50:13.772 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:13.773 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:50:13.774 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:13.774 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:50:13.964 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:50:13.972 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:50:13.972 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:50:13.975 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea6600640 1 -- 172.21.15.88:0/595996314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84005150 msgr2=0x7f5e8400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea6600640 1 --2- 172.21.15.88:0/595996314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e84005150 0x7f5e8400b9c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f5ea00cea40 tx=0x7f5e84005830 comp rx=0 tx=0).stop 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea5c00640 1 -- 172.21.15.88:0/595996314 reap_dead start 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea6600640 1 -- 172.21.15.88:0/595996314 shutdown_connections 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea6600640 1 -- 172.21.15.88:0/595996314 >> 172.21.15.88:0/595996314 conn(0x7f5ea00d44f0 msgr2=0x7f5ea00d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea6600640 1 -- 172.21.15.88:0/595996314 shutdown_connections 2024-09-21T07:50:14.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.011+0000 7f5ea6600640 1 -- 172.21.15.88:0/595996314 wait complete. 2024-09-21T07:50:14.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.013+0000 7f5ea6600640 1 Processor -- start 2024-09-21T07:50:14.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.013+0000 7f5ea6600640 1 -- start start 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.014+0000 7f5ea6600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e84005150 0x7f5ea00caed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.014+0000 7f5ea6600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea00aadc0 0x7f5ea00cdb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea6600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 0x7f5ea00ce0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea6600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00b6500 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea6600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00b6380 con 0x7f5e84005150 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea6600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea00b6680 con 0x7f5ea00aadc0 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 0x7f5ea00ce0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:14.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 0x7f5ea00ce0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37082/0 (socket says 172.21.15.88:37082) 2024-09-21T07:50:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3436416718 learned_addr learned my addr 172.21.15.88:0/3436416718 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5200640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e84005150 0x7f5ea00caed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:14.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5e97e00640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea00aadc0 0x7f5ea00cdb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea00aadc0 msgr2=0x7f5ea00cdb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea00aadc0 0x7f5ea00cdb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e84005150 msgr2=0x7f5ea00caed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e84005150 0x7f5ea00caed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3436416718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea00ab910 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.015+0000 7f5ea5200640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e84005150 0x7f5ea00caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.016+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 0x7f5ea00ce0b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800e350 tx=0x7f5e98033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.016+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98009c10 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.016+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e98044070 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.016+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea00abbf0 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.016+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e9803d450 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.016+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea014b860 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.017+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea006a820 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.018+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f5e9803d5f0 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.019+0000 7f5e9d600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5e7c078e60 0x7f5e7c07b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.019+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(65..65 src has 1..65) ==== 4943+0+0 (secure 0 0 0) 0x7f5e980bc390 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.020+0000 7f5ea5200640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5e7c078e60 0x7f5e7c07b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:14.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.020+0000 7f5ea5200640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5e7c078e60 0x7f5e7c07b370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5e90012640 tx=0x7f5e90005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:14.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.024+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e9804e020 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.220+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5ea00cf6c0 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.980+0000 7f5e9d600640 1 -- 172.21.15.88:0/3436416718 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v67) ==== 97+0+0 (secure 0 0 0) 0x7f5e98084610 con 0x7f5ea00ab1d0 2024-09-21T07:50:14.983 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.984+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5e7c078e60 msgr2=0x7f5e7c07b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.984+0000 7f5ea6600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5e7c078e60 0x7f5e7c07b370 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f5e90012640 tx=0x7f5e90005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.984+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 msgr2=0x7f5ea00ce0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.984+0000 7f5ea6600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 0x7f5ea00ce0b0 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800e350 tx=0x7f5e98033ba0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 shutdown_connections 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5e7c078e60 0x7f5e7c07b370 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea00aadc0 0x7f5ea00cdb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ab1d0 0x7f5ea00ce0b0 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 --2- 172.21.15.88:0/3436416718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e84005150 0x7f5ea00caed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:14.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 >> 172.21.15.88:0/3436416718 conn(0x7f5ea00d44f0 msgr2=0x7f5ea015f330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:14.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 shutdown_connections 2024-09-21T07:50:14.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:14.985+0000 7f5ea6600640 1 -- 172.21.15.88:0/3436416718 wait complete. 2024-09-21T07:50:15.013 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26fe00640 1 -- 172.21.15.88:0/2882978635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 msgr2=0x7fd254004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2882978635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd254004c50 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fd25c009b20 tx=0x7fd254005830 comp rx=0 tx=0).stop 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26f400640 1 -- 172.21.15.88:0/2882978635 reap_dead start 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26fe00640 1 -- 172.21.15.88:0/2882978635 shutdown_connections 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26fe00640 1 -- 172.21.15.88:0/2882978635 >> 172.21.15.88:0/2882978635 conn(0x7fd2700b1e40 msgr2=0x7fd2700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26fe00640 1 -- 172.21.15.88:0/2882978635 shutdown_connections 2024-09-21T07:50:15.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.234+0000 7fd26fe00640 1 -- 172.21.15.88:0/2882978635 wait complete. 2024-09-21T07:50:15.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.237+0000 7fd26fe00640 1 Processor -- start 2024-09-21T07:50:15.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.237+0000 7fd26fe00640 1 -- start start 2024-09-21T07:50:15.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd2700cb950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd270150c80 0x7fd2701f8910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd270151090 0x7fd2701f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2700b2f60 con 0x7fd254004800 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd27006fbb0 con 0x7fd270151090 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2700b30e0 con 0x7fd270150c80 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd2700cb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd2700cb950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37092/0 (socket says 172.21.15.88:37092) 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.238+0000 7fd26ea00640 1 -- 172.21.15.88:0/2820307957 learned_addr learned my addr 172.21.15.88:0/2820307957 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26f400640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd270151090 0x7fd2701f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26e000640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd270150c80 0x7fd2701f8910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26ea00640 1 -- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd270150c80 msgr2=0x7fd2701f8910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26ea00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd270150c80 0x7fd2701f8910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26ea00640 1 -- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd270151090 msgr2=0x7fd2701f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26ea00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd270151090 0x7fd2701f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26ea00640 1 -- 172.21.15.88:0/2820307957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27009cfb0 con 0x7fd254004800 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26f400640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd270151090 0x7fd2701f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26e000640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd270150c80 0x7fd2701f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26ea00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd2700cb950 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd260006db0 tx=0x7fd2600067a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260015070 con 0x7fd254004800 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd26000faf0 con 0x7fd254004800 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27009d290 con 0x7fd254004800 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd260011430 con 0x7fd254004800 2024-09-21T07:50:15.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.239+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2700749e0 con 0x7fd254004800 2024-09-21T07:50:15.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.240+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd27015d670 con 0x7fd254004800 2024-09-21T07:50:15.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.242+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fd2600115d0 con 0x7fd254004800 2024-09-21T07:50:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.243+0000 7fd266a00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd250078ef0 0x7fd25007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.243+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7fd26009c170 con 0x7fd254004800 2024-09-21T07:50:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.243+0000 7fd26e000640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd250078ef0 0x7fd25007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.244+0000 7fd26e000640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd250078ef0 0x7fd25007b400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd25c035000 tx=0x7fd25c02d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.248+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2600642a0 con 0x7fd254004800 2024-09-21T07:50:15.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.504+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fd2700b4600 con 0x7fd254004800 2024-09-21T07:50:15.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.505+0000 7fd266a00640 1 -- 172.21.15.88:0/2820307957 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v12) ==== 80+0+5617 (secure 0 0 0) 0x7fd260063ba0 con 0x7fd254004800 2024-09-21T07:50:15.507 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:15.507 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:50:15.507 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:50:15.507 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.507 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.508 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:15.509 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.510 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:50:15.511 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.512 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.513 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.514 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.515 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.516 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.517 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:50:15.518 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:50:15.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.alice", 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCgeu5mvQn5FhAAVOJsrbSENHWL0gKr4vrAiQ==", 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow rw path=/volumes/_nogroup/subvol30470", 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw pool=cephfs_data" 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.520 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:15.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:50:15.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:50:15.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:15.522 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.522 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:15.522 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:15.522 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd250078ef0 msgr2=0x7fd25007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd250078ef0 0x7fd25007b400 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fd25c035000 tx=0x7fd25c02d040 comp rx=0 tx=0).stop 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 msgr2=0x7fd2700cb950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd2700cb950 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd260006db0 tx=0x7fd2600067a0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 shutdown_connections 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd250078ef0 0x7fd25007b400 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd270150c80 0x7fd2701f8910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd270151090 0x7fd2701f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 --2- 172.21.15.88:0/2820307957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd254004800 0x7fd2700cb950 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 >> 172.21.15.88:0/2820307957 conn(0x7fd2700b1e40 msgr2=0x7fd2700b3590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:15.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 shutdown_connections 2024-09-21T07:50:15.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.507+0000 7fd26fe00640 1 -- 172.21.15.88:0/2820307957 wait complete. 2024-09-21T07:50:15.531 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.alice 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.752+0000 7fa86b600640 1 -- 172.21.15.88:0/2616825426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa848005350 msgr2=0x7fa84800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.752+0000 7fa86b600640 1 --2- 172.21.15.88:0/2616825426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa848005350 0x7fa84800b9c0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fa85c005fd0 tx=0x7fa848005830 comp rx=0 tx=0).stop 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.752+0000 7fa86b600640 1 -- 172.21.15.88:0/2616825426 shutdown_connections 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.752+0000 7fa86b600640 1 --2- 172.21.15.88:0/2616825426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa848005350 0x7fa84800b9c0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.752+0000 7fa86b600640 1 -- 172.21.15.88:0/2616825426 >> 172.21.15.88:0/2616825426 conn(0x7fa86406d3d0 msgr2=0x7fa8640b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.753+0000 7fa86b600640 1 -- 172.21.15.88:0/2616825426 shutdown_connections 2024-09-21T07:50:15.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.753+0000 7fa86b600640 1 -- 172.21.15.88:0/2616825426 wait complete. 2024-09-21T07:50:15.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.755+0000 7fa86b600640 1 Processor -- start 2024-09-21T07:50:15.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.755+0000 7fa86b600640 1 -- start start 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.756+0000 7fa86b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa848005350 0x7fa8640b0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.756+0000 7fa86b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 0x7fa8640b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.756+0000 7fa86b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8640c5720 0x7fa8640c5b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.756+0000 7fa86b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa864157330 con 0x7fa8640cb5e0 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.756+0000 7fa86b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8641571b0 con 0x7fa8640c5720 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.756+0000 7fa86b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa86415ce30 con 0x7fa848005350 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 0x7fa8640b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 0x7fa8640b1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37100/0 (socket says 172.21.15.88:37100) 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 -- 172.21.15.88:0/1877049137 learned_addr learned my addr 172.21.15.88:0/1877049137 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa868c00640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa848005350 0x7fa8640b0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa869600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8640c5720 0x7fa8640c5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 -- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa848005350 msgr2=0x7fa8640b0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa848005350 0x7fa8640b0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 -- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8640c5720 msgr2=0x7fa8640c5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8640c5720 0x7fa8640c5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 -- 172.21.15.88:0/1877049137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8640cc500 con 0x7fa8640cb5e0 2024-09-21T07:50:15.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa869600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8640c5720 0x7fa8640c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.757+0000 7fa85be00640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 0x7fa8640b1220 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa84c003ed0 tx=0x7fa84c001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.758+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c015070 con 0x7fa8640cb5e0 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.758+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa84c011840 con 0x7fa8640cb5e0 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.758+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa84c011c90 con 0x7fa8640cb5e0 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.758+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8640cab00 con 0x7fa8640cb5e0 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.758+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8640cb0c0 con 0x7fa8640cb5e0 2024-09-21T07:50:15.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.759+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8640cc270 con 0x7fa8640cb5e0 2024-09-21T07:50:15.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.760+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa84c018070 con 0x7fa8640cb5e0 2024-09-21T07:50:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.761+0000 7fa861600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa840078ef0 0x7fa84007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.761+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7fa84c09ba00 con 0x7fa8640cb5e0 2024-09-21T07:50:15.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.762+0000 7fa868c00640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa840078ef0 0x7fa84007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:15.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.762+0000 7fa868c00640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa840078ef0 0x7fa84007b400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa854001880 tx=0x7fa854006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:15.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:15.766+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa84c063560 con 0x7fa8640cb5e0 2024-09-21T07:50:16.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.027+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.alice"} v 0) -- 0x7fa8640afa70 con 0x7fa8640cb5e0 2024-09-21T07:50:16.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.031+0000 7fa861600640 1 -- 172.21.15.88:0/1877049137 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.alice"}]=0 v13) ==== 82+0+0 (secure 0 0 0) 0x7fa84c067f90 con 0x7fa8640cb5e0 2024-09-21T07:50:16.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa840078ef0 msgr2=0x7fa84007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa840078ef0 0x7fa84007b400 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa854001880 tx=0x7fa854006040 comp rx=0 tx=0).stop 2024-09-21T07:50:16.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 msgr2=0x7fa8640b1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 0x7fa8640b1220 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fa84c003ed0 tx=0x7fa84c001630 comp rx=0 tx=0).stop 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 shutdown_connections 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa840078ef0 0x7fa84007b400 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8640c5720 0x7fa8640c5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa8640cb5e0 0x7fa8640b1220 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 --2- 172.21.15.88:0/1877049137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa848005350 0x7fa8640b0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 >> 172.21.15.88:0/1877049137 conn(0x7fa86406d3d0 msgr2=0x7fa86415b8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 shutdown_connections 2024-09-21T07:50:16.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.034+0000 7fa86b600640 1 -- 172.21.15.88:0/1877049137 wait complete. 2024-09-21T07:50:16.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.278+0000 7f7e73000640 1 -- 172.21.15.88:0/3331683997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 msgr2=0x7f7e6c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.278+0000 7f7e73000640 1 --2- 172.21.15.88:0/3331683997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1535c0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c009830 tx=0x7f7e5c02a480 comp rx=0 tx=0).stop 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.278+0000 7f7e73000640 1 -- 172.21.15.88:0/3331683997 shutdown_connections 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.278+0000 7f7e73000640 1 --2- 172.21.15.88:0/3331683997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1535c0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.278+0000 7f7e73000640 1 -- 172.21.15.88:0/3331683997 >> 172.21.15.88:0/3331683997 conn(0x7f7e6c06d730 msgr2=0x7f7e6c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.279+0000 7f7e73000640 1 -- 172.21.15.88:0/3331683997 shutdown_connections 2024-09-21T07:50:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.279+0000 7f7e73000640 1 -- 172.21.15.88:0/3331683997 wait complete. 2024-09-21T07:50:16.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.281+0000 7f7e73000640 1 Processor -- start 2024-09-21T07:50:16.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.281+0000 7f7e73000640 1 -- start start 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.282+0000 7f7e73000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e6c0ad0e0 0x7f7e6c0b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.282+0000 7f7e73000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e6c0b3550 0x7f7e6c148880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.282+0000 7f7e73000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1f5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.282+0000 7f7e73000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c06f240 con 0x7f7e6c15d760 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.282+0000 7f7e73000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e6c0b4e30 con 0x7f7e6c0b3550 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.282+0000 7f7e73000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e6c0b4070 con 0x7f7e6c0ad0e0 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1f5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1f5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48772/0 (socket says 172.21.15.88:48772) 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 -- 172.21.15.88:0/4161731505 learned_addr learned my addr 172.21.15.88:0/4161731505 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e6be00640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e6c0ad0e0 0x7f7e6c0b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e6b400640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e6c0b3550 0x7f7e6c148880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 -- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e6c0ad0e0 msgr2=0x7f7e6c0b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e6c0ad0e0 0x7f7e6c0b1b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 -- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e6c0b3550 msgr2=0x7f7e6c148880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e6c0b3550 0x7f7e6c148880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 -- 172.21.15.88:0/4161731505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e6c156010 con 0x7f7e6c15d760 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.283+0000 7f7e71000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1f5f50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400c760 tx=0x7f7e6400cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.284+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e64003da0 con 0x7f7e6c15d760 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.284+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e6400ce10 con 0x7f7e6c15d760 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.284+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e6c0749a0 con 0x7f7e6c15d760 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.284+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e640052d0 con 0x7f7e6c15d760 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.284+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e6c1f5a30 con 0x7f7e6c15d760 2024-09-21T07:50:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.285+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e6c156280 con 0x7f7e6c15d760 2024-09-21T07:50:16.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.286+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f7e64005470 con 0x7f7e6c15d760 2024-09-21T07:50:16.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.288+0000 7f7e63e00640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7e4c078ef0 0x7f7e4c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.288+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7f7e6409ae00 con 0x7f7e6c15d760 2024-09-21T07:50:16.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.288+0000 7f7e6be00640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7e4c078ef0 0x7f7e4c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.289+0000 7f7e6be00640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7e4c078ef0 0x7f7e4c07b400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c034000 tx=0x7f7e5c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:16.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.292+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e640628e0 con 0x7f7e6c15d760 2024-09-21T07:50:16.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.480+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f7e6c1535c0 con 0x7f7e6c15d760 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.481+0000 7f7e63e00640 1 -- 172.21.15.88:0/4161731505 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v69) ==== 84+0+65 (secure 0 0 0) 0x7f7e64067310 con 0x7f7e6c15d760 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:16.483 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7e4c078ef0 msgr2=0x7f7e4c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7e4c078ef0 0x7f7e4c07b400 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c034000 tx=0x7f7e5c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 msgr2=0x7f7e6c1f5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1f5f50 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f7e6400c760 tx=0x7f7e6400cc20 comp rx=0 tx=0).stop 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 shutdown_connections 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7e4c078ef0 0x7f7e4c07b400 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e6c15d760 0x7f7e6c1f5f50 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e6c0b3550 0x7f7e6c148880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 --2- 172.21.15.88:0/4161731505 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e6c0ad0e0 0x7f7e6c0b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 >> 172.21.15.88:0/4161731505 conn(0x7f7e6c06d730 msgr2=0x7f7e6c06f420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 shutdown_connections 2024-09-21T07:50:16.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.483+0000 7f7e73000640 1 -- 172.21.15.88:0/4161731505 wait complete. 2024-09-21T07:50:16.507 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:50:16.507 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 -- 172.21.15.88:0/3267385126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 msgr2=0x7f0e3406ee80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 --2- 172.21.15.88:0/3267385126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3406ee80 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0e24009830 tx=0x7f0e2402a480 comp rx=0 tx=0).stop 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 -- 172.21.15.88:0/3267385126 shutdown_connections 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 --2- 172.21.15.88:0/3267385126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3406ee80 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 -- 172.21.15.88:0/3267385126 >> 172.21.15.88:0/3267385126 conn(0x7f0e340b1dd0 msgr2=0x7f0e340b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 -- 172.21.15.88:0/3267385126 shutdown_connections 2024-09-21T07:50:16.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.728+0000 7f0e38a00640 1 -- 172.21.15.88:0/3267385126 wait complete. 2024-09-21T07:50:16.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.731+0000 7f0e38a00640 1 Processor -- start 2024-09-21T07:50:16.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.731+0000 7f0e38a00640 1 -- start start 2024-09-21T07:50:16.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e38a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e34072e20 0x7f0e341442b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e38a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3414dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e38a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e3415f130 0x7f0e3414e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e38a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e341668c0 con 0x7f0e340b2ef0 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e38a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e34166740 con 0x7f0e34072e20 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e38a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e34166a40 con 0x7f0e3415f130 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e2aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3414dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e2aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3414dec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48794/0 (socket says 172.21.15.88:48794) 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.732+0000 7f0e2aa00640 1 -- 172.21.15.88:0/2467724746 learned_addr learned my addr 172.21.15.88:0/2467724746 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e33400640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e34072e20 0x7f0e341442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e33e00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e3415f130 0x7f0e3414e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2aa00640 1 -- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e3415f130 msgr2=0x7f0e3414e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2aa00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e3415f130 0x7f0e3414e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2aa00640 1 -- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e34072e20 msgr2=0x7f0e341442b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2aa00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e34072e20 0x7f0e341442b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2aa00640 1 -- 172.21.15.88:0/2467724746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e34144bd0 con 0x7f0e340b2ef0 2024-09-21T07:50:16.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e33e00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e3415f130 0x7f0e3414e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2aa00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3414dec0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c3a0 tx=0x7f0e1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c011790 con 0x7f0e340b2ef0 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e1c017070 con 0x7f0e340b2ef0 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e3415ccd0 con 0x7f0e340b2ef0 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.733+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c011cc0 con 0x7f0e340b2ef0 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.734+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e340b15e0 con 0x7f0e340b2ef0 2024-09-21T07:50:16.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.734+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e3406ee80 con 0x7f0e340b2ef0 2024-09-21T07:50:16.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.736+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0e1c01c460 con 0x7f0e340b2ef0 2024-09-21T07:50:16.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.737+0000 7f0e2be00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0e10078ef0 0x7f0e1007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:16.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.737+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(69..69 src has 1..69) ==== 4857+0+0 (secure 0 0 0) 0x7f0e1c09af50 con 0x7f0e340b2ef0 2024-09-21T07:50:16.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.737+0000 7f0e33400640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0e10078ef0 0x7f0e1007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:16.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.738+0000 7f0e33400640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0e10078ef0 0x7f0e1007b400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e24034000 tx=0x7f0e24004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:16.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.741+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e1c063100 con 0x7f0e340b2ef0 2024-09-21T07:50:16.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:16.926+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f0e34070050 con 0x7f0e340b2ef0 2024-09-21T07:50:17.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.034+0000 7f0e2be00640 1 -- 172.21.15.88:0/2467724746 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v70) ==== 138+0+0 (secure 0 0 0) 0x7f0e1c062a00 con 0x7f0e340b2ef0 2024-09-21T07:50:17.036 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0e10078ef0 msgr2=0x7f0e1007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0e10078ef0 0x7f0e1007b400 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f0e24034000 tx=0x7f0e24004730 comp rx=0 tx=0).stop 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 msgr2=0x7f0e3414dec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3414dec0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f0e1c00c3a0 tx=0x7f0e1c00c860 comp rx=0 tx=0).stop 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 shutdown_connections 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0e10078ef0 0x7f0e1007b400 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e3415f130 0x7f0e3414e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.036+0000 7f0e38a00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e340b2ef0 0x7f0e3414dec0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.037+0000 7f0e38a00640 1 --2- 172.21.15.88:0/2467724746 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e34072e20 0x7f0e341442b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.037+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 >> 172.21.15.88:0/2467724746 conn(0x7f0e340b1dd0 msgr2=0x7f0e341527c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:17.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.037+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 shutdown_connections 2024-09-21T07:50:17.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.037+0000 7f0e38a00640 1 -- 172.21.15.88:0/2467724746 wait complete. 2024-09-21T07:50:17.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:50:17.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 -- 172.21.15.88:0/3628021911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c15f9e0 msgr2=0x7f1e9c147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:17.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/3628021911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c15f9e0 0x7f1e9c147b90 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c070440 tx=0x7f1e8c02a440 comp rx=0 tx=0).stop 2024-09-21T07:50:17.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 -- 172.21.15.88:0/3628021911 shutdown_connections 2024-09-21T07:50:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/3628021911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e9c073540 0x7f1e9c073950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/3628021911 >> v2:172.21.15.175:3300/0 conn(0x7f1e9c0aee00 0x7f1e9c0af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/3628021911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c15f9e0 0x7f1e9c147b90 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 -- 172.21.15.88:0/3628021911 >> 172.21.15.88:0/3628021911 conn(0x7f1e9c06d1c0 msgr2=0x7f1e9c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 -- 172.21.15.88:0/3628021911 shutdown_connections 2024-09-21T07:50:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.293+0000 7f1ea0e00640 1 -- 172.21.15.88:0/3628021911 wait complete. 2024-09-21T07:50:17.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.295+0000 7f1ea0e00640 1 Processor -- start 2024-09-21T07:50:17.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.295+0000 7f1ea0e00640 1 -- start start 2024-09-21T07:50:17.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1ea0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e9c073540 0x7f1e9c0c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1ea0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 0x7f1e9c1f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1ea0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e9c15f9e0 0x7f1e9c137920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1ea0e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9c06f950 con 0x7f1e9c0aee00 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1ea0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e9c06f7d0 con 0x7f1e9c073540 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1ea0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e9c06fad0 con 0x7f1e9c15f9e0 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1e99400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 0x7f1e9c1f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1e99400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 0x7f1e9c1f9290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48820/0 (socket says 172.21.15.88:48820) 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.297+0000 7f1e99400640 1 -- 172.21.15.88:0/4229198203 learned_addr learned my addr 172.21.15.88:0/4229198203 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e9a800640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e9c15f9e0 0x7f1e9c137920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e9c073540 0x7f1e9c0c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99400640 1 -- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e9c15f9e0 msgr2=0x7f1e9c137920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99400640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e9c15f9e0 0x7f1e9c137920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99400640 1 -- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e9c073540 msgr2=0x7f1e9c0c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:17.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99400640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e9c073540 0x7f1e9c0c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:17.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99400640 1 -- 172.21.15.88:0/4229198203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e9c0c8e40 con 0x7f1e9c0aee00 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e99400640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 0x7f1e9c1f9290 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800e790 tx=0x7f1e8800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e8800eea0 con 0x7f1e9c0aee00 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e8800cd50 con 0x7f1e9c0aee00 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e88010690 con 0x7f1e9c0aee00 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e9c0c9120 con 0x7f1e9c0aee00 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.298+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e9c0c96e0 con 0x7f1e9c0aee00 2024-09-21T07:50:17.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.299+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e9c0c8bb0 con 0x7f1e9c0aee00 2024-09-21T07:50:17.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.301+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f1e88010830 con 0x7f1e9c0aee00 2024-09-21T07:50:17.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.302+0000 7f1e92000640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1e7c078f20 0x7f1e7c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:17.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.302+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 5227+0+0 (secure 0 0 0) 0x7f1e8809bb20 con 0x7f1e9c0aee00 2024-09-21T07:50:17.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.302+0000 7f1e99e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1e7c078f20 0x7f1e7c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:17.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.303+0000 7f1e99e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1e7c078f20 0x7f1e7c07b430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c033000 tx=0x7f1e8c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:17.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.307+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e88063c70 con 0x7f1e9c0aee00 2024-09-21T07:50:17.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:17.494+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1e9c06fff0 con 0x7f1e9c0aee00 2024-09-21T07:50:18.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.042+0000 7f1e92000640 1 -- 172.21.15.88:0/4229198203 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v71) ==== 170+0+0 (secure 0 0 0) 0x7f1e88063570 con 0x7f1e9c0aee00 2024-09-21T07:50:18.044 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:50:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1e7c078f20 msgr2=0x7f1e7c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1e7c078f20 0x7f1e7c07b430 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f1e8c033000 tx=0x7f1e8c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 msgr2=0x7f1e9c1f9290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 0x7f1e9c1f9290 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800e790 tx=0x7f1e8800ec50 comp rx=0 tx=0).stop 2024-09-21T07:50:18.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 shutdown_connections 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1e7c078f20 0x7f1e7c07b430 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e9c15f9e0 0x7f1e9c137920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e9c0aee00 0x7f1e9c1f9290 secure :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f1e8800e790 tx=0x7f1e8800ec50 comp rx=0 tx=0).stop 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 --2- 172.21.15.88:0/4229198203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e9c073540 0x7f1e9c0c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 >> 172.21.15.88:0/4229198203 conn(0x7f1e9c06d1c0 msgr2=0x7f1e9c0b22d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.045+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 shutdown_connections 2024-09-21T07:50:18.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.046+0000 7f1ea0e00640 1 -- 172.21.15.88:0/4229198203 wait complete. 2024-09-21T07:50:18.092 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 -- 172.21.15.88:0/1013726887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 msgr2=0x7ff6480af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 --2- 172.21.15.88:0/1013726887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff6480af0d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7ff640005fd0 tx=0x7ff64002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 -- 172.21.15.88:0/1013726887 shutdown_connections 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 --2- 172.21.15.88:0/1013726887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff6480af0d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 --2- 172.21.15.88:0/1013726887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 0x7ff6480ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 -- 172.21.15.88:0/1013726887 >> 172.21.15.88:0/1013726887 conn(0x7ff64806d620 msgr2=0x7ff6480b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 -- 172.21.15.88:0/1013726887 shutdown_connections 2024-09-21T07:50:18.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.313+0000 7ff64e200640 1 -- 172.21.15.88:0/1013726887 wait complete. 2024-09-21T07:50:18.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.316+0000 7ff64e200640 1 Processor -- start 2024-09-21T07:50:18.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.316+0000 7ff64e200640 1 -- start start 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff64e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6480b3f30 0x7ff648157330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff64e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff648155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff64e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 0x7ff648145980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff64e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff6480af9a0 con 0x7ff6480cd000 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff64e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff6480af820 con 0x7ff6480b3f30 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff64e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff6480afb20 con 0x7ff648156ca0 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.317+0000 7ff63fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff648155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff648155df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48836/0 (socket says 172.21.15.88:48836) 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 -- 172.21.15.88:0/2725060423 learned_addr learned my addr 172.21.15.88:0/2725060423 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff64d800640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 0x7ff648145980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 -- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 msgr2=0x7ff648145980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:18.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff64ce00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6480b3f30 0x7ff648157330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 0x7ff648145980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 -- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6480b3f30 msgr2=0x7ff648157330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6480b3f30 0x7ff648157330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 -- 172.21.15.88:0/2725060423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff6481f8e90 con 0x7ff6480cd000 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff64d800640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 0x7ff648145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff64ce00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6480b3f30 0x7ff648157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff63fe00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff648155df0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff630009fd0 tx=0x7ff63000ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63000ccc0 con 0x7ff6480cd000 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff63000ee70 con 0x7ff6480cd000 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff63000ce60 con 0x7ff6480cd000 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.318+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff6481f9170 con 0x7ff6480cd000 2024-09-21T07:50:18.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.319+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6481f9730 con 0x7ff6480cd000 2024-09-21T07:50:18.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.320+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff6481f8c00 con 0x7ff6480cd000 2024-09-21T07:50:18.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.321+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7ff630005260 con 0x7ff6480cd000 2024-09-21T07:50:18.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.322+0000 7ff645600640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff624078ef0 0x7ff62407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:18.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.322+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(71..71 src has 1..71) ==== 5609+0+0 (secure 0 0 0) 0x7ff63009c240 con 0x7ff6480cd000 2024-09-21T07:50:18.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.323+0000 7ff64ce00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff624078ef0 0x7ff62407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:18.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.323+0000 7ff64ce00640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff624078ef0 0x7ff62407b400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f60 tx=0x7ff638009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:18.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.326+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff630064100 con 0x7ff6480cd000 2024-09-21T07:50:18.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:18.546+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7ff6481f54c0 con 0x7ff6480cd000 2024-09-21T07:50:19.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.044+0000 7ff645600640 1 -- 172.21.15.88:0/2725060423 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v42) ==== 174+0+0 (secure 0 0 0) 0x7ff630063a00 con 0x7ff6480cd000 2024-09-21T07:50:19.047 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 10 and data pool 11 2024-09-21T07:50:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff624078ef0 msgr2=0x7ff62407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff624078ef0 0x7ff62407b400 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff638000f60 tx=0x7ff638009040 comp rx=0 tx=0).stop 2024-09-21T07:50:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 msgr2=0x7ff648155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff648155df0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7ff630009fd0 tx=0x7ff63000ec20 comp rx=0 tx=0).stop 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 shutdown_connections 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7ff624078ef0 0x7ff62407b400 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff648156ca0 0x7ff648145980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff6480cd000 0x7ff648155df0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 --2- 172.21.15.88:0/2725060423 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff6480b3f30 0x7ff648157330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.048+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 >> 172.21.15.88:0/2725060423 conn(0x7ff64806d620 msgr2=0x7ff6480b0100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:19.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.049+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 shutdown_connections 2024-09-21T07:50:19.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.050+0000 7ff64e200640 1 -- 172.21.15.88:0/2725060423 wait complete. 2024-09-21T07:50:19.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 -- 172.21.15.88:0/3596479025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 msgr2=0x7f42d40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 --2- 172.21.15.88:0/3596479025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d40ad4f0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f42d4075250 tx=0x7f42bc02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 -- 172.21.15.88:0/3596479025 shutdown_connections 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 --2- 172.21.15.88:0/3596479025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42d4160310 0x7f42d4160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 --2- 172.21.15.88:0/3596479025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d40ad4f0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 --2- 172.21.15.88:0/3596479025 >> v2:172.21.15.175:3300/0 conn(0x7f42d415d760 0x7f42d41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 -- 172.21.15.88:0/3596479025 >> 172.21.15.88:0/3596479025 conn(0x7f42d406d730 msgr2=0x7f42d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:19.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.315+0000 7f42da800640 1 -- 172.21.15.88:0/3596479025 shutdown_connections 2024-09-21T07:50:19.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.316+0000 7f42da800640 1 -- 172.21.15.88:0/3596479025 wait complete. 2024-09-21T07:50:19.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.318+0000 7f42da800640 1 Processor -- start 2024-09-21T07:50:19.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.318+0000 7f42da800640 1 -- start start 2024-09-21T07:50:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.319+0000 7f42da800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d41f5400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.319+0000 7f42da800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42d415d760 0x7f42d40b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.319+0000 7f42da800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42d4160310 0x7f42d40b1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.319+0000 7f42da800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40b4da0 con 0x7f42d40ad0e0 2024-09-21T07:50:19.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.319+0000 7f42da800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f42d40b4c20 con 0x7f42d4160310 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.319+0000 7f42da800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f42d4160c60 con 0x7f42d415d760 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d41f5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d41f5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48856/0 (socket says 172.21.15.88:48856) 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 -- 172.21.15.88:0/827941061 learned_addr learned my addr 172.21.15.88:0/827941061 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42cbe00640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42d415d760 0x7f42d40b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3e00640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42d4160310 0x7f42d40b1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 -- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42d415d760 msgr2=0x7f42d40b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42d415d760 0x7f42d40b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 -- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42d4160310 msgr2=0x7f42d40b1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42d4160310 0x7f42d40b1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 -- 172.21.15.88:0/827941061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42d41f5940 con 0x7f42d40ad0e0 2024-09-21T07:50:19.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.320+0000 7f42d3400640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d41f5400 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c9a0 tx=0x7f42c400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:19.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.321+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c4007bb0 con 0x7f42d40ad0e0 2024-09-21T07:50:19.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.321+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42c4007e40 con 0x7f42d40ad0e0 2024-09-21T07:50:19.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.321+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42c400f690 con 0x7f42d40ad0e0 2024-09-21T07:50:19.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.321+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42d414f610 con 0x7f42d40ad0e0 2024-09-21T07:50:19.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.321+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42d414fbd0 con 0x7f42d40ad0e0 2024-09-21T07:50:19.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.322+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f42d41535c0 con 0x7f42d40ad0e0 2024-09-21T07:50:19.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.323+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f42c400f830 con 0x7f42d40ad0e0 2024-09-21T07:50:19.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.324+0000 7f42cb400640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f42b0078f20 0x7f42b007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.324+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f42c409bb00 con 0x7f42d40ad0e0 2024-09-21T07:50:19.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.325+0000 7f42cbe00640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f42b0078f20 0x7f42b007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.325+0000 7f42cbe00640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f42b0078f20 0x7f42b007b430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f42bc033000 tx=0x7f42bc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:19.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.329+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42c4014030 con 0x7f42d40ad0e0 2024-09-21T07:50:19.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.513+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f42d415e1e0 con 0x7f42d40ad0e0 2024-09-21T07:50:19.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.515+0000 7f42cb400640 1 -- 172.21.15.88:0/827941061 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16693 (secure 0 0 0) 0x7f42c4063960 con 0x7f42d40ad0e0 2024-09-21T07:50:19.517 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:19.517 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":72,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:18.548082+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:16.939762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:17.507399+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:19.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f42b0078f20 msgr2=0x7f42b007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f42b0078f20 0x7f42b007b430 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f42bc033000 tx=0x7f42bc004730 comp rx=0 tx=0).stop 2024-09-21T07:50:19.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 msgr2=0x7f42d41f5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d41f5400 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f42c400c9a0 tx=0x7f42c400ce60 comp rx=0 tx=0).stop 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 shutdown_connections 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f42b0078f20 0x7f42b007b430 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f42d415d760 0x7f42d40b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f42d4160310 0x7f42d40b1ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 --2- 172.21.15.88:0/827941061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42d40ad0e0 0x7f42d41f5400 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 >> 172.21.15.88:0/827941061 conn(0x7f42d406d730 msgr2=0x7f42d40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 shutdown_connections 2024-09-21T07:50:19.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.517+0000 7f42da800640 1 -- 172.21.15.88:0/827941061 wait complete. 2024-09-21T07:50:19.541 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:19.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 -- 172.21.15.88:0/3346875538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a80730b0 msgr2=0x7f37a80adf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 --2- 172.21.15.88:0/3346875538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a80730b0 0x7f37a80adf90 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f37a0005fd0 tx=0x7f37a002bbe0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 -- 172.21.15.88:0/3346875538 shutdown_connections 2024-09-21T07:50:19.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 --2- 172.21.15.88:0/3346875538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a80730b0 0x7f37a80adf90 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 -- 172.21.15.88:0/3346875538 >> 172.21.15.88:0/3346875538 conn(0x7f37a806d730 msgr2=0x7f37a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:19.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 -- 172.21.15.88:0/3346875538 shutdown_connections 2024-09-21T07:50:19.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.758+0000 7f37b0400640 1 -- 172.21.15.88:0/3346875538 wait complete. 2024-09-21T07:50:19.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.760+0000 7f37b0400640 1 Processor -- start 2024-09-21T07:50:19.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.760+0000 7f37b0400640 1 -- start start 2024-09-21T07:50:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37b0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 0x7f37a814efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37b0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37a80730b0 0x7f37a813a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37b0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a8148c20 0x7f37a81e8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37b0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37a815aff0 con 0x7f37a80707b0 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37b0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37a80b4320 con 0x7f37a8148c20 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37b0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37a815b170 con 0x7f37a80730b0 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 0x7f37a814efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37ada00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 0x7f37a814efe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48860/0 (socket says 172.21.15.88:48860) 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37ada00640 1 -- 172.21.15.88:0/4183143361 learned_addr learned my addr 172.21.15.88:0/4183143361 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37ad000640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37a80730b0 0x7f37a813a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.762+0000 7f37ae400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a8148c20 0x7f37a81e8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37ada00640 1 -- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37a80730b0 msgr2=0x7f37a813a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37ada00640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37a80730b0 0x7f37a813a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37ada00640 1 -- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a8148c20 msgr2=0x7f37a81e8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37ada00640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a8148c20 0x7f37a81e8c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37ada00640 1 -- 172.21.15.88:0/4183143361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37a813ac70 con 0x7f37a80707b0 2024-09-21T07:50:19.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37ada00640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 0x7f37a814efe0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c370 tx=0x7f379c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c017070 con 0x7f37a80707b0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f379c0053c0 con 0x7f37a80707b0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37a8151740 con 0x7f37a80707b0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f379c003850 con 0x7f37a80707b0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.763+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37a8158ad0 con 0x7f37a80707b0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.765+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37a80ad880 con 0x7f37a80707b0 2024-09-21T07:50:19.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.766+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f379c01a070 con 0x7f37a80707b0 2024-09-21T07:50:19.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.766+0000 7f37a5600640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3788078f20 0x7f378807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:19.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.767+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 5709+0+0 (secure 0 0 0) 0x7f379c09a6a0 con 0x7f37a80707b0 2024-09-21T07:50:19.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.767+0000 7f37ad000640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3788078f20 0x7f378807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:19.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.768+0000 7f37ad000640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3788078f20 0x7f378807b430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3794007fd0 tx=0x7f3794001390 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:19.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.771+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f37a813ac70 con 0x7f37a80707b0 2024-09-21T07:50:19.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.956+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f37a80af610 con 0x7f37a80707b0 2024-09-21T07:50:19.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.957+0000 7f37a5600640 1 -- 172.21.15.88:0/4183143361 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+16693 (secure 0 0 0) 0x7f379c061f30 con 0x7f37a80707b0 2024-09-21T07:50:19.959 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:19.960 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":72,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:18.548082+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:16.939762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:17.507399+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3788078f20 msgr2=0x7f378807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3788078f20 0x7f378807b430 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f3794007fd0 tx=0x7f3794001390 comp rx=0 tx=0).stop 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 msgr2=0x7f37a814efe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 0x7f37a814efe0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f379c00c370 tx=0x7f379c00c830 comp rx=0 tx=0).stop 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 shutdown_connections 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f3788078f20 0x7f378807b430 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37a80730b0 0x7f37a813a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37a8148c20 0x7f37a81e8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 --2- 172.21.15.88:0/4183143361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37a80707b0 0x7f37a814efe0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:19.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.959+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 >> 172.21.15.88:0/4183143361 conn(0x7f37a806d730 msgr2=0x7f37a806f480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:19.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.960+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 shutdown_connections 2024-09-21T07:50:19.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:19.960+0000 7f37b0400640 1 -- 172.21.15.88:0/4183143361 wait complete. 2024-09-21T07:50:19.984 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 -- 172.21.15.88:0/2052049797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145a40 msgr2=0x7f5f24073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 --2- 172.21.15.88:0/2052049797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145a40 0x7f5f24073540 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f5f241464e0 tx=0x7f5f1002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 -- 172.21.15.88:0/2052049797 shutdown_connections 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 --2- 172.21.15.88:0/2052049797 >> v2:172.21.15.175:3300/0 conn(0x7f5f24073a80 0x7f5f24160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 --2- 172.21.15.88:0/2052049797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145a40 0x7f5f24073540 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 --2- 172.21.15.88:0/2052049797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145550 0x7f5f2406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 -- 172.21.15.88:0/2052049797 >> 172.21.15.88:0/2052049797 conn(0x7f5f2406d1c0 msgr2=0x7f5f2406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 -- 172.21.15.88:0/2052049797 shutdown_connections 2024-09-21T07:50:20.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.208+0000 7f5f2b400640 1 -- 172.21.15.88:0/2052049797 wait complete. 2024-09-21T07:50:20.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.211+0000 7f5f2b400640 1 Processor -- start 2024-09-21T07:50:20.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.211+0000 7f5f2b400640 1 -- start start 2024-09-21T07:50:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f2b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24073a80 0x7f5f24144690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f2b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 0x7f5f2415bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f2b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145a40 0x7f5f241f8e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f2b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24160c20 con 0x7f5f24145550 2024-09-21T07:50:20.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f2b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f24160aa0 con 0x7f5f24073a80 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f2b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f24160da0 con 0x7f5f24145a40 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 0x7f5f2415bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f23e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 0x7f5f2415bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48884/0 (socket says 172.21.15.88:48884) 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.212+0000 7f5f23e00640 1 -- 172.21.15.88:0/1049702025 learned_addr learned my addr 172.21.15.88:0/1049702025 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f29400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145a40 0x7f5f241f8e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f28a00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24073a80 0x7f5f24144690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f23e00640 1 -- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145a40 msgr2=0x7f5f241f8e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f23e00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145a40 0x7f5f241f8e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f23e00640 1 -- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24073a80 msgr2=0x7f5f24144690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f23e00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24073a80 0x7f5f24144690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f23e00640 1 -- 172.21.15.88:0/1049702025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f24141cd0 con 0x7f5f24145550 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f29400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145a40 0x7f5f241f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:20.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f23e00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 0x7f5f2415bd00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034040 tx=0x7f5f100039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:20.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.213+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f1003e030 con 0x7f5f24145550 2024-09-21T07:50:20.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.214+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f100047a0 con 0x7f5f24145550 2024-09-21T07:50:20.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.214+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f10035040 con 0x7f5f24145550 2024-09-21T07:50:20.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.214+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f24141fb0 con 0x7f5f24145550 2024-09-21T07:50:20.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.214+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f24142570 con 0x7f5f24145550 2024-09-21T07:50:20.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.215+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f2406fb40 con 0x7f5f24145550 2024-09-21T07:50:20.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.216+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f5f10041030 con 0x7f5f24145550 2024-09-21T07:50:20.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.217+0000 7f5f20c00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5f04078ef0 0x7f5f0407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:20.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.217+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f5f100bbbe0 con 0x7f5f24145550 2024-09-21T07:50:20.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.217+0000 7f5f28a00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5f04078ef0 0x7f5f0407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:20.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.218+0000 7f5f28a00640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5f04078ef0 0x7f5f0407b400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005fd0 tx=0x7f5f18005850 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:20.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.221+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f10083ac0 con 0x7f5f24145550 2024-09-21T07:50:20.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:20.442+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f5f240cec80 con 0x7f5f24145550 2024-09-21T07:50:20.987 INFO:tasks.ceph.mds.c.smithi088.stderr:2024-09-21T07:50:20.985+0000 7ff95fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:50:21.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.070+0000 7f5f20c00640 1 -- 172.21.15.88:0/1049702025 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) ==== 118+0+0 (secure 0 0 0) 0x7f5f100833c0 con 0x7f5f24145550 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5f04078ef0 msgr2=0x7f5f0407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5f04078ef0 0x7f5f0407b400 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5f18005fd0 tx=0x7f5f18005850 comp rx=0 tx=0).stop 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 msgr2=0x7f5f2415bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 0x7f5f2415bd00 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f5f10034040 tx=0x7f5f100039e0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 shutdown_connections 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5f04078ef0 0x7f5f0407b400 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f24145a40 0x7f5f241f8e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f24145550 0x7f5f2415bd00 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 --2- 172.21.15.88:0/1049702025 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f24073a80 0x7f5f24144690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 >> 172.21.15.88:0/1049702025 conn(0x7f5f2406d1c0 msgr2=0x7f5f240b20b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:21.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.073+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 shutdown_connections 2024-09-21T07:50:21.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.074+0000 7f5f2b400640 1 -- 172.21.15.88:0/1049702025 wait complete. 2024-09-21T07:50:21.102 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:50:21.102 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:50:21.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 -- 172.21.15.88:0/245412070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec156ca0 msgr2=0x7feaec0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 --2- 172.21.15.88:0/245412070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec156ca0 0x7feaec0ccac0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fead4009830 tx=0x7fead402a510 comp rx=0 tx=0).stop 2024-09-21T07:50:21.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 -- 172.21.15.88:0/245412070 shutdown_connections 2024-09-21T07:50:21.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 --2- 172.21.15.88:0/245412070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec156ca0 0x7feaec0ccac0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 -- 172.21.15.88:0/245412070 >> 172.21.15.88:0/245412070 conn(0x7feaec06d620 msgr2=0x7feaec0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:21.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 -- 172.21.15.88:0/245412070 shutdown_connections 2024-09-21T07:50:21.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.324+0000 7feaf1c00640 1 -- 172.21.15.88:0/245412070 wait complete. 2024-09-21T07:50:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.326+0000 7feaf1c00640 1 Processor -- start 2024-09-21T07:50:21.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.327+0000 7feaf1c00640 1 -- start start 2024-09-21T07:50:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feaf1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaec0b3f30 0x7feaec1f3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feaf1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 0x7feaec1f7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feaf1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaec156ca0 0x7feaec145570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feaf1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feaec0af7e0 con 0x7feaec0cd000 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feaf1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feaec070370 con 0x7feaec156ca0 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feaf1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feaec0af960 con 0x7feaec0b3f30 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feae3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 0x7feaec1f7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feae3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 0x7feaec1f7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48896/0 (socket says 172.21.15.88:48896) 2024-09-21T07:50:21.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.328+0000 7feae3400640 1 -- 172.21.15.88:0/907209085 learned_addr learned my addr 172.21.15.88:0/907209085 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae3400640 1 -- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaec0b3f30 msgr2=0x7feaec1f3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae3400640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaec0b3f30 0x7feaec1f3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae3400640 1 -- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaec156ca0 msgr2=0x7feaec145570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae3400640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaec156ca0 0x7feaec145570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae3400640 1 -- 172.21.15.88:0/907209085 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec09bcc0 con 0x7feaec0cd000 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae3400640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 0x7feaec1f7640 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fead402ab00 tx=0x7fead4033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4005c90 con 0x7feaec0cd000 2024-09-21T07:50:21.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fead4031e50 con 0x7feaec0cd000 2024-09-21T07:50:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec09bfa0 con 0x7feaec0cd000 2024-09-21T07:50:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.329+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fead4005260 con 0x7feaec0cd000 2024-09-21T07:50:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.330+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec09c560 con 0x7feaec0cd000 2024-09-21T07:50:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.331+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaec0ccac0 con 0x7feaec0cd000 2024-09-21T07:50:21.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.332+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fead4005400 con 0x7feaec0cd000 2024-09-21T07:50:21.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.333+0000 7feae8c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7feac8078ef0 0x7feac807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:21.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.333+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fead40bc650 con 0x7feaec0cd000 2024-09-21T07:50:21.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.333+0000 7feaebe00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7feac8078ef0 0x7feac807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:21.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.334+0000 7feaebe00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7feac8078ef0 0x7feac807b400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feadc004a30 tx=0x7feadc001670 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:21.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.338+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fead403f030 con 0x7feaec0cd000 2024-09-21T07:50:21.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:21.557+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7feaec157670 con 0x7feaec0cd000 2024-09-21T07:50:22.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.076+0000 7feae8c00640 1 -- 172.21.15.88:0/907209085 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fead4084430 con 0x7feaec0cd000 2024-09-21T07:50:22.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7feac8078ef0 msgr2=0x7feac807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7feac8078ef0 0x7feac807b400 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7feadc004a30 tx=0x7feadc001670 comp rx=0 tx=0).stop 2024-09-21T07:50:22.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 msgr2=0x7feaec1f7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 0x7feaec1f7640 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fead402ab00 tx=0x7fead4033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 shutdown_connections 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7feac8078ef0 0x7feac807b400 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaec156ca0 0x7feaec145570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaec0cd000 0x7feaec1f7640 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 --2- 172.21.15.88:0/907209085 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaec0b3f30 0x7feaec1f3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 >> 172.21.15.88:0/907209085 conn(0x7feaec06d620 msgr2=0x7feaec0b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 shutdown_connections 2024-09-21T07:50:22.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.079+0000 7feaf1c00640 1 -- 172.21.15.88:0/907209085 wait complete. 2024-09-21T07:50:22.103 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:50:22.103 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:22.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 -- 172.21.15.88:0/1060408193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ee0004800 msgr2=0x7f0ee0004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 --2- 172.21.15.88:0/1060408193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ee0004800 0x7f0ee0004c10 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4001800 tx=0x7f0ee0005830 comp rx=0 tx=0).stop 2024-09-21T07:50:22.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 -- 172.21.15.88:0/1060408193 shutdown_connections 2024-09-21T07:50:22.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 --2- 172.21.15.88:0/1060408193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ee0004800 0x7f0ee0004c10 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 -- 172.21.15.88:0/1060408193 >> 172.21.15.88:0/1060408193 conn(0x7f0efc06d730 msgr2=0x7f0efc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:22.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 -- 172.21.15.88:0/1060408193 shutdown_connections 2024-09-21T07:50:22.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.322+0000 7f0f03a00640 1 -- 172.21.15.88:0/1060408193 wait complete. 2024-09-21T07:50:22.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.324+0000 7f0f03a00640 1 Processor -- start 2024-09-21T07:50:22.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.325+0000 7f0f03a00640 1 -- start start 2024-09-21T07:50:22.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0f03a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ee0004800 0x7f0efc1f3100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0f03a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 0x7f0efc151f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0f03a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0efc152440 0x7f0efc152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0f03a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc0712b0 con 0x7f0efc154b40 2024-09-21T07:50:22.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0f03a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc071130 con 0x7f0efc152440 2024-09-21T07:50:22.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0f03a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0efc073040 con 0x7f0ee0004800 2024-09-21T07:50:22.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 0x7f0efc151f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0ef3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 0x7f0efc151f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48904/0 (socket says 172.21.15.88:48904) 2024-09-21T07:50:22.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.326+0000 7f0ef3e00640 1 -- 172.21.15.88:0/3236577387 learned_addr learned my addr 172.21.15.88:0/3236577387 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:22.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0f01000640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ee0004800 0x7f0efc1f3100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0f01a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0efc152440 0x7f0efc152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef3e00640 1 -- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ee0004800 msgr2=0x7f0efc1f3100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef3e00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ee0004800 0x7f0efc1f3100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef3e00640 1 -- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0efc152440 msgr2=0x7f0efc152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef3e00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0efc152440 0x7f0efc152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef3e00640 1 -- 172.21.15.88:0/3236577387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc148ee0 con 0x7f0efc154b40 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0f01a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0efc152440 0x7f0efc152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef3e00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 0x7f0efc151f00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4001920 tx=0x7f0ee40060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.327+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4043070 con 0x7f0efc154b40 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.328+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ee4003af0 con 0x7f0efc154b40 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.328+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40064c0 con 0x7f0efc154b40 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.328+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc1491c0 con 0x7f0efc154b40 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.328+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc158800 con 0x7f0efc154b40 2024-09-21T07:50:22.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.329+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0efc0735a0 con 0x7f0efc154b40 2024-09-21T07:50:22.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.330+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0ee4046070 con 0x7f0efc154b40 2024-09-21T07:50:22.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.331+0000 7f0ef9600640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078f20 0x7f0ed807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.331+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f0ee40bb2f0 con 0x7f0efc154b40 2024-09-21T07:50:22.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.332+0000 7f0f01000640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078f20 0x7f0ed807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.332+0000 7f0f01000640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078f20 0x7f0ed807b430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0eec001eb0 tx=0x7f0eec006040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:22.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.335+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ee4082a80 con 0x7f0efc154b40 2024-09-21T07:50:22.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.560+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0efc15e420 con 0x7f0efc154b40 2024-09-21T07:50:22.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.561+0000 7f0ef9600640 1 -- 172.21.15.88:0/3236577387 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5231 (secure 0 0 0) 0x7f0ee40874b0 con 0x7f0efc154b40 2024-09-21T07:50:22.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:22.563 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"btime":"2024-09-21T07:50:22:071724+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:22.563 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 46 2024-09-21T07:50:22.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078f20 msgr2=0x7f0ed807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078f20 0x7f0ed807b430 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f0eec001eb0 tx=0x7f0eec006040 comp rx=0 tx=0).stop 2024-09-21T07:50:22.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 msgr2=0x7f0efc151f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 0x7f0efc151f00 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4001920 tx=0x7f0ee40060e0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 shutdown_connections 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0ed8078f20 0x7f0ed807b430 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0efc152440 0x7f0efc152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0efc154b40 0x7f0efc151f00 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 --2- 172.21.15.88:0/3236577387 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ee0004800 0x7f0efc1f3100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 >> 172.21.15.88:0/3236577387 conn(0x7f0efc06d730 msgr2=0x7f0efc1543c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 shutdown_connections 2024-09-21T07:50:22.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.563+0000 7f0f03a00640 1 -- 172.21.15.88:0/3236577387 wait complete. 2024-09-21T07:50:22.586 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 -- 172.21.15.88:0/4196780418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da40704b0 msgr2=0x7f0da4073040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 --2- 172.21.15.88:0/4196780418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da40704b0 0x7f0da4073040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f0d94007350 tx=0x7f0d9402b110 comp rx=0 tx=0).stop 2024-09-21T07:50:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 -- 172.21.15.88:0/4196780418 shutdown_connections 2024-09-21T07:50:22.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 --2- 172.21.15.88:0/4196780418 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0da41626d0 0x7f0da4164af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 --2- 172.21.15.88:0/4196780418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da40704b0 0x7f0da4073040 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 -- 172.21.15.88:0/4196780418 >> 172.21.15.88:0/4196780418 conn(0x7f0da406d730 msgr2=0x7f0da40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:22.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 -- 172.21.15.88:0/4196780418 shutdown_connections 2024-09-21T07:50:22.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.805+0000 7f0dab000640 1 -- 172.21.15.88:0/4196780418 wait complete. 2024-09-21T07:50:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.807+0000 7f0dab000640 1 Processor -- start 2024-09-21T07:50:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.807+0000 7f0dab000640 1 -- start start 2024-09-21T07:50:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0dab000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0da40704b0 0x7f0da415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0dab000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0da40ad3c0 0x7f0da4163f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0dab000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 0x7f0da4162ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0dab000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40ae060 con 0x7f0da41626d0 2024-09-21T07:50:22.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0dab000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0da40adee0 con 0x7f0da40ad3c0 2024-09-21T07:50:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0dab000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0da4071130 con 0x7f0da40704b0 2024-09-21T07:50:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 0x7f0da4162ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 0x7f0da4162ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48928/0 (socket says 172.21.15.88:48928) 2024-09-21T07:50:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 -- 172.21.15.88:0/4036954970 learned_addr learned my addr 172.21.15.88:0/4036954970 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da3400640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0da40ad3c0 0x7f0da4163f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da3e00640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0da40704b0 0x7f0da415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 -- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0da40704b0 msgr2=0x7f0da415dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0da40704b0 0x7f0da415dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 -- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0da40ad3c0 msgr2=0x7f0da4163f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0da40ad3c0 0x7f0da4163f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.809+0000 7f0da9000640 1 -- 172.21.15.88:0/4036954970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0da41644c0 con 0x7f0da41626d0 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.810+0000 7f0da9000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 0x7f0da4162ae0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d9a0 tx=0x7f0d9c00de60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:22.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.810+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c017070 con 0x7f0da41626d0 2024-09-21T07:50:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.810+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d9c00be00 con 0x7f0da41626d0 2024-09-21T07:50:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.810+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d9c003bd0 con 0x7f0da41626d0 2024-09-21T07:50:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.810+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0da40b0e60 con 0x7f0da41626d0 2024-09-21T07:50:22.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.810+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0da40b1420 con 0x7f0da41626d0 2024-09-21T07:50:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.811+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0da4073040 con 0x7f0da41626d0 2024-09-21T07:50:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.812+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0d9c010460 con 0x7f0da41626d0 2024-09-21T07:50:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.813+0000 7f0d9be00640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d84078e60 0x7f0d8407b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.813+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f0d9c09b710 con 0x7f0da41626d0 2024-09-21T07:50:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.814+0000 7f0da3e00640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d84078e60 0x7f0d8407b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:22.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.814+0000 7f0da3e00640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d84078e60 0x7f0d8407b370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d94034000 tx=0x7f0d9402d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:22.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:22.818+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d9c062f50 con 0x7f0da41626d0 2024-09-21T07:50:23.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.003+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f0da41f6880 con 0x7f0da41626d0 2024-09-21T07:50:23.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.004+0000 7f0d9be00640 1 -- 172.21.15.88:0/4036954970 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v73) ==== 74+0+16680 (secure 0 0 0) 0x7f0d9c067980 con 0x7f0da41626d0 2024-09-21T07:50:23.007 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:23.007 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":73,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:20.045968+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:16.939762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:17.507399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:23.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d84078e60 msgr2=0x7f0d8407b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d84078e60 0x7f0d8407b370 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d94034000 tx=0x7f0d9402d040 comp rx=0 tx=0).stop 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 msgr2=0x7f0da4162ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 0x7f0da4162ae0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f0d9c00d9a0 tx=0x7f0d9c00de60 comp rx=0 tx=0).stop 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 shutdown_connections 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0d84078e60 0x7f0d8407b370 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0da41626d0 0x7f0da4162ae0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0da40ad3c0 0x7f0da4163f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 --2- 172.21.15.88:0/4036954970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0da40704b0 0x7f0da415dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 >> 172.21.15.88:0/4036954970 conn(0x7f0da406d730 msgr2=0x7f0da40cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 shutdown_connections 2024-09-21T07:50:23.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.007+0000 7f0dab000640 1 -- 172.21.15.88:0/4036954970 wait complete. 2024-09-21T07:50:23.034 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:50:23.034 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:50:23.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 -- 172.21.15.88:0/1560607707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 msgr2=0x7f053c0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 --2- 172.21.15.88:0/1560607707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0cfd30 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f053c0b5b30 tx=0x7f052c02a530 comp rx=0 tx=0).stop 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 -- 172.21.15.88:0/1560607707 shutdown_connections 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 --2- 172.21.15.88:0/1560607707 >> v2:172.21.15.175:3300/0 conn(0x7f053c1620e0 0x7f053c0d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 --2- 172.21.15.88:0/1560607707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0cfd30 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 -- 172.21.15.88:0/1560607707 >> 172.21.15.88:0/1560607707 conn(0x7f053c09d820 msgr2=0x7f053c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 -- 172.21.15.88:0/1560607707 shutdown_connections 2024-09-21T07:50:23.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.256+0000 7f0541200640 1 -- 172.21.15.88:0/1560607707 wait complete. 2024-09-21T07:50:23.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.258+0000 7f0541200640 1 Processor -- start 2024-09-21T07:50:23.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.258+0000 7f0541200640 1 -- start start 2024-09-21T07:50:23.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0541200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c0b9610 0x7f053c1f88e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0541200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0b3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0541200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c1620e0 0x7f053c1f6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0541200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f053c0b4360 con 0x7f053c0b9b80 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0541200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f053c0b41e0 con 0x7f053c0b9610 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0541200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f053c0b44e0 con 0x7f053c1620e0 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0532a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0b3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0532a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0b3890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48946/0 (socket says 172.21.15.88:48946) 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f0532a00640 1 -- 172.21.15.88:0/4248903253 learned_addr learned my addr 172.21.15.88:0/4248903253 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f053b400640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c0b9610 0x7f053c1f88e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.260+0000 7f053be00640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c1620e0 0x7f053c1f6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0532a00640 1 -- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c1620e0 msgr2=0x7f053c1f6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0532a00640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c1620e0 0x7f053c1f6420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0532a00640 1 -- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c0b9610 msgr2=0x7f053c1f88e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0532a00640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c0b9610 0x7f053c1f88e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0532a00640 1 -- 172.21.15.88:0/4248903253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f053c0b2dd0 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0532a00640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0b3890 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f052400d720 tx=0x7f052400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524009c30 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f052400de30 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f053c1f92c0 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0524004260 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.261+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f053c0b3090 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.263+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f053c0d1660 con 0x7f053c0b9b80 2024-09-21T07:50:23.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.264+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f0524005030 con 0x7f053c0b9b80 2024-09-21T07:50:23.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.264+0000 7f0533e00640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0518078ef0 0x7f051807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.265+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f052409b900 con 0x7f053c0b9b80 2024-09-21T07:50:23.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.265+0000 7f053b400640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0518078ef0 0x7f051807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.266+0000 7f053b400640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0518078ef0 0x7f051807b400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f052c034000 tx=0x7f052c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.269+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0524020020 con 0x7f053c0b9b80 2024-09-21T07:50:23.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.451+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f053c0d0720 con 0x7f0518078ef0 2024-09-21T07:50:23.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.457+0000 7f0533e00640 1 -- 172.21.15.88:0/4248903253 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+251321 (secure 0 0 0) 0x7f053c0d0720 con 0x7f0518078ef0 2024-09-21T07:50:23.461 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:23.463 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:50:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0518078ef0 msgr2=0x7f051807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0518078ef0 0x7f051807b400 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f052c034000 tx=0x7f052c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 msgr2=0x7f053c0b3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0b3890 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f052400d720 tx=0x7f052400dbe0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 shutdown_connections 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f0518078ef0 0x7f051807b400 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f053c1620e0 0x7f053c1f6420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f053c0b9b80 0x7f053c0b3890 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 --2- 172.21.15.88:0/4248903253 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f053c0b9610 0x7f053c1f88e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 >> 172.21.15.88:0/4248903253 conn(0x7f053c09d820 msgr2=0x7f053c0d4e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 shutdown_connections 2024-09-21T07:50:23.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.461+0000 7f0541200640 1 -- 172.21.15.88:0/4248903253 wait complete. 2024-09-21T07:50:23.484 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":203,"stamp":"2024-09-21T07:50:21.926238+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":137,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257512,"kb_used_data":7964,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717208,"statfs":{"total":773094113280,"available":772830420992,"internally_reserved":0,"allocated":8155136,"data_stored":4084741,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.889091"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088574+0000","last_change":"2024-09-21T07:50:19.074933+0000","last_active":"2024-09-21T07:50:19.088574+0000","last_peered":"2024-09-21T07:50:19.088574+0000","last_clean":"2024-09-21T07:50:19.088574+0000","last_became_active":"2024-09-21T07:50:19.074021+0000","last_became_peered":"2024-09-21T07:50:19.074021+0000","last_unstale":"2024-09-21T07:50:19.088574+0000","last_undegraded":"2024-09-21T07:50:19.088574+0000","last_fullsized":"2024-09-21T07:50:19.088574+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:59.831046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.092567+0000","last_change":"2024-09-21T07:50:19.084678+0000","last_active":"2024-09-21T07:50:19.092567+0000","last_peered":"2024-09-21T07:50:19.092567+0000","last_clean":"2024-09-21T07:50:19.092567+0000","last_became_active":"2024-09-21T07:50:19.084357+0000","last_became_peered":"2024-09-21T07:50:19.084357+0000","last_unstale":"2024-09-21T07:50:19.092567+0000","last_undegraded":"2024-09-21T07:50:19.092567+0000","last_fullsized":"2024-09-21T07:50:19.092567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:22.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.058556+0000","last_change":"2024-09-21T07:50:18.058556+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.058556+0000","last_undegraded":"2024-09-21T07:50:18.058556+0000","last_fullsized":"2024-09-21T07:50:18.058556+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.058204+0000","last_change":"2024-09-21T07:50:18.058204+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.058204+0000","last_undegraded":"2024-09-21T07:50:18.058204+0000","last_fullsized":"2024-09-21T07:50:18.058204+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.086866+0000","last_change":"2024-09-21T07:50:19.083337+0000","last_active":"2024-09-21T07:50:19.086866+0000","last_peered":"2024-09-21T07:50:19.086866+0000","last_clean":"2024-09-21T07:50:19.086866+0000","last_became_active":"2024-09-21T07:50:19.082960+0000","last_became_peered":"2024-09-21T07:50:19.082960+0000","last_unstale":"2024-09-21T07:50:19.086866+0000","last_undegraded":"2024-09-21T07:50:19.086866+0000","last_fullsized":"2024-09-21T07:50:19.086866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:34.596878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095691+0000","last_change":"2024-09-21T07:50:19.091545+0000","last_active":"2024-09-21T07:50:19.095691+0000","last_peered":"2024-09-21T07:50:19.095691+0000","last_clean":"2024-09-21T07:50:19.095691+0000","last_became_active":"2024-09-21T07:50:19.091135+0000","last_became_peered":"2024-09-21T07:50:19.091135+0000","last_unstale":"2024-09-21T07:50:19.095691+0000","last_undegraded":"2024-09-21T07:50:19.095691+0000","last_fullsized":"2024-09-21T07:50:19.095691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:37.914081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051024+0000","last_change":"2024-09-21T07:50:19.055075+0000","last_active":"2024-09-21T07:50:20.051024+0000","last_peered":"2024-09-21T07:50:20.051024+0000","last_clean":"2024-09-21T07:50:20.051024+0000","last_became_active":"2024-09-21T07:50:19.054798+0000","last_became_peered":"2024-09-21T07:50:19.054798+0000","last_unstale":"2024-09-21T07:50:20.051024+0000","last_undegraded":"2024-09-21T07:50:20.051024+0000","last_fullsized":"2024-09-21T07:50:20.051024+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:13.626965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088999+0000","last_change":"2024-09-21T07:50:19.074960+0000","last_active":"2024-09-21T07:50:19.088999+0000","last_peered":"2024-09-21T07:50:19.088999+0000","last_clean":"2024-09-21T07:50:19.088999+0000","last_became_active":"2024-09-21T07:50:19.074095+0000","last_became_peered":"2024-09-21T07:50:19.074095+0000","last_unstale":"2024-09-21T07:50:19.088999+0000","last_undegraded":"2024-09-21T07:50:19.088999+0000","last_fullsized":"2024-09-21T07:50:19.088999+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:14.142105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.058955+0000","last_change":"2024-09-21T07:50:18.058955+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.058955+0000","last_undegraded":"2024-09-21T07:50:18.058955+0000","last_fullsized":"2024-09-21T07:50:18.058955+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051221+0000","last_change":"2024-09-21T07:50:19.064321+0000","last_active":"2024-09-21T07:50:20.051221+0000","last_peered":"2024-09-21T07:50:20.051221+0000","last_clean":"2024-09-21T07:50:20.051221+0000","last_became_active":"2024-09-21T07:50:19.063463+0000","last_became_peered":"2024-09-21T07:50:19.063463+0000","last_unstale":"2024-09-21T07:50:20.051221+0000","last_undegraded":"2024-09-21T07:50:20.051221+0000","last_fullsized":"2024-09-21T07:50:20.051221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:33.735607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089288+0000","last_change":"2024-09-21T07:50:19.077422+0000","last_active":"2024-09-21T07:50:19.089288+0000","last_peered":"2024-09-21T07:50:19.089288+0000","last_clean":"2024-09-21T07:50:19.089288+0000","last_became_active":"2024-09-21T07:50:19.076521+0000","last_became_peered":"2024-09-21T07:50:19.076521+0000","last_unstale":"2024-09-21T07:50:19.089288+0000","last_undegraded":"2024-09-21T07:50:19.089288+0000","last_fullsized":"2024-09-21T07:50:19.089288+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:03.102804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.084420+0000","last_change":"2024-09-21T07:50:19.080129+0000","last_active":"2024-09-21T07:50:19.084420+0000","last_peered":"2024-09-21T07:50:19.084420+0000","last_clean":"2024-09-21T07:50:19.084420+0000","last_became_active":"2024-09-21T07:50:19.079644+0000","last_became_peered":"2024-09-21T07:50:19.079644+0000","last_unstale":"2024-09-21T07:50:19.084420+0000","last_undegraded":"2024-09-21T07:50:19.084420+0000","last_fullsized":"2024-09-21T07:50:19.084420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:39.607077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.059750+0000","last_change":"2024-09-21T07:50:18.059750+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.059750+0000","last_undegraded":"2024-09-21T07:50:18.059750+0000","last_fullsized":"2024-09-21T07:50:18.059750+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.076857+0000","last_change":"2024-09-21T07:50:19.072199+0000","last_active":"2024-09-21T07:50:19.076857+0000","last_peered":"2024-09-21T07:50:19.076857+0000","last_clean":"2024-09-21T07:50:19.076857+0000","last_became_active":"2024-09-21T07:50:19.070583+0000","last_became_peered":"2024-09-21T07:50:19.070583+0000","last_unstale":"2024-09-21T07:50:19.076857+0000","last_undegraded":"2024-09-21T07:50:19.076857+0000","last_fullsized":"2024-09-21T07:50:19.076857+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:50.938596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.072340+0000","last_change":"2024-09-21T07:50:19.064362+0000","last_active":"2024-09-21T07:50:19.072340+0000","last_peered":"2024-09-21T07:50:19.072340+0000","last_clean":"2024-09-21T07:50:19.072340+0000","last_became_active":"2024-09-21T07:50:19.063322+0000","last_became_peered":"2024-09-21T07:50:19.063322+0000","last_unstale":"2024-09-21T07:50:19.072340+0000","last_undegraded":"2024-09-21T07:50:19.072340+0000","last_fullsized":"2024-09-21T07:50:19.072340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:51.029552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089025+0000","last_change":"2024-09-21T07:50:19.082296+0000","last_active":"2024-09-21T07:50:19.089025+0000","last_peered":"2024-09-21T07:50:19.089025+0000","last_clean":"2024-09-21T07:50:19.089025+0000","last_became_active":"2024-09-21T07:50:19.081652+0000","last_became_peered":"2024-09-21T07:50:19.081652+0000","last_unstale":"2024-09-21T07:50:19.089025+0000","last_undegraded":"2024-09-21T07:50:19.089025+0000","last_fullsized":"2024-09-21T07:50:19.089025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:40.314898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.070556+0000","last_change":"2024-09-21T07:50:19.063930+0000","last_active":"2024-09-21T07:50:19.070556+0000","last_peered":"2024-09-21T07:50:19.070556+0000","last_clean":"2024-09-21T07:50:19.070556+0000","last_became_active":"2024-09-21T07:50:19.063520+0000","last_became_peered":"2024-09-21T07:50:19.063520+0000","last_unstale":"2024-09-21T07:50:19.070556+0000","last_undegraded":"2024-09-21T07:50:19.070556+0000","last_fullsized":"2024-09-21T07:50:19.070556+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.456835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.070193+0000","last_change":"2024-09-21T07:50:19.062381+0000","last_active":"2024-09-21T07:50:19.070193+0000","last_peered":"2024-09-21T07:50:19.070193+0000","last_clean":"2024-09-21T07:50:19.070193+0000","last_became_active":"2024-09-21T07:50:19.062050+0000","last_became_peered":"2024-09-21T07:50:19.062050+0000","last_unstale":"2024-09-21T07:50:19.070193+0000","last_undegraded":"2024-09-21T07:50:19.070193+0000","last_fullsized":"2024-09-21T07:50:19.070193+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:23.146741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089313+0000","last_change":"2024-09-21T07:50:19.078093+0000","last_active":"2024-09-21T07:50:19.089313+0000","last_peered":"2024-09-21T07:50:19.089313+0000","last_clean":"2024-09-21T07:50:19.089313+0000","last_became_active":"2024-09-21T07:50:19.077809+0000","last_became_peered":"2024-09-21T07:50:19.077809+0000","last_unstale":"2024-09-21T07:50:19.089313+0000","last_undegraded":"2024-09-21T07:50:19.089313+0000","last_fullsized":"2024-09-21T07:50:19.089313+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:09.089424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.076247+0000","last_change":"2024-09-21T07:50:19.074153+0000","last_active":"2024-09-21T07:50:19.076247+0000","last_peered":"2024-09-21T07:50:19.076247+0000","last_clean":"2024-09-21T07:50:19.076247+0000","last_became_active":"2024-09-21T07:50:19.072750+0000","last_became_peered":"2024-09-21T07:50:19.072750+0000","last_unstale":"2024-09-21T07:50:19.076247+0000","last_undegraded":"2024-09-21T07:50:19.076247+0000","last_fullsized":"2024-09-21T07:50:19.076247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.360151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088384+0000","last_change":"2024-09-21T07:50:19.079831+0000","last_active":"2024-09-21T07:50:19.088384+0000","last_peered":"2024-09-21T07:50:19.088384+0000","last_clean":"2024-09-21T07:50:19.088384+0000","last_became_active":"2024-09-21T07:50:19.078464+0000","last_became_peered":"2024-09-21T07:50:19.078464+0000","last_unstale":"2024-09-21T07:50:19.088384+0000","last_undegraded":"2024-09-21T07:50:19.088384+0000","last_fullsized":"2024-09-21T07:50:19.088384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:44.178216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.082349+0000","last_change":"2024-09-21T07:50:19.072081+0000","last_active":"2024-09-21T07:50:19.082349+0000","last_peered":"2024-09-21T07:50:19.082349+0000","last_clean":"2024-09-21T07:50:19.082349+0000","last_became_active":"2024-09-21T07:50:19.071800+0000","last_became_peered":"2024-09-21T07:50:19.071800+0000","last_unstale":"2024-09-21T07:50:19.082349+0000","last_undegraded":"2024-09-21T07:50:19.082349+0000","last_fullsized":"2024-09-21T07:50:19.082349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:25.260802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050678+0000","last_change":"2024-09-21T07:50:19.079780+0000","last_active":"2024-09-21T07:50:20.050678+0000","last_peered":"2024-09-21T07:50:20.050678+0000","last_clean":"2024-09-21T07:50:20.050678+0000","last_became_active":"2024-09-21T07:50:19.079407+0000","last_became_peered":"2024-09-21T07:50:19.079407+0000","last_unstale":"2024-09-21T07:50:20.050678+0000","last_undegraded":"2024-09-21T07:50:20.050678+0000","last_fullsized":"2024-09-21T07:50:20.050678+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:10.828365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088563+0000","last_change":"2024-09-21T07:50:19.074330+0000","last_active":"2024-09-21T07:50:19.088563+0000","last_peered":"2024-09-21T07:50:19.088563+0000","last_clean":"2024-09-21T07:50:19.088563+0000","last_became_active":"2024-09-21T07:50:19.073987+0000","last_became_peered":"2024-09-21T07:50:19.073987+0000","last_unstale":"2024-09-21T07:50:19.088563+0000","last_undegraded":"2024-09-21T07:50:19.088563+0000","last_fullsized":"2024-09-21T07:50:19.088563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:05.401336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051259+0000","last_change":"2024-09-21T07:50:19.083611+0000","last_active":"2024-09-21T07:50:20.051259+0000","last_peered":"2024-09-21T07:50:20.051259+0000","last_clean":"2024-09-21T07:50:20.051259+0000","last_became_active":"2024-09-21T07:50:19.083100+0000","last_became_peered":"2024-09-21T07:50:19.083100+0000","last_unstale":"2024-09-21T07:50:20.051259+0000","last_undegraded":"2024-09-21T07:50:20.051259+0000","last_fullsized":"2024-09-21T07:50:20.051259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:53.711994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089253+0000","last_change":"2024-09-21T07:50:19.077593+0000","last_active":"2024-09-21T07:50:19.089253+0000","last_peered":"2024-09-21T07:50:19.089253+0000","last_clean":"2024-09-21T07:50:19.089253+0000","last_became_active":"2024-09-21T07:50:19.077293+0000","last_became_peered":"2024-09-21T07:50:19.077293+0000","last_unstale":"2024-09-21T07:50:19.089253+0000","last_undegraded":"2024-09-21T07:50:19.089253+0000","last_fullsized":"2024-09-21T07:50:19.089253+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:55.104639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.083779+0000","last_change":"2024-09-21T07:50:19.079290+0000","last_active":"2024-09-21T07:50:19.083779+0000","last_peered":"2024-09-21T07:50:19.083779+0000","last_clean":"2024-09-21T07:50:19.083779+0000","last_became_active":"2024-09-21T07:50:19.078227+0000","last_became_peered":"2024-09-21T07:50:19.078227+0000","last_unstale":"2024-09-21T07:50:19.083779+0000","last_undegraded":"2024-09-21T07:50:19.083779+0000","last_fullsized":"2024-09-21T07:50:19.083779+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:47.078670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.090328+0000","last_change":"2024-09-21T07:50:19.082952+0000","last_active":"2024-09-21T07:50:19.090328+0000","last_peered":"2024-09-21T07:50:19.090328+0000","last_clean":"2024-09-21T07:50:19.090328+0000","last_became_active":"2024-09-21T07:50:19.082576+0000","last_became_peered":"2024-09-21T07:50:19.082576+0000","last_unstale":"2024-09-21T07:50:19.090328+0000","last_undegraded":"2024-09-21T07:50:19.090328+0000","last_fullsized":"2024-09-21T07:50:19.090328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:32.091613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.058359+0000","last_change":"2024-09-21T07:50:18.058359+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.058359+0000","last_undegraded":"2024-09-21T07:50:18.058359+0000","last_fullsized":"2024-09-21T07:50:18.058359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089406+0000","last_change":"2024-09-21T07:50:19.069884+0000","last_active":"2024-09-21T07:50:19.089406+0000","last_peered":"2024-09-21T07:50:19.089406+0000","last_clean":"2024-09-21T07:50:19.089406+0000","last_became_active":"2024-09-21T07:50:19.069600+0000","last_became_peered":"2024-09-21T07:50:19.069600+0000","last_unstale":"2024-09-21T07:50:19.089406+0000","last_undegraded":"2024-09-21T07:50:19.089406+0000","last_fullsized":"2024-09-21T07:50:19.089406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:43.356105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089133+0000","last_change":"2024-09-21T07:50:19.081019+0000","last_active":"2024-09-21T07:50:19.089133+0000","last_peered":"2024-09-21T07:50:19.089133+0000","last_clean":"2024-09-21T07:50:19.089133+0000","last_became_active":"2024-09-21T07:50:19.080702+0000","last_became_peered":"2024-09-21T07:50:19.080702+0000","last_unstale":"2024-09-21T07:50:19.089133+0000","last_undegraded":"2024-09-21T07:50:19.089133+0000","last_fullsized":"2024-09-21T07:50:19.089133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:07.448041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089359+0000","last_change":"2024-09-21T07:50:19.081992+0000","last_active":"2024-09-21T07:50:19.089359+0000","last_peered":"2024-09-21T07:50:19.089359+0000","last_clean":"2024-09-21T07:50:19.089359+0000","last_became_active":"2024-09-21T07:50:19.081588+0000","last_became_peered":"2024-09-21T07:50:19.081588+0000","last_unstale":"2024-09-21T07:50:19.089359+0000","last_undegraded":"2024-09-21T07:50:19.089359+0000","last_fullsized":"2024-09-21T07:50:19.089359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:05.117390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.092179+0000","last_change":"2024-09-21T07:50:19.087010+0000","last_active":"2024-09-21T07:50:19.092179+0000","last_peered":"2024-09-21T07:50:19.092179+0000","last_clean":"2024-09-21T07:50:19.092179+0000","last_became_active":"2024-09-21T07:50:19.086720+0000","last_became_peered":"2024-09-21T07:50:19.086720+0000","last_unstale":"2024-09-21T07:50:19.092179+0000","last_undegraded":"2024-09-21T07:50:19.092179+0000","last_fullsized":"2024-09-21T07:50:19.092179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:54.692860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"72'4","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.086121+0000","last_change":"2024-09-21T07:50:18.073367+0000","last_active":"2024-09-21T07:50:19.086121+0000","last_peered":"2024-09-21T07:50:19.086121+0000","last_clean":"2024-09-21T07:50:19.086121+0000","last_became_active":"2024-09-21T07:50:18.073050+0000","last_became_peered":"2024-09-21T07:50:18.073050+0000","last_unstale":"2024-09-21T07:50:19.086121+0000","last_undegraded":"2024-09-21T07:50:19.086121+0000","last_fullsized":"2024-09-21T07:50:19.086121+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:21.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.058433+0000","last_change":"2024-09-21T07:50:18.058433+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.058433+0000","last_undegraded":"2024-09-21T07:50:18.058433+0000","last_fullsized":"2024-09-21T07:50:18.058433+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075760+0000","last_change":"2024-09-21T07:50:18.067461+0000","last_active":"2024-09-21T07:50:19.075760+0000","last_peered":"2024-09-21T07:50:19.075760+0000","last_clean":"2024-09-21T07:50:19.075760+0000","last_became_active":"2024-09-21T07:50:18.067141+0000","last_became_peered":"2024-09-21T07:50:18.067141+0000","last_unstale":"2024-09-21T07:50:19.075760+0000","last_undegraded":"2024-09-21T07:50:19.075760+0000","last_fullsized":"2024-09-21T07:50:19.075760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:03.733100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095670+0000","last_change":"2024-09-21T07:50:19.089763+0000","last_active":"2024-09-21T07:50:19.095670+0000","last_peered":"2024-09-21T07:50:19.095670+0000","last_clean":"2024-09-21T07:50:19.095670+0000","last_became_active":"2024-09-21T07:50:19.089389+0000","last_became_peered":"2024-09-21T07:50:19.089389+0000","last_unstale":"2024-09-21T07:50:19.095670+0000","last_undegraded":"2024-09-21T07:50:19.095670+0000","last_fullsized":"2024-09-21T07:50:19.095670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:47.561075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088433+0000","last_change":"2024-09-21T07:50:18.076445+0000","last_active":"2024-09-21T07:50:19.088433+0000","last_peered":"2024-09-21T07:50:19.088433+0000","last_clean":"2024-09-21T07:50:19.088433+0000","last_became_active":"2024-09-21T07:50:18.076188+0000","last_became_peered":"2024-09-21T07:50:18.076188+0000","last_unstale":"2024-09-21T07:50:19.088433+0000","last_undegraded":"2024-09-21T07:50:19.088433+0000","last_fullsized":"2024-09-21T07:50:19.088433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:55.164877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051096+0000","last_change":"2024-09-21T07:50:19.075753+0000","last_active":"2024-09-21T07:50:20.051096+0000","last_peered":"2024-09-21T07:50:20.051096+0000","last_clean":"2024-09-21T07:50:20.051096+0000","last_became_active":"2024-09-21T07:50:19.075399+0000","last_became_peered":"2024-09-21T07:50:19.075399+0000","last_unstale":"2024-09-21T07:50:20.051096+0000","last_undegraded":"2024-09-21T07:50:20.051096+0000","last_fullsized":"2024-09-21T07:50:20.051096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:34.198178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085753+0000","last_change":"2024-09-21T07:50:18.074894+0000","last_active":"2024-09-21T07:50:19.085753+0000","last_peered":"2024-09-21T07:50:19.085753+0000","last_clean":"2024-09-21T07:50:19.085753+0000","last_became_active":"2024-09-21T07:50:18.074115+0000","last_became_peered":"2024-09-21T07:50:18.074115+0000","last_unstale":"2024-09-21T07:50:19.085753+0000","last_undegraded":"2024-09-21T07:50:19.085753+0000","last_fullsized":"2024-09-21T07:50:19.085753+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:12.718692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.087755+0000","last_change":"2024-09-21T07:50:19.063111+0000","last_active":"2024-09-21T07:50:19.087755+0000","last_peered":"2024-09-21T07:50:19.087755+0000","last_clean":"2024-09-21T07:50:19.087755+0000","last_became_active":"2024-09-21T07:50:19.060509+0000","last_became_peered":"2024-09-21T07:50:19.060509+0000","last_unstale":"2024-09-21T07:50:19.087755+0000","last_undegraded":"2024-09-21T07:50:19.087755+0000","last_fullsized":"2024-09-21T07:50:19.087755+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:36.691011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085722+0000","last_change":"2024-09-21T07:50:18.062122+0000","last_active":"2024-09-21T07:50:19.085722+0000","last_peered":"2024-09-21T07:50:19.085722+0000","last_clean":"2024-09-21T07:50:19.085722+0000","last_became_active":"2024-09-21T07:50:18.061825+0000","last_became_peered":"2024-09-21T07:50:18.061825+0000","last_unstale":"2024-09-21T07:50:19.085722+0000","last_undegraded":"2024-09-21T07:50:19.085722+0000","last_fullsized":"2024-09-21T07:50:19.085722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:16.961875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095033+0000","last_change":"2024-09-21T07:50:19.088559+0000","last_active":"2024-09-21T07:50:19.095033+0000","last_peered":"2024-09-21T07:50:19.095033+0000","last_clean":"2024-09-21T07:50:19.095033+0000","last_became_active":"2024-09-21T07:50:19.088224+0000","last_became_peered":"2024-09-21T07:50:19.088224+0000","last_unstale":"2024-09-21T07:50:19.095033+0000","last_undegraded":"2024-09-21T07:50:19.095033+0000","last_fullsized":"2024-09-21T07:50:19.095033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:05.554264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050628+0000","last_change":"2024-09-21T07:50:18.069259+0000","last_active":"2024-09-21T07:50:20.050628+0000","last_peered":"2024-09-21T07:50:20.050628+0000","last_clean":"2024-09-21T07:50:20.050628+0000","last_became_active":"2024-09-21T07:50:18.067538+0000","last_became_peered":"2024-09-21T07:50:18.067538+0000","last_unstale":"2024-09-21T07:50:20.050628+0000","last_undegraded":"2024-09-21T07:50:20.050628+0000","last_fullsized":"2024-09-21T07:50:20.050628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:57.461370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089192+0000","last_change":"2024-09-21T07:50:19.085649+0000","last_active":"2024-09-21T07:50:19.089192+0000","last_peered":"2024-09-21T07:50:19.089192+0000","last_clean":"2024-09-21T07:50:19.089192+0000","last_became_active":"2024-09-21T07:50:19.085349+0000","last_became_peered":"2024-09-21T07:50:19.085349+0000","last_unstale":"2024-09-21T07:50:19.089192+0000","last_undegraded":"2024-09-21T07:50:19.089192+0000","last_fullsized":"2024-09-21T07:50:19.089192+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:03.655502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.071903+0000","last_change":"2024-09-21T07:50:18.077241+0000","last_active":"2024-09-21T07:50:19.071903+0000","last_peered":"2024-09-21T07:50:19.071903+0000","last_clean":"2024-09-21T07:50:19.071903+0000","last_became_active":"2024-09-21T07:50:18.076913+0000","last_became_peered":"2024-09-21T07:50:18.076913+0000","last_unstale":"2024-09-21T07:50:19.071903+0000","last_undegraded":"2024-09-21T07:50:19.071903+0000","last_fullsized":"2024-09-21T07:50:19.071903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:14.586470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075971+0000","last_change":"2024-09-21T07:50:19.072081+0000","last_active":"2024-09-21T07:50:19.075971+0000","last_peered":"2024-09-21T07:50:19.075971+0000","last_clean":"2024-09-21T07:50:19.075971+0000","last_became_active":"2024-09-21T07:50:19.070907+0000","last_became_peered":"2024-09-21T07:50:19.070907+0000","last_unstale":"2024-09-21T07:50:19.075971+0000","last_undegraded":"2024-09-21T07:50:19.075971+0000","last_fullsized":"2024-09-21T07:50:19.075971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:20.195607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.052703+0000","last_change":"2024-09-21T07:50:18.067552+0000","last_active":"2024-09-21T07:50:19.052703+0000","last_peered":"2024-09-21T07:50:19.052703+0000","last_clean":"2024-09-21T07:50:19.052703+0000","last_became_active":"2024-09-21T07:50:18.067203+0000","last_became_peered":"2024-09-21T07:50:18.067203+0000","last_unstale":"2024-09-21T07:50:19.052703+0000","last_undegraded":"2024-09-21T07:50:19.052703+0000","last_fullsized":"2024-09-21T07:50:19.052703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:38.698170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051367+0000","last_change":"2024-09-21T07:50:19.088944+0000","last_active":"2024-09-21T07:50:20.051367+0000","last_peered":"2024-09-21T07:50:20.051367+0000","last_clean":"2024-09-21T07:50:20.051367+0000","last_became_active":"2024-09-21T07:50:19.086571+0000","last_became_peered":"2024-09-21T07:50:19.086571+0000","last_unstale":"2024-09-21T07:50:20.051367+0000","last_undegraded":"2024-09-21T07:50:20.051367+0000","last_fullsized":"2024-09-21T07:50:20.051367+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:18.836068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.054711+0000","last_change":"2024-09-21T07:50:18.073914+0000","last_active":"2024-09-21T07:50:19.054711+0000","last_peered":"2024-09-21T07:50:19.054711+0000","last_clean":"2024-09-21T07:50:19.054711+0000","last_became_active":"2024-09-21T07:50:18.072897+0000","last_became_peered":"2024-09-21T07:50:18.072897+0000","last_unstale":"2024-09-21T07:50:19.054711+0000","last_undegraded":"2024-09-21T07:50:19.054711+0000","last_fullsized":"2024-09-21T07:50:19.054711+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:37.533903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095251+0000","last_change":"2024-09-21T07:50:19.090698+0000","last_active":"2024-09-21T07:50:19.095251+0000","last_peered":"2024-09-21T07:50:19.095251+0000","last_clean":"2024-09-21T07:50:19.095251+0000","last_became_active":"2024-09-21T07:50:19.090341+0000","last_became_peered":"2024-09-21T07:50:19.090341+0000","last_unstale":"2024-09-21T07:50:19.095251+0000","last_undegraded":"2024-09-21T07:50:19.095251+0000","last_fullsized":"2024-09-21T07:50:19.095251+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:45.742021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.070269+0000","last_change":"2024-09-21T07:50:18.075151+0000","last_active":"2024-09-21T07:50:19.070269+0000","last_peered":"2024-09-21T07:50:19.070269+0000","last_clean":"2024-09-21T07:50:19.070269+0000","last_became_active":"2024-09-21T07:50:18.074662+0000","last_became_peered":"2024-09-21T07:50:18.074662+0000","last_unstale":"2024-09-21T07:50:19.070269+0000","last_undegraded":"2024-09-21T07:50:19.070269+0000","last_fullsized":"2024-09-21T07:50:19.070269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:19.721961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088880+0000","last_change":"2024-09-21T07:50:19.076994+0000","last_active":"2024-09-21T07:50:19.088880+0000","last_peered":"2024-09-21T07:50:19.088880+0000","last_clean":"2024-09-21T07:50:19.088880+0000","last_became_active":"2024-09-21T07:50:19.076684+0000","last_became_peered":"2024-09-21T07:50:19.076684+0000","last_unstale":"2024-09-21T07:50:19.088880+0000","last_undegraded":"2024-09-21T07:50:19.088880+0000","last_fullsized":"2024-09-21T07:50:19.088880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:05.896870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.084687+0000","last_change":"2024-09-21T07:50:18.061318+0000","last_active":"2024-09-21T07:50:19.084687+0000","last_peered":"2024-09-21T07:50:19.084687+0000","last_clean":"2024-09-21T07:50:19.084687+0000","last_became_active":"2024-09-21T07:50:18.060246+0000","last_became_peered":"2024-09-21T07:50:18.060246+0000","last_unstale":"2024-09-21T07:50:19.084687+0000","last_undegraded":"2024-09-21T07:50:19.084687+0000","last_fullsized":"2024-09-21T07:50:19.084687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:37.243809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.083751+0000","last_change":"2024-09-21T07:50:19.078951+0000","last_active":"2024-09-21T07:50:19.083751+0000","last_peered":"2024-09-21T07:50:19.083751+0000","last_clean":"2024-09-21T07:50:19.083751+0000","last_became_active":"2024-09-21T07:50:19.077959+0000","last_became_peered":"2024-09-21T07:50:19.077959+0000","last_unstale":"2024-09-21T07:50:19.083751+0000","last_undegraded":"2024-09-21T07:50:19.083751+0000","last_fullsized":"2024-09-21T07:50:19.083751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:28.514449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"72'6","reported_seq":19,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.093201+0000","last_change":"2024-09-21T07:50:18.069448+0000","last_active":"2024-09-21T07:50:19.093201+0000","last_peered":"2024-09-21T07:50:19.093201+0000","last_clean":"2024-09-21T07:50:19.093201+0000","last_became_active":"2024-09-21T07:50:18.069165+0000","last_became_peered":"2024-09-21T07:50:18.069165+0000","last_unstale":"2024-09-21T07:50:19.093201+0000","last_undegraded":"2024-09-21T07:50:19.093201+0000","last_fullsized":"2024-09-21T07:50:19.093201+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:17.833026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075041+0000","last_change":"2024-09-21T07:50:19.070204+0000","last_active":"2024-09-21T07:50:19.075041+0000","last_peered":"2024-09-21T07:50:19.075041+0000","last_clean":"2024-09-21T07:50:19.075041+0000","last_became_active":"2024-09-21T07:50:19.069160+0000","last_became_peered":"2024-09-21T07:50:19.069160+0000","last_unstale":"2024-09-21T07:50:19.075041+0000","last_undegraded":"2024-09-21T07:50:19.075041+0000","last_fullsized":"2024-09-21T07:50:19.075041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:17.506660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.091124+0000","last_change":"2024-09-21T07:50:18.071213+0000","last_active":"2024-09-21T07:50:19.091124+0000","last_peered":"2024-09-21T07:50:19.091124+0000","last_clean":"2024-09-21T07:50:19.091124+0000","last_became_active":"2024-09-21T07:50:18.070905+0000","last_became_peered":"2024-09-21T07:50:18.070905+0000","last_unstale":"2024-09-21T07:50:19.091124+0000","last_undegraded":"2024-09-21T07:50:19.091124+0000","last_fullsized":"2024-09-21T07:50:19.091124+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:13.121735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050884+0000","last_change":"2024-09-21T07:50:19.068866+0000","last_active":"2024-09-21T07:50:20.050884+0000","last_peered":"2024-09-21T07:50:20.050884+0000","last_clean":"2024-09-21T07:50:20.050884+0000","last_became_active":"2024-09-21T07:50:19.066134+0000","last_became_peered":"2024-09-21T07:50:19.066134+0000","last_unstale":"2024-09-21T07:50:20.050884+0000","last_undegraded":"2024-09-21T07:50:20.050884+0000","last_fullsized":"2024-09-21T07:50:20.050884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:28.552933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050620+0000","last_change":"2024-09-21T07:50:18.060992+0000","last_active":"2024-09-21T07:50:20.050620+0000","last_peered":"2024-09-21T07:50:20.050620+0000","last_clean":"2024-09-21T07:50:20.050620+0000","last_became_active":"2024-09-21T07:50:18.060683+0000","last_became_peered":"2024-09-21T07:50:18.060683+0000","last_unstale":"2024-09-21T07:50:20.050620+0000","last_undegraded":"2024-09-21T07:50:20.050620+0000","last_fullsized":"2024-09-21T07:50:20.050620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:40.804735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075893+0000","last_change":"2024-09-21T07:50:19.073051+0000","last_active":"2024-09-21T07:50:19.075893+0000","last_peered":"2024-09-21T07:50:19.075893+0000","last_clean":"2024-09-21T07:50:19.075893+0000","last_became_active":"2024-09-21T07:50:19.072685+0000","last_became_peered":"2024-09-21T07:50:19.072685+0000","last_unstale":"2024-09-21T07:50:19.075893+0000","last_undegraded":"2024-09-21T07:50:19.075893+0000","last_fullsized":"2024-09-21T07:50:19.075893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:33.524928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050889+0000","last_change":"2024-09-21T07:50:18.066959+0000","last_active":"2024-09-21T07:50:20.050889+0000","last_peered":"2024-09-21T07:50:20.050889+0000","last_clean":"2024-09-21T07:50:20.050889+0000","last_became_active":"2024-09-21T07:50:18.065168+0000","last_became_peered":"2024-09-21T07:50:18.065168+0000","last_unstale":"2024-09-21T07:50:20.050889+0000","last_undegraded":"2024-09-21T07:50:20.050889+0000","last_fullsized":"2024-09-21T07:50:20.050889+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:07.675249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.056808+0000","last_change":"2024-09-21T07:50:18.056808+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.056808+0000","last_undegraded":"2024-09-21T07:50:18.056808+0000","last_fullsized":"2024-09-21T07:50:18.056808+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"72'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050387+0000","last_change":"2024-09-21T07:50:18.067636+0000","last_active":"2024-09-21T07:50:20.050387+0000","last_peered":"2024-09-21T07:50:20.050387+0000","last_clean":"2024-09-21T07:50:20.050387+0000","last_became_active":"2024-09-21T07:50:18.067199+0000","last_became_peered":"2024-09-21T07:50:18.067199+0000","last_unstale":"2024-09-21T07:50:20.050387+0000","last_undegraded":"2024-09-21T07:50:20.050387+0000","last_fullsized":"2024-09-21T07:50:20.050387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:16.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085500+0000","last_change":"2024-09-21T07:50:19.078007+0000","last_active":"2024-09-21T07:50:19.085500+0000","last_peered":"2024-09-21T07:50:19.085500+0000","last_clean":"2024-09-21T07:50:19.085500+0000","last_became_active":"2024-09-21T07:50:19.077731+0000","last_became_peered":"2024-09-21T07:50:19.077731+0000","last_unstale":"2024-09-21T07:50:19.085500+0000","last_undegraded":"2024-09-21T07:50:19.085500+0000","last_fullsized":"2024-09-21T07:50:19.085500+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:19.277913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088074+0000","last_change":"2024-09-21T07:50:18.074253+0000","last_active":"2024-09-21T07:50:19.088074+0000","last_peered":"2024-09-21T07:50:19.088074+0000","last_clean":"2024-09-21T07:50:19.088074+0000","last_became_active":"2024-09-21T07:50:18.073286+0000","last_became_peered":"2024-09-21T07:50:18.073286+0000","last_unstale":"2024-09-21T07:50:19.088074+0000","last_undegraded":"2024-09-21T07:50:19.088074+0000","last_fullsized":"2024-09-21T07:50:19.088074+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:18.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089057+0000","last_change":"2024-09-21T07:50:19.080794+0000","last_active":"2024-09-21T07:50:19.089057+0000","last_peered":"2024-09-21T07:50:19.089057+0000","last_clean":"2024-09-21T07:50:19.089057+0000","last_became_active":"2024-09-21T07:50:19.079531+0000","last_became_peered":"2024-09-21T07:50:19.079531+0000","last_unstale":"2024-09-21T07:50:19.089057+0000","last_undegraded":"2024-09-21T07:50:19.089057+0000","last_fullsized":"2024-09-21T07:50:19.089057+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:46.908903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085634+0000","last_change":"2024-09-21T07:50:18.070731+0000","last_active":"2024-09-21T07:50:19.085634+0000","last_peered":"2024-09-21T07:50:19.085634+0000","last_clean":"2024-09-21T07:50:19.085634+0000","last_became_active":"2024-09-21T07:50:18.070442+0000","last_became_peered":"2024-09-21T07:50:18.070442+0000","last_unstale":"2024-09-21T07:50:19.085634+0000","last_undegraded":"2024-09-21T07:50:19.085634+0000","last_fullsized":"2024-09-21T07:50:19.085634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:42.710809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.056825+0000","last_change":"2024-09-21T07:50:18.056825+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.056825+0000","last_undegraded":"2024-09-21T07:50:18.056825+0000","last_fullsized":"2024-09-21T07:50:18.056825+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.090950+0000","last_change":"2024-09-21T07:50:18.068881+0000","last_active":"2024-09-21T07:50:19.090950+0000","last_peered":"2024-09-21T07:50:19.090950+0000","last_clean":"2024-09-21T07:50:19.090950+0000","last_became_active":"2024-09-21T07:50:18.068518+0000","last_became_peered":"2024-09-21T07:50:18.068518+0000","last_unstale":"2024-09-21T07:50:19.090950+0000","last_undegraded":"2024-09-21T07:50:19.090950+0000","last_fullsized":"2024-09-21T07:50:19.090950+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.708952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":1,"reported_epoch":71,"state":"creating+peering","last_fresh":"2024-09-21T07:50:18.056904+0000","last_change":"2024-09-21T07:50:18.056904+0000","last_active":"2024-09-21T07:50:18.036056+0000","last_peered":"2024-09-21T07:50:18.036056+0000","last_clean":"2024-09-21T07:50:18.036056+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:18.056904+0000","last_undegraded":"2024-09-21T07:50:18.056904+0000","last_fullsized":"2024-09-21T07:50:18.056904+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.063575+0000","last_change":"2024-09-21T07:50:18.076027+0000","last_active":"2024-09-21T07:50:19.063575+0000","last_peered":"2024-09-21T07:50:19.063575+0000","last_clean":"2024-09-21T07:50:19.063575+0000","last_became_active":"2024-09-21T07:50:18.074533+0000","last_became_peered":"2024-09-21T07:50:18.074533+0000","last_unstale":"2024-09-21T07:50:19.063575+0000","last_undegraded":"2024-09-21T07:50:19.063575+0000","last_fullsized":"2024-09-21T07:50:19.063575+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:44.738979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.084144+0000","last_change":"2024-09-21T07:50:19.079008+0000","last_active":"2024-09-21T07:50:19.084144+0000","last_peered":"2024-09-21T07:50:19.084144+0000","last_clean":"2024-09-21T07:50:19.084144+0000","last_became_active":"2024-09-21T07:50:19.076734+0000","last_became_peered":"2024-09-21T07:50:19.076734+0000","last_unstale":"2024-09-21T07:50:19.084144+0000","last_undegraded":"2024-09-21T07:50:19.084144+0000","last_fullsized":"2024-09-21T07:50:19.084144+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:45.401544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.069008+0000","last_change":"2024-09-21T07:50:18.064073+0000","last_active":"2024-09-21T07:50:19.069008+0000","last_peered":"2024-09-21T07:50:19.069008+0000","last_clean":"2024-09-21T07:50:19.069008+0000","last_became_active":"2024-09-21T07:50:18.063748+0000","last_became_peered":"2024-09-21T07:50:18.063748+0000","last_unstale":"2024-09-21T07:50:19.069008+0000","last_undegraded":"2024-09-21T07:50:19.069008+0000","last_fullsized":"2024-09-21T07:50:19.069008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:34.797244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.081628+0000","last_change":"2024-09-21T07:50:19.077796+0000","last_active":"2024-09-21T07:50:19.081628+0000","last_peered":"2024-09-21T07:50:19.081628+0000","last_clean":"2024-09-21T07:50:19.081628+0000","last_became_active":"2024-09-21T07:50:19.077444+0000","last_became_peered":"2024-09-21T07:50:19.077444+0000","last_unstale":"2024-09-21T07:50:19.081628+0000","last_undegraded":"2024-09-21T07:50:19.081628+0000","last_fullsized":"2024-09-21T07:50:19.081628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:46.990022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.065919+0000","last_change":"2024-09-21T07:50:18.066123+0000","last_active":"2024-09-21T07:50:19.065919+0000","last_peered":"2024-09-21T07:50:19.065919+0000","last_clean":"2024-09-21T07:50:19.065919+0000","last_became_active":"2024-09-21T07:50:18.065469+0000","last_became_peered":"2024-09-21T07:50:18.065469+0000","last_unstale":"2024-09-21T07:50:19.065919+0000","last_undegraded":"2024-09-21T07:50:19.065919+0000","last_fullsized":"2024-09-21T07:50:19.065919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:54.340852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.072368+0000","last_change":"2024-09-21T07:50:19.066123+0000","last_active":"2024-09-21T07:50:19.072368+0000","last_peered":"2024-09-21T07:50:19.072368+0000","last_clean":"2024-09-21T07:50:19.072368+0000","last_became_active":"2024-09-21T07:50:19.064472+0000","last_became_peered":"2024-09-21T07:50:19.064472+0000","last_unstale":"2024-09-21T07:50:19.072368+0000","last_undegraded":"2024-09-21T07:50:19.072368+0000","last_fullsized":"2024-09-21T07:50:19.072368+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.600298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.053159+0000","last_change":"2024-09-21T07:50:18.061608+0000","last_active":"2024-09-21T07:50:19.053159+0000","last_peered":"2024-09-21T07:50:19.053159+0000","last_clean":"2024-09-21T07:50:19.053159+0000","last_became_active":"2024-09-21T07:50:18.060830+0000","last_became_peered":"2024-09-21T07:50:18.060830+0000","last_unstale":"2024-09-21T07:50:19.053159+0000","last_undegraded":"2024-09-21T07:50:19.053159+0000","last_fullsized":"2024-09-21T07:50:19.053159+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:36.360850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.083851+0000","last_change":"2024-09-21T07:50:19.079284+0000","last_active":"2024-09-21T07:50:19.083851+0000","last_peered":"2024-09-21T07:50:19.083851+0000","last_clean":"2024-09-21T07:50:19.083851+0000","last_became_active":"2024-09-21T07:50:19.077925+0000","last_became_peered":"2024-09-21T07:50:19.077925+0000","last_unstale":"2024-09-21T07:50:19.083851+0000","last_undegraded":"2024-09-21T07:50:19.083851+0000","last_fullsized":"2024-09-21T07:50:19.083851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.377395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.087959+0000","last_change":"2024-09-21T07:50:18.073593+0000","last_active":"2024-09-21T07:50:19.087959+0000","last_peered":"2024-09-21T07:50:19.087959+0000","last_clean":"2024-09-21T07:50:19.087959+0000","last_became_active":"2024-09-21T07:50:18.072486+0000","last_became_peered":"2024-09-21T07:50:18.072486+0000","last_unstale":"2024-09-21T07:50:19.087959+0000","last_undegraded":"2024-09-21T07:50:19.087959+0000","last_fullsized":"2024-09-21T07:50:19.087959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:37.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051409+0000","last_change":"2024-09-21T07:50:19.080985+0000","last_active":"2024-09-21T07:50:20.051409+0000","last_peered":"2024-09-21T07:50:20.051409+0000","last_clean":"2024-09-21T07:50:20.051409+0000","last_became_active":"2024-09-21T07:50:19.080697+0000","last_became_peered":"2024-09-21T07:50:19.080697+0000","last_unstale":"2024-09-21T07:50:20.051409+0000","last_undegraded":"2024-09-21T07:50:20.051409+0000","last_fullsized":"2024-09-21T07:50:20.051409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:20.725003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.057788+0000","last_change":"2024-09-21T07:50:18.070317+0000","last_active":"2024-09-21T07:50:19.057788+0000","last_peered":"2024-09-21T07:50:19.057788+0000","last_clean":"2024-09-21T07:50:19.057788+0000","last_became_active":"2024-09-21T07:50:18.069817+0000","last_became_peered":"2024-09-21T07:50:18.069817+0000","last_unstale":"2024-09-21T07:50:19.057788+0000","last_undegraded":"2024-09-21T07:50:19.057788+0000","last_fullsized":"2024-09-21T07:50:19.057788+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:57.683732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085552+0000","last_change":"2024-09-21T07:50:19.075367+0000","last_active":"2024-09-21T07:50:19.085552+0000","last_peered":"2024-09-21T07:50:19.085552+0000","last_clean":"2024-09-21T07:50:19.085552+0000","last_became_active":"2024-09-21T07:50:19.075061+0000","last_became_peered":"2024-09-21T07:50:19.075061+0000","last_unstale":"2024-09-21T07:50:19.085552+0000","last_undegraded":"2024-09-21T07:50:19.085552+0000","last_fullsized":"2024-09-21T07:50:19.085552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:17.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.056338+0000","last_change":"2024-09-21T07:50:18.066377+0000","last_active":"2024-09-21T07:50:19.056338+0000","last_peered":"2024-09-21T07:50:19.056338+0000","last_clean":"2024-09-21T07:50:19.056338+0000","last_became_active":"2024-09-21T07:50:18.064476+0000","last_became_peered":"2024-09-21T07:50:18.064476+0000","last_unstale":"2024-09-21T07:50:19.056338+0000","last_undegraded":"2024-09-21T07:50:19.056338+0000","last_fullsized":"2024-09-21T07:50:19.056338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:20.414897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051322+0000","last_change":"2024-09-21T07:50:19.076524+0000","last_active":"2024-09-21T07:50:20.051322+0000","last_peered":"2024-09-21T07:50:20.051322+0000","last_clean":"2024-09-21T07:50:20.051322+0000","last_became_active":"2024-09-21T07:50:19.076227+0000","last_became_peered":"2024-09-21T07:50:19.076227+0000","last_unstale":"2024-09-21T07:50:20.051322+0000","last_undegraded":"2024-09-21T07:50:20.051322+0000","last_fullsized":"2024-09-21T07:50:20.051322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:55.343894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.055646+0000","last_change":"2024-09-21T07:50:18.060400+0000","last_active":"2024-09-21T07:50:19.055646+0000","last_peered":"2024-09-21T07:50:19.055646+0000","last_clean":"2024-09-21T07:50:19.055646+0000","last_became_active":"2024-09-21T07:50:18.060072+0000","last_became_peered":"2024-09-21T07:50:18.060072+0000","last_unstale":"2024-09-21T07:50:19.055646+0000","last_undegraded":"2024-09-21T07:50:19.055646+0000","last_fullsized":"2024-09-21T07:50:19.055646+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:47.250673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.073534+0000","last_change":"2024-09-21T07:50:19.068280+0000","last_active":"2024-09-21T07:50:19.073534+0000","last_peered":"2024-09-21T07:50:19.073534+0000","last_clean":"2024-09-21T07:50:19.073534+0000","last_became_active":"2024-09-21T07:50:19.067952+0000","last_became_peered":"2024-09-21T07:50:19.067952+0000","last_unstale":"2024-09-21T07:50:19.073534+0000","last_undegraded":"2024-09-21T07:50:19.073534+0000","last_fullsized":"2024-09-21T07:50:19.073534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:50.539444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.055000+0000","last_change":"2024-09-21T07:50:18.059750+0000","last_active":"2024-09-21T07:50:19.055000+0000","last_peered":"2024-09-21T07:50:19.055000+0000","last_clean":"2024-09-21T07:50:19.055000+0000","last_became_active":"2024-09-21T07:50:18.059129+0000","last_became_peered":"2024-09-21T07:50:18.059129+0000","last_unstale":"2024-09-21T07:50:19.055000+0000","last_undegraded":"2024-09-21T07:50:19.055000+0000","last_fullsized":"2024-09-21T07:50:19.055000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:59.705236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089307+0000","last_change":"2024-09-21T07:50:19.082778+0000","last_active":"2024-09-21T07:50:19.089307+0000","last_peered":"2024-09-21T07:50:19.089307+0000","last_clean":"2024-09-21T07:50:19.089307+0000","last_became_active":"2024-09-21T07:50:19.082115+0000","last_became_peered":"2024-09-21T07:50:19.082115+0000","last_unstale":"2024-09-21T07:50:19.089307+0000","last_undegraded":"2024-09-21T07:50:19.089307+0000","last_fullsized":"2024-09-21T07:50:19.089307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:44.964526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.055533+0000","last_change":"2024-09-21T07:50:18.065924+0000","last_active":"2024-09-21T07:50:19.055533+0000","last_peered":"2024-09-21T07:50:19.055533+0000","last_clean":"2024-09-21T07:50:19.055533+0000","last_became_active":"2024-09-21T07:50:18.065586+0000","last_became_peered":"2024-09-21T07:50:18.065586+0000","last_unstale":"2024-09-21T07:50:19.055533+0000","last_undegraded":"2024-09-21T07:50:19.055533+0000","last_fullsized":"2024-09-21T07:50:19.055533+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:50.365660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.087164+0000","last_change":"2024-09-21T07:50:19.084053+0000","last_active":"2024-09-21T07:50:19.087164+0000","last_peered":"2024-09-21T07:50:19.087164+0000","last_clean":"2024-09-21T07:50:19.087164+0000","last_became_active":"2024-09-21T07:50:19.083630+0000","last_became_peered":"2024-09-21T07:50:19.083630+0000","last_unstale":"2024-09-21T07:50:19.087164+0000","last_undegraded":"2024-09-21T07:50:19.087164+0000","last_fullsized":"2024-09-21T07:50:19.087164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:05.011541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.067034+0000","last_change":"2024-09-21T07:50:18.066456+0000","last_active":"2024-09-21T07:50:19.067034+0000","last_peered":"2024-09-21T07:50:19.067034+0000","last_clean":"2024-09-21T07:50:19.067034+0000","last_became_active":"2024-09-21T07:50:18.065846+0000","last_became_peered":"2024-09-21T07:50:18.065846+0000","last_unstale":"2024-09-21T07:50:19.067034+0000","last_undegraded":"2024-09-21T07:50:19.067034+0000","last_fullsized":"2024-09-21T07:50:19.067034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:17.833026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085966+0000","last_change":"2024-09-21T07:50:19.077017+0000","last_active":"2024-09-21T07:50:19.085966+0000","last_peered":"2024-09-21T07:50:19.085966+0000","last_clean":"2024-09-21T07:50:19.085966+0000","last_became_active":"2024-09-21T07:50:19.076697+0000","last_became_peered":"2024-09-21T07:50:19.076697+0000","last_unstale":"2024-09-21T07:50:19.085966+0000","last_undegraded":"2024-09-21T07:50:19.085966+0000","last_fullsized":"2024-09-21T07:50:19.085966+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:52.088889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.061213+0000","last_change":"2024-09-21T07:50:18.071782+0000","last_active":"2024-09-21T07:50:19.061213+0000","last_peered":"2024-09-21T07:50:19.061213+0000","last_clean":"2024-09-21T07:50:19.061213+0000","last_became_active":"2024-09-21T07:50:18.070867+0000","last_became_peered":"2024-09-21T07:50:18.070867+0000","last_unstale":"2024-09-21T07:50:19.061213+0000","last_undegraded":"2024-09-21T07:50:19.061213+0000","last_fullsized":"2024-09-21T07:50:19.061213+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:31.088571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.062427+0000","last_change":"2024-09-21T07:50:19.059767+0000","last_active":"2024-09-21T07:50:19.062427+0000","last_peered":"2024-09-21T07:50:19.062427+0000","last_clean":"2024-09-21T07:50:19.062427+0000","last_became_active":"2024-09-21T07:50:19.059475+0000","last_became_peered":"2024-09-21T07:50:19.059475+0000","last_unstale":"2024-09-21T07:50:19.062427+0000","last_undegraded":"2024-09-21T07:50:19.062427+0000","last_fullsized":"2024-09-21T07:50:19.062427+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:46.099340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051006+0000","last_change":"2024-09-21T07:50:18.059765+0000","last_active":"2024-09-21T07:50:20.051006+0000","last_peered":"2024-09-21T07:50:20.051006+0000","last_clean":"2024-09-21T07:50:20.051006+0000","last_became_active":"2024-09-21T07:50:18.059438+0000","last_became_peered":"2024-09-21T07:50:18.059438+0000","last_unstale":"2024-09-21T07:50:20.051006+0000","last_undegraded":"2024-09-21T07:50:20.051006+0000","last_fullsized":"2024-09-21T07:50:20.051006+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:28.150183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":209,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050408+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:50:20.050408+0000","last_peered":"2024-09-21T07:50:20.050408+0000","last_clean":"2024-09-21T07:50:20.050408+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:50:20.050408+0000","last_undegraded":"2024-09-21T07:50:20.050408+0000","last_fullsized":"2024-09-21T07:50:20.050408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":3208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476795,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":362785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476795,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":363241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476792,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1470464,"data_stored":959327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476792,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":868352,"data_stored":362729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":880640,"data_stored":364305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1462272,"data_stored":953117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509496,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":363353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542200,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32024,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339816,"statfs":{"total":96636764160,"available":96603971584,"internally_reserved":0,"allocated":851968,"data_stored":355884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:50:23.488 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:50:23.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 -- 172.21.15.88:0/1217241786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140d1580 msgr2=0x7fa9140d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 --2- 172.21.15.88:0/1217241786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140d1580 0x7fa9140d1990 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fa91408d440 tx=0x7fa90c02a980 comp rx=0 tx=0).stop 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 -- 172.21.15.88:0/1217241786 shutdown_connections 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 --2- 172.21.15.88:0/1217241786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140d1580 0x7fa9140d1990 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 --2- 172.21.15.88:0/1217241786 >> v2:172.21.15.175:3300/0 conn(0x7fa9140b9e00 0x7fa914162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 -- 172.21.15.88:0/1217241786 >> 172.21.15.88:0/1217241786 conn(0x7fa91408bfe0 msgr2=0x7fa91409d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 -- 172.21.15.88:0/1217241786 shutdown_connections 2024-09-21T07:50:23.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.742+0000 7fa919a00640 1 -- 172.21.15.88:0/1217241786 wait complete. 2024-09-21T07:50:23.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.744+0000 7fa919a00640 1 Processor -- start 2024-09-21T07:50:23.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.745+0000 7fa919a00640 1 -- start start 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9140b3160 0x7fa91415fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 0x7fa91415cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140d1580 0x7fa9141f6020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9140d44b0 con 0x7fa9140b9e00 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa9140d4330 con 0x7fa9140d1580 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa9140d4630 con 0x7fa9140b3160 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa90b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 0x7fa91415cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa90b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 0x7fa91415cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48966/0 (socket says 172.21.15.88:48966) 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa90b400640 1 -- 172.21.15.88:0/2933797148 learned_addr learned my addr 172.21.15.88:0/2933797148 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa919000640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140d1580 0x7fa9141f6020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.746+0000 7fa913e00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9140b3160 0x7fa91415fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa90b400640 1 -- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9140b3160 msgr2=0x7fa91415fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa90b400640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9140b3160 0x7fa91415fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa90b400640 1 -- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140d1580 msgr2=0x7fa9141f6020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa90b400640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140d1580 0x7fa9141f6020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa90b400640 1 -- 172.21.15.88:0/2933797148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9140a1b90 con 0x7fa9140b9e00 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa913e00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9140b3160 0x7fa91415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa90b400640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 0x7fa91415cdd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00b3f0 tx=0x7fa8fc00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc015020 con 0x7fa9140b9e00 2024-09-21T07:50:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa8fc011920 con 0x7fa9140b9e00 2024-09-21T07:50:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8fc011d70 con 0x7fa9140b9e00 2024-09-21T07:50:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9140a23d0 con 0x7fa9140b9e00 2024-09-21T07:50:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.747+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9140a20f0 con 0x7fa9140b9e00 2024-09-21T07:50:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.749+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9140d4b50 con 0x7fa9140b9e00 2024-09-21T07:50:23.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.750+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa8fc018020 con 0x7fa9140b9e00 2024-09-21T07:50:23.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.751+0000 7fa910c00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa8f0078f20 0x7fa8f007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:23.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.751+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fa8fc09afa0 con 0x7fa9140b9e00 2024-09-21T07:50:23.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.751+0000 7fa913e00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa8f0078f20 0x7fa8f007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:23.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.752+0000 7fa913e00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa8f0078f20 0x7fa8f007b430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa904005fd0 tx=0x7fa90400f360 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:23.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.755+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa8fc01f030 con 0x7fa9140b9e00 2024-09-21T07:50:23.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.936+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa9141602c0 con 0x7fa8f0078f20 2024-09-21T07:50:23.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.942+0000 7fa910c00640 1 -- 172.21.15.88:0/2933797148 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252058 (secure 0 0 0) 0x7fa9141602c0 con 0x7fa8f0078f20 2024-09-21T07:50:23.947 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:23.948 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa8f0078f20 msgr2=0x7fa8f007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa8f0078f20 0x7fa8f007b430 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fa904005fd0 tx=0x7fa90400f360 comp rx=0 tx=0).stop 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 msgr2=0x7fa91415cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 0x7fa91415cdd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7fa8fc00b3f0 tx=0x7fa8fc00b8b0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 shutdown_connections 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa8f0078f20 0x7fa8f007b430 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa9140d1580 0x7fa9141f6020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9140b9e00 0x7fa91415cdd0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.946+0000 7fa919a00640 1 --2- 172.21.15.88:0/2933797148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa9140b3160 0x7fa91415fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.947+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 >> 172.21.15.88:0/2933797148 conn(0x7fa91408bfe0 msgr2=0x7fa9140d24a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.947+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 shutdown_connections 2024-09-21T07:50:23.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:23.947+0000 7fa919a00640 1 -- 172.21.15.88:0/2933797148 wait complete. 2024-09-21T07:50:23.971 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":204,"stamp":"2024-09-21T07:50:23.926916+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":145,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":110,"ondisk_log_size":110,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":155,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":257544,"kb_used_data":8000,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754717176,"statfs":{"total":773094113280,"available":772830388224,"internally_reserved":0,"allocated":8192000,"data_stored":4099428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001708"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088574+0000","last_change":"2024-09-21T07:50:19.074933+0000","last_active":"2024-09-21T07:50:19.088574+0000","last_peered":"2024-09-21T07:50:19.088574+0000","last_clean":"2024-09-21T07:50:19.088574+0000","last_became_active":"2024-09-21T07:50:19.074021+0000","last_became_peered":"2024-09-21T07:50:19.074021+0000","last_unstale":"2024-09-21T07:50:19.088574+0000","last_undegraded":"2024-09-21T07:50:19.088574+0000","last_fullsized":"2024-09-21T07:50:19.088574+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:59.831046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.092567+0000","last_change":"2024-09-21T07:50:19.084678+0000","last_active":"2024-09-21T07:50:19.092567+0000","last_peered":"2024-09-21T07:50:19.092567+0000","last_clean":"2024-09-21T07:50:19.092567+0000","last_became_active":"2024-09-21T07:50:19.084357+0000","last_became_peered":"2024-09-21T07:50:19.084357+0000","last_unstale":"2024-09-21T07:50:19.092567+0000","last_undegraded":"2024-09-21T07:50:19.092567+0000","last_fullsized":"2024-09-21T07:50:19.092567+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:22.547300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773420+0000","last_change":"2024-09-21T07:50:19.085467+0000","last_active":"2024-09-21T07:50:21.773420+0000","last_peered":"2024-09-21T07:50:21.773420+0000","last_clean":"2024-09-21T07:50:21.773420+0000","last_became_active":"2024-09-21T07:50:19.085119+0000","last_became_peered":"2024-09-21T07:50:19.085119+0000","last_unstale":"2024-09-21T07:50:21.773420+0000","last_undegraded":"2024-09-21T07:50:21.773420+0000","last_fullsized":"2024-09-21T07:50:21.773420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:46:08.484540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773361+0000","last_change":"2024-09-21T07:50:19.085215+0000","last_active":"2024-09-21T07:50:21.773361+0000","last_peered":"2024-09-21T07:50:21.773361+0000","last_clean":"2024-09-21T07:50:21.773361+0000","last_became_active":"2024-09-21T07:50:19.084826+0000","last_became_peered":"2024-09-21T07:50:19.084826+0000","last_unstale":"2024-09-21T07:50:21.773361+0000","last_undegraded":"2024-09-21T07:50:21.773361+0000","last_fullsized":"2024-09-21T07:50:21.773361+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:29:00.332602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.086866+0000","last_change":"2024-09-21T07:50:19.083337+0000","last_active":"2024-09-21T07:50:19.086866+0000","last_peered":"2024-09-21T07:50:19.086866+0000","last_clean":"2024-09-21T07:50:19.086866+0000","last_became_active":"2024-09-21T07:50:19.082960+0000","last_became_peered":"2024-09-21T07:50:19.082960+0000","last_unstale":"2024-09-21T07:50:19.086866+0000","last_undegraded":"2024-09-21T07:50:19.086866+0000","last_fullsized":"2024-09-21T07:50:19.086866+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:08:34.596878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095691+0000","last_change":"2024-09-21T07:50:19.091545+0000","last_active":"2024-09-21T07:50:19.095691+0000","last_peered":"2024-09-21T07:50:19.095691+0000","last_clean":"2024-09-21T07:50:19.095691+0000","last_became_active":"2024-09-21T07:50:19.091135+0000","last_became_peered":"2024-09-21T07:50:19.091135+0000","last_unstale":"2024-09-21T07:50:19.095691+0000","last_undegraded":"2024-09-21T07:50:19.095691+0000","last_fullsized":"2024-09-21T07:50:19.095691+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:37.914081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051024+0000","last_change":"2024-09-21T07:50:19.055075+0000","last_active":"2024-09-21T07:50:20.051024+0000","last_peered":"2024-09-21T07:50:20.051024+0000","last_clean":"2024-09-21T07:50:20.051024+0000","last_became_active":"2024-09-21T07:50:19.054798+0000","last_became_peered":"2024-09-21T07:50:19.054798+0000","last_unstale":"2024-09-21T07:50:20.051024+0000","last_undegraded":"2024-09-21T07:50:20.051024+0000","last_fullsized":"2024-09-21T07:50:20.051024+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:54:13.626965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088999+0000","last_change":"2024-09-21T07:50:19.074960+0000","last_active":"2024-09-21T07:50:19.088999+0000","last_peered":"2024-09-21T07:50:19.088999+0000","last_clean":"2024-09-21T07:50:19.088999+0000","last_became_active":"2024-09-21T07:50:19.074095+0000","last_became_peered":"2024-09-21T07:50:19.074095+0000","last_unstale":"2024-09-21T07:50:19.088999+0000","last_undegraded":"2024-09-21T07:50:19.088999+0000","last_fullsized":"2024-09-21T07:50:19.088999+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:14.142105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773629+0000","last_change":"2024-09-21T07:50:19.067315+0000","last_active":"2024-09-21T07:50:21.773629+0000","last_peered":"2024-09-21T07:50:21.773629+0000","last_clean":"2024-09-21T07:50:21.773629+0000","last_became_active":"2024-09-21T07:50:19.067034+0000","last_became_peered":"2024-09-21T07:50:19.067034+0000","last_unstale":"2024-09-21T07:50:21.773629+0000","last_undegraded":"2024-09-21T07:50:21.773629+0000","last_fullsized":"2024-09-21T07:50:21.773629+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:37.518500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051221+0000","last_change":"2024-09-21T07:50:19.064321+0000","last_active":"2024-09-21T07:50:20.051221+0000","last_peered":"2024-09-21T07:50:20.051221+0000","last_clean":"2024-09-21T07:50:20.051221+0000","last_became_active":"2024-09-21T07:50:19.063463+0000","last_became_peered":"2024-09-21T07:50:19.063463+0000","last_unstale":"2024-09-21T07:50:20.051221+0000","last_undegraded":"2024-09-21T07:50:20.051221+0000","last_fullsized":"2024-09-21T07:50:20.051221+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:51:33.735607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089288+0000","last_change":"2024-09-21T07:50:19.077422+0000","last_active":"2024-09-21T07:50:19.089288+0000","last_peered":"2024-09-21T07:50:19.089288+0000","last_clean":"2024-09-21T07:50:19.089288+0000","last_became_active":"2024-09-21T07:50:19.076521+0000","last_became_peered":"2024-09-21T07:50:19.076521+0000","last_unstale":"2024-09-21T07:50:19.089288+0000","last_undegraded":"2024-09-21T07:50:19.089288+0000","last_fullsized":"2024-09-21T07:50:19.089288+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:03.102804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.084420+0000","last_change":"2024-09-21T07:50:19.080129+0000","last_active":"2024-09-21T07:50:19.084420+0000","last_peered":"2024-09-21T07:50:19.084420+0000","last_clean":"2024-09-21T07:50:19.084420+0000","last_became_active":"2024-09-21T07:50:19.079644+0000","last_became_peered":"2024-09-21T07:50:19.079644+0000","last_unstale":"2024-09-21T07:50:19.084420+0000","last_undegraded":"2024-09-21T07:50:19.084420+0000","last_fullsized":"2024-09-21T07:50:19.084420+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:39.607077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773678+0000","last_change":"2024-09-21T07:50:19.076526+0000","last_active":"2024-09-21T07:50:21.773678+0000","last_peered":"2024-09-21T07:50:21.773678+0000","last_clean":"2024-09-21T07:50:21.773678+0000","last_became_active":"2024-09-21T07:50:19.076180+0000","last_became_peered":"2024-09-21T07:50:19.076180+0000","last_unstale":"2024-09-21T07:50:21.773678+0000","last_undegraded":"2024-09-21T07:50:21.773678+0000","last_fullsized":"2024-09-21T07:50:21.773678+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:21.783970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.076857+0000","last_change":"2024-09-21T07:50:19.072199+0000","last_active":"2024-09-21T07:50:19.076857+0000","last_peered":"2024-09-21T07:50:19.076857+0000","last_clean":"2024-09-21T07:50:19.076857+0000","last_became_active":"2024-09-21T07:50:19.070583+0000","last_became_peered":"2024-09-21T07:50:19.070583+0000","last_unstale":"2024-09-21T07:50:19.076857+0000","last_undegraded":"2024-09-21T07:50:19.076857+0000","last_fullsized":"2024-09-21T07:50:19.076857+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:01:50.938596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.072340+0000","last_change":"2024-09-21T07:50:19.064362+0000","last_active":"2024-09-21T07:50:19.072340+0000","last_peered":"2024-09-21T07:50:19.072340+0000","last_clean":"2024-09-21T07:50:19.072340+0000","last_became_active":"2024-09-21T07:50:19.063322+0000","last_became_peered":"2024-09-21T07:50:19.063322+0000","last_unstale":"2024-09-21T07:50:19.072340+0000","last_undegraded":"2024-09-21T07:50:19.072340+0000","last_fullsized":"2024-09-21T07:50:19.072340+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:51.029552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089025+0000","last_change":"2024-09-21T07:50:19.082296+0000","last_active":"2024-09-21T07:50:19.089025+0000","last_peered":"2024-09-21T07:50:19.089025+0000","last_clean":"2024-09-21T07:50:19.089025+0000","last_became_active":"2024-09-21T07:50:19.081652+0000","last_became_peered":"2024-09-21T07:50:19.081652+0000","last_unstale":"2024-09-21T07:50:19.089025+0000","last_undegraded":"2024-09-21T07:50:19.089025+0000","last_fullsized":"2024-09-21T07:50:19.089025+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:40.314898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.070556+0000","last_change":"2024-09-21T07:50:19.063930+0000","last_active":"2024-09-21T07:50:19.070556+0000","last_peered":"2024-09-21T07:50:19.070556+0000","last_clean":"2024-09-21T07:50:19.070556+0000","last_became_active":"2024-09-21T07:50:19.063520+0000","last_became_peered":"2024-09-21T07:50:19.063520+0000","last_unstale":"2024-09-21T07:50:19.070556+0000","last_undegraded":"2024-09-21T07:50:19.070556+0000","last_fullsized":"2024-09-21T07:50:19.070556+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:40.456835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.070193+0000","last_change":"2024-09-21T07:50:19.062381+0000","last_active":"2024-09-21T07:50:19.070193+0000","last_peered":"2024-09-21T07:50:19.070193+0000","last_clean":"2024-09-21T07:50:19.070193+0000","last_became_active":"2024-09-21T07:50:19.062050+0000","last_became_peered":"2024-09-21T07:50:19.062050+0000","last_unstale":"2024-09-21T07:50:19.070193+0000","last_undegraded":"2024-09-21T07:50:19.070193+0000","last_fullsized":"2024-09-21T07:50:19.070193+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:49:23.146741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089313+0000","last_change":"2024-09-21T07:50:19.078093+0000","last_active":"2024-09-21T07:50:19.089313+0000","last_peered":"2024-09-21T07:50:19.089313+0000","last_clean":"2024-09-21T07:50:19.089313+0000","last_became_active":"2024-09-21T07:50:19.077809+0000","last_became_peered":"2024-09-21T07:50:19.077809+0000","last_unstale":"2024-09-21T07:50:19.089313+0000","last_undegraded":"2024-09-21T07:50:19.089313+0000","last_fullsized":"2024-09-21T07:50:19.089313+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:09.089424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.076247+0000","last_change":"2024-09-21T07:50:19.074153+0000","last_active":"2024-09-21T07:50:19.076247+0000","last_peered":"2024-09-21T07:50:19.076247+0000","last_clean":"2024-09-21T07:50:19.076247+0000","last_became_active":"2024-09-21T07:50:19.072750+0000","last_became_peered":"2024-09-21T07:50:19.072750+0000","last_unstale":"2024-09-21T07:50:19.076247+0000","last_undegraded":"2024-09-21T07:50:19.076247+0000","last_fullsized":"2024-09-21T07:50:19.076247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:06.360151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088384+0000","last_change":"2024-09-21T07:50:19.079831+0000","last_active":"2024-09-21T07:50:19.088384+0000","last_peered":"2024-09-21T07:50:19.088384+0000","last_clean":"2024-09-21T07:50:19.088384+0000","last_became_active":"2024-09-21T07:50:19.078464+0000","last_became_peered":"2024-09-21T07:50:19.078464+0000","last_unstale":"2024-09-21T07:50:19.088384+0000","last_undegraded":"2024-09-21T07:50:19.088384+0000","last_fullsized":"2024-09-21T07:50:19.088384+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:44.178216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.082349+0000","last_change":"2024-09-21T07:50:19.072081+0000","last_active":"2024-09-21T07:50:19.082349+0000","last_peered":"2024-09-21T07:50:19.082349+0000","last_clean":"2024-09-21T07:50:19.082349+0000","last_became_active":"2024-09-21T07:50:19.071800+0000","last_became_peered":"2024-09-21T07:50:19.071800+0000","last_unstale":"2024-09-21T07:50:19.082349+0000","last_undegraded":"2024-09-21T07:50:19.082349+0000","last_fullsized":"2024-09-21T07:50:19.082349+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:52:25.260802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050678+0000","last_change":"2024-09-21T07:50:19.079780+0000","last_active":"2024-09-21T07:50:20.050678+0000","last_peered":"2024-09-21T07:50:20.050678+0000","last_clean":"2024-09-21T07:50:20.050678+0000","last_became_active":"2024-09-21T07:50:19.079407+0000","last_became_peered":"2024-09-21T07:50:19.079407+0000","last_unstale":"2024-09-21T07:50:20.050678+0000","last_undegraded":"2024-09-21T07:50:20.050678+0000","last_fullsized":"2024-09-21T07:50:20.050678+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:10.828365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088563+0000","last_change":"2024-09-21T07:50:19.074330+0000","last_active":"2024-09-21T07:50:19.088563+0000","last_peered":"2024-09-21T07:50:19.088563+0000","last_clean":"2024-09-21T07:50:19.088563+0000","last_became_active":"2024-09-21T07:50:19.073987+0000","last_became_peered":"2024-09-21T07:50:19.073987+0000","last_unstale":"2024-09-21T07:50:19.088563+0000","last_undegraded":"2024-09-21T07:50:19.088563+0000","last_fullsized":"2024-09-21T07:50:19.088563+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:05.401336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051259+0000","last_change":"2024-09-21T07:50:19.083611+0000","last_active":"2024-09-21T07:50:20.051259+0000","last_peered":"2024-09-21T07:50:20.051259+0000","last_clean":"2024-09-21T07:50:20.051259+0000","last_became_active":"2024-09-21T07:50:19.083100+0000","last_became_peered":"2024-09-21T07:50:19.083100+0000","last_unstale":"2024-09-21T07:50:20.051259+0000","last_undegraded":"2024-09-21T07:50:20.051259+0000","last_fullsized":"2024-09-21T07:50:20.051259+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:53.711994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089253+0000","last_change":"2024-09-21T07:50:19.077593+0000","last_active":"2024-09-21T07:50:19.089253+0000","last_peered":"2024-09-21T07:50:19.089253+0000","last_clean":"2024-09-21T07:50:19.089253+0000","last_became_active":"2024-09-21T07:50:19.077293+0000","last_became_peered":"2024-09-21T07:50:19.077293+0000","last_unstale":"2024-09-21T07:50:19.089253+0000","last_undegraded":"2024-09-21T07:50:19.089253+0000","last_fullsized":"2024-09-21T07:50:19.089253+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:55.104639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.083779+0000","last_change":"2024-09-21T07:50:19.079290+0000","last_active":"2024-09-21T07:50:19.083779+0000","last_peered":"2024-09-21T07:50:19.083779+0000","last_clean":"2024-09-21T07:50:19.083779+0000","last_became_active":"2024-09-21T07:50:19.078227+0000","last_became_peered":"2024-09-21T07:50:19.078227+0000","last_unstale":"2024-09-21T07:50:19.083779+0000","last_undegraded":"2024-09-21T07:50:19.083779+0000","last_fullsized":"2024-09-21T07:50:19.083779+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:14:47.078670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.090328+0000","last_change":"2024-09-21T07:50:19.082952+0000","last_active":"2024-09-21T07:50:19.090328+0000","last_peered":"2024-09-21T07:50:19.090328+0000","last_clean":"2024-09-21T07:50:19.090328+0000","last_became_active":"2024-09-21T07:50:19.082576+0000","last_became_peered":"2024-09-21T07:50:19.082576+0000","last_unstale":"2024-09-21T07:50:19.090328+0000","last_undegraded":"2024-09-21T07:50:19.090328+0000","last_fullsized":"2024-09-21T07:50:19.090328+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:32.091613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773745+0000","last_change":"2024-09-21T07:50:19.088485+0000","last_active":"2024-09-21T07:50:21.773745+0000","last_peered":"2024-09-21T07:50:21.773745+0000","last_clean":"2024-09-21T07:50:21.773745+0000","last_became_active":"2024-09-21T07:50:19.088218+0000","last_became_peered":"2024-09-21T07:50:19.088218+0000","last_unstale":"2024-09-21T07:50:21.773745+0000","last_undegraded":"2024-09-21T07:50:21.773745+0000","last_fullsized":"2024-09-21T07:50:21.773745+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:40:23.821068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089406+0000","last_change":"2024-09-21T07:50:19.069884+0000","last_active":"2024-09-21T07:50:19.089406+0000","last_peered":"2024-09-21T07:50:19.089406+0000","last_clean":"2024-09-21T07:50:19.089406+0000","last_became_active":"2024-09-21T07:50:19.069600+0000","last_became_peered":"2024-09-21T07:50:19.069600+0000","last_unstale":"2024-09-21T07:50:19.089406+0000","last_undegraded":"2024-09-21T07:50:19.089406+0000","last_fullsized":"2024-09-21T07:50:19.089406+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:43.356105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089133+0000","last_change":"2024-09-21T07:50:19.081019+0000","last_active":"2024-09-21T07:50:19.089133+0000","last_peered":"2024-09-21T07:50:19.089133+0000","last_clean":"2024-09-21T07:50:19.089133+0000","last_became_active":"2024-09-21T07:50:19.080702+0000","last_became_peered":"2024-09-21T07:50:19.080702+0000","last_unstale":"2024-09-21T07:50:19.089133+0000","last_undegraded":"2024-09-21T07:50:19.089133+0000","last_fullsized":"2024-09-21T07:50:19.089133+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:07.448041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089359+0000","last_change":"2024-09-21T07:50:19.081992+0000","last_active":"2024-09-21T07:50:19.089359+0000","last_peered":"2024-09-21T07:50:19.089359+0000","last_clean":"2024-09-21T07:50:19.089359+0000","last_became_active":"2024-09-21T07:50:19.081588+0000","last_became_peered":"2024-09-21T07:50:19.081588+0000","last_unstale":"2024-09-21T07:50:19.089359+0000","last_undegraded":"2024-09-21T07:50:19.089359+0000","last_fullsized":"2024-09-21T07:50:19.089359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:05.117390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.092179+0000","last_change":"2024-09-21T07:50:19.087010+0000","last_active":"2024-09-21T07:50:19.092179+0000","last_peered":"2024-09-21T07:50:19.092179+0000","last_clean":"2024-09-21T07:50:19.092179+0000","last_became_active":"2024-09-21T07:50:19.086720+0000","last_became_peered":"2024-09-21T07:50:19.086720+0000","last_unstale":"2024-09-21T07:50:19.092179+0000","last_undegraded":"2024-09-21T07:50:19.092179+0000","last_fullsized":"2024-09-21T07:50:19.092179+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:54.692860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1f","version":"72'4","reported_seq":17,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.086121+0000","last_change":"2024-09-21T07:50:18.073367+0000","last_active":"2024-09-21T07:50:19.086121+0000","last_peered":"2024-09-21T07:50:19.086121+0000","last_clean":"2024-09-21T07:50:19.086121+0000","last_became_active":"2024-09-21T07:50:18.073050+0000","last_became_peered":"2024-09-21T07:50:18.073050+0000","last_unstale":"2024-09-21T07:50:19.086121+0000","last_undegraded":"2024-09-21T07:50:19.086121+0000","last_fullsized":"2024-09-21T07:50:19.086121+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:37:21.544258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773673+0000","last_change":"2024-09-21T07:50:19.081779+0000","last_active":"2024-09-21T07:50:21.773673+0000","last_peered":"2024-09-21T07:50:21.773673+0000","last_clean":"2024-09-21T07:50:21.773673+0000","last_became_active":"2024-09-21T07:50:19.081246+0000","last_became_peered":"2024-09-21T07:50:19.081246+0000","last_unstale":"2024-09-21T07:50:21.773673+0000","last_undegraded":"2024-09-21T07:50:21.773673+0000","last_fullsized":"2024-09-21T07:50:21.773673+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:41:48.806757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1e","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075760+0000","last_change":"2024-09-21T07:50:18.067461+0000","last_active":"2024-09-21T07:50:19.075760+0000","last_peered":"2024-09-21T07:50:19.075760+0000","last_clean":"2024-09-21T07:50:19.075760+0000","last_became_active":"2024-09-21T07:50:18.067141+0000","last_became_peered":"2024-09-21T07:50:18.067141+0000","last_unstale":"2024-09-21T07:50:19.075760+0000","last_undegraded":"2024-09-21T07:50:19.075760+0000","last_fullsized":"2024-09-21T07:50:19.075760+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:03.733100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095670+0000","last_change":"2024-09-21T07:50:19.089763+0000","last_active":"2024-09-21T07:50:19.095670+0000","last_peered":"2024-09-21T07:50:19.095670+0000","last_clean":"2024-09-21T07:50:19.095670+0000","last_became_active":"2024-09-21T07:50:19.089389+0000","last_became_peered":"2024-09-21T07:50:19.089389+0000","last_unstale":"2024-09-21T07:50:19.095670+0000","last_undegraded":"2024-09-21T07:50:19.095670+0000","last_fullsized":"2024-09-21T07:50:19.095670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:47.561075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1d","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088433+0000","last_change":"2024-09-21T07:50:18.076445+0000","last_active":"2024-09-21T07:50:19.088433+0000","last_peered":"2024-09-21T07:50:19.088433+0000","last_clean":"2024-09-21T07:50:19.088433+0000","last_became_active":"2024-09-21T07:50:18.076188+0000","last_became_peered":"2024-09-21T07:50:18.076188+0000","last_unstale":"2024-09-21T07:50:19.088433+0000","last_undegraded":"2024-09-21T07:50:19.088433+0000","last_fullsized":"2024-09-21T07:50:19.088433+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:55.164877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051096+0000","last_change":"2024-09-21T07:50:19.075753+0000","last_active":"2024-09-21T07:50:20.051096+0000","last_peered":"2024-09-21T07:50:20.051096+0000","last_clean":"2024-09-21T07:50:20.051096+0000","last_became_active":"2024-09-21T07:50:19.075399+0000","last_became_peered":"2024-09-21T07:50:19.075399+0000","last_unstale":"2024-09-21T07:50:20.051096+0000","last_undegraded":"2024-09-21T07:50:20.051096+0000","last_fullsized":"2024-09-21T07:50:20.051096+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:29:34.198178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1c","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085753+0000","last_change":"2024-09-21T07:50:18.074894+0000","last_active":"2024-09-21T07:50:19.085753+0000","last_peered":"2024-09-21T07:50:19.085753+0000","last_clean":"2024-09-21T07:50:19.085753+0000","last_became_active":"2024-09-21T07:50:18.074115+0000","last_became_peered":"2024-09-21T07:50:18.074115+0000","last_unstale":"2024-09-21T07:50:19.085753+0000","last_undegraded":"2024-09-21T07:50:19.085753+0000","last_fullsized":"2024-09-21T07:50:19.085753+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:12.718692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.087755+0000","last_change":"2024-09-21T07:50:19.063111+0000","last_active":"2024-09-21T07:50:19.087755+0000","last_peered":"2024-09-21T07:50:19.087755+0000","last_clean":"2024-09-21T07:50:19.087755+0000","last_became_active":"2024-09-21T07:50:19.060509+0000","last_became_peered":"2024-09-21T07:50:19.060509+0000","last_unstale":"2024-09-21T07:50:19.087755+0000","last_undegraded":"2024-09-21T07:50:19.087755+0000","last_fullsized":"2024-09-21T07:50:19.087755+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:36.691011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.1b","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085722+0000","last_change":"2024-09-21T07:50:18.062122+0000","last_active":"2024-09-21T07:50:19.085722+0000","last_peered":"2024-09-21T07:50:19.085722+0000","last_clean":"2024-09-21T07:50:19.085722+0000","last_became_active":"2024-09-21T07:50:18.061825+0000","last_became_peered":"2024-09-21T07:50:18.061825+0000","last_unstale":"2024-09-21T07:50:19.085722+0000","last_undegraded":"2024-09-21T07:50:19.085722+0000","last_fullsized":"2024-09-21T07:50:19.085722+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:16.961875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095033+0000","last_change":"2024-09-21T07:50:19.088559+0000","last_active":"2024-09-21T07:50:19.095033+0000","last_peered":"2024-09-21T07:50:19.095033+0000","last_clean":"2024-09-21T07:50:19.095033+0000","last_became_active":"2024-09-21T07:50:19.088224+0000","last_became_peered":"2024-09-21T07:50:19.088224+0000","last_unstale":"2024-09-21T07:50:19.095033+0000","last_undegraded":"2024-09-21T07:50:19.095033+0000","last_fullsized":"2024-09-21T07:50:19.095033+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:05.554264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1a","version":"72'1","reported_seq":16,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050628+0000","last_change":"2024-09-21T07:50:18.069259+0000","last_active":"2024-09-21T07:50:20.050628+0000","last_peered":"2024-09-21T07:50:20.050628+0000","last_clean":"2024-09-21T07:50:20.050628+0000","last_became_active":"2024-09-21T07:50:18.067538+0000","last_became_peered":"2024-09-21T07:50:18.067538+0000","last_unstale":"2024-09-21T07:50:20.050628+0000","last_undegraded":"2024-09-21T07:50:20.050628+0000","last_fullsized":"2024-09-21T07:50:20.050628+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:57.461370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089192+0000","last_change":"2024-09-21T07:50:19.085649+0000","last_active":"2024-09-21T07:50:19.089192+0000","last_peered":"2024-09-21T07:50:19.089192+0000","last_clean":"2024-09-21T07:50:19.089192+0000","last_became_active":"2024-09-21T07:50:19.085349+0000","last_became_peered":"2024-09-21T07:50:19.085349+0000","last_unstale":"2024-09-21T07:50:19.089192+0000","last_undegraded":"2024-09-21T07:50:19.089192+0000","last_fullsized":"2024-09-21T07:50:19.089192+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:03.655502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.071903+0000","last_change":"2024-09-21T07:50:18.077241+0000","last_active":"2024-09-21T07:50:19.071903+0000","last_peered":"2024-09-21T07:50:19.071903+0000","last_clean":"2024-09-21T07:50:19.071903+0000","last_became_active":"2024-09-21T07:50:18.076913+0000","last_became_peered":"2024-09-21T07:50:18.076913+0000","last_unstale":"2024-09-21T07:50:19.071903+0000","last_undegraded":"2024-09-21T07:50:19.071903+0000","last_fullsized":"2024-09-21T07:50:19.071903+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:28:14.586470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075971+0000","last_change":"2024-09-21T07:50:19.072081+0000","last_active":"2024-09-21T07:50:19.075971+0000","last_peered":"2024-09-21T07:50:19.075971+0000","last_clean":"2024-09-21T07:50:19.075971+0000","last_became_active":"2024-09-21T07:50:19.070907+0000","last_became_peered":"2024-09-21T07:50:19.070907+0000","last_unstale":"2024-09-21T07:50:19.075971+0000","last_undegraded":"2024-09-21T07:50:19.075971+0000","last_fullsized":"2024-09-21T07:50:19.075971+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:06:20.195607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.052703+0000","last_change":"2024-09-21T07:50:18.067552+0000","last_active":"2024-09-21T07:50:19.052703+0000","last_peered":"2024-09-21T07:50:19.052703+0000","last_clean":"2024-09-21T07:50:19.052703+0000","last_became_active":"2024-09-21T07:50:18.067203+0000","last_became_peered":"2024-09-21T07:50:18.067203+0000","last_unstale":"2024-09-21T07:50:19.052703+0000","last_undegraded":"2024-09-21T07:50:19.052703+0000","last_fullsized":"2024-09-21T07:50:19.052703+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:38.698170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051367+0000","last_change":"2024-09-21T07:50:19.088944+0000","last_active":"2024-09-21T07:50:20.051367+0000","last_peered":"2024-09-21T07:50:20.051367+0000","last_clean":"2024-09-21T07:50:20.051367+0000","last_became_active":"2024-09-21T07:50:19.086571+0000","last_became_peered":"2024-09-21T07:50:19.086571+0000","last_unstale":"2024-09-21T07:50:20.051367+0000","last_undegraded":"2024-09-21T07:50:20.051367+0000","last_fullsized":"2024-09-21T07:50:20.051367+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:18.836068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.054711+0000","last_change":"2024-09-21T07:50:18.073914+0000","last_active":"2024-09-21T07:50:19.054711+0000","last_peered":"2024-09-21T07:50:19.054711+0000","last_clean":"2024-09-21T07:50:19.054711+0000","last_became_active":"2024-09-21T07:50:18.072897+0000","last_became_peered":"2024-09-21T07:50:18.072897+0000","last_unstale":"2024-09-21T07:50:19.054711+0000","last_undegraded":"2024-09-21T07:50:19.054711+0000","last_fullsized":"2024-09-21T07:50:19.054711+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:37.533903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.095251+0000","last_change":"2024-09-21T07:50:19.090698+0000","last_active":"2024-09-21T07:50:19.095251+0000","last_peered":"2024-09-21T07:50:19.095251+0000","last_clean":"2024-09-21T07:50:19.095251+0000","last_became_active":"2024-09-21T07:50:19.090341+0000","last_became_peered":"2024-09-21T07:50:19.090341+0000","last_unstale":"2024-09-21T07:50:19.095251+0000","last_undegraded":"2024-09-21T07:50:19.095251+0000","last_fullsized":"2024-09-21T07:50:19.095251+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:45.742021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.070269+0000","last_change":"2024-09-21T07:50:18.075151+0000","last_active":"2024-09-21T07:50:19.070269+0000","last_peered":"2024-09-21T07:50:19.070269+0000","last_clean":"2024-09-21T07:50:19.070269+0000","last_became_active":"2024-09-21T07:50:18.074662+0000","last_became_peered":"2024-09-21T07:50:18.074662+0000","last_unstale":"2024-09-21T07:50:19.070269+0000","last_undegraded":"2024-09-21T07:50:19.070269+0000","last_fullsized":"2024-09-21T07:50:19.070269+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:19.721961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088880+0000","last_change":"2024-09-21T07:50:19.076994+0000","last_active":"2024-09-21T07:50:19.088880+0000","last_peered":"2024-09-21T07:50:19.088880+0000","last_clean":"2024-09-21T07:50:19.088880+0000","last_became_active":"2024-09-21T07:50:19.076684+0000","last_became_peered":"2024-09-21T07:50:19.076684+0000","last_unstale":"2024-09-21T07:50:19.088880+0000","last_undegraded":"2024-09-21T07:50:19.088880+0000","last_fullsized":"2024-09-21T07:50:19.088880+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:05.896870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.15","version":"72'3","reported_seq":16,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.084687+0000","last_change":"2024-09-21T07:50:18.061318+0000","last_active":"2024-09-21T07:50:19.084687+0000","last_peered":"2024-09-21T07:50:19.084687+0000","last_clean":"2024-09-21T07:50:19.084687+0000","last_became_active":"2024-09-21T07:50:18.060246+0000","last_became_peered":"2024-09-21T07:50:18.060246+0000","last_unstale":"2024-09-21T07:50:19.084687+0000","last_undegraded":"2024-09-21T07:50:19.084687+0000","last_fullsized":"2024-09-21T07:50:19.084687+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:37.243809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.083751+0000","last_change":"2024-09-21T07:50:19.078951+0000","last_active":"2024-09-21T07:50:19.083751+0000","last_peered":"2024-09-21T07:50:19.083751+0000","last_clean":"2024-09-21T07:50:19.083751+0000","last_became_active":"2024-09-21T07:50:19.077959+0000","last_became_peered":"2024-09-21T07:50:19.077959+0000","last_unstale":"2024-09-21T07:50:19.083751+0000","last_undegraded":"2024-09-21T07:50:19.083751+0000","last_fullsized":"2024-09-21T07:50:19.083751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:28:28.514449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.14","version":"72'6","reported_seq":19,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.093201+0000","last_change":"2024-09-21T07:50:18.069448+0000","last_active":"2024-09-21T07:50:19.093201+0000","last_peered":"2024-09-21T07:50:19.093201+0000","last_clean":"2024-09-21T07:50:19.093201+0000","last_became_active":"2024-09-21T07:50:18.069165+0000","last_became_peered":"2024-09-21T07:50:18.069165+0000","last_unstale":"2024-09-21T07:50:19.093201+0000","last_undegraded":"2024-09-21T07:50:19.093201+0000","last_fullsized":"2024-09-21T07:50:19.093201+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:17.833026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075041+0000","last_change":"2024-09-21T07:50:19.070204+0000","last_active":"2024-09-21T07:50:19.075041+0000","last_peered":"2024-09-21T07:50:19.075041+0000","last_clean":"2024-09-21T07:50:19.075041+0000","last_became_active":"2024-09-21T07:50:19.069160+0000","last_became_peered":"2024-09-21T07:50:19.069160+0000","last_unstale":"2024-09-21T07:50:19.075041+0000","last_undegraded":"2024-09-21T07:50:19.075041+0000","last_fullsized":"2024-09-21T07:50:19.075041+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:41:17.506660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.13","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.091124+0000","last_change":"2024-09-21T07:50:18.071213+0000","last_active":"2024-09-21T07:50:19.091124+0000","last_peered":"2024-09-21T07:50:19.091124+0000","last_clean":"2024-09-21T07:50:19.091124+0000","last_became_active":"2024-09-21T07:50:18.070905+0000","last_became_peered":"2024-09-21T07:50:18.070905+0000","last_unstale":"2024-09-21T07:50:19.091124+0000","last_undegraded":"2024-09-21T07:50:19.091124+0000","last_fullsized":"2024-09-21T07:50:19.091124+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:13.121735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050884+0000","last_change":"2024-09-21T07:50:19.068866+0000","last_active":"2024-09-21T07:50:20.050884+0000","last_peered":"2024-09-21T07:50:20.050884+0000","last_clean":"2024-09-21T07:50:20.050884+0000","last_became_active":"2024-09-21T07:50:19.066134+0000","last_became_peered":"2024-09-21T07:50:19.066134+0000","last_unstale":"2024-09-21T07:50:20.050884+0000","last_undegraded":"2024-09-21T07:50:20.050884+0000","last_fullsized":"2024-09-21T07:50:20.050884+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:28.552933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050620+0000","last_change":"2024-09-21T07:50:18.060992+0000","last_active":"2024-09-21T07:50:20.050620+0000","last_peered":"2024-09-21T07:50:20.050620+0000","last_clean":"2024-09-21T07:50:20.050620+0000","last_became_active":"2024-09-21T07:50:18.060683+0000","last_became_peered":"2024-09-21T07:50:18.060683+0000","last_unstale":"2024-09-21T07:50:20.050620+0000","last_undegraded":"2024-09-21T07:50:20.050620+0000","last_fullsized":"2024-09-21T07:50:20.050620+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:50:40.804735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.075893+0000","last_change":"2024-09-21T07:50:19.073051+0000","last_active":"2024-09-21T07:50:19.075893+0000","last_peered":"2024-09-21T07:50:19.075893+0000","last_clean":"2024-09-21T07:50:19.075893+0000","last_became_active":"2024-09-21T07:50:19.072685+0000","last_became_peered":"2024-09-21T07:50:19.072685+0000","last_unstale":"2024-09-21T07:50:19.075893+0000","last_undegraded":"2024-09-21T07:50:19.075893+0000","last_fullsized":"2024-09-21T07:50:19.075893+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:33.524928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":15,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050889+0000","last_change":"2024-09-21T07:50:18.066959+0000","last_active":"2024-09-21T07:50:20.050889+0000","last_peered":"2024-09-21T07:50:20.050889+0000","last_clean":"2024-09-21T07:50:20.050889+0000","last_became_active":"2024-09-21T07:50:18.065168+0000","last_became_peered":"2024-09-21T07:50:18.065168+0000","last_unstale":"2024-09-21T07:50:20.050889+0000","last_undegraded":"2024-09-21T07:50:20.050889+0000","last_fullsized":"2024-09-21T07:50:20.050889+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:07.675249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773371+0000","last_change":"2024-09-21T07:50:19.070034+0000","last_active":"2024-09-21T07:50:21.773371+0000","last_peered":"2024-09-21T07:50:21.773371+0000","last_clean":"2024-09-21T07:50:21.773371+0000","last_became_active":"2024-09-21T07:50:19.069649+0000","last_became_peered":"2024-09-21T07:50:19.069649+0000","last_unstale":"2024-09-21T07:50:21.773371+0000","last_undegraded":"2024-09-21T07:50:21.773371+0000","last_fullsized":"2024-09-21T07:50:21.773371+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:01.099624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.10","version":"72'8","reported_seq":23,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050387+0000","last_change":"2024-09-21T07:50:18.067636+0000","last_active":"2024-09-21T07:50:20.050387+0000","last_peered":"2024-09-21T07:50:20.050387+0000","last_clean":"2024-09-21T07:50:20.050387+0000","last_became_active":"2024-09-21T07:50:18.067199+0000","last_became_peered":"2024-09-21T07:50:18.067199+0000","last_unstale":"2024-09-21T07:50:20.050387+0000","last_undegraded":"2024-09-21T07:50:20.050387+0000","last_fullsized":"2024-09-21T07:50:20.050387+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:16.631127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085500+0000","last_change":"2024-09-21T07:50:19.078007+0000","last_active":"2024-09-21T07:50:19.085500+0000","last_peered":"2024-09-21T07:50:19.085500+0000","last_clean":"2024-09-21T07:50:19.085500+0000","last_became_active":"2024-09-21T07:50:19.077731+0000","last_became_peered":"2024-09-21T07:50:19.077731+0000","last_unstale":"2024-09-21T07:50:19.085500+0000","last_undegraded":"2024-09-21T07:50:19.085500+0000","last_fullsized":"2024-09-21T07:50:19.085500+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:19.277913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.f","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.088074+0000","last_change":"2024-09-21T07:50:18.074253+0000","last_active":"2024-09-21T07:50:19.088074+0000","last_peered":"2024-09-21T07:50:19.088074+0000","last_clean":"2024-09-21T07:50:19.088074+0000","last_became_active":"2024-09-21T07:50:18.073286+0000","last_became_peered":"2024-09-21T07:50:18.073286+0000","last_unstale":"2024-09-21T07:50:19.088074+0000","last_undegraded":"2024-09-21T07:50:19.088074+0000","last_fullsized":"2024-09-21T07:50:19.088074+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:09:18.477891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089057+0000","last_change":"2024-09-21T07:50:19.080794+0000","last_active":"2024-09-21T07:50:19.089057+0000","last_peered":"2024-09-21T07:50:19.089057+0000","last_clean":"2024-09-21T07:50:19.089057+0000","last_became_active":"2024-09-21T07:50:19.079531+0000","last_became_peered":"2024-09-21T07:50:19.079531+0000","last_unstale":"2024-09-21T07:50:19.089057+0000","last_undegraded":"2024-09-21T07:50:19.089057+0000","last_fullsized":"2024-09-21T07:50:19.089057+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:46.908903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.e","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085634+0000","last_change":"2024-09-21T07:50:18.070731+0000","last_active":"2024-09-21T07:50:19.085634+0000","last_peered":"2024-09-21T07:50:19.085634+0000","last_clean":"2024-09-21T07:50:19.085634+0000","last_became_active":"2024-09-21T07:50:18.070442+0000","last_became_peered":"2024-09-21T07:50:18.070442+0000","last_unstale":"2024-09-21T07:50:19.085634+0000","last_undegraded":"2024-09-21T07:50:19.085634+0000","last_fullsized":"2024-09-21T07:50:19.085634+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:59:42.710809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773643+0000","last_change":"2024-09-21T07:50:19.066353+0000","last_active":"2024-09-21T07:50:21.773643+0000","last_peered":"2024-09-21T07:50:21.773643+0000","last_clean":"2024-09-21T07:50:21.773643+0000","last_became_active":"2024-09-21T07:50:19.066041+0000","last_became_peered":"2024-09-21T07:50:19.066041+0000","last_unstale":"2024-09-21T07:50:21.773643+0000","last_undegraded":"2024-09-21T07:50:21.773643+0000","last_fullsized":"2024-09-21T07:50:21.773643+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:22.487304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.d","version":"72'1","reported_seq":14,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.090950+0000","last_change":"2024-09-21T07:50:18.068881+0000","last_active":"2024-09-21T07:50:19.090950+0000","last_peered":"2024-09-21T07:50:19.090950+0000","last_clean":"2024-09-21T07:50:19.090950+0000","last_became_active":"2024-09-21T07:50:18.068518+0000","last_became_peered":"2024-09-21T07:50:18.068518+0000","last_unstale":"2024-09-21T07:50:19.090950+0000","last_undegraded":"2024-09-21T07:50:19.090950+0000","last_fullsized":"2024-09-21T07:50:19.090950+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:52.708952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:21.773381+0000","last_change":"2024-09-21T07:50:19.080746+0000","last_active":"2024-09-21T07:50:21.773381+0000","last_peered":"2024-09-21T07:50:21.773381+0000","last_clean":"2024-09-21T07:50:21.773381+0000","last_became_active":"2024-09-21T07:50:19.079605+0000","last_became_peered":"2024-09-21T07:50:19.079605+0000","last_unstale":"2024-09-21T07:50:21.773381+0000","last_undegraded":"2024-09-21T07:50:21.773381+0000","last_fullsized":"2024-09-21T07:50:21.773381+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:23.550587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.063575+0000","last_change":"2024-09-21T07:50:18.076027+0000","last_active":"2024-09-21T07:50:19.063575+0000","last_peered":"2024-09-21T07:50:19.063575+0000","last_clean":"2024-09-21T07:50:19.063575+0000","last_became_active":"2024-09-21T07:50:18.074533+0000","last_became_peered":"2024-09-21T07:50:18.074533+0000","last_unstale":"2024-09-21T07:50:19.063575+0000","last_undegraded":"2024-09-21T07:50:19.063575+0000","last_fullsized":"2024-09-21T07:50:19.063575+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:44.738979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.084144+0000","last_change":"2024-09-21T07:50:19.079008+0000","last_active":"2024-09-21T07:50:19.084144+0000","last_peered":"2024-09-21T07:50:19.084144+0000","last_clean":"2024-09-21T07:50:19.084144+0000","last_became_active":"2024-09-21T07:50:19.076734+0000","last_became_peered":"2024-09-21T07:50:19.076734+0000","last_unstale":"2024-09-21T07:50:19.084144+0000","last_undegraded":"2024-09-21T07:50:19.084144+0000","last_fullsized":"2024-09-21T07:50:19.084144+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:45.401544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.069008+0000","last_change":"2024-09-21T07:50:18.064073+0000","last_active":"2024-09-21T07:50:19.069008+0000","last_peered":"2024-09-21T07:50:19.069008+0000","last_clean":"2024-09-21T07:50:19.069008+0000","last_became_active":"2024-09-21T07:50:18.063748+0000","last_became_peered":"2024-09-21T07:50:18.063748+0000","last_unstale":"2024-09-21T07:50:19.069008+0000","last_undegraded":"2024-09-21T07:50:19.069008+0000","last_fullsized":"2024-09-21T07:50:19.069008+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:16:34.797244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.081628+0000","last_change":"2024-09-21T07:50:19.077796+0000","last_active":"2024-09-21T07:50:19.081628+0000","last_peered":"2024-09-21T07:50:19.081628+0000","last_clean":"2024-09-21T07:50:19.081628+0000","last_became_active":"2024-09-21T07:50:19.077444+0000","last_became_peered":"2024-09-21T07:50:19.077444+0000","last_unstale":"2024-09-21T07:50:19.081628+0000","last_undegraded":"2024-09-21T07:50:19.081628+0000","last_fullsized":"2024-09-21T07:50:19.081628+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:46.990022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.065919+0000","last_change":"2024-09-21T07:50:18.066123+0000","last_active":"2024-09-21T07:50:19.065919+0000","last_peered":"2024-09-21T07:50:19.065919+0000","last_clean":"2024-09-21T07:50:19.065919+0000","last_became_active":"2024-09-21T07:50:18.065469+0000","last_became_peered":"2024-09-21T07:50:18.065469+0000","last_unstale":"2024-09-21T07:50:19.065919+0000","last_undegraded":"2024-09-21T07:50:19.065919+0000","last_fullsized":"2024-09-21T07:50:19.065919+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:54.340852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.072368+0000","last_change":"2024-09-21T07:50:19.066123+0000","last_active":"2024-09-21T07:50:19.072368+0000","last_peered":"2024-09-21T07:50:19.072368+0000","last_clean":"2024-09-21T07:50:19.072368+0000","last_became_active":"2024-09-21T07:50:19.064472+0000","last_became_peered":"2024-09-21T07:50:19.064472+0000","last_unstale":"2024-09-21T07:50:19.072368+0000","last_undegraded":"2024-09-21T07:50:19.072368+0000","last_fullsized":"2024-09-21T07:50:19.072368+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:36:36.600298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.053159+0000","last_change":"2024-09-21T07:50:18.061608+0000","last_active":"2024-09-21T07:50:19.053159+0000","last_peered":"2024-09-21T07:50:19.053159+0000","last_clean":"2024-09-21T07:50:19.053159+0000","last_became_active":"2024-09-21T07:50:18.060830+0000","last_became_peered":"2024-09-21T07:50:18.060830+0000","last_unstale":"2024-09-21T07:50:19.053159+0000","last_undegraded":"2024-09-21T07:50:19.053159+0000","last_fullsized":"2024-09-21T07:50:19.053159+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:58:36.360850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.083851+0000","last_change":"2024-09-21T07:50:19.079284+0000","last_active":"2024-09-21T07:50:19.083851+0000","last_peered":"2024-09-21T07:50:19.083851+0000","last_clean":"2024-09-21T07:50:19.083851+0000","last_became_active":"2024-09-21T07:50:19.077925+0000","last_became_peered":"2024-09-21T07:50:19.077925+0000","last_unstale":"2024-09-21T07:50:19.083851+0000","last_undegraded":"2024-09-21T07:50:19.083851+0000","last_fullsized":"2024-09-21T07:50:19.083851+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:36.377395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.7","version":"72'2","reported_seq":15,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.087959+0000","last_change":"2024-09-21T07:50:18.073593+0000","last_active":"2024-09-21T07:50:19.087959+0000","last_peered":"2024-09-21T07:50:19.087959+0000","last_clean":"2024-09-21T07:50:19.087959+0000","last_became_active":"2024-09-21T07:50:18.072486+0000","last_became_peered":"2024-09-21T07:50:18.072486+0000","last_unstale":"2024-09-21T07:50:19.087959+0000","last_undegraded":"2024-09-21T07:50:19.087959+0000","last_fullsized":"2024-09-21T07:50:19.087959+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:37.825778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051409+0000","last_change":"2024-09-21T07:50:19.080985+0000","last_active":"2024-09-21T07:50:20.051409+0000","last_peered":"2024-09-21T07:50:20.051409+0000","last_clean":"2024-09-21T07:50:20.051409+0000","last_became_active":"2024-09-21T07:50:19.080697+0000","last_became_peered":"2024-09-21T07:50:19.080697+0000","last_unstale":"2024-09-21T07:50:20.051409+0000","last_undegraded":"2024-09-21T07:50:20.051409+0000","last_fullsized":"2024-09-21T07:50:20.051409+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:20.725003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.057788+0000","last_change":"2024-09-21T07:50:18.070317+0000","last_active":"2024-09-21T07:50:19.057788+0000","last_peered":"2024-09-21T07:50:19.057788+0000","last_clean":"2024-09-21T07:50:19.057788+0000","last_became_active":"2024-09-21T07:50:18.069817+0000","last_became_peered":"2024-09-21T07:50:18.069817+0000","last_unstale":"2024-09-21T07:50:19.057788+0000","last_undegraded":"2024-09-21T07:50:19.057788+0000","last_fullsized":"2024-09-21T07:50:19.057788+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:54:57.683732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085552+0000","last_change":"2024-09-21T07:50:19.075367+0000","last_active":"2024-09-21T07:50:19.085552+0000","last_peered":"2024-09-21T07:50:19.085552+0000","last_clean":"2024-09-21T07:50:19.085552+0000","last_became_active":"2024-09-21T07:50:19.075061+0000","last_became_peered":"2024-09-21T07:50:19.075061+0000","last_unstale":"2024-09-21T07:50:19.085552+0000","last_undegraded":"2024-09-21T07:50:19.085552+0000","last_fullsized":"2024-09-21T07:50:19.085552+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:21:17.485867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.056338+0000","last_change":"2024-09-21T07:50:18.066377+0000","last_active":"2024-09-21T07:50:19.056338+0000","last_peered":"2024-09-21T07:50:19.056338+0000","last_clean":"2024-09-21T07:50:19.056338+0000","last_became_active":"2024-09-21T07:50:18.064476+0000","last_became_peered":"2024-09-21T07:50:18.064476+0000","last_unstale":"2024-09-21T07:50:19.056338+0000","last_undegraded":"2024-09-21T07:50:19.056338+0000","last_fullsized":"2024-09-21T07:50:19.056338+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:44:20.414897+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":13,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051322+0000","last_change":"2024-09-21T07:50:19.076524+0000","last_active":"2024-09-21T07:50:20.051322+0000","last_peered":"2024-09-21T07:50:20.051322+0000","last_clean":"2024-09-21T07:50:20.051322+0000","last_became_active":"2024-09-21T07:50:19.076227+0000","last_became_peered":"2024-09-21T07:50:19.076227+0000","last_unstale":"2024-09-21T07:50:20.051322+0000","last_undegraded":"2024-09-21T07:50:20.051322+0000","last_fullsized":"2024-09-21T07:50:20.051322+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:55.343894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.055646+0000","last_change":"2024-09-21T07:50:18.060400+0000","last_active":"2024-09-21T07:50:19.055646+0000","last_peered":"2024-09-21T07:50:19.055646+0000","last_clean":"2024-09-21T07:50:19.055646+0000","last_became_active":"2024-09-21T07:50:18.060072+0000","last_became_peered":"2024-09-21T07:50:18.060072+0000","last_unstale":"2024-09-21T07:50:19.055646+0000","last_undegraded":"2024-09-21T07:50:19.055646+0000","last_fullsized":"2024-09-21T07:50:19.055646+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:35:47.250673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.073534+0000","last_change":"2024-09-21T07:50:19.068280+0000","last_active":"2024-09-21T07:50:19.073534+0000","last_peered":"2024-09-21T07:50:19.073534+0000","last_clean":"2024-09-21T07:50:19.073534+0000","last_became_active":"2024-09-21T07:50:19.067952+0000","last_became_peered":"2024-09-21T07:50:19.067952+0000","last_unstale":"2024-09-21T07:50:19.073534+0000","last_undegraded":"2024-09-21T07:50:19.073534+0000","last_fullsized":"2024-09-21T07:50:19.073534+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:50.539444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.055000+0000","last_change":"2024-09-21T07:50:18.059750+0000","last_active":"2024-09-21T07:50:19.055000+0000","last_peered":"2024-09-21T07:50:19.055000+0000","last_clean":"2024-09-21T07:50:19.055000+0000","last_became_active":"2024-09-21T07:50:18.059129+0000","last_became_peered":"2024-09-21T07:50:18.059129+0000","last_unstale":"2024-09-21T07:50:19.055000+0000","last_undegraded":"2024-09-21T07:50:19.055000+0000","last_fullsized":"2024-09-21T07:50:19.055000+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:51:59.705236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.089307+0000","last_change":"2024-09-21T07:50:19.082778+0000","last_active":"2024-09-21T07:50:19.089307+0000","last_peered":"2024-09-21T07:50:19.089307+0000","last_clean":"2024-09-21T07:50:19.089307+0000","last_became_active":"2024-09-21T07:50:19.082115+0000","last_became_peered":"2024-09-21T07:50:19.082115+0000","last_unstale":"2024-09-21T07:50:19.089307+0000","last_undegraded":"2024-09-21T07:50:19.089307+0000","last_fullsized":"2024-09-21T07:50:19.089307+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:14:44.964526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.055533+0000","last_change":"2024-09-21T07:50:18.065924+0000","last_active":"2024-09-21T07:50:19.055533+0000","last_peered":"2024-09-21T07:50:19.055533+0000","last_clean":"2024-09-21T07:50:19.055533+0000","last_became_active":"2024-09-21T07:50:18.065586+0000","last_became_peered":"2024-09-21T07:50:18.065586+0000","last_unstale":"2024-09-21T07:50:19.055533+0000","last_undegraded":"2024-09-21T07:50:19.055533+0000","last_fullsized":"2024-09-21T07:50:19.055533+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:50.365660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.087164+0000","last_change":"2024-09-21T07:50:19.084053+0000","last_active":"2024-09-21T07:50:19.087164+0000","last_peered":"2024-09-21T07:50:19.087164+0000","last_clean":"2024-09-21T07:50:19.087164+0000","last_became_active":"2024-09-21T07:50:19.083630+0000","last_became_peered":"2024-09-21T07:50:19.083630+0000","last_unstale":"2024-09-21T07:50:19.087164+0000","last_undegraded":"2024-09-21T07:50:19.087164+0000","last_fullsized":"2024-09-21T07:50:19.087164+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:16:05.011541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.067034+0000","last_change":"2024-09-21T07:50:18.066456+0000","last_active":"2024-09-21T07:50:19.067034+0000","last_peered":"2024-09-21T07:50:19.067034+0000","last_clean":"2024-09-21T07:50:19.067034+0000","last_became_active":"2024-09-21T07:50:18.065846+0000","last_became_peered":"2024-09-21T07:50:18.065846+0000","last_unstale":"2024-09-21T07:50:19.067034+0000","last_undegraded":"2024-09-21T07:50:19.067034+0000","last_fullsized":"2024-09-21T07:50:19.067034+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:10:17.833026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.085966+0000","last_change":"2024-09-21T07:50:19.077017+0000","last_active":"2024-09-21T07:50:19.085966+0000","last_peered":"2024-09-21T07:50:19.085966+0000","last_clean":"2024-09-21T07:50:19.085966+0000","last_became_active":"2024-09-21T07:50:19.076697+0000","last_became_peered":"2024-09-21T07:50:19.076697+0000","last_unstale":"2024-09-21T07:50:19.085966+0000","last_undegraded":"2024-09-21T07:50:19.085966+0000","last_fullsized":"2024-09-21T07:50:19.085966+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:52.088889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":13,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.061213+0000","last_change":"2024-09-21T07:50:18.071782+0000","last_active":"2024-09-21T07:50:19.061213+0000","last_peered":"2024-09-21T07:50:19.061213+0000","last_clean":"2024-09-21T07:50:19.061213+0000","last_became_active":"2024-09-21T07:50:18.070867+0000","last_became_peered":"2024-09-21T07:50:18.070867+0000","last_unstale":"2024-09-21T07:50:19.061213+0000","last_undegraded":"2024-09-21T07:50:19.061213+0000","last_fullsized":"2024-09-21T07:50:19.061213+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:31.088571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":72,"state":"active+clean","last_fresh":"2024-09-21T07:50:19.062427+0000","last_change":"2024-09-21T07:50:19.059767+0000","last_active":"2024-09-21T07:50:19.062427+0000","last_peered":"2024-09-21T07:50:19.062427+0000","last_clean":"2024-09-21T07:50:19.062427+0000","last_became_active":"2024-09-21T07:50:19.059475+0000","last_became_peered":"2024-09-21T07:50:19.059475+0000","last_unstale":"2024-09-21T07:50:19.062427+0000","last_undegraded":"2024-09-21T07:50:19.062427+0000","last_fullsized":"2024-09-21T07:50:19.062427+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:18.036056+0000","last_clean_scrub_stamp":"2024-09-21T07:50:18.036056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:46.099340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.b","version":"72'2","reported_seq":17,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.051006+0000","last_change":"2024-09-21T07:50:18.059765+0000","last_active":"2024-09-21T07:50:20.051006+0000","last_peered":"2024-09-21T07:50:20.051006+0000","last_clean":"2024-09-21T07:50:20.051006+0000","last_became_active":"2024-09-21T07:50:18.059438+0000","last_became_peered":"2024-09-21T07:50:18.059438+0000","last_unstale":"2024-09-21T07:50:20.051006+0000","last_undegraded":"2024-09-21T07:50:20.051006+0000","last_fullsized":"2024-09-21T07:50:20.051006+0000","mapping_epoch":70,"log_start":"0'0","ondisk_log_start":"0'0","created":70,"last_epoch_clean":71,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:17.033014+0000","last_clean_scrub_stamp":"2024-09-21T07:50:17.033014+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:28.150183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"19'72","reported_seq":209,"reported_epoch":73,"state":"active+clean","last_fresh":"2024-09-21T07:50:20.050408+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:50:20.050408+0000","last_peered":"2024-09-21T07:50:20.050408+0000","last_clean":"2024-09-21T07:50:20.050408+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:50:20.050408+0000","last_undegraded":"2024-09-21T07:50:20.050408+0000","last_fullsized":"2024-09-21T07:50:20.050408+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":72,"log_dups_size":0,"ondisk_log_size":72,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":66,"num_read_kb":55,"num_write":101,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":16,"seq":68719476795,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":362785,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476795,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32040,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339800,"statfs":{"total":96636764160,"available":96603955200,"internally_reserved":0,"allocated":868352,"data_stored":363241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476792,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32632,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339208,"statfs":{"total":96636764160,"available":96603348992,"internally_reserved":0,"allocated":1470464,"data_stored":959327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476792,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32044,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339796,"statfs":{"total":96636764160,"available":96603951104,"internally_reserved":0,"allocated":868352,"data_stored":362729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509495,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":860,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":880640,"data_stored":364305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509496,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1462272,"data_stored":953117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509496,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32048,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339792,"statfs":{"total":96636764160,"available":96603947008,"internally_reserved":0,"allocated":876544,"data_stored":363353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":7,"up_from":14,"seq":60129542201,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32056,"kb_used_data":868,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339784,"statfs":{"total":96636764160,"available":96603938816,"internally_reserved":0,"allocated":888832,"data_stored":370571,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:50:23.975 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:50:23.976 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:50:24.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 -- 172.21.15.88:0/1873742614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2040af4e0 msgr2=0x7fd20415d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 --2- 172.21.15.88:0/1873742614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2040af4e0 0x7fd20415d790 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8005fd0 tx=0x7fd1f802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 -- 172.21.15.88:0/1873742614 shutdown_connections 2024-09-21T07:50:24.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 --2- 172.21.15.88:0/1873742614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2040cd240 0x7fd2040b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 --2- 172.21.15.88:0/1873742614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2040af4e0 0x7fd20415d790 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 -- 172.21.15.88:0/1873742614 >> 172.21.15.88:0/1873742614 conn(0x7fd2040b1dd0 msgr2=0x7fd2040b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:24.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 -- 172.21.15.88:0/1873742614 shutdown_connections 2024-09-21T07:50:24.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.240+0000 7fd20a400640 1 -- 172.21.15.88:0/1873742614 wait complete. 2024-09-21T07:50:24.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.242+0000 7fd20a400640 1 Processor -- start 2024-09-21T07:50:24.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.242+0000 7fd20a400640 1 -- start start 2024-09-21T07:50:24.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.243+0000 7fd20a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2040af4e0 0x7fd2041f8280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd20a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 0x7fd2041ea7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd20a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2041f8a10 0x7fd2041f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd20a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2040b4500 con 0x7fd204160370 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd20a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2041609d0 con 0x7fd2041f8a10 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd20a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2040b4680 con 0x7fd2040af4e0 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 0x7fd2041ea7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 0x7fd2041ea7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:48986/0 (socket says 172.21.15.88:48986) 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 -- 172.21.15.88:0/633975430 learned_addr learned my addr 172.21.15.88:0/633975430 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd209000640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2040af4e0 0x7fd2041f8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd209a00640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2041f8a10 0x7fd2041f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 -- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2040af4e0 msgr2=0x7fd2041f8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2040af4e0 0x7fd2041f8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 -- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2041f8a10 msgr2=0x7fd2041f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2041f8a10 0x7fd2041f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.244+0000 7fd203e00640 1 -- 172.21.15.88:0/633975430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2041f87c0 con 0x7fd204160370 2024-09-21T07:50:24.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.245+0000 7fd203e00640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 0x7fd2041ea7b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000c3d0 tx=0x7fd1f000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:24.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.245+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0017070 con 0x7fd204160370 2024-09-21T07:50:24.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.245+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd1f00053c0 con 0x7fd204160370 2024-09-21T07:50:24.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.245+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2041f9360 con 0x7fd204160370 2024-09-21T07:50:24.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.245+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1f0003850 con 0x7fd204160370 2024-09-21T07:50:24.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.245+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd20415f240 con 0x7fd204160370 2024-09-21T07:50:24.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.246+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2040b4b30 con 0x7fd204160370 2024-09-21T07:50:24.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.247+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fd1f001a070 con 0x7fd204160370 2024-09-21T07:50:24.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.248+0000 7fd200c00640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd1e4078f20 0x7fd1e407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.248+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fd1f009a670 con 0x7fd204160370 2024-09-21T07:50:24.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.249+0000 7fd209000640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd1e4078f20 0x7fd1e407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.250+0000 7fd209000640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd1e4078f20 0x7fd1e407b430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8033000 tx=0x7fd1f8004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:24.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.253+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2041f9360 con 0x7fd204160370 2024-09-21T07:50:24.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.511+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd2041eb400 con 0x7fd204160370 2024-09-21T07:50:24.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.514+0000 7fd200c00640 1 -- 172.21.15.88:0/633975430 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v14) ==== 190+0+147 (secure 0 0 0) 0x7fd2041f87c0 con 0x7fd204160370 2024-09-21T07:50:24.516 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:50:24.516 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:50:24.516 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:50:24.517 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:50:24.517 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:50:24.517 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:50:24.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd1e4078f20 msgr2=0x7fd1e407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd1e4078f20 0x7fd1e407b430 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fd1f8033000 tx=0x7fd1f8004730 comp rx=0 tx=0).stop 2024-09-21T07:50:24.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 msgr2=0x7fd2041ea7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 0x7fd2041ea7b0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7fd1f000c3d0 tx=0x7fd1f000c890 comp rx=0 tx=0).stop 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 shutdown_connections 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd1e4078f20 0x7fd1e407b430 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2041f8a10 0x7fd2041f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd204160370 0x7fd2041ea7b0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 --2- 172.21.15.88:0/633975430 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2040af4e0 0x7fd2041f8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 >> 172.21.15.88:0/633975430 conn(0x7fd2040b1dd0 msgr2=0x7fd20406fa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 shutdown_connections 2024-09-21T07:50:24.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.517+0000 7fd20a400640 1 -- 172.21.15.88:0/633975430 wait complete. 2024-09-21T07:50:24.541 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 -- 172.21.15.88:0/3482277981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7470003a50 msgr2=0x7f7470003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 --2- 172.21.15.88:0/3482277981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7470003a50 0x7f7470003ec0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f747c006fc0 tx=0x7f7470005830 comp rx=0 tx=0).stop 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 -- 172.21.15.88:0/3482277981 shutdown_connections 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 --2- 172.21.15.88:0/3482277981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7470003a50 0x7f7470003ec0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 -- 172.21.15.88:0/3482277981 >> 172.21.15.88:0/3482277981 conn(0x7f748c09aa50 msgr2=0x7f748c0b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 -- 172.21.15.88:0/3482277981 shutdown_connections 2024-09-21T07:50:24.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.760+0000 7f7491800640 1 -- 172.21.15.88:0/3482277981 wait complete. 2024-09-21T07:50:24.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.762+0000 7f7491800640 1 Processor -- start 2024-09-21T07:50:24.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.763+0000 7f7491800640 1 -- start start 2024-09-21T07:50:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f7491800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7470003a50 0x7f748c0d03a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f7491800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 0x7f748c15d100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f7491800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f748c151910 0x7f748c151d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f7491800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0d1290 con 0x7f748c15ccf0 2024-09-21T07:50:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f7491800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f748c0d1110 con 0x7f7470003a50 2024-09-21T07:50:24.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f7491800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f748c0d1410 con 0x7f748c151910 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f748b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 0x7f748c15d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f748b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 0x7f748c15d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49004/0 (socket says 172.21.15.88:49004) 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.764+0000 7f748b400640 1 -- 172.21.15.88:0/2780914338 learned_addr learned my addr 172.21.15.88:0/2780914338 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f7490e00640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f748c151910 0x7f748c151d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748be00640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7470003a50 0x7f748c0d03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748b400640 1 -- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f748c151910 msgr2=0x7f748c151d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748b400640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f748c151910 0x7f748c151d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748b400640 1 -- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7470003a50 msgr2=0x7f748c0d03a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748b400640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7470003a50 0x7f748c0d03a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748b400640 1 -- 172.21.15.88:0/2780914338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f748c15da00 con 0x7f748c15ccf0 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f7490e00640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f748c151910 0x7f748c151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:24.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f748b400640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 0x7f748c15d100 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7478003fb0 tx=0x7f74780016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:24.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478015070 con 0x7f748c15ccf0 2024-09-21T07:50:24.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7478011900 con 0x7f748c15ccf0 2024-09-21T07:50:24.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.765+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7478011d50 con 0x7f748c15ccf0 2024-09-21T07:50:24.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.766+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f748c15dce0 con 0x7f748c15ccf0 2024-09-21T07:50:24.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.766+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f748c15e2a0 con 0x7f748c15ccf0 2024-09-21T07:50:24.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.767+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f748c15f320 con 0x7f748c15ccf0 2024-09-21T07:50:24.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.768+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f7478018070 con 0x7f748c15ccf0 2024-09-21T07:50:24.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.769+0000 7f7483e00640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f746c078ef0 0x7f746c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:24.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.769+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f747809c0a0 con 0x7f748c15ccf0 2024-09-21T07:50:24.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.770+0000 7f748be00640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f746c078ef0 0x7f746c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:24.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.770+0000 7f748be00640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f746c078ef0 0x7f746c07b400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f747c033000 tx=0x7f747c000fe0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:24.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.773+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f74780638c0 con 0x7f748c15ccf0 2024-09-21T07:50:24.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.994+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f748c0d0ef0 con 0x7f748c15ccf0 2024-09-21T07:50:24.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.994+0000 7f7483e00640 1 -- 172.21.15.88:0/2780914338 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) ==== 94+0+5231 (secure 0 0 0) 0x7f74780682f0 con 0x7f748c15ccf0 2024-09-21T07:50:24.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:24.997 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":46,"btime":"2024-09-21T07:50:22:071724+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:24.997 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 46 2024-09-21T07:50:24.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f746c078ef0 msgr2=0x7f746c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f746c078ef0 0x7f746c07b400 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f747c033000 tx=0x7f747c000fe0 comp rx=0 tx=0).stop 2024-09-21T07:50:24.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 msgr2=0x7f748c15d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:24.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 0x7f748c15d100 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f7478003fb0 tx=0x7f74780016f0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 shutdown_connections 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f746c078ef0 0x7f746c07b400 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f748c151910 0x7f748c151d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f748c15ccf0 0x7f748c15d100 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 --2- 172.21.15.88:0/2780914338 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7470003a50 0x7f748c0d03a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 >> 172.21.15.88:0/2780914338 conn(0x7f748c09aa50 msgr2=0x7f748c071840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 shutdown_connections 2024-09-21T07:50:25.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:24.997+0000 7f7491800640 1 -- 172.21.15.88:0/2780914338 wait complete. 2024-09-21T07:50:25.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:50:18.548042+0000', 'modified': '2024-09-21T07:50:21.559601+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6186}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6186': {'gid': 6186, 'name': 'c', 'rank': 0, 'incarnation': 42, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/2332018233', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 2332018233}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 2332018233}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6186, 'qdb_cluster': [6186]} 2024-09-21T07:50:25.427 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:50:25.427 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:50:25.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 -- 172.21.15.88:0/1932115740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f90070cc0 msgr2=0x7f6f90156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 --2- 172.21.15.88:0/1932115740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f90070cc0 0x7f6f90156e40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6f900b4fa0 tx=0x7f6f7c02a540 comp rx=0 tx=0).stop 2024-09-21T07:50:25.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 -- 172.21.15.88:0/1932115740 shutdown_connections 2024-09-21T07:50:25.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 --2- 172.21.15.88:0/1932115740 >> v2:172.21.15.175:3300/0 conn(0x7f6f9015ce30 0x7f6f9015d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 --2- 172.21.15.88:0/1932115740 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f90070cc0 0x7f6f90156e40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 -- 172.21.15.88:0/1932115740 >> 172.21.15.88:0/1932115740 conn(0x7f6f9006d3d0 msgr2=0x7f6f900b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 -- 172.21.15.88:0/1932115740 shutdown_connections 2024-09-21T07:50:25.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.647+0000 7f6f97200640 1 -- 172.21.15.88:0/1932115740 wait complete. 2024-09-21T07:50:25.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.649+0000 7f6f97200640 1 Processor -- start 2024-09-21T07:50:25.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.649+0000 7f6f97200640 1 -- start start 2024-09-21T07:50:25.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f97200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 0x7f6f900c9d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f97200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9015ce30 0x7f6f900ae6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f97200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f900b3af0 0x7f6f900b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f97200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900cd050 con 0x7f6f900cfc80 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f97200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900cced0 con 0x7f6f900b3af0 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f97200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f900cd1d0 con 0x7f6f9015ce30 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 0x7f6f900c9d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 0x7f6f900c9d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49010/0 (socket says 172.21.15.88:49010) 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 -- 172.21.15.88:0/1518621234 learned_addr learned my addr 172.21.15.88:0/1518621234 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8f400640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9015ce30 0x7f6f900ae6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f95200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f900b3af0 0x7f6f900b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 -- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9015ce30 msgr2=0x7f6f900ae6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9015ce30 0x7f6f900ae6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 -- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f900b3af0 msgr2=0x7f6f900b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.651+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f900b3af0 0x7f6f900b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f8fe00640 1 -- 172.21.15.88:0/1518621234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f900ca260 con 0x7f6f900cfc80 2024-09-21T07:50:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 0x7f6f900c9d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000c760 tx=0x7f6f8000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f87e00640 1 -- 172.21.15.88:0/1518621234 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80003b70 con 0x7f6f900cfc80 2024-09-21T07:50:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f87e00640 1 -- 172.21.15.88:0/1518621234 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f80003d30 con 0x7f6f900cfc80 2024-09-21T07:50:25.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f87e00640 1 -- 172.21.15.88:0/1518621234 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8000aab0 con 0x7f6f900cfc80 2024-09-21T07:50:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f900b4440 con 0x7f6f900cfc80 2024-09-21T07:50:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.652+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f901f3b80 con 0x7f6f900cfc80 2024-09-21T07:50:25.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.655+0000 7f6f87e00640 1 -- 172.21.15.88:0/1518621234 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f6f8000ac50 con 0x7f6f900cfc80 2024-09-21T07:50:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.655+0000 7f6f87e00640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f70078f20 0x7f6f7007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.656+0000 7f6f87e00640 1 -- 172.21.15.88:0/1518621234 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f6f8009b2c0 con 0x7f6f900cfc80 2024-09-21T07:50:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.656+0000 7f6f9e2a0740 1 Processor -- start 2024-09-21T07:50:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.656+0000 7f6f9e2a0740 1 -- start start 2024-09-21T07:50:25.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.656+0000 7f6f8f400640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f70078f20 0x7f6f7007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.657+0000 7f6f8f400640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f70078f20 0x7f6f7007b430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c033000 tx=0x7f6f7c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda41d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda422c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda12370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed997430 con 0x55c5ed64c590 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed9875d0 con 0x55c5ed66d270 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55c5ed976a00 con 0x55c5ed6443b0 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda12370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda12370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49016/0 (socket says 172.21.15.88:49016) 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 -- 172.21.15.88:0/2495687033 learned_addr learned my addr 172.21.15.88:0/2495687033 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f8f400640 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda422c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda41d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 -- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 msgr2=0x55c5eda422c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda422c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 -- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 msgr2=0x55c5eda41d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda41d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.658+0000 7f6f95200640 1 -- 172.21.15.88:0/2495687033 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5ed93f520 con 0x55c5ed64c590 2024-09-21T07:50:25.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.659+0000 7f6f95200640 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda12370 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009ad0 tx=0x7f6f88009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e398ee4b0c4eb0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.659+0000 7f6f84c00640 1 -- 172.21.15.88:0/2495687033 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88011da0 con 0x55c5ed64c590 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.659+0000 7f6f84c00640 1 -- 172.21.15.88:0/2495687033 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f88012450 con 0x55c5ed64c590 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.659+0000 7f6f84c00640 1 -- 172.21.15.88:0/2495687033 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f88012840 con 0x55c5ed64c590 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/2495687033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 msgr2=0x55c5eda12370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda12370 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f6f88009ad0 tx=0x7f6f88009f90 comp rx=0 tx=0).stop 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/2495687033 shutdown_connections 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda422c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda12370 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/2495687033 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda41d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/2495687033 >> 172.21.15.88:0/2495687033 conn(0x55c5ed6613c0 msgr2=0x55c5eda17d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/2495687033 shutdown_connections 2024-09-21T07:50:25.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.660+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/2495687033 wait complete. 2024-09-21T07:50:25.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.661+0000 7f6f9e2a0740 1 -- start start 2024-09-21T07:50:25.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.661+0000 7f6f9e2a0740 1 Processor -- start 2024-09-21T07:50:25.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed6443b0 0x55c5eda6c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda384e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 0x55c5eda40300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed91d330 con 0x55c5ed66d270 2024-09-21T07:50:25.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5ed97a810 con 0x55c5ed6443b0 2024-09-21T07:50:25.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55c5ed6e7620 con 0x55c5ed64c590 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f9e2a0740 10 client.0 _ll_register_callbacks cb 0x55c5ed922420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 0x55c5eda40300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 0x55c5eda40300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49018/0 (socket says 172.21.15.88:49018) 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 -- 172.21.15.88:0/240256826 learned_addr learned my addr 172.21.15.88:0/240256826 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed6443b0 0x55c5eda6c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f8f400640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 -- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 msgr2=0x55c5eda384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 -- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed6443b0 msgr2=0x55c5eda6c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed6443b0 0x55c5eda6c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.663+0000 7f6f95200640 1 -- 172.21.15.88:0/240256826 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5ed6d8850 con 0x55c5ed66d270 2024-09-21T07:50:25.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f95200640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 0x55c5eda40300 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6f88018000 tx=0x7f6f88009890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cac1b304ba9c6d44 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f79600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f79600640 1 -- 172.21.15.88:0/240256826 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f8802a020 con 0x55c5ed66d270 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f79600640 1 -- 172.21.15.88:0/240256826 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f6f88012b90 con 0x55c5ed66d270 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f88012450 con 0x55c5ed66d270 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f79600640 1 -- 172.21.15.88:0/240256826 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6f88012d90 con 0x55c5ed66d270 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f9e2a0740 10 client.6315 fetch_fsmap learned FSMap version 46 2024-09-21T07:50:25.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.664+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55c5ed74f6b0 con 0x55c5ed66d270 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f79600640 1 -- 172.21.15.88:0/240256826 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f88012450 con 0x55c5ed66d270 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f79600640 1 -- 172.21.15.88:0/240256826 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f6f88037080 con 0x55c5ed66d270 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f9e2a0740 10 client.6315 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f9e2a0740 10 client.6315 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f9e2a0740 20 client.6315 populate_metadata read hostname 'smithi088' 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5ed62be40 0x55c5eda3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f9e2a0740 4 client.6315 mds_command: new command op to 6186 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:25.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 --> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c5ed989520 con 0x55c5ed62be40 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.665+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5ed62be40 0x55c5eda3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.666+0000 7f6f79600640 10 client.6315 ms_handle_connect on v2:172.21.15.88:6834/2332018233 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.666+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5ed62be40 0x55c5eda3afe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e2050d51eb6b9e6 server_cookie=612ea29bf4b23f60 in_seq=0 out_seq=0 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.666+0000 7f6f79600640 1 -- 172.21.15.88:0/240256826 <== mds.0 v2:172.21.15.88:6834/2332018233 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c5ed989520 con 0x55c5ed62be40 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.666+0000 7f6f79600640 10 client.6315 handle_command_reply: tid=0 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.666+0000 7f6f9e2a0740 1 client.6315 shutdown 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 msgr2=0x55c5eda40300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 0x55c5eda40300 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6f88018000 tx=0x7f6f88009890 comp rx=0 tx=0).stop 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 shutdown_connections 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5ed62be40 0x55c5eda3afe0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed66d270 0x55c5eda40300 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/240256826 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed6443b0 0x55c5eda6c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 >> 172.21.15.88:0/240256826 conn(0x55c5ed6613c0 msgr2=0x55c5eda60110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f79600640 0 client.6315 ms_handle_reset on v2:172.21.15.88:6834/2332018233 2024-09-21T07:50:25.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 shutdown_connections 2024-09-21T07:50:25.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/240256826 wait complete. 2024-09-21T07:50:25.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.667+0000 7f6f9e2a0740 20 client.6315 trim_cache size 0 max 16384 2024-09-21T07:50:25.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.686+0000 7f6f9e2a0740 1 Processor -- start 2024-09-21T07:50:25.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.686+0000 7f6f9e2a0740 1 -- start start 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.687+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda3f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.687+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda3afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.687+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda3b520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.687+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900cbf50 con 0x55c5ed64c590 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.687+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f900740d0 con 0x55c5ed66d270 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.687+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f90074720 con 0x55c5ed6443b0 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda3afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda3afe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49026/0 (socket says 172.21.15.88:49026) 2024-09-21T07:50:25.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 -- 172.21.15.88:0/1901927714 learned_addr learned my addr 172.21.15.88:0/1901927714 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda3f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f95200640 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda3b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 -- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 msgr2=0x55c5eda3f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda3f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 -- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 msgr2=0x55c5eda3b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda3b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 -- 172.21.15.88:0/1901927714 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c5eda18a60 con 0x55c5ed64c590 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.688+0000 7f6f8f400640 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda3afe0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02ab30 tx=0x7f6f7c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383b4e449a326a47 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f7be00640 1 -- 172.21.15.88:0/1901927714 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c04e070 con 0x55c5ed64c590 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f7be00640 1 -- 172.21.15.88:0/1901927714 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f7c0039a0 con 0x55c5ed64c590 2024-09-21T07:50:25.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f7be00640 1 -- 172.21.15.88:0/1901927714 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f7c035040 con 0x55c5ed64c590 2024-09-21T07:50:25.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/1901927714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 msgr2=0x55c5eda3afe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda3afe0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c02ab30 tx=0x7f6f7c03c000 comp rx=0 tx=0).stop 2024-09-21T07:50:25.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/1901927714 shutdown_connections 2024-09-21T07:50:25.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda3b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed64c590 0x55c5eda3afe0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/1901927714 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed6443b0 0x55c5eda3f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.689+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/1901927714 >> 172.21.15.88:0/1901927714 conn(0x55c5ed6613c0 msgr2=0x55c5eda382d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.690+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/1901927714 shutdown_connections 2024-09-21T07:50:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.690+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/1901927714 wait complete. 2024-09-21T07:50:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.690+0000 7f6f9e2a0740 1 -- start start 2024-09-21T07:50:25.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.691+0000 7f6f9e2a0740 1 Processor -- start 2024-09-21T07:50:25.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 0x55c5eda6b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda3a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda600d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c5eda38a20 con 0x55c5ed6443b0 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55c5eda576d0 con 0x55c5ed66d270 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55c5eda6f830 con 0x55c5ed64c590 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f9e2a0740 10 client.0 _ll_register_callbacks cb 0x7f6f88011eb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 0x55c5eda6b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f8fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 0x55c5eda6b9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49038/0 (socket says 172.21.15.88:49038) 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f8fe00640 1 -- 172.21.15.88:0/182498907 learned_addr learned my addr 172.21.15.88:0/182498907 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f95200640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda600d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.692+0000 7f6f8f400640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda3a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f8fe00640 1 -- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 msgr2=0x55c5eda3a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda3a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f8fe00640 1 -- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 msgr2=0x55c5eda600d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda600d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f8fe00640 1 -- 172.21.15.88:0/182498907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c5eda7e950 con 0x55c5ed6443b0 2024-09-21T07:50:25.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f8fe00640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 0x55c5eda6b9e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6f8005fca0 tx=0x7f6f8009b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=404934420ae8299e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f79600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f79600640 1 -- 172.21.15.88:0/182498907 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f80063130 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f79600640 1 -- 172.21.15.88:0/182498907 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f6f80067130 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f7c0039a0 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f79600640 1 -- 172.21.15.88:0/182498907 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) ==== 24+0+0 (secure 0 0 0) 0x7f6f80062a30 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f9e2a0740 10 client.6321 fetch_fsmap learned FSMap version 46 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.693+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) -- 0x55c5edadd100 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f79600640 1 -- 172.21.15.88:0/182498907 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f7c0039a0 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f79600640 1 -- 172.21.15.88:0/182498907 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 46) ==== 3502+0+0 (secure 0 0 0) 0x7f6f8006bf30 con 0x55c5ed6443b0 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f9e2a0740 10 client.6321 fetch_fsmap finished waiting for FSMap version 46 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f9e2a0740 10 client.6321 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f9e2a0740 20 client.6321 populate_metadata read hostname 'smithi088' 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5edae0600 0x55c5eda12320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f9e2a0740 4 client.6321 mds_command: new command op to 6186 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 --> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] -- command(tid 0: {"prefix": "status"}) -- 0x55c5ed984fd0 con 0x55c5edae0600 2024-09-21T07:50:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.694+0000 7f6f8f400640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5edae0600 0x55c5eda12320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f8f400640 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5edae0600 0x55c5eda12320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f107b64f9e892ada server_cookie=fe2a6dc52e73a270 in_seq=0 out_seq=0 2024-09-21T07:50:25.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f79600640 10 client.6321 ms_handle_connect on v2:172.21.15.88:6834/2332018233 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f79600640 1 -- 172.21.15.88:0/182498907 <== mds.0 v2:172.21.15.88:6834/2332018233 1 ==== command_reply(tid 0: 0 ) ==== 8+0+324 (crc 0 0 0) 0x55c5ed984fd0 con 0x55c5edae0600 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f79600640 10 client.6321 handle_command_reply: tid=0 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 client.6321 shutdown 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 msgr2=0x55c5eda6b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 0x55c5eda6b9e0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6f8005fca0 tx=0x7f6f8009b040 comp rx=0 tx=0).stop 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 shutdown_connections 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:6834/2332018233,v1:172.21.15.88:6835/2332018233] conn(0x55c5edae0600 0x55c5eda12320 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55c5ed64c590 0x55c5eda3a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55c5ed66d270 0x55c5eda600d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f79600640 0 client.6321 ms_handle_reset on v2:172.21.15.88:6834/2332018233 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 --2- 172.21.15.88:0/182498907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c5ed6443b0 0x55c5eda6b9e0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 >> 172.21.15.88:0/182498907 conn(0x55c5ed6613c0 msgr2=0x55c5eda40040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 shutdown_connections 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 1 -- 172.21.15.88:0/182498907 wait complete. 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.695+0000 7f6f9e2a0740 20 client.6321 trim_cache size 0 max 16384 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:50:25.701 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6186, 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6495953270000001, 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 46, 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 73, 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 73, 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.71868175 2024-09-21T07:50:25.702 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f70078f20 msgr2=0x7f6f7007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f70078f20 0x7f6f7007b430 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f6f7c033000 tx=0x7f6f7c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 msgr2=0x7f6f900c9d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 0x7f6f900c9d20 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f6f8000c760 tx=0x7f6f8000cc20 comp rx=0 tx=0).stop 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 shutdown_connections 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6f70078f20 0x7f6f7007b430 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f9015ce30 0x7f6f900ae6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f900b3af0 0x7f6f900b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 --2- 172.21.15.88:0/1518621234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f900cfc80 0x7f6f900c9d20 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 >> 172.21.15.88:0/1518621234 conn(0x7f6f9006d3d0 msgr2=0x7f6f901603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 shutdown_connections 2024-09-21T07:50:25.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.696+0000 7f6f97200640 1 -- 172.21.15.88:0/1518621234 wait complete. 2024-09-21T07:50:25.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:50:25.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 -- 172.21.15.88:0/618957285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d0415e9a0 msgr2=0x7f6d040d08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 --2- 172.21.15.88:0/618957285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d0415e9a0 0x7f6d040d08c0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc005fd0 tx=0x7f6cfc02a820 comp rx=0 tx=0).stop 2024-09-21T07:50:25.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 -- 172.21.15.88:0/618957285 shutdown_connections 2024-09-21T07:50:25.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 --2- 172.21.15.88:0/618957285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d0415e9a0 0x7f6d040d08c0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 -- 172.21.15.88:0/618957285 >> 172.21.15.88:0/618957285 conn(0x7f6d04071070 msgr2=0x7f6d04071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:25.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 -- 172.21.15.88:0/618957285 shutdown_connections 2024-09-21T07:50:25.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.939+0000 7f6d03200640 1 -- 172.21.15.88:0/618957285 wait complete. 2024-09-21T07:50:25.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.941+0000 7f6d03200640 1 Processor -- start 2024-09-21T07:50:25.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.941+0000 7f6d03200640 1 -- start start 2024-09-21T07:50:25.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d03200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d040d0350 0x7f6d040c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d03200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d0415e9a0 0x7f6d0415ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d03200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 0x7f6d0415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d03200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d040cdc60 con 0x7f6d04159fc0 2024-09-21T07:50:25.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d03200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6d040cdae0 con 0x7f6d040d0350 2024-09-21T07:50:25.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d03200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6d040cdde0 con 0x7f6d0415e9a0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 0x7f6d0415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 0x7f6d0415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55820/0 (socket says 172.21.15.88:55820) 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 -- 172.21.15.88:0/528661861 learned_addr learned my addr 172.21.15.88:0/528661861 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6cf9400640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d0415e9a0 0x7f6d0415ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d01e00640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d040d0350 0x7f6d040c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 -- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d0415e9a0 msgr2=0x7f6d0415ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d0415e9a0 0x7f6d0415ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 -- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d040d0350 msgr2=0x7f6d040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d040d0350 0x7f6d040c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.943+0000 7f6d02800640 1 -- 172.21.15.88:0/528661861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d041582f0 con 0x7f6d04159fc0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.944+0000 7f6d02800640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 0x7f6d0415b1c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc006010 tx=0x7f6cfc032c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.944+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc037040 con 0x7f6d04159fc0 2024-09-21T07:50:25.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.944+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cfc042070 con 0x7f6d04159fc0 2024-09-21T07:50:25.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.944+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d041585d0 con 0x7f6d04159fc0 2024-09-21T07:50:25.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.944+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc0042a0 con 0x7f6d04159fc0 2024-09-21T07:50:25.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.944+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d04155da0 con 0x7f6d04159fc0 2024-09-21T07:50:25.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.945+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d040d08c0 con 0x7f6d04159fc0 2024-09-21T07:50:25.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.946+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f6cfc004440 con 0x7f6d04159fc0 2024-09-21T07:50:25.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.947+0000 7f6cfaa00640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ce0078f50 0x7f6ce007b460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:25.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.947+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f6cfc0bbf50 con 0x7f6d04159fc0 2024-09-21T07:50:25.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.948+0000 7f6d01e00640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ce0078f50 0x7f6ce007b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:25.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.948+0000 7f6d01e00640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ce0078f50 0x7f6ce007b460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006fd0 tx=0x7f6cf4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:25.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:25.952+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cfc083f50 con 0x7f6d04159fc0 2024-09-21T07:50:26.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.180+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f6d0406e2f0 con 0x7f6d04159fc0 2024-09-21T07:50:26.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.181+0000 7f6cfaa00640 1 -- 172.21.15.88:0/528661861 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7f6cfc083850 con 0x7f6d04159fc0 2024-09-21T07:50:26.183 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:50:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.183+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ce0078f50 msgr2=0x7f6ce007b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.183+0000 7f6d03200640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ce0078f50 0x7f6ce007b460 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4006fd0 tx=0x7f6cf4008040 comp rx=0 tx=0).stop 2024-09-21T07:50:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.183+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 msgr2=0x7f6d0415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.183+0000 7f6d03200640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 0x7f6d0415b1c0 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc006010 tx=0x7f6cfc032c90 comp rx=0 tx=0).stop 2024-09-21T07:50:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 shutdown_connections 2024-09-21T07:50:26.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f6ce0078f50 0x7f6ce007b460 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:26.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6d0415e9a0 0x7f6d0415ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:26.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d04159fc0 0x7f6d0415b1c0 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:26.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 --2- 172.21.15.88:0/528661861 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6d040d0350 0x7f6d040c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:26.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 >> 172.21.15.88:0/528661861 conn(0x7f6d04071070 msgr2=0x7f6d040725e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:26.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 shutdown_connections 2024-09-21T07:50:26.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26.184+0000 7f6d03200640 1 -- 172.21.15.88:0/528661861 wait complete. 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:50:26.208 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:50:26.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:26.209 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:50:26.244 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:50:26.244 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:50:26.245 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42468sec preferred_lft 42468sec 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.246 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:50:26.247 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:50:26.247 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:50:26.247 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:26.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:26.247 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:50:26.338 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:50:26.340 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:26.340 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:50:26.377 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:50:26.377 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:50:26.377 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:50:26.377 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:50:26.377 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:50:26.377 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:50:26.378 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:50:26.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:50:26.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:50:26.378 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:50:26.378 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:50:26.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:50:26.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:26 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:50:26.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:26.572 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:50:26.607 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:50:26.607 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:50:26.608 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:50:26.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:26.608 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:50:26.656 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:26.698 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:26.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:50:26.704 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:50:26.780 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:50:26.780 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:50:26.780 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:50:26.870 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:50:26.871 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:26.871 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:50:27.065 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:50:27.065 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:50:27.065 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:50:27.087 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:50:27.087 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:50:27.087 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:50:27.087 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:50:27.087 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:50:27.126 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:50:27.126 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:50:27.126 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:50:27.126 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:50:27.126 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:50:27.127 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:50:27.127 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:50:27.127 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:50:27.128 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:50:27.128 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:50:27.128 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:50:27.128 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:50:27.128 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:50:27.131 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:50:27.131 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:27.131 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:50:27.187 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:50:27.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:50:27.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:50:27.282 INFO:teuthology.orchestra.run.smithi088.stdout:client5654 2024-09-21T07:50:27.282 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:27.282 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:27.318 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.576+0000 7fc190a00640 1 -- 172.21.15.88:0/2668376479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c15d760 msgr2=0x7fc18c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.576+0000 7fc190a00640 1 --2- 172.21.15.88:0/2668376479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c15d760 0x7fc18c1535c0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7fc17c005fd0 tx=0x7fc17c02a540 comp rx=0 tx=0).stop 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.576+0000 7fc190a00640 1 -- 172.21.15.88:0/2668376479 shutdown_connections 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.576+0000 7fc190a00640 1 --2- 172.21.15.88:0/2668376479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc18c0ad0e0 0x7fc18c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.576+0000 7fc190a00640 1 --2- 172.21.15.88:0/2668376479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c15d760 0x7fc18c1535c0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.576+0000 7fc190a00640 1 -- 172.21.15.88:0/2668376479 >> 172.21.15.88:0/2668376479 conn(0x7fc18c06d730 msgr2=0x7fc18c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:27.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.577+0000 7fc190a00640 1 -- 172.21.15.88:0/2668376479 shutdown_connections 2024-09-21T07:50:27.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.577+0000 7fc190a00640 1 -- 172.21.15.88:0/2668376479 wait complete. 2024-09-21T07:50:27.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.579+0000 7fc190a00640 1 Processor -- start 2024-09-21T07:50:27.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.579+0000 7fc190a00640 1 -- start start 2024-09-21T07:50:27.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc190a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 0x7fc18c14e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc190a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc18c15d760 0x7fc18c1f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc190a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc18c160310 0x7fc18c1f9390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc190a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c06f220 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc190a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc18c06f0a0 con 0x7fc18c15d760 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc190a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc18c0b4c20 con 0x7fc18c160310 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc189e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 0x7fc18c14e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc189e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 0x7fc18c14e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55832/0 (socket says 172.21.15.88:55832) 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.580+0000 7fc189e00640 1 -- 172.21.15.88:0/1232588633 learned_addr learned my addr 172.21.15.88:0/1232588633 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc18a800640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc18c160310 0x7fc18c1f9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189400640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc18c15d760 0x7fc18c1f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189e00640 1 -- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc18c160310 msgr2=0x7fc18c1f9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189e00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc18c160310 0x7fc18c1f9390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189e00640 1 -- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc18c15d760 msgr2=0x7fc18c1f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189e00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc18c15d760 0x7fc18c1f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189e00640 1 -- 172.21.15.88:0/1232588633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc18c14f180 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc189e00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 0x7fc18c14e090 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc17c034040 tx=0x7fc17c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.581+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c03e030 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.582+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc17c003d90 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.582+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc17c035040 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.582+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc18c14e8e0 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.582+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc18c14eea0 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.583+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc18c160fa0 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.584+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fc17c041030 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.585+0000 7fc182000640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc16c078ef0 0x7fc16c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.585+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fc17c0bba90 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.585+0000 7fc189400640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc16c078ef0 0x7fc16c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:27.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.586+0000 7fc189400640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc16c078ef0 0x7fc16c07b400 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc18c15d3c0 tx=0x7fc178005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:27.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.589+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc17c083a00 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.811+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc18c1609d0 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.812+0000 7fc182000640 1 -- 172.21.15.88:0/1232588633 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) ==== 53+0+73 (secure 0 0 0) 0x7fc17c083300 con 0x7fc18c0ad0e0 2024-09-21T07:50:27.814 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.814+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc16c078ef0 msgr2=0x7fc16c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.814+0000 7fc190a00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc16c078ef0 0x7fc16c07b400 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fc18c15d3c0 tx=0x7fc178005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.814+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 msgr2=0x7fc18c14e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.814+0000 7fc190a00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 0x7fc18c14e090 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fc17c034040 tx=0x7fc17c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 shutdown_connections 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fc16c078ef0 0x7fc16c07b400 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc18c160310 0x7fc18c1f9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc18c15d760 0x7fc18c1f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 --2- 172.21.15.88:0/1232588633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc18c0ad0e0 0x7fc18c14e090 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 >> 172.21.15.88:0/1232588633 conn(0x7fc18c06d730 msgr2=0x7fc18c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 shutdown_connections 2024-09-21T07:50:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:27.815+0000 7fc190a00640 1 -- 172.21.15.88:0/1232588633 wait complete. 2024-09-21T07:50:27.840 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:50:27.840 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:50:27.840 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:50:27.841 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:50:27.841 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:50:27.841 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:50:27.841 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:50:27.841 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:50:27.841 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:50:27.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:27.841 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:50:27.884 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42417sec preferred_lft 42417sec 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:50:27.885 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:50:27.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:27.887 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:50:27.994 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:50:27.997 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:27.997 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:50:28.036 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:50:28.036 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:50:28.036 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:50:28.036 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:50:28.036 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:50:28.037 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:50:28.037 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:50:28.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:50:28.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:50:28.037 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:50:28.037 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:50:28.160 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:50:28 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:50:28.230 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:50:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:50:28.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:28.238 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:50:28.275 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:50:28.275 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:50:28.275 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:50:28.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:28.275 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:50:28.371 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:50:28.372 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:50:28.406 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:50:28.406 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:50:28.407 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:50:28.481 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:50:28.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:28.481 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:50:28.674 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:50:28.674 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:50:28.674 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:50:28.694 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:50:28.694 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:50:28.694 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:50:28.694 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:50:28.694 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:50:28.723 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:50:28.724 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:50:28.725 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:50:28.725 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:50:28.725 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:50:28.725 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:50:28.728 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:50:28.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:28.728 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:50:28.783 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:50:28.861 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:50:28 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:50:28.865 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:50:28 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:50:28.873 INFO:teuthology.orchestra.run.smithi175.stdout:client6133 2024-09-21T07:50:28.873 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:28.873 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:28.908 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:50:29.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.140+0000 7f53ac000640 1 -- 172.21.15.88:0/3238202236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a40af100 msgr2=0x7f53a40af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.140+0000 7f53ac000640 1 --2- 172.21.15.88:0/3238202236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a40af100 0x7f53a40af510 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f53a40d0d00 tx=0x7f539c02a540 comp rx=0 tx=0).stop 2024-09-21T07:50:29.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.140+0000 7f53ac000640 1 -- 172.21.15.88:0/3238202236 shutdown_connections 2024-09-21T07:50:29.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.140+0000 7f53ac000640 1 --2- 172.21.15.88:0/3238202236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a40af100 0x7f53a40af510 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.141+0000 7f53ac000640 1 --2- 172.21.15.88:0/3238202236 >> v2:172.21.15.175:3300/0 conn(0x7f53a406d4b0 0x7f53a40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.141+0000 7f53ac000640 1 -- 172.21.15.88:0/3238202236 >> 172.21.15.88:0/3238202236 conn(0x7f53a40b2540 msgr2=0x7f53a40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.141+0000 7f53ac000640 1 -- 172.21.15.88:0/3238202236 shutdown_connections 2024-09-21T07:50:29.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.141+0000 7f53ac000640 1 -- 172.21.15.88:0/3238202236 wait complete. 2024-09-21T07:50:29.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.143+0000 7f53ac000640 1 Processor -- start 2024-09-21T07:50:29.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.143+0000 7f53ac000640 1 -- start start 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.144+0000 7f53ac000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53a40af100 0x7f53a41f5250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.144+0000 7f53ac000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53a40cc830 0x7f53a415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.144+0000 7f53ac000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 0x7f53a415bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.144+0000 7f53ac000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40b3620 con 0x7f53a415b940 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.144+0000 7f53ac000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f53a40b34a0 con 0x7f53a40cc830 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.144+0000 7f53ac000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f53a40b37a0 con 0x7f53a40af100 2024-09-21T07:50:29.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 0x7f53a415bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 0x7f53a415bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55846/0 (socket says 172.21.15.88:55846) 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 -- 172.21.15.88:0/1993483707 learned_addr learned my addr 172.21.15.88:0/1993483707 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53a8c00640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53a40cc830 0x7f53a415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53a9600640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53a40af100 0x7f53a41f5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 -- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53a40af100 msgr2=0x7f53a41f5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53a40af100 0x7f53a41f5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 -- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53a40cc830 msgr2=0x7f53a415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53a40cc830 0x7f53a415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 -- 172.21.15.88:0/1993483707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53a415c290 con 0x7f53a415b940 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.145+0000 7f53aa000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 0x7f53a415bd50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f539c0098f0 tx=0x7f539c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.146+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c005c90 con 0x7f53a415b940 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.146+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f539c031e50 con 0x7f53a415b940 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.146+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f539c005240 con 0x7f53a415b940 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.146+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53a41f5790 con 0x7f53a415b940 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.146+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53a415fc10 con 0x7f53a415b940 2024-09-21T07:50:29.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.147+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53a415f6a0 con 0x7f53a415b940 2024-09-21T07:50:29.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.148+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f539c0053e0 con 0x7f53a415b940 2024-09-21T07:50:29.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.149+0000 7f53a1600640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5384078ef0 0x7f538407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.149+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f539c0bc580 con 0x7f53a415b940 2024-09-21T07:50:29.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.149+0000 7f53a9600640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5384078ef0 0x7f538407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.150+0000 7f53a9600640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5384078ef0 0x7f538407b400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53980020f0 tx=0x7f5398009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.154+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f53a415c290 con 0x7f53a415b940 2024-09-21T07:50:29.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.377+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f53a40b3cc0 con 0x7f53a415b940 2024-09-21T07:50:29.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.430+0000 7f53a1600640 1 -- 172.21.15.88:0/1993483707 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v47) ==== 136+0+0 (secure 0 0 0) 0x7f539c03f030 con 0x7f53a415b940 2024-09-21T07:50:29.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.432+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5384078ef0 msgr2=0x7f538407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.432+0000 7f53ac000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5384078ef0 0x7f538407b400 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f53980020f0 tx=0x7f5398009040 comp rx=0 tx=0).stop 2024-09-21T07:50:29.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.432+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 msgr2=0x7f53a415bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.432+0000 7f53ac000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 0x7f53a415bd50 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f539c0098f0 tx=0x7f539c033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:29.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 shutdown_connections 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f5384078ef0 0x7f538407b400 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53a415b940 0x7f53a415bd50 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f53a40cc830 0x7f53a415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 --2- 172.21.15.88:0/1993483707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f53a40af100 0x7f53a41f5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 >> 172.21.15.88:0/1993483707 conn(0x7f53a40b2540 msgr2=0x7f53a40997b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 shutdown_connections 2024-09-21T07:50:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.433+0000 7f53ac000640 1 -- 172.21.15.88:0/1993483707 wait complete. 2024-09-21T07:50:29.457 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:50:29.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.675+0000 7f72aae00640 1 -- 172.21.15.88:0/548593552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a40705b0 msgr2=0x7f72a40709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.675+0000 7f72aae00640 1 --2- 172.21.15.88:0/548593552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a40705b0 0x7f72a40709c0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f728c009830 tx=0x7f728c02a480 comp rx=0 tx=0).stop 2024-09-21T07:50:29.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.676+0000 7f72aae00640 1 -- 172.21.15.88:0/548593552 shutdown_connections 2024-09-21T07:50:29.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.676+0000 7f72aae00640 1 --2- 172.21.15.88:0/548593552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a40705b0 0x7f72a40709c0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.676+0000 7f72aae00640 1 --2- 172.21.15.88:0/548593552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a40cfc80 0x7f72a4156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.676+0000 7f72aae00640 1 -- 172.21.15.88:0/548593552 >> 172.21.15.88:0/548593552 conn(0x7f72a406d3d0 msgr2=0x7f72a40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.676+0000 7f72aae00640 1 -- 172.21.15.88:0/548593552 shutdown_connections 2024-09-21T07:50:29.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.676+0000 7f72aae00640 1 -- 172.21.15.88:0/548593552 wait complete. 2024-09-21T07:50:29.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.678+0000 7f72aae00640 1 Processor -- start 2024-09-21T07:50:29.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.678+0000 7f72aae00640 1 -- start start 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.679+0000 7f72aae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a40705b0 0x7f72a4074280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.679+0000 7f72aae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a40cfc80 0x7f72a40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.679+0000 7f72aae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 0x7f72a40b9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.679+0000 7f72aae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f72a40cced0 con 0x7f72a415ce30 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.679+0000 7f72aae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f72a40d0170 con 0x7f72a40cfc80 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.679+0000 7f72aae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f72a40cd050 con 0x7f72a40705b0 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 0x7f72a40b9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a40705b0 0x7f72a4074280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 0x7f72a40b9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55870/0 (socket says 172.21.15.88:55870) 2024-09-21T07:50:29.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 -- 172.21.15.88:0/74309817 learned_addr learned my addr 172.21.15.88:0/74309817 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:29.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f729b400640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a40cfc80 0x7f72a40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 -- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a40705b0 msgr2=0x7f72a4074280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a40705b0 0x7f72a4074280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 -- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a40cfc80 msgr2=0x7f72a40b8b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a40cfc80 0x7f72a40b8b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.680+0000 7f72a8e00640 1 -- 172.21.15.88:0/74309817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72a41f58a0 con 0x7f72a415ce30 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.681+0000 7f72a8e00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 0x7f72a40b9070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f729c00cb10 tx=0x7f729c00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.681+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c016020 con 0x7f72a415ce30 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.681+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f729c00ad00 con 0x7f72a415ce30 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.681+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72a41f9310 con 0x7f72a415ce30 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.681+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f729c012680 con 0x7f72a415ce30 2024-09-21T07:50:29.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.681+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72a41f98d0 con 0x7f72a415ce30 2024-09-21T07:50:29.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.682+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f72a40cd570 con 0x7f72a415ce30 2024-09-21T07:50:29.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.683+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f729c019020 con 0x7f72a415ce30 2024-09-21T07:50:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.684+0000 7f72a0c00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7280078ef0 0x7f728007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:29.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.684+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f729c09b790 con 0x7f72a415ce30 2024-09-21T07:50:29.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.685+0000 7f72a3e00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7280078ef0 0x7f728007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:29.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.686+0000 7f72a3e00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7280078ef0 0x7f728007b400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f72a409e510 tx=0x7f7294005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:29.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.689+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f729c020020 con 0x7f72a415ce30 2024-09-21T07:50:29.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.900+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f72a41f5a70 con 0x7f7280078ef0 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.903+0000 7f72a0c00640 1 -- 172.21.15.88:0/74309817 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f72a41f5a70 con 0x7f7280078ef0 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:29.905 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7280078ef0 msgr2=0x7f728007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7280078ef0 0x7f728007b400 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f72a409e510 tx=0x7f7294005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 msgr2=0x7f72a40b9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:29.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 0x7f72a40b9070 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f729c00cb10 tx=0x7f729c00a060 comp rx=0 tx=0).stop 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 shutdown_connections 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f7280078ef0 0x7f728007b400 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f72a415ce30 0x7f72a40b9070 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f72a40cfc80 0x7f72a40b8b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 --2- 172.21.15.88:0/74309817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f72a40705b0 0x7f72a4074280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 >> 172.21.15.88:0/74309817 conn(0x7f72a406d3d0 msgr2=0x7f72a415e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 shutdown_connections 2024-09-21T07:50:29.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:29.905+0000 7f72aae00640 1 -- 172.21.15.88:0/74309817 wait complete. 2024-09-21T07:50:29.933 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 -- 172.21.15.88:0/2453950116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140cd1d0 msgr2=0x7f2b140af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 --2- 172.21.15.88:0/2453950116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140cd1d0 0x7f2b140af0d0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f2b0c005fd0 tx=0x7f2b0c02aaa0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 -- 172.21.15.88:0/2453950116 shutdown_connections 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 --2- 172.21.15.88:0/2453950116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140cd1d0 0x7f2b140af0d0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 --2- 172.21.15.88:0/2453950116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b140b3f30 0x7f2b14156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 -- 172.21.15.88:0/2453950116 >> 172.21.15.88:0/2453950116 conn(0x7f2b1406d620 msgr2=0x7f2b140b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.154+0000 7f2b13200640 1 -- 172.21.15.88:0/2453950116 shutdown_connections 2024-09-21T07:50:30.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.155+0000 7f2b13200640 1 -- 172.21.15.88:0/2453950116 wait complete. 2024-09-21T07:50:30.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.157+0000 7f2b13200640 1 Processor -- start 2024-09-21T07:50:30.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.157+0000 7f2b13200640 1 -- start start 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b13200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 0x7f2b141550b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b13200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b140cd1d0 0x7f2b141f5120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b13200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b1415d6e0 0x7f2b141f4260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b13200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2b140af9a0 con 0x7f2b140b3f30 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b13200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b140af820 con 0x7f2b140cd1d0 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b13200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b140afb20 con 0x7f2b1415d6e0 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 0x7f2b141550b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b11e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 0x7f2b141550b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55896/0 (socket says 172.21.15.88:55896) 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.158+0000 7f2b11e00640 1 -- 172.21.15.88:0/2551538261 learned_addr learned my addr 172.21.15.88:0/2551538261 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11400640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b140cd1d0 0x7f2b141f5120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b12800640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b1415d6e0 0x7f2b141f4260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11e00640 1 -- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b1415d6e0 msgr2=0x7f2b141f4260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11e00640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b1415d6e0 0x7f2b141f4260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11e00640 1 -- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b140cd1d0 msgr2=0x7f2b141f5120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11e00640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b140cd1d0 0x7f2b141f5120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11e00640 1 -- 172.21.15.88:0/2551538261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b141f5660 con 0x7f2b140b3f30 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b11e00640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 0x7f2b141550b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400d6d0 tx=0x7f2b0400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b04017070 con 0x7f2b140b3f30 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b04005330 con 0x7f2b140b3f30 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b1415c670 con 0x7f2b140b3f30 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.159+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b040097c0 con 0x7f2b140b3f30 2024-09-21T07:50:30.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.160+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b140d0c70 con 0x7f2b140b3f30 2024-09-21T07:50:30.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.161+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b141f5830 con 0x7f2b140b3f30 2024-09-21T07:50:30.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.162+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f2b04009b70 con 0x7f2b140b3f30 2024-09-21T07:50:30.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.163+0000 7f2b0a000640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2af4078e90 0x7f2af407b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.163+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f2b0409b6d0 con 0x7f2b140b3f30 2024-09-21T07:50:30.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.163+0000 7f2b11400640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2af4078e90 0x7f2af407b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.164+0000 7f2b11400640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2af4078e90 0x7f2af407b3a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2b1415fe90 tx=0x7f2b00009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.167+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b04062f10 con 0x7f2b140b3f30 2024-09-21T07:50:30.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.353+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2b14156ca0 con 0x7f2b140b3f30 2024-09-21T07:50:30.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.353+0000 7f2b0a000640 1 -- 172.21.15.88:0/2551538261 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v38)=0 v38) ==== 122+0+0 (secure 0 0 0) 0x7f2b04067940 con 0x7f2b140b3f30 2024-09-21T07:50:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.355+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2af4078e90 msgr2=0x7f2af407b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2af4078e90 0x7f2af407b3a0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2b1415fe90 tx=0x7f2b00009040 comp rx=0 tx=0).stop 2024-09-21T07:50:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 msgr2=0x7f2b141550b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 0x7f2b141550b0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f2b0400d6d0 tx=0x7f2b0400db90 comp rx=0 tx=0).stop 2024-09-21T07:50:30.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 shutdown_connections 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f2af4078e90 0x7f2af407b3a0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b1415d6e0 0x7f2b141f4260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b140cd1d0 0x7f2b141f5120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 --2- 172.21.15.88:0/2551538261 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b140b3f30 0x7f2b141550b0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 >> 172.21.15.88:0/2551538261 conn(0x7f2b1406d620 msgr2=0x7f2b1415f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 shutdown_connections 2024-09-21T07:50:30.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.356+0000 7f2b13200640 1 -- 172.21.15.88:0/2551538261 wait complete. 2024-09-21T07:50:30.380 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:30.380 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:30.415 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:30.415 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:30.491 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:50:30.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 -- 172.21.15.88:0/443724715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815c510 msgr2=0x7fa808160ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 --2- 172.21.15.88:0/443724715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815c510 0x7fa808160ac0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc005fd0 tx=0x7fa7fc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 -- 172.21.15.88:0/443724715 shutdown_connections 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 --2- 172.21.15.88:0/443724715 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa808074490 0x7fa8080b3870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 --2- 172.21.15.88:0/443724715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815c510 0x7fa808160ac0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 -- 172.21.15.88:0/443724715 >> 172.21.15.88:0/443724715 conn(0x7fa8080d1470 msgr2=0x7fa808073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 -- 172.21.15.88:0/443724715 shutdown_connections 2024-09-21T07:50:30.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.749+0000 7fa80ea00640 1 -- 172.21.15.88:0/443724715 wait complete. 2024-09-21T07:50:30.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.751+0000 7fa80ea00640 1 Processor -- start 2024-09-21T07:50:30.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.751+0000 7fa80ea00640 1 -- start start 2024-09-21T07:50:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8080cdcf0 0x7fa80806fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa80815c510 0x7fa808071080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 0x7fa80815b620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80ea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa8080ccf80 con 0x7fa80815b210 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa8080ce310 con 0x7fa8080cdcf0 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa8080b3db0 con 0x7fa80815c510 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 0x7fa80815b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 0x7fa80815b620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55914/0 (socket says 172.21.15.88:55914) 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 -- 172.21.15.88:0/3556400509 learned_addr learned my addr 172.21.15.88:0/3556400509 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80cc00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa80815c510 0x7fa808071080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80d600640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8080cdcf0 0x7fa80806fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 -- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa80815c510 msgr2=0x7fa808071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa80815c510 0x7fa808071080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 -- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8080cdcf0 msgr2=0x7fa80806fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8080cdcf0 0x7fa80806fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.753+0000 7fa80e000640 1 -- 172.21.15.88:0/3556400509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa80815bb60 con 0x7fa80815b210 2024-09-21T07:50:30.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.754+0000 7fa80e000640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 0x7fa80815b620 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e990 tx=0x7fa80000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.754+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800004d20 con 0x7fa80815b210 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.754+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa80000caa0 con 0x7fa80815b210 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.754+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa800005020 con 0x7fa80815b210 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.754+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8081f5310 con 0x7fa80815b210 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.754+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8081f58d0 con 0x7fa80815b210 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.755+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8080b42a0 con 0x7fa80815b210 2024-09-21T07:50:30.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.756+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fa8000051c0 con 0x7fa80815b210 2024-09-21T07:50:30.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.757+0000 7fa805600640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa7e8078f20 0x7fa7e807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:30.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.757+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fa80009bcd0 con 0x7fa80815b210 2024-09-21T07:50:30.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.758+0000 7fa80d600640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa7e8078f20 0x7fa7e807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:30.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.759+0000 7fa80d600640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa7e8078f20 0x7fa7e807b430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc033000 tx=0x7fa7fc004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:30.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.762+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa80815bb60 con 0x7fa80815b210 2024-09-21T07:50:30.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.960+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa8080701a0 con 0x7fa80815b210 2024-09-21T07:50:30.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.961+0000 7fa805600640 1 -- 172.21.15.88:0/3556400509 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) ==== 80+0+0 (secure 0 0 0) 0x7fa800016030 con 0x7fa80815b210 2024-09-21T07:50:30.963 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:50:30.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa7e8078f20 msgr2=0x7fa7e807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa7e8078f20 0x7fa7e807b430 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa7fc033000 tx=0x7fa7fc004730 comp rx=0 tx=0).stop 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 msgr2=0x7fa80815b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 0x7fa80815b620 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fa80000e990 tx=0x7fa80000ee50 comp rx=0 tx=0).stop 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 shutdown_connections 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fa7e8078f20 0x7fa7e807b430 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa80815c510 0x7fa808071080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa80815b210 0x7fa80815b620 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 --2- 172.21.15.88:0/3556400509 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa8080cdcf0 0x7fa80806fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 >> 172.21.15.88:0/3556400509 conn(0x7fa8080d1470 msgr2=0x7fa8080b5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 shutdown_connections 2024-09-21T07:50:30.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:30.963+0000 7fa80ea00640 1 -- 172.21.15.88:0/3556400509 wait complete. 2024-09-21T07:50:30.988 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:50:30.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:30.988 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:50:31.051 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:50:31.051 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:31.051 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:50:31.088 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:31.088 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:31.127 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:31.128 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:31.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612e00640 1 -- 172.21.15.88:0/120029761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 msgr2=0x7fe5f40040b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612e00640 1 --2- 172.21.15.88:0/120029761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe5f40040b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fe600001830 tx=0x7fe5f4005830 comp rx=0 tx=0).stop 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612400640 1 -- 172.21.15.88:0/120029761 reap_dead start 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612e00640 1 -- 172.21.15.88:0/120029761 shutdown_connections 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612e00640 1 -- 172.21.15.88:0/120029761 >> 172.21.15.88:0/120029761 conn(0x7fe60c0b1e40 msgr2=0x7fe60c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612e00640 1 -- 172.21.15.88:0/120029761 shutdown_connections 2024-09-21T07:50:31.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.424+0000 7fe612e00640 1 -- 172.21.15.88:0/120029761 wait complete. 2024-09-21T07:50:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.427+0000 7fe612e00640 1 Processor -- start 2024-09-21T07:50:31.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.427+0000 7fe612e00640 1 -- start start 2024-09-21T07:50:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe612e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe60c1f4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe612e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c0cf480 0x7fe60c151510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe612e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe60c0b0770 0x7fe60c0b0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe612e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0b30e0 con 0x7fe5f4003c60 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe612e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe60c0b2f60 con 0x7fe60c0cf480 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe612e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe60c0b3260 con 0x7fe60c0b0770 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.428+0000 7fe611a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe60c1f4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe60c1f4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55940/0 (socket says 172.21.15.88:55940) 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 -- 172.21.15.88:0/1538611065 learned_addr learned my addr 172.21.15.88:0/1538611065 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611000640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c0cf480 0x7fe60c151510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe612400640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe60c0b0770 0x7fe60c0b0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 -- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe60c0b0770 msgr2=0x7fe60c0b0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe60c0b0770 0x7fe60c0b0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 -- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c0cf480 msgr2=0x7fe60c151510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c0cf480 0x7fe60c151510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 -- 172.21.15.88:0/1538611065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60c0b10c0 con 0x7fe5f4003c60 2024-09-21T07:50:31.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.429+0000 7fe611a00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe60c1f4c60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe600000f60 tx=0x7fe600006060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.430+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600043070 con 0x7fe5f4003c60 2024-09-21T07:50:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.430+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6000038e0 con 0x7fe5f4003c60 2024-09-21T07:50:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.430+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600003d30 con 0x7fe5f4003c60 2024-09-21T07:50:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.430+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60c1f5460 con 0x7fe5f4003c60 2024-09-21T07:50:31.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.430+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe60c1f5a20 con 0x7fe5f4003c60 2024-09-21T07:50:31.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.431+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe60c15b3e0 con 0x7fe5f4003c60 2024-09-21T07:50:31.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.432+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fe6000303c0 con 0x7fe5f4003c60 2024-09-21T07:50:31.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.433+0000 7fe609600640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe5ec078ef0 0x7fe5ec07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.433+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fe6000bb1a0 con 0x7fe5f4003c60 2024-09-21T07:50:31.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.434+0000 7fe611000640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe5ec078ef0 0x7fe5ec07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.434+0000 7fe611000640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe5ec078ef0 0x7fe5ec07b400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0cf140 tx=0x7fe5f8001040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:31.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.438+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe60c0b10c0 con 0x7fe5f4003c60 2024-09-21T07:50:31.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.635+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fe60c0cf890 con 0x7fe5f4003c60 2024-09-21T07:50:31.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.636+0000 7fe609600640 1 -- 172.21.15.88:0/1538611065 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v73) ==== 80+0+0 (secure 0 0 0) 0x7fe600082ac0 con 0x7fe5f4003c60 2024-09-21T07:50:31.638 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.638+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe5ec078ef0 msgr2=0x7fe5ec07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.638+0000 7fe612e00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe5ec078ef0 0x7fe5ec07b400 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe60c0cf140 tx=0x7fe5f8001040 comp rx=0 tx=0).stop 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.638+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 msgr2=0x7fe60c1f4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.638+0000 7fe612e00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe60c1f4c60 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7fe600000f60 tx=0x7fe600006060 comp rx=0 tx=0).stop 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 shutdown_connections 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fe5ec078ef0 0x7fe5ec07b400 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe60c0b0770 0x7fe60c0b0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe60c0cf480 0x7fe60c151510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 --2- 172.21.15.88:0/1538611065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5f4003c60 0x7fe60c1f4c60 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 >> 172.21.15.88:0/1538611065 conn(0x7fe60c0b1e40 msgr2=0x7fe60c15c100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 shutdown_connections 2024-09-21T07:50:31.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.639+0000 7fe612e00640 1 -- 172.21.15.88:0/1538611065 wait complete. 2024-09-21T07:50:31.662 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:50:31.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:31.662 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:50:31.731 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:50:31.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:50:31.731 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:50:31.768 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 -- 172.21.15.88:0/1044610732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340af170 msgr2=0x7f88340b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 --2- 172.21.15.88:0/1044610732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340af170 0x7f88340b2f60 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f88340cfe20 tx=0x7f881c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 -- 172.21.15.88:0/1044610732 shutdown_connections 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 --2- 172.21.15.88:0/1044610732 >> v2:172.21.15.175:3300/0 conn(0x7f883415a0f0 0x7f883415d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 --2- 172.21.15.88:0/1044610732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340af170 0x7f88340b2f60 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 -- 172.21.15.88:0/1044610732 >> 172.21.15.88:0/1044610732 conn(0x7f88340b1e40 msgr2=0x7f88340b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 -- 172.21.15.88:0/1044610732 shutdown_connections 2024-09-21T07:50:31.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.989+0000 7f8833200640 1 -- 172.21.15.88:0/1044610732 wait complete. 2024-09-21T07:50:31.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.992+0000 7f8833200640 1 Processor -- start 2024-09-21T07:50:31.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.992+0000 7f8833200640 1 -- start start 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.993+0000 7f8833200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 0x7f88341e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.993+0000 7f8833200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88340af170 0x7f88341471e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.993+0000 7f8833200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f883415a0f0 0x7f8834144260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.993+0000 7f8833200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f883406f290 con 0x7f88340ad9c0 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.993+0000 7f8833200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8834070c80 con 0x7f88340af170 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.993+0000 7f8833200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f883406f410 con 0x7f883415a0f0 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 0x7f88341e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 0x7f88341e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55956/0 (socket says 172.21.15.88:55956) 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 -- 172.21.15.88:0/2598538682 learned_addr learned my addr 172.21.15.88:0/2598538682 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8832800640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f883415a0f0 0x7f8834144260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8829400640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88340af170 0x7f88341471e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 -- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f883415a0f0 msgr2=0x7f8834144260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f883415a0f0 0x7f8834144260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 -- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88340af170 msgr2=0x7f88341471e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88340af170 0x7f88341471e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 -- 172.21.15.88:0/2598538682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88341f5010 con 0x7f88340ad9c0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.994+0000 7f8831e00640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 0x7f88341e6e40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f882400c3c0 tx=0x7f882400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.995+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824017070 con 0x7f88340ad9c0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.995+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f88240035f0 con 0x7f88340ad9c0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.995+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8824003a80 con 0x7f88340ad9c0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.995+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8834156870 con 0x7f88340ad9c0 2024-09-21T07:50:31.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.995+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8834158f40 con 0x7f88340ad9c0 2024-09-21T07:50:31.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.996+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f883406f930 con 0x7f88340ad9c0 2024-09-21T07:50:31.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.997+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f882401a070 con 0x7f88340ad9c0 2024-09-21T07:50:32.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.999+0000 7f882aa00640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8810078ef0 0x7f881007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.999+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f882409a5a0 con 0x7f88340ad9c0 2024-09-21T07:50:32.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:31.999+0000 7f8829400640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8810078ef0 0x7f881007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.000+0000 7f8829400640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8810078ef0 0x7f881007b400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f881c033000 tx=0x7f881c004730 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.003+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8824061ec0 con 0x7f88340ad9c0 2024-09-21T07:50:32.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.224+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8834159880 con 0x7f88340ad9c0 2024-09-21T07:50:32.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.225+0000 7f882aa00640 1 -- 172.21.15.88:0/2598538682 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f88240668f0 con 0x7f88340ad9c0 2024-09-21T07:50:32.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:32.227 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:32.227 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:32.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8810078ef0 msgr2=0x7f881007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8810078ef0 0x7f881007b400 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f881c033000 tx=0x7f881c004730 comp rx=0 tx=0).stop 2024-09-21T07:50:32.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 msgr2=0x7f88341e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 0x7f88341e6e40 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f882400c3c0 tx=0x7f882400c880 comp rx=0 tx=0).stop 2024-09-21T07:50:32.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 shutdown_connections 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8810078ef0 0x7f881007b400 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f883415a0f0 0x7f8834144260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f88340af170 0x7f88341471e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 --2- 172.21.15.88:0/2598538682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88340ad9c0 0x7f88341e6e40 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 >> 172.21.15.88:0/2598538682 conn(0x7f88340b1e40 msgr2=0x7f883415bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 shutdown_connections 2024-09-21T07:50:32.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.227+0000 7f8833200640 1 -- 172.21.15.88:0/2598538682 wait complete. 2024-09-21T07:50:32.251 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:50:32.494 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-09-21T07:50:32.494 DEBUG:tasks.cephfs.filesystem:_json_asok output [] 2024-09-21T07:50:32.494 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol53650 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1528a00640 1 -- 172.21.15.88:0/2005279572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1508004db0 msgr2=0x7f1508005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1528a00640 1 --2- 172.21.15.88:0/2005279572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1508004db0 0x7f1508005280 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f151c005fd0 tx=0x7f1508005830 comp rx=0 tx=0).stop 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1523200640 1 -- 172.21.15.88:0/2005279572 reap_dead start 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1528a00640 1 -- 172.21.15.88:0/2005279572 shutdown_connections 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1528a00640 1 -- 172.21.15.88:0/2005279572 >> 172.21.15.88:0/2005279572 conn(0x7f152406d620 msgr2=0x7f15240b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1528a00640 1 -- 172.21.15.88:0/2005279572 shutdown_connections 2024-09-21T07:50:32.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.715+0000 7f1528a00640 1 -- 172.21.15.88:0/2005279572 wait complete. 2024-09-21T07:50:32.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.717+0000 7f1528a00640 1 Processor -- start 2024-09-21T07:50:32.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.718+0000 7f1528a00640 1 -- start start 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1528a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1508004db0 0x7f15241f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1528a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 0x7f15241ef610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1528a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1524072eb0 0x7f15240732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1528a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15240af0d0 con 0x7f152415f320 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1528a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15240cd520 con 0x7f1524072eb0 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1528a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15240af250 con 0x7f1508004db0 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1519e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 0x7f15241ef610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1519e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 0x7f15241ef610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55964/0 (socket says 172.21.15.88:55964) 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.719+0000 7f1519e00640 1 -- 172.21.15.88:0/1559835818 learned_addr learned my addr 172.21.15.88:0/1559835818 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1519e00640 1 -- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1508004db0 msgr2=0x7f15241f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1519e00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1508004db0 0x7f15241f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1519e00640 1 -- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1524072eb0 msgr2=0x7f15240732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1519e00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1524072eb0 0x7f15240732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1519e00640 1 -- 172.21.15.88:0/1559835818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1524073800 con 0x7f152415f320 2024-09-21T07:50:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1519e00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 0x7f15241ef610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f150c006450 tx=0x7f150c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c015070 con 0x7f152415f320 2024-09-21T07:50:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f150c011840 con 0x7f152415f320 2024-09-21T07:50:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15241f56d0 con 0x7f152415f320 2024-09-21T07:50:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f150c011c50 con 0x7f152415f320 2024-09-21T07:50:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.720+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15241f5970 con 0x7f152415f320 2024-09-21T07:50:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.722+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15240af7e0 con 0x7f152415f320 2024-09-21T07:50:32.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.723+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f150c018070 con 0x7f152415f320 2024-09-21T07:50:32.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.724+0000 7f151b400640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1500078f20 0x7f150007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:32.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.724+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f150c09c020 con 0x7f152415f320 2024-09-21T07:50:32.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.724+0000 7f1522800640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1500078f20 0x7f150007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:32.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.725+0000 7f1522800640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1500078f20 0x7f150007b430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1514006fd0 tx=0x7f15140068a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:32.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.728+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f150c063e40 con 0x7f152415f320 2024-09-21T07:50:32.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:32.939+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 --> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol53650", "target": ["mon-mgr", ""]}) -- 0x7f152409e420 con 0x7f1500078f20 2024-09-21T07:50:32.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:50:32.950+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:32.950 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:50:32.950+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:32.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:50:32.950+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:32.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:50:32.950+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:32.951 INFO:tasks.ceph.mgr.x.smithi175.stderr:2024-09-21T07:50:32.950+0000 7ff29d600640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:33.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.001+0000 7f151b400640 1 -- 172.21.15.88:0/1559835818 <== mgr.4111 v2:172.21.15.175:6800/3294411971 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f152409e420 con 0x7f1500078f20 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1500078f20 msgr2=0x7f150007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1500078f20 0x7f150007b430 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f1514006fd0 tx=0x7f15140068a0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 msgr2=0x7f15241ef610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 0x7f15241ef610 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f150c006450 tx=0x7f150c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 shutdown_connections 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f1500078f20 0x7f150007b430 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1524072eb0 0x7f15240732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f152415f320 0x7f15241ef610 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 --2- 172.21.15.88:0/1559835818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1508004db0 0x7f15241f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 >> 172.21.15.88:0/1559835818 conn(0x7f152406d620 msgr2=0x7f1524068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 shutdown_connections 2024-09-21T07:50:33.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.004+0000 7f1528a00640 1 -- 172.21.15.88:0/1559835818 wait complete. 2024-09-21T07:50:33.031 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0f200640 1 -- 172.21.15.88:0/44986218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8eec003a00 msgr2=0x7f8eec003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0f200640 1 --2- 172.21.15.88:0/44986218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8eec003a00 0x7f8eec003e70 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009b50 tx=0x7f8eec005830 comp rx=0 tx=0).stop 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0d200640 1 -- 172.21.15.88:0/44986218 reap_dead start 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0f200640 1 -- 172.21.15.88:0/44986218 shutdown_connections 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0f200640 1 -- 172.21.15.88:0/44986218 >> 172.21.15.88:0/44986218 conn(0x7f8f0806d730 msgr2=0x7f8f080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0f200640 1 -- 172.21.15.88:0/44986218 shutdown_connections 2024-09-21T07:50:33.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.248+0000 7f8f0f200640 1 -- 172.21.15.88:0/44986218 wait complete. 2024-09-21T07:50:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.251+0000 7f8f0f200640 1 Processor -- start 2024-09-21T07:50:33.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.251+0000 7f8f0f200640 1 -- start start 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f0f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8eec003a00 0x7f8f080731f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f0f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 0x7f8f0814a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f0f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f0815b7f0 0x7f8f0815bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f0f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0806f220 con 0x7f8f0814ab70 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f0f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8f0806f0a0 con 0x7f8f0815b7f0 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f0f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8f080b4c20 con 0x7f8eec003a00 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 0x7f8f0814a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 0x7f8f0814a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55970/0 (socket says 172.21.15.88:55970) 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.252+0000 7f8f07400640 1 -- 172.21.15.88:0/2490023432 learned_addr learned my addr 172.21.15.88:0/2490023432 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f0d200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f0815b7f0 0x7f8f0815bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07e00640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8eec003a00 0x7f8f080731f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07400640 1 -- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8eec003a00 msgr2=0x7f8f080731f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07400640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8eec003a00 0x7f8f080731f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07400640 1 -- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f0815b7f0 msgr2=0x7f8f0815bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07400640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f0815b7f0 0x7f8f0815bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07400640 1 -- 172.21.15.88:0/2490023432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f0815c140 con 0x7f8f0814ab70 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f07400640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 0x7f8f0814a350 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009460 tx=0x7f8ef4011c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef4004460 con 0x7f8f0814ab70 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ef4016070 con 0x7f8f0814ab70 2024-09-21T07:50:33.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f08154880 con 0x7f8f0814ab70 2024-09-21T07:50:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.253+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ef400fba0 con 0x7f8f0814ab70 2024-09-21T07:50:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.254+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f08154e40 con 0x7f8f0814ab70 2024-09-21T07:50:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.254+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f08160fa0 con 0x7f8f0814ab70 2024-09-21T07:50:33.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.256+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7f8ef4018030 con 0x7f8f0814ab70 2024-09-21T07:50:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.257+0000 7f8effe00640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8ee8078f20 0x7f8ee807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.257+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f8ef409bc30 con 0x7f8f0814ab70 2024-09-21T07:50:33.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.257+0000 7f8f07e00640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8ee8078f20 0x7f8ee807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.258+0000 7f8f07e00640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8ee8078f20 0x7f8ee807b430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009b50 tx=0x7f8ef802d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:33.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.261+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8f0815c140 con 0x7f8f0814ab70 2024-09-21T07:50:33.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.484+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8f0815ee50 con 0x7f8f0814ab70 2024-09-21T07:50:33.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.484+0000 7f8effe00640 1 -- 172.21.15.88:0/2490023432 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f8ef4063a10 con 0x7f8f0814ab70 2024-09-21T07:50:33.487 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:33.487 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:33.487 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.486+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8ee8078f20 msgr2=0x7f8ee807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.486+0000 7f8f0f200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8ee8078f20 0x7f8ee807b430 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8ef8009b50 tx=0x7f8ef802d040 comp rx=0 tx=0).stop 2024-09-21T07:50:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.486+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 msgr2=0x7f8f0814a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.486+0000 7f8f0f200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 0x7f8f0814a350 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f8ef4009460 tx=0x7f8ef4011c30 comp rx=0 tx=0).stop 2024-09-21T07:50:33.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 shutdown_connections 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f8ee8078f20 0x7f8ee807b430 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8f0815b7f0 0x7f8f0815bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8f0814ab70 0x7f8f0814a350 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 --2- 172.21.15.88:0/2490023432 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8eec003a00 0x7f8f080731f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 >> 172.21.15.88:0/2490023432 conn(0x7f8f0806d730 msgr2=0x7f8f080cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 shutdown_connections 2024-09-21T07:50:33.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.487+0000 7f8f0f200640 1 -- 172.21.15.88:0/2490023432 wait complete. 2024-09-21T07:50:33.515 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:50:33.755 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":6390,"entity":{"name":{"type":"client","num":6390},"addr":{"type":"any","addr":"172.21.15.175:0","nonce":691722558}},"state":"open","num_leases":1,"num_caps":7,"request_load_avg":15,"uptime":0.78098481900000005,"requests_in_flight":2,"num_completed_requests":0,"num_completed_flushes":1,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":21.961269484181756,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":15,"last_trim_completed_flushes_tid":2,"delegated_inos":[{"start":"0x10000000005","length":500}],"inst":"client.6390 172.21.15.175:0/691722558","completed_requests":[],"prealloc_inos":[{"start":"0x10000000005","length":996}],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"x","hostname":"smithi175","pid":"35989","root":"/"}}] 2024-09-21T07:50:33.756 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "x", "hostname": "smithi175", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35989", "root": "/" }, "completed_requests": [], "delegated_inos": [ { "length": 500, "start": "0x10000000005" } ], "entity": { "addr": { "addr": "172.21.15.175:0", "nonce": 691722558, "type": "any" }, "name": { "num": 6390, "type": "client" } }, "id": 6390, "inst": "client.6390 172.21.15.175:0/691722558", "last_trim_completed_flushes_tid": 2, "last_trim_completed_requests_tid": 15, "num_caps": 7, "num_completed_flushes": 1, "num_completed_requests": 0, "num_leases": 1, "prealloc_inos": [ { "length": 996, "start": "0x10000000005" } ], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 15, "requests_in_flight": 2, "session_cache_liveness": { "halflife": 300, "value": 21.961269484181756 }, "state": "open", "uptime": 0.780984819 } ] 2024-09-21T07:50:33.756 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd490800640 1 -- 172.21.15.88:0/638811415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd470004d40 msgr2=0x7fd470005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd490800640 1 --2- 172.21.15.88:0/638811415 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd470004d40 0x7fd470005150 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fd4880aea60 tx=0x7fd470005830 comp rx=0 tx=0).stop 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd48e800640 1 -- 172.21.15.88:0/638811415 reap_dead start 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd490800640 1 -- 172.21.15.88:0/638811415 shutdown_connections 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd490800640 1 -- 172.21.15.88:0/638811415 >> 172.21.15.88:0/638811415 conn(0x7fd48806d730 msgr2=0x7fd4880b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd490800640 1 -- 172.21.15.88:0/638811415 shutdown_connections 2024-09-21T07:50:33.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.974+0000 7fd490800640 1 -- 172.21.15.88:0/638811415 wait complete. 2024-09-21T07:50:33.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.976+0000 7fd490800640 1 Processor -- start 2024-09-21T07:50:33.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.977+0000 7fd490800640 1 -- start start 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd490800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd470004d40 0x7fd4881f43e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd490800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4881f6760 0x7fd4881f3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd490800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 0x7fd488158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd490800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd48806f220 con 0x7fd4881f3ca0 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd490800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd48806f0a0 con 0x7fd470004d40 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd490800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4880b4c20 con 0x7fd4881f6760 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd48e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 0x7fd488158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd48e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 0x7fd488158360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55982/0 (socket says 172.21.15.88:55982) 2024-09-21T07:50:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd48e800640 1 -- 172.21.15.88:0/3262689408 learned_addr learned my addr 172.21.15.88:0/3262689408 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.978+0000 7fd48d400640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4881f6760 0x7fd4881f3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48de00640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd470004d40 0x7fd4881f43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48e800640 1 -- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4881f6760 msgr2=0x7fd4881f3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48e800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4881f6760 0x7fd4881f3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48e800640 1 -- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd470004d40 msgr2=0x7fd4881f43e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48e800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd470004d40 0x7fd4881f43e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48e800640 1 -- 172.21.15.88:0/3262689408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4881578c0 con 0x7fd4881f3ca0 2024-09-21T07:50:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd48e800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 0x7fd488158360 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd48402b660 tx=0x7fd484004a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400c3a0 con 0x7fd4881f3ca0 2024-09-21T07:50:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd484004d40 con 0x7fd4881f3ca0 2024-09-21T07:50:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48402bbd0 con 0x7fd4881f3ca0 2024-09-21T07:50:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.979+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd488150db0 con 0x7fd4881f3ca0 2024-09-21T07:50:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.980+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd488157e40 con 0x7fd4881f3ca0 2024-09-21T07:50:33.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.981+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd488150b20 con 0x7fd4881f3ca0 2024-09-21T07:50:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.982+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 11) ==== 101747+0+0 (secure 0 0 0) 0x7fd484006030 con 0x7fd4881f3ca0 2024-09-21T07:50:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.983+0000 7fd482000640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd46c078f20 0x7fd46c07b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.983+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7fd48403d020 con 0x7fd4881f3ca0 2024-09-21T07:50:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.984+0000 7fd48de00640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd46c078f20 0x7fd46c07b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.984+0000 7fd48de00640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd46c078f20 0x7fd46c07b430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009cd0 tx=0x7fd47c005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:33.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:33.987+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4880ada30 con 0x7fd4881f3ca0 2024-09-21T07:50:34.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.222+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fd48815e680 con 0x7fd4881f3ca0 2024-09-21T07:50:34.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.229+0000 7fd482000640 1 -- 172.21.15.88:0/3262689408 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v11) ==== 81+0+530962 (secure 0 0 0) 0x7fd48815e680 con 0x7fd4881f3ca0 2024-09-21T07:50:34.238 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "epoch": 11, 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "active_gid": 4111, 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "active_name": "x", 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "active_addrs": { 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:6800", 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3294411971 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.239 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v1", 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:6801", 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3294411971 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "active_addr": "172.21.15.175:6801/3294411971", 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "active_change": "2024-09-21T07:45:43.841149+0000", 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "available": true, 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: "standbys": [ 2024-09-21T07:50:34.240 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "gid": 4116, 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "y", 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr_features": 4540719139924082687, 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "available_modules": [ 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "alerts", 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.241 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.242 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.243 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.244 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.245 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.246 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_destination": { 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_destination", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.247 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_from_name": { 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_from_name", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.248 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "Ceph", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Email From: name", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_host": { 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_host", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.249 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SMTP server", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_password": { 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_password", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.250 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Password to authenticate with", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_port": { 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_port", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.251 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "465", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SMTP port", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_sender": { 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_sender", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.252 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_ssl": { 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_ssl", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.253 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_user": { 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_user", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.254 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "User to authenticate as", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.255 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.256 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "balancer", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "active": { 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "active", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:50:34.257 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "begin_time": { 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "begin_time", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0000", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:50:34.258 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "begin_weekday": { 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "begin_weekday", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "0", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "6", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:50:34.259 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_compat_max_iterations": { 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "25", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "250", 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.260 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_compat_metrics": { 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crush_compat_metrics", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.261 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_compat_step": { 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crush_compat_step", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0.5", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "0.001", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "0.999", 2024-09-21T07:50:34.262 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "end_time": { 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "end_time", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "2359", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.263 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "end_weekday": { 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "end_weekday", 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.264 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "0", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "6", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.265 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.266 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.267 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.268 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.269 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "min_score": { 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "min_score", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.270 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": { 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mode", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.271 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "upmap", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "crush-compat", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "none", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "read", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap-read" 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Balancer mode", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.272 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_ids": { 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pool_ids", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.273 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.274 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.275 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap_max_deviation": { 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "upmap_max_deviation", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:50:34.276 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap_max_optimizations": { 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:50:34.277 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephadm", 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "agent_down_multiplier": { 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "agent_down_multiplier", 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.278 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3.0", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "agent_refresh_rate": { 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "agent_refresh_rate", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.279 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "20", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "agent_starting_port": { 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "agent_starting_port", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.280 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "4721", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "allow_ptrace": { 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "allow_ptrace", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.281 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "autotune_interval": { 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "autotune_interval", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.282 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0.7", 2024-09-21T07:50:34.283 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "cephadm_log_destination": { 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephadm_log_destination", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.284 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "file", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "file,syslog", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "syslog" 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "cgroups_split": { 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cgroups_split", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.285 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "config_checks_enabled": { 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "config_checks_enabled", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.286 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "config_dashboard": { 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "config_dashboard", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.287 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_alertmanager": { 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_alertmanager", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.288 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_base": { 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_base", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.289 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_elasticsearch": { 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.290 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "elasticsearch container image", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_grafana": { 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_grafana", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.291 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_haproxy": { 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_haproxy", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.292 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "HAproxy container image", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_jaeger_agent": { 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.293 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_jaeger_collector": { 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.294 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_jaeger_query": { 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.295 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Jaeger query container image", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_keepalived": { 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_keepalived", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:50:34.296 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Keepalived container image", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_loki": { 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_loki", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.297 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Loki container image", 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_nginx": { 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_nginx", 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.298 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Nginx container image", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_node_exporter": { 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_node_exporter", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.299 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_nvmeof": { 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_nvmeof", 2024-09-21T07:50:34.300 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Nvme-of container image", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:50:34.301 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.302 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_prometheus": { 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_prometheus", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.303 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_promtail": { 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_promtail", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Promtail container image", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.304 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_samba": { 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_samba", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.305 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_samba_metrics": { 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.306 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_snmp_gateway": { 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.307 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "container_init": { 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_init", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.308 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "daemon_cache_timeout": { 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.309 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.310 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "900", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "default_registry": { 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "default_registry", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.311 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "device_cache_timeout": { 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "device_cache_timeout", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.312 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "1800", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "device_enhanced_scan": { 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "device_enhanced_scan", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.313 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "facts_cache_timeout": { 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "facts_cache_timeout", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.314 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "grafana_dashboards_path": { 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.315 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "host_check_interval": { 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "host_check_interval", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.316 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "hw_monitoring": { 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "hw_monitoring", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.317 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "inventory_list_all": { 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "inventory_list_all", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.318 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.319 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.320 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "log_refresh_metadata": { 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_refresh_metadata", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.321 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.322 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.323 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.324 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.325 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "*", 2024-09-21T07:50:34.326 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "max_count_per_host": { 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_count_per_host", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.327 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "max_osd_draining_count": { 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_osd_draining_count", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.328 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "migration_current": { 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "migration_current", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.329 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "internal - do not modify", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": { 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mode", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "root", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.330 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "cephadm-package", 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "root" 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "oob_default_addr": { 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "oob_default_addr", 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.331 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "169.254.1.1", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "prometheus_alerts_path": { 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.332 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_insecure": { 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_insecure", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.333 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_password": { 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_password", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.334 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_url": { 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_url", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.335 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_username": { 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_username", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.336 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "secure_monitoring_stack": { 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.337 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "service_discovery_port": { 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "service_discovery_port", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8765", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.338 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.339 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "ssh_config_file": { 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssh_config_file", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.340 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.341 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "ssh_keepalive_interval": { 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "7", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.342 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "use_agent": { 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "use_agent", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.343 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "use_repo_digest": { 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "use_repo_digest", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.344 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_on_failed_host_check": { 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.345 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_on_stray_daemons": { 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.346 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_on_stray_hosts": { 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.347 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crash", 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.348 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.349 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.350 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.351 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.352 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "retain_interval": { 2024-09-21T07:50:34.353 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "retain_interval", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "31536000", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.354 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_recent_interval": { 2024-09-21T07:50:34.355 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_recent_interval", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "1209600", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.356 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "dashboard", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.357 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.358 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.359 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.360 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.361 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.362 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.363 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.364 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.365 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.366 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.367 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.368 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.369 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.370 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "admin", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.371 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.372 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_URL": { 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.373 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "admin", 2024-09-21T07:50:34.374 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.375 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.376 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.377 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.378 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "[]", 2024-09-21T07:50:34.379 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.380 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "{}", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.381 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.382 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.383 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.384 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.385 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.386 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.387 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.388 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:50:34.389 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:50:34.390 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:50:34.391 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:50:34.392 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:50:34.393 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:50:34.394 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "45", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:50:34.395 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:50:34.396 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "admin", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:50:34.397 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:50:34.398 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:50:34.399 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.400 INFO:teuthology.orchestra.run.smithi088.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.401 INFO:teuthology.orchestra.run.smithi088.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.402 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.403 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "cross_origin_url": { 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cross_origin_url", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.404 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "crt_file": { 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crt_file", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.405 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "debug": { 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "debug", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.406 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "jwt_token_ttl": { 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "jwt_token_ttl", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "28800", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.407 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "key_file": { 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "key_file", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.408 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.409 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.410 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.411 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.412 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.413 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "motd": { 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "motd", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "The message of the day", 2024-09-21T07:50:34.414 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "server_addr": { 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_addr", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "::", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.416 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "server_port": { 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_port", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8080", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.417 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.418 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "ssl": { 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssl", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.419 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "ssl_server_port": { 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssl_server_port", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8443", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.420 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "sso_oauth2": { 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sso_oauth2", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.421 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_behaviour": { 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_behaviour", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "redirect", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "redirect" 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.422 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_error_status_code": { 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_error_status_code", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "500", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "400", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "599", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.423 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "url_prefix": { 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "url_prefix", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.424 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "devicehealth", 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "enable_monitoring": { 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enable_monitoring", 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.425 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "monitor device health metrics", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.426 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.427 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.428 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.429 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.430 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "mark_out_threshold": { 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mark_out_threshold", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.431 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "2419200", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_name": { 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pool_name", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.432 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "device_health_metrics", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "retention_period": { 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "retention_period", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.433 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "15552000", 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "scrape_frequency": { 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "scrape_frequency", 2024-09-21T07:50:34.434 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "86400", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "self_heal": { 2024-09-21T07:50:34.435 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "self_heal", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.436 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.437 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_threshold": { 2024-09-21T07:50:34.438 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_threshold", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "7257600", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.439 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "diskprediction_local", 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.440 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.441 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.442 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.443 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.444 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "predict_interval": { 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "predict_interval", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.445 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "86400", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "predictor_model": { 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "predictor_model", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.446 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "prophetstor", 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.447 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.448 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.449 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "influx", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": false, 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "batch_size": { 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "batch_size", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5000", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.450 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "database": { 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "database", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "ceph", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.451 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": { 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "hostname", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.452 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "30", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "5", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.453 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.454 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.455 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.456 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.457 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.458 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "password": { 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "password", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.459 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "port": { 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "port", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8086", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "InfluxDB server port", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.460 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.461 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "ssl": { 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssl", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "false", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.462 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "threads": { 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "threads", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "32", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.463 INFO:teuthology.orchestra.run.smithi088.stdout: "username": { 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "username", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "verify_ssl": { 2024-09-21T07:50:34.464 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "verify_ssl", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "true", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.465 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "insights", 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.466 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.467 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.468 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.469 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.470 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.471 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "iostat", 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.472 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.473 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.474 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.475 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.476 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.477 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.478 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "localpool", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "failure_domain": { 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "failure_domain", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "host", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.479 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.480 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.482 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.483 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.484 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "min_size": { 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "min_size", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.485 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "num_rep": { 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "num_rep", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.486 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_num": { 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pg_num", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "128", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.487 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "prefix": { 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "prefix", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.488 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.489 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "subtree": { 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "subtree", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "rack", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.490 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mds_autoscaler", 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.491 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.492 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.493 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.494 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.495 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.496 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.497 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mirroring", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.498 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.499 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.500 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.501 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.502 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.503 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "nfs", 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.504 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.505 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.506 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.507 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.509 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.510 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "orchestrator", 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "fail_fs": { 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "fail_fs", 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.511 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.512 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.513 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.514 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.515 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.516 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator": { 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "orchestrator", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "cephadm", 2024-09-21T07:50:34.517 INFO:teuthology.orchestra.run.smithi088.stdout: "rook", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "test_orchestrator" 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Orchestrator backend", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.518 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "osd_perf_query", 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.519 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.520 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.521 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.522 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.523 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.524 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "osd_support", 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.525 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.526 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.527 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.528 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.529 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.530 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.531 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pg_autoscaler", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.532 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.533 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.534 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.535 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.536 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.537 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.538 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "threshold": { 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "threshold", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3.0", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1.0", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "scaling threshold", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.539 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "progress", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "allow_pg_recovery_event": { 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.540 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": { 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enabled", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.541 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.542 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.543 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.544 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.545 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.546 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "max_completed_events": { 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_completed_events", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "50", 2024-09-21T07:50:34.547 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.548 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.549 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "prometheus", 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.550 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "cache": { 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cache", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.551 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "exclude_perf_counters": { 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "exclude_perf_counters", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.552 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.553 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.554 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.555 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.556 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.557 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_stats_pools": { 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_stats_pools", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.558 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "300", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.559 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "scrape_interval": { 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "scrape_interval", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "15.0", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.560 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "server_addr": { 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_addr", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "::", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.561 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "server_port": { 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_port", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "9283", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.562 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.563 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "stale_cache_strategy": { 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "stale_cache_strategy", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "log", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.564 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_behaviour": { 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_behaviour", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "default", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.565 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "default", 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "error" 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_error_status_code": { 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_error_status_code", 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.566 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "500", 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "400", 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "599", 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.567 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_support", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.568 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.569 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.570 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.571 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.572 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "max_concurrent_snap_create": { 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.573 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.574 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.575 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "trash_purge_schedule": { 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "trash_purge_schedule", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.576 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "restful", 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "enable_auth": { 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enable_auth", 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.577 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "key_file": { 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "key_file", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.578 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.579 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.580 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.581 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.582 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.583 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "max_requests": { 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_requests", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.584 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "500", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "server_addr": { 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_addr", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.585 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "server_port": { 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_port", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.586 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.587 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rgw", 2024-09-21T07:50:34.588 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.589 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.590 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.591 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.592 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.593 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:50:34.594 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.595 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rook", 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.596 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.597 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.598 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.599 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.600 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.601 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "storage_class": { 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "storage_class", 2024-09-21T07:50:34.602 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "local", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.603 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "selftest", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.604 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.605 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.606 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "roption1": { 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "roption1", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "roption2": { 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "roption2", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "xyz", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption1": { 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption1", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption2": { 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption2", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption3": { 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption3", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption4": { 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption4", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption5": { 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption5", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption6": { 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption6", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption7": { 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption7", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "42", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "testkey": { 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "testkey", 2024-09-21T07:50:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "testlkey": { 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "testlkey", 2024-09-21T07:50:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "testnewline": { 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "testnewline", 2024-09-21T07:50:34.621 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.622 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smb", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "internal_store_backend": { 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "internal_store_backend", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.623 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.624 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.625 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.626 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.627 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.628 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.629 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "update_orchestration": { 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "update_orchestration", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.630 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "snap_schedule", 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "allow_m_granularity": { 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "allow_m_granularity", 2024-09-21T07:50:34.631 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "dump_on_update": { 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "dump_on_update", 2024-09-21T07:50:34.632 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.633 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.634 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.635 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.636 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.637 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.638 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.639 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "stats", 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.640 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.641 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.642 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.643 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.644 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.645 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "status", 2024-09-21T07:50:34.646 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.647 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.648 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.649 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.650 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.651 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.652 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "telegraf", 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "address": { 2024-09-21T07:50:34.653 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "address", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.654 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "15", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.656 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.657 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.658 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.659 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.660 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "telemetry", 2024-09-21T07:50:34.661 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_basic": { 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_basic", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.662 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_crash": { 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_crash", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.663 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_device": { 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_device", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.664 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_ident": { 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_ident", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_perf": { 2024-09-21T07:50:34.665 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_perf", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.666 INFO:teuthology.orchestra.run.smithi088.stdout: "contact": { 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "contact", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.667 INFO:teuthology.orchestra.run.smithi088.stdout: "description": { 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "description", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "device_url": { 2024-09-21T07:50:34.668 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "device_url", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": { 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enabled", 2024-09-21T07:50:34.669 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.670 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "24", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "8", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "last_opt_revision": { 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "last_opt_revision", 2024-09-21T07:50:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "1", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "leaderboard": { 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "leaderboard", 2024-09-21T07:50:34.672 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "leaderboard_description": { 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "leaderboard_description", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.673 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.674 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.675 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.676 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.677 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.678 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "organization": { 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "organization", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.679 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "proxy": { 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "proxy", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.680 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.681 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "url": { 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "url", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.682 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "test_orchestrator", 2024-09-21T07:50:34.683 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.684 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.685 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.686 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.687 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.688 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.689 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.690 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.691 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.692 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.693 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.694 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "max_concurrent_clones": { 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_concurrent_clones", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "4", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.695 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "periodic_async_work": { 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "periodic_async_work", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Periodically check for async work", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.696 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot_clone_delay": { 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.697 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.698 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.699 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix", 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "discovery_interval": { 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "discovery_interval", 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.700 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "100", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "identifier": { 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "identifier", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.701 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.702 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.703 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.704 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.705 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.706 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.707 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.708 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "zabbix_host": { 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix_host", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.709 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "zabbix_port": { 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix_port", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.710 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10051", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "zabbix_sender": { 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix_sender", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.711 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.712 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "modules": [ 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "iostat", 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "nfs", 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "restful" 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "available_modules": [ 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "alerts", 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.713 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.714 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.715 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.716 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.717 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.718 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.719 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_destination": { 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_destination", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.720 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_from_name": { 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_from_name", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "Ceph", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Email From: name", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.721 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_host": { 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_host", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SMTP server", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.722 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_password": { 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_password", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Password to authenticate with", 2024-09-21T07:50:34.723 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_port": { 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_port", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "465", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.724 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SMTP port", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_sender": { 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_sender", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.725 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SMTP envelope sender", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_ssl": { 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_ssl", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.726 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "smtp_user": { 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smtp_user", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.727 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "User to authenticate as", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.728 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "balancer", 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "active": { 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "active", 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.729 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "begin_time": { 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "begin_time", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.730 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0000", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "begin_weekday": { 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "begin_weekday", 2024-09-21T07:50:34.731 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "0", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "6", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_compat_max_iterations": { 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crush_compat_max_iterations", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.732 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "25", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "250", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_compat_metrics": { 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crush_compat_metrics", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.733 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "crush_compat_step": { 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crush_compat_step", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.734 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0.5", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "0.001", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "0.999", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "aggressiveness of optimization", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "end_time": { 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "end_time", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.735 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "2359", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "end_weekday": { 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "end_weekday", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.736 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "0", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "6", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.737 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.738 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.739 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.740 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.741 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.742 INFO:teuthology.orchestra.run.smithi088.stdout: "min_score": { 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "min_score", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.743 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": { 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mode", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "upmap", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "crush-compat", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "none", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "read", 2024-09-21T07:50:34.744 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap-read" 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Balancer mode", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_ids": { 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pool_ids", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.745 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.746 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.747 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap_max_deviation": { 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "upmap_max_deviation", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.748 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "upmap_max_optimizations": { 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "upmap_max_optimizations", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.749 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephadm", 2024-09-21T07:50:34.750 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "agent_down_multiplier": { 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "agent_down_multiplier", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3.0", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.751 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "agent_refresh_rate": { 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "agent_refresh_rate", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "20", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.752 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "agent_starting_port": { 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "agent_starting_port", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "4721", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.753 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "allow_ptrace": { 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "allow_ptrace", 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.754 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "autotune_interval": { 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "autotune_interval", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.755 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "autotune_memory_target_ratio": { 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0.7", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.756 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "cephadm_log_destination": { 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephadm_log_destination", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "file", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "file,syslog", 2024-09-21T07:50:34.757 INFO:teuthology.orchestra.run.smithi088.stdout: "syslog" 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "cgroups_split": { 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cgroups_split", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.758 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "config_checks_enabled": { 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "config_checks_enabled", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.759 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "config_dashboard": { 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "config_dashboard", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.760 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_alertmanager": { 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_alertmanager", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.761 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_base": { 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_base", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.762 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Container image name, without the tag", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_elasticsearch": { 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_elasticsearch", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.763 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "elasticsearch container image", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_grafana": { 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_grafana", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.764 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_haproxy": { 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_haproxy", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "HAproxy container image", 2024-09-21T07:50:34.765 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_jaeger_agent": { 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_jaeger_agent", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.766 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Jaeger agent container image", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_jaeger_collector": { 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_jaeger_collector", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.767 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Jaeger collector container image", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_jaeger_query": { 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_jaeger_query", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.768 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Jaeger query container image", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_keepalived": { 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_keepalived", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.769 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Keepalived container image", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_loki": { 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_loki", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.770 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Loki container image", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_nginx": { 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_nginx", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.771 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Nginx container image", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_node_exporter": { 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_node_exporter", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.772 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_nvmeof": { 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_nvmeof", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.773 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Nvme-of container image", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_oauth2_proxy": { 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.774 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "oauth2-proxy container image", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_prometheus": { 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_prometheus", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.775 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Prometheus container image", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_promtail": { 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_promtail", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.776 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Promtail container image", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_samba": { 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_samba", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.777 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Samba/SMB container image", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_samba_metrics": { 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_samba_metrics", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.778 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "container_image_snmp_gateway": { 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_image_snmp_gateway", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.779 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "SNMP Gateway container image", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "container_init": { 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "container_init", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.780 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "daemon_cache_timeout": { 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "daemon_cache_timeout", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T07:50:34.781 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "default_cephadm_command_timeout": { 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "900", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.782 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "default_registry": { 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "default_registry", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "docker.io", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.783 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "device_cache_timeout": { 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "device_cache_timeout", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "1800", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.784 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "seconds to cache device inventory", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "device_enhanced_scan": { 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "device_enhanced_scan", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.785 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "facts_cache_timeout": { 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "facts_cache_timeout", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.786 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "seconds to cache host facts data", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "grafana_dashboards_path": { 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "grafana_dashboards_path", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.787 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "host_check_interval": { 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "host_check_interval", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to perform a host check", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.788 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "hw_monitoring": { 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "hw_monitoring", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.789 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "inventory_list_all": { 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "inventory_list_all", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.790 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.791 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "log_refresh_metadata": { 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_refresh_metadata", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.792 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.793 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.794 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.795 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.796 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.797 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "*", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "max_count_per_host": { 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_count_per_host", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.798 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "max number of daemons per service per host", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "max_osd_draining_count": { 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_osd_draining_count", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.799 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "migration_current": { 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "migration_current", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.800 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "internal - do not modify", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": { 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mode", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "root", 2024-09-21T07:50:34.801 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "cephadm-package", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "root" 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "oob_default_addr": { 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "oob_default_addr", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.802 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "169.254.1.1", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "prometheus_alerts_path": { 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "prometheus_alerts_path", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.803 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_insecure": { 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_insecure", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.804 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_password": { 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_password", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.805 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_url": { 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_url", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.806 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "registry_username": { 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "registry_username", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.807 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "secure_monitoring_stack": { 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "secure_monitoring_stack", 2024-09-21T07:50:34.808 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "service_discovery_port": { 2024-09-21T07:50:34.809 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "service_discovery_port", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8765", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "cephadm service discovery port", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.810 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.811 INFO:teuthology.orchestra.run.smithi088.stdout: "ssh_config_file": { 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssh_config_file", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.812 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "ssh_keepalive_count_max": { 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.813 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "ssh_keepalive_interval": { 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssh_keepalive_interval", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "7", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.814 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "use_agent": { 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "use_agent", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.815 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "use_repo_digest": { 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "use_repo_digest", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.816 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_on_failed_host_check": { 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_on_failed_host_check", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.817 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_on_stray_daemons": { 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_on_stray_daemons", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.818 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_on_stray_hosts": { 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_on_stray_hosts", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.819 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crash", 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.820 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.821 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.822 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.823 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.824 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "retain_interval": { 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "retain_interval", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "31536000", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.825 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.826 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_recent_interval": { 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_recent_interval", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "1209600", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.827 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "dashboard", 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T07:50:34.828 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.829 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.830 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "AUDIT_API_ENABLED": { 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.831 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.832 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.833 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.834 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.835 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.836 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.837 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.838 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.839 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.840 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.841 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "admin", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.842 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.843 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_URL": { 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_URL", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.844 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "admin", 2024-09-21T07:50:34.845 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.846 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.847 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.848 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.849 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.850 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "[]", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T07:50:34.851 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "{}", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T07:50:34.852 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T07:50:34.853 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T07:50:34.854 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.855 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.856 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.857 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.858 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.859 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.860 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.861 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.862 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.863 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.864 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.865 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "45", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.866 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.867 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "admin", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.868 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.869 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.870 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.871 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:34.872 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.873 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "cross_origin_url": { 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cross_origin_url", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.874 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "crt_file": { 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "crt_file", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.875 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "debug": { 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "debug", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.876 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Enable/disable debug options", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "jwt_token_ttl": { 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "jwt_token_ttl", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.877 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "28800", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "key_file": { 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "key_file", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.878 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.879 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.880 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.881 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.882 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.883 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "motd": { 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "motd", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.884 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "The message of the day", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "redirect_resolve_ip_addr": { 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.885 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "server_addr": { 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_addr", 2024-09-21T07:50:34.886 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "::", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "server_port": { 2024-09-21T07:50:34.887 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_port", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8080", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.888 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.889 INFO:teuthology.orchestra.run.smithi088.stdout: "ssl": { 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssl", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.890 INFO:teuthology.orchestra.run.smithi088.stdout: "ssl_server_port": { 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssl_server_port", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8443", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.891 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "sso_oauth2": { 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sso_oauth2", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.892 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_behaviour": { 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_behaviour", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "redirect", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "redirect" 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.893 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_error_status_code": { 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_error_status_code", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "500", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "400", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "599", 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.894 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "url_prefix": { 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "url_prefix", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.895 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "devicehealth", 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "enable_monitoring": { 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enable_monitoring", 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.896 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "monitor device health metrics", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.897 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.898 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.899 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.900 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.901 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "mark_out_threshold": { 2024-09-21T07:50:34.902 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mark_out_threshold", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "2419200", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.903 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_name": { 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pool_name", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "device_health_metrics", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.904 INFO:teuthology.orchestra.run.smithi088.stdout: "retention_period": { 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "retention_period", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "15552000", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how long to retain device health metrics", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.905 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "scrape_frequency": { 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "scrape_frequency", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "86400", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.906 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "self_heal": { 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "self_heal", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.907 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.908 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.909 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "warn_threshold": { 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "warn_threshold", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "7257600", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T07:50:34.910 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "diskprediction_local", 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.911 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.912 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.913 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.914 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.915 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.916 INFO:teuthology.orchestra.run.smithi088.stdout: "predict_interval": { 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "predict_interval", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "86400", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.917 INFO:teuthology.orchestra.run.smithi088.stdout: "predictor_model": { 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "predictor_model", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "prophetstor", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.918 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "600", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.919 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.920 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "influx", 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": false, 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "influxdb python module not found", 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "batch_size": { 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "batch_size", 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.921 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5000", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "database": { 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "database", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.922 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "ceph", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "hostname": { 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "hostname", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.923 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "InfluxDB server hostname", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.924 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "30", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "5", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.925 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.926 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.927 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.928 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.929 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "password": { 2024-09-21T07:50:34.930 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "password", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "password of InfluxDB server user", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.931 INFO:teuthology.orchestra.run.smithi088.stdout: "port": { 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "port", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "8086", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "InfluxDB server port", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.932 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.933 INFO:teuthology.orchestra.run.smithi088.stdout: "ssl": { 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "ssl", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "false", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.934 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "threads": { 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "threads", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "32", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.935 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "username": { 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "username", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "username of InfluxDB server user", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.936 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "verify_ssl": { 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "verify_ssl", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "true", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.937 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "insights", 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.938 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.939 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.940 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.941 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.942 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.943 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.944 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "iostat", 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.945 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.946 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.947 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.948 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.949 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.950 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.951 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "localpool", 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.952 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "failure_domain": { 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "failure_domain", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "host", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "failure domain for any created local pool", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.953 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.954 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.955 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.956 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.957 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.958 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "min_size": { 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "min_size", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.959 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "default min_size for any created local pool", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "num_rep": { 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "num_rep", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.960 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "default replica count for any created local pool", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_num": { 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pg_num", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.961 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "128", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "prefix": { 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "prefix", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.962 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "name prefix for any created local pool", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.963 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "subtree": { 2024-09-21T07:50:34.964 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "subtree", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "rack", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.965 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mds_autoscaler", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.966 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.967 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.968 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.969 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.970 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.971 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mirroring", 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.972 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.973 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.974 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.975 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.976 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.977 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.978 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "nfs", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.979 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.980 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.981 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.982 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.983 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.984 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "orchestrator", 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.985 INFO:teuthology.orchestra.run.smithi088.stdout: "fail_fs": { 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "fail_fs", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.986 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.987 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.988 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.989 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.990 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator": { 2024-09-21T07:50:34.991 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "orchestrator", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "cephadm", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "rook", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "test_orchestrator" 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Orchestrator backend", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.992 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.993 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "osd_perf_query", 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.994 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.995 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.996 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:34.997 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:34.998 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.000 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "osd_support", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.001 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.002 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.003 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.004 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.005 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.006 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "pg_autoscaler", 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.007 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.008 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.009 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.010 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.011 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.012 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.013 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.014 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "threshold": { 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "threshold", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "3.0", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1.0", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "scaling threshold", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.015 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "progress", 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "allow_pg_recovery_event": { 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "allow_pg_recovery_event", 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.016 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": { 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enabled", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.017 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.018 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.019 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.020 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.021 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.022 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "max_completed_events": { 2024-09-21T07:50:35.023 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_completed_events", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "50", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "number of past completed events to remember", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "sleep_interval": { 2024-09-21T07:50:35.024 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sleep_interval", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "how long the module is going to sleep", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.025 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.026 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "prometheus", 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "cache": { 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cache", 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.027 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "exclude_perf_counters": { 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "exclude_perf_counters", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.028 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.029 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.030 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.031 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.032 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.033 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_stats_pools": { 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_stats_pools", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.034 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.035 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "300", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "scrape_interval": { 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "scrape_interval", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.036 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "15.0", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "server_addr": { 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_addr", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.037 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "::", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "server_port": { 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_port", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.038 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "9283", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.039 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "stale_cache_strategy": { 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "stale_cache_strategy", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.040 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "log", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_behaviour": { 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_behaviour", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "default", 2024-09-21T07:50:35.041 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "default", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "error" 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "standby_error_status_code": { 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "standby_error_status_code", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.042 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "500", 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "400", 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "599", 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_support", 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.043 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.044 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.045 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.046 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.047 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "max_concurrent_snap_create": { 2024-09-21T07:50:35.048 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_concurrent_snap_create", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "mirror_snapshot_schedule": { 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T07:50:35.049 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.050 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "trash_purge_schedule": { 2024-09-21T07:50:35.051 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "trash_purge_schedule", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.052 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "restful", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "enable_auth": { 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enable_auth", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.053 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "key_file": { 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "key_file", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.055 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.056 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.057 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.058 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.059 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "max_requests": { 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_requests", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "500", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.060 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "server_addr": { 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_addr", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.061 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "server_port": { 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "server_port", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.062 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.063 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rgw", 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.064 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.065 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.066 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.067 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.068 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.069 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "5", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.070 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rook", 2024-09-21T07:50:35.071 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.072 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.073 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.074 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.075 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.076 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.077 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "storage_class": { 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "storage_class", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "local", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.078 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "selftest", 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.079 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.080 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.081 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.082 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.083 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.084 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "roption1": { 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "roption1", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.085 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "roption2": { 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "roption2", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "xyz", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.086 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption1": { 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption1", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.087 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption2": { 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption2", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.088 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption3": { 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption3", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "float", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.089 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption4": { 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption4", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.090 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption5": { 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption5", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.091 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption6": { 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption6", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.092 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "rwoption7": { 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rwoption7", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "1", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "42", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.093 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.094 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "testkey": { 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "testkey", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.095 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "testlkey": { 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "testlkey", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.096 INFO:teuthology.orchestra.run.smithi088.stdout: "testnewline": { 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "testnewline", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.097 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "smb", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "internal_store_backend": { 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "internal_store_backend", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.098 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.099 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.100 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.101 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.102 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.103 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.104 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "update_orchestration": { 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "update_orchestration", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.105 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "snap_schedule", 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "allow_m_granularity": { 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "allow_m_granularity", 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.106 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "dump_on_update": { 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "dump_on_update", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.107 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "dump database to debug log on update", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.108 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.109 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.110 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.111 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.112 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.113 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "stats", 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.114 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.115 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.116 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.117 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.118 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.119 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.120 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "status", 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.121 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.122 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.123 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.124 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.125 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.126 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "telegraf", 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.127 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "address": { 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "address", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.128 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "15", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.129 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.130 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.131 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.132 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.133 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.134 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.135 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "telemetry", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_basic": { 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_basic", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.136 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_crash": { 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_crash", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T07:50:35.137 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_device": { 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_device", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.138 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_ident": { 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_ident", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T07:50:35.139 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "channel_perf": { 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "channel_perf", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.140 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "contact": { 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "contact", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.141 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "description": { 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "description", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.142 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "device_url": { 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "device_url", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.143 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "enabled": { 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "enabled", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:35.144 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "24", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "8", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "last_opt_revision": { 2024-09-21T07:50:35.145 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "last_opt_revision", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "1", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "leaderboard": { 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "leaderboard", 2024-09-21T07:50:35.146 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "leaderboard_description": { 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "leaderboard_description", 2024-09-21T07:50:35.147 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.148 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.149 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.150 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.151 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.152 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "organization": { 2024-09-21T07:50:35.153 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "organization", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "proxy": { 2024-09-21T07:50:35.154 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "proxy", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.155 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "url": { 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "url", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.156 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "test_orchestrator", 2024-09-21T07:50:35.157 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.158 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.159 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.160 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.161 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.162 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.163 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.164 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.165 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.166 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.167 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.168 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "max_concurrent_clones": { 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "max_concurrent_clones", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "4", 2024-09-21T07:50:35.169 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "periodic_async_work": { 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "periodic_async_work", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.170 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Periodically check for async work", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot_clone_delay": { 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "snapshot_clone_delay", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.171 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot_clone_no_wait": { 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.172 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "True", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.173 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix", 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "can_run": true, 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "error_string": "", 2024-09-21T07:50:35.174 INFO:teuthology.orchestra.run.smithi088.stdout: "module_options": { 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "discovery_interval": { 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "discovery_interval", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "uint", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "100", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.175 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "identifier": { 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "identifier", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.176 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "interval": { 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "interval", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "secs", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "60", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.177 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "log_level": { 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_level", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.178 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster": { 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster", 2024-09-21T07:50:35.179 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_cluster_level": { 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_cluster_level", 2024-09-21T07:50:35.180 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "info", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [ 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "critical", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "debug", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "error", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "info", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "warning" 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.181 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "log_to_file": { 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "log_to_file", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "bool", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "False", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.182 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "sqlite3_killpoint": { 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "sqlite3_killpoint", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "dev", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 1, 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "0", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.183 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "zabbix_host": { 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix_host", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.184 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "zabbix_port": { 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix_port", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "int", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "10051", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.185 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "zabbix_sender": { 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "zabbix_sender", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "str", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "level": "advanced", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "flags": 0, 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "min": "", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "max": "", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "enum_allowed": [], 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "desc": "", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "long_desc": "", 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "tags": [], 2024-09-21T07:50:35.186 INFO:teuthology.orchestra.run.smithi088.stdout: "see_also": [] 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "services": {}, 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "always_on_modules": { 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "octopus": [ 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "balancer", 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "devicehealth", 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator", 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_autoscaler", 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "progress", 2024-09-21T07:50:35.187 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_support", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "status", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "telemetry", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "volumes" 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "pacific": [ 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "balancer", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "devicehealth", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_autoscaler", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "progress", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_support", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "status", 2024-09-21T07:50:35.188 INFO:teuthology.orchestra.run.smithi088.stdout: "telemetry", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "volumes" 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "quincy": [ 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "balancer", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "devicehealth", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_autoscaler", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "progress", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_support", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "status", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "telemetry", 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "volumes" 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "reef": [ 2024-09-21T07:50:35.189 INFO:teuthology.orchestra.run.smithi088.stdout: "balancer", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "devicehealth", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_autoscaler", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "progress", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_support", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "status", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "telemetry", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "volumes" 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "squid": [ 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "balancer", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "crash", 2024-09-21T07:50:35.190 INFO:teuthology.orchestra.run.smithi088.stdout: "devicehealth", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "orchestrator", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "pg_autoscaler", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "progress", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "rbd_support", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "status", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "telemetry", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "volumes" 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "last_failure_osd_epoch": 0, 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "active_clients": [ 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "devicehealth", 2024-09-21T07:50:35.191 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3741301681 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "libcephsqlite", 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.192 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1554442378 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "rbd_support", 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2139739944 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.193 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1671168534 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.194 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2524320784 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 1647382363 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.195 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3845646698 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.196 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 569875066 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 3468185230 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:35.197 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "volumes", 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: "addrvec": [ 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "v2", 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: "addr": "172.21.15.175:0", 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: "nonce": 2931692545 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:35.198 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:50:35.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.235+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd46c078f20 msgr2=0x7fd46c07b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.235+0000 7fd490800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd46c078f20 0x7fd46c07b430 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fd47c009cd0 tx=0x7fd47c005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.235+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 msgr2=0x7fd488158360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.235+0000 7fd490800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 0x7fd488158360 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fd48402b660 tx=0x7fd484004a10 comp rx=0 tx=0).stop 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 shutdown_connections 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7fd46c078f20 0x7fd46c07b430 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4881f6760 0x7fd4881f3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4881f3ca0 0x7fd488158360 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 --2- 172.21.15.88:0/3262689408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd470004d40 0x7fd4881f43e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 >> 172.21.15.88:0/3262689408 conn(0x7fd48806d730 msgr2=0x7fd488160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 shutdown_connections 2024-09-21T07:50:35.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:34.236+0000 7fd490800640 1 -- 172.21.15.88:0/3262689408 wait complete. 2024-09-21T07:50:35.205 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr fail x 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 -- 172.21.15.88:0/3116434605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180705b0 msgr2=0x7f4b180709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 --2- 172.21.15.88:0/3116434605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180705b0 0x7f4b180709c0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009830 tx=0x7f4b0402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 -- 172.21.15.88:0/3116434605 shutdown_connections 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 --2- 172.21.15.88:0/3116434605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180705b0 0x7f4b180709c0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 --2- 172.21.15.88:0/3116434605 >> v2:172.21.15.175:3300/0 conn(0x7f4b180cfc80 0x7f4b18156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 -- 172.21.15.88:0/3116434605 >> 172.21.15.88:0/3116434605 conn(0x7f4b1806d3d0 msgr2=0x7f4b180b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 -- 172.21.15.88:0/3116434605 shutdown_connections 2024-09-21T07:50:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.424+0000 7f4b1f800640 1 -- 172.21.15.88:0/3116434605 wait complete. 2024-09-21T07:50:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.427+0000 7f4b1f800640 1 Processor -- start 2024-09-21T07:50:35.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.427+0000 7f4b1f800640 1 -- start start 2024-09-21T07:50:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b1f800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b180705b0 0x7f4b180c9e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b1f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 0x7f4b18155850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b1f800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b1815ce30 0x7f4b180c7f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b1f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b180cced0 con 0x7f4b180cfc80 2024-09-21T07:50:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b1f800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4b180d0170 con 0x7f4b180705b0 2024-09-21T07:50:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b1f800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4b180cd050 con 0x7f4b1815ce30 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.428+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 0x7f4b18155850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 0x7f4b18155850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56012/0 (socket says 172.21.15.88:56012) 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 -- 172.21.15.88:0/2646347531 learned_addr learned my addr 172.21.15.88:0/2646347531 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b1ce00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b180705b0 0x7f4b180c9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b1d800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b1815ce30 0x7f4b180c7f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 -- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b1815ce30 msgr2=0x7f4b180c7f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b1815ce30 0x7f4b180c7f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 -- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b180705b0 msgr2=0x7f4b180c9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b180705b0 0x7f4b180c9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 -- 172.21.15.88:0/2646347531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b180c6860 con 0x7f4b180cfc80 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b1d800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b1815ce30 0x7f4b180c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b17e00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 0x7f4b18155850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009950 tx=0x7f4b04033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.429+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04005c90 con 0x7f4b180cfc80 2024-09-21T07:50:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.430+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4b04031e50 con 0x7f4b180cfc80 2024-09-21T07:50:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.430+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b04005260 con 0x7f4b180cfc80 2024-09-21T07:50:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.430+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b18155d90 con 0x7f4b180cfc80 2024-09-21T07:50:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.430+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b180c6db0 con 0x7f4b180cfc80 2024-09-21T07:50:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.431+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b180ca3f0 con 0x7f4b180cfc80 2024-09-21T07:50:35.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.432+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 12) ==== 101798+0+0 (secure 0 0 0) 0x7f4b04005400 con 0x7f4b180cfc80 2024-09-21T07:50:35.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.433+0000 7f4b14c00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078fb0 0x7f4af807b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.433+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(73..73 src has 1..73) ==== 5709+0+0 (secure 0 0 0) 0x7f4b040bc610 con 0x7f4b180cfc80 2024-09-21T07:50:35.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.434+0000 7f4b1ce00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078fb0 0x7f4af807b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.434+0000 7f4b1ce00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078fb0 0x7f4af807b4c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b1809e510 tx=0x7f4b08005f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:35.439 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.437+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4b0403f030 con 0x7f4b180cfc80 2024-09-21T07:50:35.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.673+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "mgr fail", "who": "x"} v 0) -- 0x7f4b180cd5b0 con 0x7f4b180cfc80 2024-09-21T07:50:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.677+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "mgr fail", "who": "x"}]=0 v13) ==== 68+0+0 (secure 0 0 0) 0x7f4b040843b0 con 0x7f4b180cfc80 2024-09-21T07:50:35.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.680+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mgrmap(e 13) ==== 50742+0+0 (secure 0 0 0) 0x7f4b0407ff20 con 0x7f4b180cfc80 2024-09-21T07:50:35.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.680+0000 7f4b14c00640 1 -- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078fb0 msgr2=0x7f4af807b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.680+0000 7f4b14c00640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078fb0 0x7f4af807b4c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f4b1809e510 tx=0x7f4b08005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 msgr2=0x7f4b18155850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 0x7f4b18155850 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f4b04009950 tx=0x7f4b04033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 shutdown_connections 2024-09-21T07:50:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:6800/3294411971,v1:172.21.15.175:6801/3294411971] conn(0x7f4af8078fb0 0x7f4af807b4c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4b1815ce30 0x7f4b180c7f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b180cfc80 0x7f4b18155850 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 --2- 172.21.15.88:0/2646347531 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4b180705b0 0x7f4b180c9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 >> 172.21.15.88:0/2646347531 conn(0x7f4b1806d3d0 msgr2=0x7f4b180ad2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.681+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 shutdown_connections 2024-09-21T07:50:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.682+0000 7f4b1f800640 1 -- 172.21.15.88:0/2646347531 wait complete. 2024-09-21T07:50:35.729 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:35.819 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.817+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.820 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.817+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.820 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.817+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.820 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.817+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.820 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.817+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.821 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.819+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.821 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.819+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.821 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.819+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.821 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.819+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.821 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:50:35.819+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:50:35.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 -- 172.21.15.88:0/3849002538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3818165480 msgr2=0x7f38181678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 --2- 172.21.15.88:0/3849002538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3818165480 0x7f38181678a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f3810009be0 tx=0x7f381002a890 comp rx=0 tx=0).stop 2024-09-21T07:50:35.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 -- 172.21.15.88:0/3849002538 shutdown_connections 2024-09-21T07:50:35.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 --2- 172.21.15.88:0/3849002538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3818165480 0x7f38181678a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 -- 172.21.15.88:0/3849002538 >> 172.21.15.88:0/3849002538 conn(0x7f381806d730 msgr2=0x7f38180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:35.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 -- 172.21.15.88:0/3849002538 shutdown_connections 2024-09-21T07:50:35.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.980+0000 7f381ce00640 1 -- 172.21.15.88:0/3849002538 wait complete. 2024-09-21T07:50:35.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.982+0000 7f381ce00640 1 Processor -- start 2024-09-21T07:50:35.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.982+0000 7f381ce00640 1 -- start start 2024-09-21T07:50:35.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f381ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 0x7f381814a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f381ce00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381815d760 0x7f38180d0270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f381ce00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818165480 0x7f38180d07b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f381ce00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3818167de0 con 0x7f38180ad0e0 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f381ce00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38180b4e30 con 0x7f3818165480 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f381ce00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3818167f60 con 0x7f381815d760 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f3815e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 0x7f381814a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f3815e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 0x7f381814a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56314/0 (socket says 172.21.15.88:56314) 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f3815e00640 1 -- 172.21.15.88:0/2885745536 learned_addr learned my addr 172.21.15.88:0/2885745536 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f3815400640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381815d760 0x7f38180d0270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.984+0000 7f3816800640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818165480 0x7f38180d07b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3815e00640 1 -- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381815d760 msgr2=0x7f38180d0270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3815e00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381815d760 0x7f38180d0270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3815e00640 1 -- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818165480 msgr2=0x7f38180d07b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:35.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3815e00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818165480 0x7f38180d07b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3815e00640 1 -- 172.21.15.88:0/2885745536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38180cf130 con 0x7f38180ad0e0 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3816800640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818165480 0x7f38180d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f3815e00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 0x7f381814a170 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f380800c3c0 tx=0x7f380800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808017070 con 0x7f38180ad0e0 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38080035f0 con 0x7f38180ad0e0 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38180ceb70 con 0x7f38180ad0e0 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808003a80 con 0x7f38180ad0e0 2024-09-21T07:50:35.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.985+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f381814adc0 con 0x7f38180ad0e0 2024-09-21T07:50:35.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.986+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38181535c0 con 0x7f38180ad0e0 2024-09-21T07:50:35.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.987+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 13) ==== 50742+0+0 (secure 0 0 0) 0x7f380801a070 con 0x7f38180ad0e0 2024-09-21T07:50:35.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.987+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f38080527a0 con 0x7f38180ad0e0 2024-09-21T07:50:35.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:35.992+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f380801ad70 con 0x7f38180ad0e0 2024-09-21T07:50:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.218+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f381814f1a0 con 0x7f38180ad0e0 2024-09-21T07:50:36.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.218+0000 7f380e000640 1 -- 172.21.15.88:0/2885745536 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f3808054cf0 con 0x7f38180ad0e0 2024-09-21T07:50:36.221 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:36.221 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:36.221 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:36.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 msgr2=0x7f381814a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:36.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 0x7f381814a170 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f380800c3c0 tx=0x7f380800c880 comp rx=0 tx=0).stop 2024-09-21T07:50:36.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 shutdown_connections 2024-09-21T07:50:36.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f381815d760 0x7f38180d0270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3818165480 0x7f38180d07b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 --2- 172.21.15.88:0/2885745536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38180ad0e0 0x7f381814a170 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:36.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 >> 172.21.15.88:0/2885745536 conn(0x7f381806d730 msgr2=0x7f3818068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:36.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 shutdown_connections 2024-09-21T07:50:36.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:36.221+0000 7f381ce00640 1 -- 172.21.15.88:0/2885745536 wait complete. 2024-09-21T07:50:36.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.c.asok --format=json session ls 2024-09-21T07:50:36.488 INFO:teuthology.orchestra.run.smithi088.stdout:[{"id":6438,"entity":{"name":{"type":"client","num":6438},"addr":{"type":"any","addr":"172.21.15.88:0","nonce":2542555506}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.64498746200000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9970425079195109,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.6438 172.21.15.88:0/2542555506","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"y","hostname":"smithi088","pid":"35875","root":"/"}},{"id":5738,"entity":{"name":{"type":"client","num":5738},"addr":{"type":"any","addr":"172.21.15.88:0","nonce":1238326277}},"state":"open","num_leases":0,"num_caps":2,"request_load_avg":2,"uptime":0.64498746200000001,"requests_in_flight":0,"num_completed_requests":0,"num_completed_flushes":0,"reconnecting":false,"recall_caps":{"value":0,"halflife":60},"release_caps":{"value":0,"halflife":60},"recall_caps_throttle":{"value":0,"halflife":1.5},"recall_caps_throttle2o":{"value":0,"halflife":0.5},"session_cache_liveness":{"value":1.9970425079195109,"halflife":300},"cap_acquisition":{"value":0,"halflife":30},"last_trim_completed_requests_tid":2,"last_trim_completed_flushes_tid":0,"delegated_inos":[],"inst":"client.5738 172.21.15.88:0/1238326277","completed_requests":[],"prealloc_inos":[],"client_metadata":{"client_features":{"feature_bits":"0x00000000003fffff"},"metric_spec":{"metric_flags":{"feature_bits":"0x000000000000ffff"}},"ceph_sha1":"1843221895adccfb3b692060251dd6a0183bc410","ceph_version":"ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)","entity_id":"y","hostname":"smithi088","pid":"35875","root":"/"}}] 2024-09-21T07:50:36.489 DEBUG:tasks.cephfs.filesystem:_json_asok output [ { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi088", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35875", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.88:0", "nonce": 2542555506, "type": "any" }, "name": { "num": 6438, "type": "client" } }, "id": 6438, "inst": "client.6438 172.21.15.88:0/2542555506", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9970425079195109 }, "state": "open", "uptime": 0.644987462 }, { "cap_acquisition": { "halflife": 30, "value": 0 }, "client_metadata": { "ceph_sha1": "1843221895adccfb3b692060251dd6a0183bc410", "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", "client_features": { "feature_bits": "0x00000000003fffff" }, "entity_id": "y", "hostname": "smithi088", "metric_spec": { "metric_flags": { "feature_bits": "0x000000000000ffff" } }, "pid": "35875", "root": "/" }, "completed_requests": [], "delegated_inos": [], "entity": { "addr": { "addr": "172.21.15.88:0", "nonce": 1238326277, "type": "any" }, "name": { "num": 5738, "type": "client" } }, "id": 5738, "inst": "client.5738 172.21.15.88:0/1238326277", "last_trim_completed_flushes_tid": 0, "last_trim_completed_requests_tid": 2, "num_caps": 2, "num_completed_flushes": 0, "num_completed_requests": 0, "num_leases": 0, "prealloc_inos": [], "recall_caps": { "halflife": 60, "value": 0 }, "recall_caps_throttle": { "halflife": 1.5, "value": 0 }, "recall_caps_throttle2o": { "halflife": 0.5, "value": 0 }, "reconnecting": false, "release_caps": { "halflife": 60, "value": 0 }, "request_load_avg": 2, "requests_in_flight": 0, "session_cache_liveness": { "halflife": 300, "value": 1.9970425079195109 }, "state": "open", "uptime": 0.644987462 } ] 2024-09-21T07:50:36.489 DEBUG:tasks.ceph_test_case:wait_until_true: success in 0s and 0 retries 2024-09-21T07:50:36.489 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:50:36.489 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:36.578 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:36.669 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:50:36.669 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:36.670 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:36.670 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:36.710 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:36.710 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:36.710 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:36.710 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:36.746 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:36.746 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:36.746 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:36.746 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:36.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:36.786 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:50:36.786 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:36.786 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:36.822 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:36.822 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:37.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 -- 172.21.15.88:0/2636001642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 msgr2=0x7f1084004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 --2- 172.21.15.88:0/2636001642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f1084004c10 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f1088001800 tx=0x7f1084005830 comp rx=0 tx=0).stop 2024-09-21T07:50:37.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 -- 172.21.15.88:0/2636001642 shutdown_connections 2024-09-21T07:50:37.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 --2- 172.21.15.88:0/2636001642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f1084004c10 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 -- 172.21.15.88:0/2636001642 >> 172.21.15.88:0/2636001642 conn(0x7f10a006d730 msgr2=0x7f10a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 -- 172.21.15.88:0/2636001642 shutdown_connections 2024-09-21T07:50:37.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.046+0000 7f10a5800640 1 -- 172.21.15.88:0/2636001642 wait complete. 2024-09-21T07:50:37.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.048+0000 7f10a5800640 1 Processor -- start 2024-09-21T07:50:37.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.049+0000 7f10a5800640 1 -- start start 2024-09-21T07:50:37.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f10a5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f10a014acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f10a5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a00b1500 0x7f10a014b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f10a5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a01f9000 0x7f10a01f9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f10a5800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10a00ad4f0 con 0x7f1084004800 2024-09-21T07:50:37.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f10a5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10a00b4e30 con 0x7f10a00b1500 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f10a5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10a00ad670 con 0x7f10a01f9000 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f109ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f10a014acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f109ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f10a014acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56334/0 (socket says 172.21.15.88:56334) 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f109ea00640 1 -- 172.21.15.88:0/3074277724 learned_addr learned my addr 172.21.15.88:0/3074277724 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f1096000640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a00b1500 0x7f10a014b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.050+0000 7f109f400640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a01f9000 0x7f10a01f9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f109ea00640 1 -- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a01f9000 msgr2=0x7f10a01f9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f109ea00640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a01f9000 0x7f10a01f9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f109ea00640 1 -- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a00b1500 msgr2=0x7f10a014b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f109ea00640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a00b1500 0x7f10a014b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f109ea00640 1 -- 172.21.15.88:0/3074277724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a015c1a0 con 0x7f1084004800 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f109ea00640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f10a014acb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1090006630 tx=0x7f10900039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090015070 con 0x7f1084004800 2024-09-21T07:50:37.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1090011920 con 0x7f1084004800 2024-09-21T07:50:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1090011d70 con 0x7f1084004800 2024-09-21T07:50:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a015c480 con 0x7f1084004800 2024-09-21T07:50:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.051+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a0073300 con 0x7f1084004800 2024-09-21T07:50:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.052+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a0050f60 con 0x7f1084004800 2024-09-21T07:50:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.053+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 14) ==== 50869+0+0 (secure 0 0 0) 0x7f1090018070 con 0x7f1084004800 2024-09-21T07:50:37.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.053+0000 7f1097400640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f107c03e6a0 0x7f107c040bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.053+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f1090054670 con 0x7f1084004800 2024-09-21T07:50:37.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.053+0000 7f1096000640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f107c03e6a0 0x7f107c040bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.054+0000 7f1096000640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f107c03e6a0 0x7f107c040bb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1088032000 tx=0x7f1088001360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.058+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10a0050f60 con 0x7f1084004800 2024-09-21T07:50:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.281+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f10a01535c0 con 0x7f1084004800 2024-09-21T07:50:37.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.281+0000 7f1097400640 1 -- 172.21.15.88:0/3074277724 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f109001bc00 con 0x7f1084004800 2024-09-21T07:50:37.284 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:37.284 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:37.284 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:37.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f107c03e6a0 msgr2=0x7f107c040bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f107c03e6a0 0x7f107c040bb0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1088032000 tx=0x7f1088001360 comp rx=0 tx=0).stop 2024-09-21T07:50:37.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 msgr2=0x7f10a014acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f10a014acb0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f1090006630 tx=0x7f10900039c0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 shutdown_connections 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f107c03e6a0 0x7f107c040bb0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10a01f9000 0x7f10a01f9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10a00b1500 0x7f10a014b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 --2- 172.21.15.88:0/3074277724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1084004800 0x7f10a014acb0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 >> 172.21.15.88:0/3074277724 conn(0x7f10a006d730 msgr2=0x7f10a01527f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 shutdown_connections 2024-09-21T07:50:37.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.284+0000 7f10a5800640 1 -- 172.21.15.88:0/3074277724 wait complete. 2024-09-21T07:50:37.308 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 -- 172.21.15.88:0/4066251028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e80ad0e0 msgr2=0x7f29e80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 --2- 172.21.15.88:0/4066251028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e80ad0e0 0x7f29e80ad4f0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f29d4009830 tx=0x7f29d402a520 comp rx=0 tx=0).stop 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 -- 172.21.15.88:0/4066251028 shutdown_connections 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 --2- 172.21.15.88:0/4066251028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e80ad0e0 0x7f29e80ad4f0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 -- 172.21.15.88:0/4066251028 >> 172.21.15.88:0/4066251028 conn(0x7f29e806d730 msgr2=0x7f29e80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 -- 172.21.15.88:0/4066251028 shutdown_connections 2024-09-21T07:50:37.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.525+0000 7f29ed400640 1 -- 172.21.15.88:0/4066251028 wait complete. 2024-09-21T07:50:37.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.528+0000 7f29ed400640 1 Processor -- start 2024-09-21T07:50:37.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.528+0000 7f29ed400640 1 -- start start 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29ed400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29e80ad0e0 0x7f29e81f62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29ed400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e80b3550 0x7f29e81f3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29ed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 0x7f29e8156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29ed400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29e80b4da0 con 0x7f29e815d760 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29ed400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29e80b4c20 con 0x7f29e80b3550 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29ed400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29e80b4070 con 0x7f29e80ad0e0 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29e7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 0x7f29e8156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29e7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 0x7f29e8156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56348/0 (socket says 172.21.15.88:56348) 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29e7200640 1 -- 172.21.15.88:0/247827750 learned_addr learned my addr 172.21.15.88:0/247827750 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.529+0000 7f29e5e00640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e80b3550 0x7f29e81f3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e6800640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29e80ad0e0 0x7f29e81f62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e7200640 1 -- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29e80ad0e0 msgr2=0x7f29e81f62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e7200640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29e80ad0e0 0x7f29e81f62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e7200640 1 -- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e80b3550 msgr2=0x7f29e81f3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e7200640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e80b3550 0x7f29e81f3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e7200640 1 -- 172.21.15.88:0/247827750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29e809a790 con 0x7f29e815d760 2024-09-21T07:50:37.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29e7200640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 0x7f29e8156010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29e000c700 tx=0x7f29e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e0003b70 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29e0003d10 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29e000aab0 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29e809aa70 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.530+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29e809b030 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.531+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29e809a500 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.532+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 14) ==== 50869+0+0 (secure 0 0 0) 0x7f29e000ac50 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.532+0000 7f29dea00640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29c803e670 0x7f29c8040b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.532+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f29e0053520 con 0x7f29e815d760 2024-09-21T07:50:37.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.533+0000 7f29e6800640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29c803e670 0x7f29c8040b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:37.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.533+0000 7f29e6800640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29c803e670 0x7f29c8040b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f29d80046b0 tx=0x7f29d8007e90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:37.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.537+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f29e000ce10 con 0x7f29e815d760 2024-09-21T07:50:37.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.691+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f29e0019d10 con 0x7f29e815d760 2024-09-21T07:50:37.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.762+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f29e80ad4f0 con 0x7f29e815d760 2024-09-21T07:50:37.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.763+0000 7f29dea00640 1 -- 172.21.15.88:0/247827750 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f29e001f160 con 0x7f29e815d760 2024-09-21T07:50:37.765 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:37.765 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:37.765 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:37.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29c803e670 msgr2=0x7f29c8040b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29c803e670 0x7f29c8040b80 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f29d80046b0 tx=0x7f29d8007e90 comp rx=0 tx=0).stop 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 msgr2=0x7f29e8156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 0x7f29e8156010 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f29e000c700 tx=0x7f29e000cbc0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 shutdown_connections 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29c803e670 0x7f29c8040b80 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29e815d760 0x7f29e8156010 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f29e80b3550 0x7f29e81f3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 --2- 172.21.15.88:0/247827750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29e80ad0e0 0x7f29e81f62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 >> 172.21.15.88:0/247827750 conn(0x7f29e806d730 msgr2=0x7f29e80b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 shutdown_connections 2024-09-21T07:50:37.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:37.765+0000 7f29ed400640 1 -- 172.21.15.88:0/247827750 wait complete. 2024-09-21T07:50:37.797 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:38.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 -- 172.21.15.88:0/432100168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8280ad0e0 msgr2=0x7fb8280ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 --2- 172.21.15.88:0/432100168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8280ad0e0 0x7fb8280ad4f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb814009830 tx=0x7fb81402a480 comp rx=0 tx=0).stop 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 -- 172.21.15.88:0/432100168 shutdown_connections 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 --2- 172.21.15.88:0/432100168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8280ad0e0 0x7fb8280ad4f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 --2- 172.21.15.88:0/432100168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb82815d760 0x7fb8281535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 -- 172.21.15.88:0/432100168 >> 172.21.15.88:0/432100168 conn(0x7fb82806d730 msgr2=0x7fb8280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 -- 172.21.15.88:0/432100168 shutdown_connections 2024-09-21T07:50:38.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.017+0000 7fb82f400640 1 -- 172.21.15.88:0/432100168 wait complete. 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.019+0000 7fb82f400640 1 Processor -- start 2024-09-21T07:50:38.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.019+0000 7fb82f400640 1 -- start start 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8280ad0e0 0x7fb828159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 0x7fb82809dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb828160310 0x7fb82815baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb82806f220 con 0x7fb82815d760 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb82806f0a0 con 0x7fb828160310 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8280b4c20 con 0x7fb8280ad0e0 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 0x7fb82809dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 0x7fb82809dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56376/0 (socket says 172.21.15.88:56376) 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 -- 172.21.15.88:0/3998013452 learned_addr learned my addr 172.21.15.88:0/3998013452 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:38.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82ca00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8280ad0e0 0x7fb828159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb82d400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb828160310 0x7fb82815baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 -- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8280ad0e0 msgr2=0x7fb828159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8280ad0e0 0x7fb828159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 -- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb828160310 msgr2=0x7fb82815baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb828160310 0x7fb82815baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.021+0000 7fb827e00640 1 -- 172.21.15.88:0/3998013452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb828158960 con 0x7fb82815d760 2024-09-21T07:50:38.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.022+0000 7fb827e00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 0x7fb82809dbc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb814009950 tx=0x7fb814033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.022+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814005c90 con 0x7fb82815d760 2024-09-21T07:50:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.022+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb814031e50 con 0x7fb82815d760 2024-09-21T07:50:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.022+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb814005260 con 0x7fb82815d760 2024-09-21T07:50:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.022+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb828158c40 con 0x7fb82815d760 2024-09-21T07:50:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.022+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb828159200 con 0x7fb82815d760 2024-09-21T07:50:38.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.023+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb828160fa0 con 0x7fb82815d760 2024-09-21T07:50:38.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.024+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7fb814005400 con 0x7fb82815d760 2024-09-21T07:50:38.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.024+0000 7fb824c00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb80803e8b0 0x7fb808040dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.024+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7fb814074c80 con 0x7fb82815d760 2024-09-21T07:50:38.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.024+0000 7fb82ca00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb80803e8b0 0x7fb808040dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.025+0000 7fb82ca00640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb80803e8b0 0x7fb808040dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b30 tx=0x7fb81c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.029+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb81403f030 con 0x7fb82815d760 2024-09-21T07:50:38.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.214+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb82806e060 con 0x7fb82815d760 2024-09-21T07:50:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.216+0000 7fb824c00640 1 -- 172.21.15.88:0/3998013452 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17502 (secure 0 0 0) 0x7fb814033e10 con 0x7fb82815d760 2024-09-21T07:50:38.218 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:38.218 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":74,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:35.675032+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:16.939762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:17.507399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6800/3294411971":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3741301681":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3468185230":"2024-09-22T07:50:35.674965+0000","172.21.15.175:6801/3294411971":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1554442378":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/569875066":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1647382363":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2139739944":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/691722558":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1671168534":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2524320784":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3845646698":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2931692545":"2024-09-22T07:50:35.674965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:38.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb80803e8b0 msgr2=0x7fb808040dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb80803e8b0 0x7fb808040dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009b30 tx=0x7fb81c005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 msgr2=0x7fb82809dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 0x7fb82809dbc0 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fb814009950 tx=0x7fb814033c20 comp rx=0 tx=0).stop 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 shutdown_connections 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb80803e8b0 0x7fb808040dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb828160310 0x7fb82815baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb82815d760 0x7fb82809dbc0 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 --2- 172.21.15.88:0/3998013452 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8280ad0e0 0x7fb828159fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 >> 172.21.15.88:0/3998013452 conn(0x7fb82806d730 msgr2=0x7fb8280b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 shutdown_connections 2024-09-21T07:50:38.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.218+0000 7fb82f400640 1 -- 172.21.15.88:0/3998013452 wait complete. 2024-09-21T07:50:38.246 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:50:38.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:38.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 -- 172.21.15.88:0/2049161605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f132015f910 msgr2=0x7f132015bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 --2- 172.21.15.88:0/2049161605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f132015f910 0x7f132015bd90 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f1320072810 tx=0x7f131802a9a0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 -- 172.21.15.88:0/2049161605 shutdown_connections 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 --2- 172.21.15.88:0/2049161605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f132015f910 0x7f132015bd90 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 --2- 172.21.15.88:0/2049161605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f13200b2750 0x7f132015f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 --2- 172.21.15.88:0/2049161605 >> v2:172.21.15.175:3300/0 conn(0x7f13200aebc0 0x7f1320152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 -- 172.21.15.88:0/2049161605 >> 172.21.15.88:0/2049161605 conn(0x7f132006d730 msgr2=0x7f13200b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 -- 172.21.15.88:0/2049161605 shutdown_connections 2024-09-21T07:50:38.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.463+0000 7f1328200640 1 -- 172.21.15.88:0/2049161605 wait complete. 2024-09-21T07:50:38.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.465+0000 7f1328200640 1 Processor -- start 2024-09-21T07:50:38.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.466+0000 7f1328200640 1 -- start start 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1328200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 0x7f132014fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1328200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13200b2750 0x7f1320099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1328200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132015f910 0x7f13200cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1328200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f132015c660 con 0x7f13200aebc0 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1328200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f132015c4e0 con 0x7f13200b2750 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1328200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f132015c7e0 con 0x7f132015f910 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1325800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 0x7f132014fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1325800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 0x7f132014fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56400/0 (socket says 172.21.15.88:56400) 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1325800640 1 -- 172.21.15.88:0/423573764 learned_addr learned my addr 172.21.15.88:0/423573764 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:38.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1324e00640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13200b2750 0x7f1320099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.467+0000 7f1326200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132015f910 0x7f13200cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1325800640 1 -- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132015f910 msgr2=0x7f13200cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1325800640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132015f910 0x7f13200cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1325800640 1 -- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13200b2750 msgr2=0x7f1320099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1325800640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13200b2750 0x7f1320099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1325800640 1 -- 172.21.15.88:0/423573764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13201507c0 con 0x7f13200aebc0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1325800640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 0x7f132014fde0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f131400e710 tx=0x7f131400ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f131400ca50 con 0x7f13200aebc0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f131400cce0 con 0x7f13200aebc0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1314010690 con 0x7f13200aebc0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1320150aa0 con 0x7f13200aebc0 2024-09-21T07:50:38.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.468+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1320151060 con 0x7f13200aebc0 2024-09-21T07:50:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.469+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f132015f3d0 con 0x7f13200aebc0 2024-09-21T07:50:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.470+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f1314010a40 con 0x7f13200aebc0 2024-09-21T07:50:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.470+0000 7f131d600640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f130003e8e0 0x7f1300040df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.470+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f1314054800 con 0x7f13200aebc0 2024-09-21T07:50:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.470+0000 7f1324e00640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f130003e8e0 0x7f1300040df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.471+0000 7f1324e00640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f130003e8e0 0x7f1300040df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1320152470 tx=0x7f130c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.475+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1314020360 con 0x7f13200aebc0 2024-09-21T07:50:38.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.698+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f132015f670 con 0x7f13200aebc0 2024-09-21T07:50:38.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.698+0000 7f131d600640 1 -- 172.21.15.88:0/423573764 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f1314010d30 con 0x7f13200aebc0 2024-09-21T07:50:38.700 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:38.701 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f130003e8e0 msgr2=0x7f1300040df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f130003e8e0 0x7f1300040df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f1320152470 tx=0x7f130c005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 msgr2=0x7f132014fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 0x7f132014fde0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f131400e710 tx=0x7f131400ebd0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 shutdown_connections 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f130003e8e0 0x7f1300040df0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f132015f910 0x7f13200cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f13200b2750 0x7f1320099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 --2- 172.21.15.88:0/423573764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13200aebc0 0x7f132014fde0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 >> 172.21.15.88:0/423573764 conn(0x7f132006d730 msgr2=0x7f132015dcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 shutdown_connections 2024-09-21T07:50:38.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.701+0000 7f1328200640 1 -- 172.21.15.88:0/423573764 wait complete. 2024-09-21T07:50:38.725 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 -- 172.21.15.88:0/4173523942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45040ad0e0 msgr2=0x7f45040ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 --2- 172.21.15.88:0/4173523942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45040ad0e0 0x7f45040ad4f0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f44f0005fd0 tx=0x7f44f002a540 comp rx=0 tx=0).stop 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 -- 172.21.15.88:0/4173523942 shutdown_connections 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 --2- 172.21.15.88:0/4173523942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45040ad0e0 0x7f45040ad4f0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 --2- 172.21.15.88:0/4173523942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f450415d760 0x7f45041535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 -- 172.21.15.88:0/4173523942 >> 172.21.15.88:0/4173523942 conn(0x7f450406d730 msgr2=0x7f45040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 -- 172.21.15.88:0/4173523942 shutdown_connections 2024-09-21T07:50:38.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.944+0000 7f450c600640 1 -- 172.21.15.88:0/4173523942 wait complete. 2024-09-21T07:50:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.946+0000 7f450c600640 1 Processor -- start 2024-09-21T07:50:38.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.947+0000 7f450c600640 1 -- start start 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45040ad0e0 0x7f450414d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450c600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 0x7f45041f91a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4504160310 0x7f45041f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450c600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f450406f220 con 0x7f450415d760 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f450406f0a0 con 0x7f4504160310 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f45040b4c20 con 0x7f45040ad0e0 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f4509200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 0x7f45041f91a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f4509200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 0x7f45041f91a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56410/0 (socket says 172.21.15.88:56410) 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f4509200640 1 -- 172.21.15.88:0/1753058121 learned_addr learned my addr 172.21.15.88:0/1753058121 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:38.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f4509c00640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45040ad0e0 0x7f450414d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.948+0000 7f450a600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4504160310 0x7f45041f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4509200640 1 -- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45040ad0e0 msgr2=0x7f450414d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4509200640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45040ad0e0 0x7f450414d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4509200640 1 -- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4504160310 msgr2=0x7f45041f96e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:38.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4509200640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4504160310 0x7f45041f96e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:38.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4509200640 1 -- 172.21.15.88:0/1753058121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45040b1690 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4509200640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 0x7f45041f91a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f44f0000c00 tx=0x7f44f0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.949+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f003e030 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.950+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f44f0003e00 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.950+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44f0035040 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.950+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f450414e4d0 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.950+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f450414ea90 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.951+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f450414e240 con 0x7f450415d760 2024-09-21T07:50:38.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.951+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f44f0041030 con 0x7f450415d760 2024-09-21T07:50:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.951+0000 7f4501600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f44e403e8e0 0x7f44e4040df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.951+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f44f00743d0 con 0x7f450415d760 2024-09-21T07:50:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.952+0000 7f4509c00640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f44e403e8e0 0x7f44e4040df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:38.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.952+0000 7f4509c00640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f44e403e8e0 0x7f44e4040df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005fd0 tx=0x7f44f80095e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:38.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:38.957+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f44f004bd60 con 0x7f450415d760 2024-09-21T07:50:39.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.179+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f45040d0c30 con 0x7f450415d760 2024-09-21T07:50:39.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.180+0000 7f4501600640 1 -- 172.21.15.88:0/1753058121 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 47 v47) ==== 94+0+5231 (secure 0 0 0) 0x7f45040d0c30 con 0x7f450415d760 2024-09-21T07:50:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:39.182 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":47,"btime":"2024-09-21T07:50:29:429641+0000","default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:18.548042+0000","modified":"2024-09-21T07:50:21.559601+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6186},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6186":{"gid":6186,"name":"c","rank":0,"incarnation":42,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2332018233","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":2332018233},{"type":"v1","addr":"172.21.15.88:6835","nonce":2332018233}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6186,"qdb_cluster":[6186]},"id":5}]} 2024-09-21T07:50:39.183 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 47 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f44e403e8e0 msgr2=0x7f44e4040df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f44e403e8e0 0x7f44e4040df0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f44f8005fd0 tx=0x7f44f80095e0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 msgr2=0x7f45041f91a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 0x7f45041f91a0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f44f0000c00 tx=0x7f44f0003910 comp rx=0 tx=0).stop 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 shutdown_connections 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f44e403e8e0 0x7f44e4040df0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4504160310 0x7f45041f96e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f450415d760 0x7f45041f91a0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 --2- 172.21.15.88:0/1753058121 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45040ad0e0 0x7f450414d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.182+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 >> 172.21.15.88:0/1753058121 conn(0x7f450406d730 msgr2=0x7f45040cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.183+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 shutdown_connections 2024-09-21T07:50:39.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.183+0000 7f450c600640 1 -- 172.21.15.88:0/1753058121 wait complete. 2024-09-21T07:50:39.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 -- 172.21.15.88:0/3835439148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100ad0e0 msgr2=0x7f7f100ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 --2- 172.21.15.88:0/3835439148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100ad0e0 0x7f7f100ad4f0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f7efc005fd0 tx=0x7f7efc02a540 comp rx=0 tx=0).stop 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 -- 172.21.15.88:0/3835439148 shutdown_connections 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 --2- 172.21.15.88:0/3835439148 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f100b3550 0x7f7f100b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 --2- 172.21.15.88:0/3835439148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100ad0e0 0x7f7f100ad4f0 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 -- 172.21.15.88:0/3835439148 >> 172.21.15.88:0/3835439148 conn(0x7f7f1006d730 msgr2=0x7f7f100b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 -- 172.21.15.88:0/3835439148 shutdown_connections 2024-09-21T07:50:39.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.424+0000 7f7f17200640 1 -- 172.21.15.88:0/3835439148 wait complete. 2024-09-21T07:50:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.427+0000 7f7f17200640 1 Processor -- start 2024-09-21T07:50:39.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.427+0000 7f7f17200640 1 -- start start 2024-09-21T07:50:39.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f17200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f100ad0e0 0x7f7f10156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f17200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 0x7f7f101f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f17200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f1015d760 0x7f7f1015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f17200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f100b4da0 con 0x7f7f100b3550 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f17200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f100b4c20 con 0x7f7f1015d760 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f17200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7f100b3ea0 con 0x7f7f100ad0e0 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.428+0000 7f7f0f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 0x7f7f101f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 0x7f7f101f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56432/0 (socket says 172.21.15.88:56432) 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 -- 172.21.15.88:0/1873304076 learned_addr learned my addr 172.21.15.88:0/1873304076 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f15200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f1015d760 0x7f7f1015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0fe00640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f100ad0e0 0x7f7f10156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 -- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f100ad0e0 msgr2=0x7f7f10156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f100ad0e0 0x7f7f10156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 -- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f1015d760 msgr2=0x7f7f1015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f1015d760 0x7f7f1015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 -- 172.21.15.88:0/1873304076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1014f4b0 con 0x7f7f100b3550 2024-09-21T07:50:39.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.429+0000 7f7f0f400640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 0x7f7f101f5070 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7efc034040 tx=0x7f7efc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:39.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.430+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc03e030 con 0x7f7f100b3550 2024-09-21T07:50:39.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.430+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7efc003d90 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.430+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7efc035040 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.430+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1014f790 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.430+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1014fd50 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.431+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f100b4370 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.431+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 15) ==== 51077+0+0 (secure 0 0 0) 0x7f7efc041030 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.431+0000 7f7f07e00640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ef003e8e0 0x7f7ef0040df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.432+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f7efc074410 con 0x7f7f100b3550 2024-09-21T07:50:39.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.432+0000 7f7f0fe00640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ef003e8e0 0x7f7ef0040df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.432+0000 7f7f0fe00640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ef003e8e0 0x7f7ef0040df0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f00001720 tx=0x7f7f00009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:39.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.437+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7efc04bd60 con 0x7f7f100b3550 2024-09-21T07:50:39.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.623+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f10069700 con 0x7f7f100b3550 2024-09-21T07:50:39.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.624+0000 7f7f07e00640 1 -- 172.21.15.88:0/1873304076 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v74) ==== 74+0+17502 (secure 0 0 0) 0x7f7efc0740e0 con 0x7f7f100b3550 2024-09-21T07:50:39.627 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:39.627 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":74,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:35.675032+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:16.939762+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"72","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:17.507399+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"73","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6800/3294411971":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3741301681":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3468185230":"2024-09-22T07:50:35.674965+0000","172.21.15.175:6801/3294411971":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1554442378":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/569875066":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1647382363":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2139739944":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/691722558":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1671168534":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2524320784":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3845646698":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2931692545":"2024-09-22T07:50:35.674965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:39.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ef003e8e0 msgr2=0x7f7ef0040df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ef003e8e0 0x7f7ef0040df0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f7f00001720 tx=0x7f7f00009040 comp rx=0 tx=0).stop 2024-09-21T07:50:39.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 msgr2=0x7f7f101f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 0x7f7f101f5070 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f7efc034040 tx=0x7f7efc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 shutdown_connections 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ef003e8e0 0x7f7ef0040df0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f1015d760 0x7f7f1015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f100b3550 0x7f7f101f5070 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 --2- 172.21.15.88:0/1873304076 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f100ad0e0 0x7f7f10156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 >> 172.21.15.88:0/1873304076 conn(0x7f7f1006d730 msgr2=0x7f7f1015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 shutdown_connections 2024-09-21T07:50:39.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.627+0000 7f7f17200640 1 -- 172.21.15.88:0/1873304076 wait complete. 2024-09-21T07:50:39.650 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:50:39.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 -- 172.21.15.88:0/1440755823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f828815ce30 msgr2=0x7f828815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 --2- 172.21.15.88:0/1440755823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f828815ce30 0x7f828815d240 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f828806dd90 tx=0x7f828002a980 comp rx=0 tx=0).stop 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 -- 172.21.15.88:0/1440755823 shutdown_connections 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 --2- 172.21.15.88:0/1440755823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f828815ce30 0x7f828815d240 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 --2- 172.21.15.88:0/1440755823 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82880705b0 0x7f82880709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 --2- 172.21.15.88:0/1440755823 >> v2:172.21.15.175:3300/0 conn(0x7f82880cfc80 0x7f8288156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 -- 172.21.15.88:0/1440755823 >> 172.21.15.88:0/1440755823 conn(0x7f828806d3d0 msgr2=0x7f82880b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 -- 172.21.15.88:0/1440755823 shutdown_connections 2024-09-21T07:50:39.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.874+0000 7f828f600640 1 -- 172.21.15.88:0/1440755823 wait complete. 2024-09-21T07:50:39.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.876+0000 7f828f600640 1 Processor -- start 2024-09-21T07:50:39.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.877+0000 7f828f600640 1 -- start start 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828f600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 0x7f82881f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828f600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82880cfc80 0x7f82881f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828f600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828815ce30 0x7f828814bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828f600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82880cce40 con 0x7f82880705b0 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828f600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82880cccc0 con 0x7f828815ce30 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828f600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82880ccfc0 con 0x7f82880cfc80 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 0x7f82881f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 0x7f82881f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56452/0 (socket says 172.21.15.88:56452) 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f828cc00640 1 -- 172.21.15.88:0/595999485 learned_addr learned my addr 172.21.15.88:0/595999485 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:39.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.878+0000 7f8287e00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82880cfc80 0x7f82881f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828d600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828815ce30 0x7f828814bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828cc00640 1 -- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82880cfc80 msgr2=0x7f82881f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828cc00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82880cfc80 0x7f82881f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828cc00640 1 -- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828815ce30 msgr2=0x7f828814bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828cc00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828815ce30 0x7f828814bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828cc00640 1 -- 172.21.15.88:0/595999485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f828814f890 con 0x7f82880705b0 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828d600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828815ce30 0x7f828814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:39.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828cc00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 0x7f82881f5580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8278009fd0 tx=0x7f827800ec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:39.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827800ccc0 con 0x7f82880705b0 2024-09-21T07:50:39.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f827800ee70 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82880ae420 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f827800ce60 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.879+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82880c64c0 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.880+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82880c6720 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.881+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 51133+0+0 (secure 0 0 0) 0x7f8278004280 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.881+0000 7f8284c00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f826803e940 0x7f8268040e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.881+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6268+0+0 (secure 0 0 0) 0x7f8278054650 con 0x7f82880705b0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.881+0000 7f8287e00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f826803e940 0x7f8268040e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:39.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.882+0000 7f8287e00640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f826803e940 0x7f8268040e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8288159a40 tx=0x7f8274005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:39.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:39.886+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8278021440 con 0x7f82880705b0 2024-09-21T07:50:40.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.109+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f82880cd550 con 0x7f82880705b0 2024-09-21T07:50:40.111 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:50:40.110+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:50:40.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.113+0000 7f8284c00640 1 -- 172.21.15.88:0/595999485 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v48) ==== 161+0+0 (secure 0 0 0) 0x7f8278020580 con 0x7f82880705b0 2024-09-21T07:50:40.115 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.115+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f826803e940 msgr2=0x7f8268040e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.115+0000 7f828f600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f826803e940 0x7f8268040e50 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f8288159a40 tx=0x7f8274005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.115+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 msgr2=0x7f82881f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.115+0000 7f828f600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 0x7f82881f5580 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7f8278009fd0 tx=0x7f827800ec20 comp rx=0 tx=0).stop 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 shutdown_connections 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f826803e940 0x7f8268040e50 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82880cfc80 0x7f82881f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f828815ce30 0x7f828814bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 --2- 172.21.15.88:0/595999485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82880705b0 0x7f82881f5580 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 >> 172.21.15.88:0/595999485 conn(0x7f828806d3d0 msgr2=0x7f828815aef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 shutdown_connections 2024-09-21T07:50:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.116+0000 7f828f600640 1 -- 172.21.15.88:0/595999485 wait complete. 2024-09-21T07:50:40.129 INFO:tasks.ceph.mds.c.smithi088.stderr: -8981> 2024-09-21T07:50:20.985+0000 7ff95fe00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:50:40.141 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:50:40.254 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:50:40.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.366+0000 7fa31b400640 1 -- 172.21.15.88:0/2172197735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314073c50 msgr2=0x7fa3140cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.366+0000 7fa31b400640 1 --2- 172.21.15.88:0/2172197735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314073c50 0x7fa3140cd610 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fa30c005fd0 tx=0x7fa30c02a940 comp rx=0 tx=0).stop 2024-09-21T07:50:40.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.367+0000 7fa31b400640 1 -- 172.21.15.88:0/2172197735 shutdown_connections 2024-09-21T07:50:40.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.367+0000 7fa31b400640 1 --2- 172.21.15.88:0/2172197735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314073c50 0x7fa3140cd610 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.367+0000 7fa31b400640 1 --2- 172.21.15.88:0/2172197735 >> v2:172.21.15.88:3300/0 conn(0x7fa314145550 0x7fa31406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.367+0000 7fa31b400640 1 -- 172.21.15.88:0/2172197735 >> 172.21.15.88:0/2172197735 conn(0x7fa31406d1c0 msgr2=0x7fa31406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:40.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.367+0000 7fa31b400640 1 -- 172.21.15.88:0/2172197735 shutdown_connections 2024-09-21T07:50:40.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.367+0000 7fa31b400640 1 -- 172.21.15.88:0/2172197735 wait complete. 2024-09-21T07:50:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.369+0000 7fa31b400640 1 Processor -- start 2024-09-21T07:50:40.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.369+0000 7fa31b400640 1 -- start start 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.370+0000 7fa31b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 0x7fa3141379e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.370+0000 7fa31b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314145a40 0x7fa3141f68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa31b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3141f6e10 0x7fa3141f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa31b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140ce150 con 0x7fa314145550 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa31b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3140cdfd0 con 0x7fa3141f6e10 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa31b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa3140ce2d0 con 0x7fa314145a40 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 0x7fa3141379e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 0x7fa3141379e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56466/0 (socket says 172.21.15.88:56466) 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 -- 172.21.15.88:0/293688041 learned_addr learned my addr 172.21.15.88:0/293688041 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:40.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa313e00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314145a40 0x7fa3141f68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa319400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3141f6e10 0x7fa3141f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 -- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314145a40 msgr2=0x7fa3141f68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314145a40 0x7fa3141f68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 -- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3141f6e10 msgr2=0x7fa3141f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3141f6e10 0x7fa3141f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.371+0000 7fa318a00640 1 -- 172.21.15.88:0/293688041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa314138330 con 0x7fa314145550 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.372+0000 7fa318a00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 0x7fa3141379e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e9c0 tx=0x7fa30800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.372+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa30800cd80 con 0x7fa314145550 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.372+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa308004020 con 0x7fa314145550 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.372+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa31415dfd0 con 0x7fa314145550 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.372+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308004450 con 0x7fa314145550 2024-09-21T07:50:40.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.372+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3141f4230 con 0x7fa314145550 2024-09-21T07:50:40.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.373+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3140ce7f0 con 0x7fa314145550 2024-09-21T07:50:40.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.374+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 51133+0+0 (secure 0 0 0) 0x7fa3080035f0 con 0x7fa314145550 2024-09-21T07:50:40.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.374+0000 7fa310c00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f403e970 0x7fa2f4040e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:40.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.374+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6354+0+0 (secure 0 0 0) 0x7fa308054940 con 0x7fa314145550 2024-09-21T07:50:40.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.374+0000 7fa313e00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f403e970 0x7fa2f4040e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:40.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.375+0000 7fa313e00640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f403e970 0x7fa2f4040e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3141f0c60 tx=0x7fa300009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:40.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.378+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa308009460 con 0x7fa314145550 2024-09-21T07:50:40.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.601+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fa3141e63b0 con 0x7fa314145550 2024-09-21T07:50:40.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:40.752+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fa308010d10 con 0x7fa314145550 2024-09-21T07:50:41.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.115+0000 7fa310c00640 1 -- 172.21.15.88:0/293688041 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v49) ==== 104+0+0 (secure 0 0 0) 0x7fa308066590 con 0x7fa314145550 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f403e970 msgr2=0x7fa2f4040e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f403e970 0x7fa2f4040e80 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fa3141f0c60 tx=0x7fa300009040 comp rx=0 tx=0).stop 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 msgr2=0x7fa3141379e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 0x7fa3141379e0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7fa30800e9c0 tx=0x7fa30800ee80 comp rx=0 tx=0).stop 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 shutdown_connections 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f403e970 0x7fa2f4040e80 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa314145a40 0x7fa3141f68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa3141f6e10 0x7fa3141f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 --2- 172.21.15.88:0/293688041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa314145550 0x7fa3141379e0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 >> 172.21.15.88:0/293688041 conn(0x7fa31406d1c0 msgr2=0x7fa3140690b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 shutdown_connections 2024-09-21T07:50:41.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.118+0000 7fa31b400640 1 -- 172.21.15.88:0/293688041 wait complete. 2024-09-21T07:50:41.146 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:50:41.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 -- 172.21.15.88:0/1733260451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19406d4b0 msgr2=0x7fa1940b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 --2- 172.21.15.88:0/1733260451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19406d4b0 0x7fa1940b4620 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7fa194071040 tx=0x7fa18002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 -- 172.21.15.88:0/1733260451 shutdown_connections 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 --2- 172.21.15.88:0/1733260451 >> v2:172.21.15.175:3300/0 conn(0x7fa1940af100 0x7fa1940af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 --2- 172.21.15.88:0/1733260451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa19406d4b0 0x7fa1940b4620 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 -- 172.21.15.88:0/1733260451 >> 172.21.15.88:0/1733260451 conn(0x7fa1940b2540 msgr2=0x7fa1940d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 -- 172.21.15.88:0/1733260451 shutdown_connections 2024-09-21T07:50:41.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.368+0000 7fa19c200640 1 -- 172.21.15.88:0/1733260451 wait complete. 2024-09-21T07:50:41.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.370+0000 7fa19c200640 1 Processor -- start 2024-09-21T07:50:41.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.370+0000 7fa19c200640 1 -- start start 2024-09-21T07:50:41.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 0x7fa1941f66c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1940cc830 0x7fa1941e8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa194072ff0 0x7fa1940c90e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1940b3620 con 0x7fa1940af100 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1940b34a0 con 0x7fa194072ff0 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1940b37a0 con 0x7fa1940cc830 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 0x7fa1941f66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 0x7fa1941f66c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56494/0 (socket says 172.21.15.88:56494) 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 -- 172.21.15.88:0/2178735895 learned_addr learned my addr 172.21.15.88:0/2178735895 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa198e00640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1940cc830 0x7fa1941e8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa19a200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa194072ff0 0x7fa1940c90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 -- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1940cc830 msgr2=0x7fa1941e8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1940cc830 0x7fa1941e8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 -- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa194072ff0 msgr2=0x7fa1940c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa194072ff0 0x7fa1940c90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:41.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.372+0000 7fa199800640 1 -- 172.21.15.88:0/2178735895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1941e72c0 con 0x7fa1940af100 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa198e00640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1940cc830 0x7fa1941e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa19a200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa194072ff0 0x7fa1940c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa199800640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 0x7fa1941f66c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c3a0 tx=0x7fa18800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188017070 con 0x7fa1940af100 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa1880053c0 con 0x7fa1940af100 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa188003850 con 0x7fa1940af100 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1941e75a0 con 0x7fa1940af100 2024-09-21T07:50:41.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.373+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1941e7b60 con 0x7fa1940af100 2024-09-21T07:50:41.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.374+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1940b3cc0 con 0x7fa1940af100 2024-09-21T07:50:41.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.376+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fa18801a070 con 0x7fa1940af100 2024-09-21T07:50:41.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.377+0000 7fa191600640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa174078cb0 0x7fa17407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:41.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.377+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 6354+0+0 (secure 0 0 0) 0x7fa18809a620 con 0x7fa1940af100 2024-09-21T07:50:41.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.377+0000 7fa198e00640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa174078cb0 0x7fa17407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:41.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.377+0000 7fa198e00640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa174078cb0 0x7fa17407b1c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa180033000 tx=0x7fa180004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:41.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.381+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa1880625a0 con 0x7fa1940af100 2024-09-21T07:50:41.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:41.569+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa1940d1030 con 0x7fa1940af100 2024-09-21T07:50:42.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.122+0000 7fa191600640 1 -- 172.21.15.88:0/2178735895 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v76) ==== 181+0+0 (secure 0 0 0) 0x7fa188061ea0 con 0x7fa1940af100 2024-09-21T07:50:42.124 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:50:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa174078cb0 msgr2=0x7fa17407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa174078cb0 0x7fa17407b1c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fa180033000 tx=0x7fa180004730 comp rx=0 tx=0).stop 2024-09-21T07:50:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 msgr2=0x7fa1941f66c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 0x7fa1941f66c0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fa18800c3a0 tx=0x7fa18800c860 comp rx=0 tx=0).stop 2024-09-21T07:50:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 shutdown_connections 2024-09-21T07:50:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa174078cb0 0x7fa17407b1c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa1940cc830 0x7fa1941e8360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa194072ff0 0x7fa1940c90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.126+0000 7fa19c200640 1 --2- 172.21.15.88:0/2178735895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1940af100 0x7fa1941f66c0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.127+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 >> 172.21.15.88:0/2178735895 conn(0x7fa1940b2540 msgr2=0x7fa194068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:42.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.127+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 shutdown_connections 2024-09-21T07:50:42.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.127+0000 7fa19c200640 1 -- 172.21.15.88:0/2178735895 wait complete. 2024-09-21T07:50:42.152 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:50:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 -- 172.21.15.88:0/675559204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd406cda0 msgr2=0x7f6fd406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 --2- 172.21.15.88:0/675559204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd406cda0 0x7f6fd406d1b0 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f6fd40d01c0 tx=0x7f6fc002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 -- 172.21.15.88:0/675559204 shutdown_connections 2024-09-21T07:50:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 --2- 172.21.15.88:0/675559204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd406cda0 0x7f6fd406d1b0 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 --2- 172.21.15.88:0/675559204 >> v2:172.21.15.175:3300/0 conn(0x7f6fd40cc830 0x7f6fd40b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 -- 172.21.15.88:0/675559204 >> 172.21.15.88:0/675559204 conn(0x7f6fd40b2540 msgr2=0x7f6fd40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:42.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 -- 172.21.15.88:0/675559204 shutdown_connections 2024-09-21T07:50:42.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.376+0000 7f6fd9c00640 1 -- 172.21.15.88:0/675559204 wait complete. 2024-09-21T07:50:42.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.378+0000 7f6fd9c00640 1 Processor -- start 2024-09-21T07:50:42.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.379+0000 7f6fd9c00640 1 -- start start 2024-09-21T07:50:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd9c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fd406cda0 0x7f6fd40ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 0x7f6fd4156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd9c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fd40cc830 0x7f6fd40cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd9c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd40b35f0 con 0x7f6fd40af100 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd9c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd40b3470 con 0x7f6fd40cc830 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd9c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6fd40b3770 con 0x7f6fd406cda0 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 0x7f6fd4156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 0x7f6fd4156200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56518/0 (socket says 172.21.15.88:56518) 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.380+0000 7f6fd2000640 1 -- 172.21.15.88:0/905734887 learned_addr learned my addr 172.21.15.88:0/905734887 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd3400640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fd40cc830 0x7f6fd40cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:42.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2a00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fd406cda0 0x7f6fd40ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2000640 1 -- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fd406cda0 msgr2=0x7f6fd40ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2000640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fd406cda0 0x7f6fd40ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2000640 1 -- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fd40cc830 msgr2=0x7f6fd40cf130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2000640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fd40cc830 0x7f6fd40cf130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2000640 1 -- 172.21.15.88:0/905734887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd40ca6d0 con 0x7f6fd40af100 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd2000640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 0x7f6fd4156200 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0034040 tx=0x7f6fc00039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc003e030 con 0x7f6fd40af100 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fc00047a0 con 0x7f6fd40af100 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fc0035040 con 0x7f6fd40af100 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd9c00640 1 -- 172.21.15.88:0/905734887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd4156cc0 con 0x7f6fd40af100 2024-09-21T07:50:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.381+0000 7f6fd9c00640 1 -- 172.21.15.88:0/905734887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd41f9410 con 0x7f6fd40af100 2024-09-21T07:50:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.383+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd40b41c0 con 0x7f6fd40af100 2024-09-21T07:50:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.384+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f6fc0041030 con 0x7f6fd40af100 2024-09-21T07:50:42.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.385+0000 7f6fcaa00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fb4078cb0 0x7f6fb407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:42.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.385+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(76..76 src has 1..76) ==== 5916+0+0 (secure 0 0 0) 0x7f6fc00bb800 con 0x7f6fd40af100 2024-09-21T07:50:42.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.385+0000 7f6fd2a00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fb4078cb0 0x7f6fb407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:42.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.386+0000 7f6fd2a00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fb4078cb0 0x7f6fb407b1c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005fd0 tx=0x7f6fc4005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:42.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.390+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fd40b41c0 con 0x7f6fd40af100 2024-09-21T07:50:42.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:42.577+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f6fd40b3c90 con 0x7f6fd40af100 2024-09-21T07:50:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.124+0000 7f6fcaa00640 1 -- 172.21.15.88:0/905734887 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v77) ==== 169+0+0 (secure 0 0 0) 0x7f6fc00839d0 con 0x7f6fd40af100 2024-09-21T07:50:43.125 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:50:43.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.127+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fb4078cb0 msgr2=0x7f6fb407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.127+0000 7f6fbfe00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fb4078cb0 0x7f6fb407b1c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4005fd0 tx=0x7f6fc4005850 comp rx=0 tx=0).stop 2024-09-21T07:50:43.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.127+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 msgr2=0x7f6fd4156200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.127+0000 7f6fbfe00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 0x7f6fd4156200 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f6fc0034040 tx=0x7f6fc00039e0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.128+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 shutdown_connections 2024-09-21T07:50:43.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.128+0000 7f6fbfe00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fb4078cb0 0x7f6fb407b1c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.128+0000 7f6fbfe00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fd40cc830 0x7f6fd40cf130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.128+0000 7f6fbfe00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fd40af100 0x7f6fd4156200 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.128+0000 7f6fbfe00640 1 --2- 172.21.15.88:0/905734887 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fd406cda0 0x7f6fd40ca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.128+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 >> 172.21.15.88:0/905734887 conn(0x7f6fd40b2540 msgr2=0x7f6fd40b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:43.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.129+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 shutdown_connections 2024-09-21T07:50:43.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.131+0000 7f6fbfe00640 1 -- 172.21.15.88:0/905734887 wait complete. 2024-09-21T07:50:43.162 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:50:43.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 -- 172.21.15.88:0/790704005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 msgr2=0x7fee880b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 --2- 172.21.15.88:0/790704005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee880b2f60 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fee880cfe20 tx=0x7fee7002a420 comp rx=0 tx=0).stop 2024-09-21T07:50:43.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 -- 172.21.15.88:0/790704005 shutdown_connections 2024-09-21T07:50:43.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 --2- 172.21.15.88:0/790704005 >> v2:172.21.15.175:3300/0 conn(0x7fee8815a0f0 0x7fee8815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 --2- 172.21.15.88:0/790704005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee880b2f60 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 -- 172.21.15.88:0/790704005 >> 172.21.15.88:0/790704005 conn(0x7fee880b1e40 msgr2=0x7fee880b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:43.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 -- 172.21.15.88:0/790704005 shutdown_connections 2024-09-21T07:50:43.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.382+0000 7fee87e00640 1 -- 172.21.15.88:0/790704005 wait complete. 2024-09-21T07:50:43.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.384+0000 7fee87e00640 1 Processor -- start 2024-09-21T07:50:43.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.384+0000 7fee87e00640 1 -- start start 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee880ad9c0 0x7fee8814c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee881f92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee8815a0f0 0x7fee881f9800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee8806f290 con 0x7fee880af170 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee88070c80 con 0x7fee8815a0f0 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee8806f410 con 0x7fee880ad9c0 2024-09-21T07:50:43.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee7e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee881f92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee7e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee881f92c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56538/0 (socket says 172.21.15.88:56538) 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee7e000640 1 -- 172.21.15.88:0/1810118507 learned_addr learned my addr 172.21.15.88:0/1810118507 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee87400640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee8815a0f0 0x7fee881f9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee86a00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee880ad9c0 0x7fee8814c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee7e000640 1 -- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee880ad9c0 msgr2=0x7fee8814c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.386+0000 7fee7e000640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee880ad9c0 0x7fee8814c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7e000640 1 -- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee8815a0f0 msgr2=0x7fee881f9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7e000640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee8815a0f0 0x7fee881f9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7e000640 1 -- 172.21.15.88:0/1810118507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee88157ed0 con 0x7fee880af170 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee87400640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee8815a0f0 0x7fee881f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee86a00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee880ad9c0 0x7fee8814c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:43.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7e000640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee881f92c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fee70009920 tx=0x7fee70033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70031d60 con 0x7fee880af170 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee70005020 con 0x7fee880af170 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70005470 con 0x7fee880af170 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee881fa120 con 0x7fee880af170 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.387+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee881586b0 con 0x7fee880af170 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.388+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee880d0ea0 con 0x7fee880af170 2024-09-21T07:50:43.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.389+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fee700047a0 con 0x7fee880af170 2024-09-21T07:50:43.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.390+0000 7fee7f400640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee64078ce0 0x7fee6407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.391+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7fee700bc1e0 con 0x7fee880af170 2024-09-21T07:50:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.391+0000 7fee86a00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee64078ce0 0x7fee6407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.391+0000 7fee86a00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee64078ce0 0x7fee6407b1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fee780020f0 tx=0x7fee78009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:43.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.395+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee880d0ea0 con 0x7fee880af170 2024-09-21T07:50:43.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.582+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fee8815d3b0 con 0x7fee880af170 2024-09-21T07:50:43.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.587+0000 7fee7f400640 1 -- 172.21.15.88:0/1810118507 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v39) ==== 71+0+0 (secure 0 0 0) 0x7fee8815d3b0 con 0x7fee880af170 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee64078ce0 msgr2=0x7fee6407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee64078ce0 0x7fee6407b1f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fee780020f0 tx=0x7fee78009040 comp rx=0 tx=0).stop 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 msgr2=0x7fee881f92c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee881f92c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7fee70009920 tx=0x7fee70033d60 comp rx=0 tx=0).stop 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 shutdown_connections 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee64078ce0 0x7fee6407b1f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee8815a0f0 0x7fee881f9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee880af170 0x7fee881f92c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 --2- 172.21.15.88:0/1810118507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee880ad9c0 0x7fee8814c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 >> 172.21.15.88:0/1810118507 conn(0x7fee880b1e40 msgr2=0x7fee880b0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 shutdown_connections 2024-09-21T07:50:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.590+0000 7fee87e00640 1 -- 172.21.15.88:0/1810118507 wait complete. 2024-09-21T07:50:43.617 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction' 2024-09-21T07:50:43.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.838+0000 7f6fcae00640 1 -- 172.21.15.88:0/1489870619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc40aebc0 msgr2=0x7f6fc4152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.838+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1489870619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc40aebc0 0x7f6fc4152920 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4005fd0 tx=0x7f6fb402a490 comp rx=0 tx=0).stop 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.839+0000 7f6fcae00640 1 -- 172.21.15.88:0/1489870619 shutdown_connections 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.839+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1489870619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc40b2750 0x7f6fc415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.839+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1489870619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc40aebc0 0x7f6fc4152920 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.839+0000 7f6fcae00640 1 -- 172.21.15.88:0/1489870619 >> 172.21.15.88:0/1489870619 conn(0x7f6fc406d730 msgr2=0x7f6fc40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.839+0000 7f6fcae00640 1 -- 172.21.15.88:0/1489870619 shutdown_connections 2024-09-21T07:50:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.839+0000 7f6fcae00640 1 -- 172.21.15.88:0/1489870619 wait complete. 2024-09-21T07:50:43.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.841+0000 7f6fcae00640 1 Processor -- start 2024-09-21T07:50:43.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.841+0000 7f6fcae00640 1 -- start start 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.842+0000 7f6fcae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc40aebc0 0x7f6fc41f63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.842+0000 7f6fcae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc40b2750 0x7f6fc4148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.842+0000 7f6fcae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 0x7f6fc40ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.842+0000 7f6fcae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc415c4a0 con 0x7f6fc415f910 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.842+0000 7f6fcae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6fc40b4a20 con 0x7f6fc40aebc0 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.842+0000 7f6fcae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6fc415c620 con 0x7f6fc40b2750 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 0x7f6fc40ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 0x7f6fc40ca640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56556/0 (socket says 172.21.15.88:56556) 2024-09-21T07:50:43.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 -- 172.21.15.88:0/1641383522 learned_addr learned my addr 172.21.15.88:0/1641383522 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc3e00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc40aebc0 0x7f6fc41f63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc3400640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc40b2750 0x7f6fc4148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 -- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc40b2750 msgr2=0x7f6fc4148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc40b2750 0x7f6fc4148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 -- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc40aebc0 msgr2=0x7f6fc41f63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc40aebc0 0x7f6fc41f63a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 -- 172.21.15.88:0/1641383522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fc4152170 con 0x7f6fc415f910 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.843+0000 7f6fc8e00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 0x7f6fc40ca640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00b970 tx=0x7f6fbc00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.844+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc019070 con 0x7f6fc415f910 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.844+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6fbc00cc30 con 0x7f6fc415f910 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.844+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fbc0146a0 con 0x7f6fc415f910 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.844+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fc40b1c50 con 0x7f6fc415f910 2024-09-21T07:50:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.844+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fc40b13b0 con 0x7f6fc415f910 2024-09-21T07:50:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.845+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fc415cbc0 con 0x7f6fc415f910 2024-09-21T07:50:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.846+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f6fbc01c070 con 0x7f6fc415f910 2024-09-21T07:50:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.847+0000 7f6fbbe00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fa4078ce0 0x7f6fa407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.847+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7f6fbc09a990 con 0x7f6fc415f910 2024-09-21T07:50:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.847+0000 7f6fc3e00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fa4078ce0 0x7f6fa407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:43.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.848+0000 7f6fc3e00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fa4078ce0 0x7f6fa407b1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4033000 tx=0x7f6fb4004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:43.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:43.852+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6fbc062750 con 0x7f6fc415f910 2024-09-21T07:50:44.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.107+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]} v 0) -- 0x7f6fc415b5f0 con 0x7f6fc415f910 2024-09-21T07:50:44.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.126+0000 7f6fbbe00640 1 -- 172.21.15.88:0/1641383522 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_mgr_eviction"]}]=0 v263) ==== 131+0+0 (secure 0 0 0) 0x7f6fbc067180 con 0x7f6fc415f910 2024-09-21T07:50:44.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fa4078ce0 msgr2=0x7f6fa407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fa4078ce0 0x7f6fa407b1f0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f6fb4033000 tx=0x7f6fb4004730 comp rx=0 tx=0).stop 2024-09-21T07:50:44.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 msgr2=0x7f6fc40ca640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 0x7f6fc40ca640 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f6fbc00b970 tx=0x7f6fbc00be30 comp rx=0 tx=0).stop 2024-09-21T07:50:44.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 shutdown_connections 2024-09-21T07:50:44.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6fa4078ce0 0x7f6fa407b1f0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6fc40b2750 0x7f6fc4148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6fc415f910 0x7f6fc40ca640 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 --2- 172.21.15.88:0/1641383522 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6fc40aebc0 0x7f6fc41f63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 >> 172.21.15.88:0/1641383522 conn(0x7f6fc406d730 msgr2=0x7f6fc40cb5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:44.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 shutdown_connections 2024-09-21T07:50:44.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.129+0000 7f6fcae00640 1 -- 172.21.15.88:0/1641383522 wait complete. 2024-09-21T07:50:44.156 INFO:tasks.cephfs_test_runner:test_mgr_eviction (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-21T07:50:44.156 INFO:tasks.cephfs_test_runner:Starting test: test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) Test the creating vols, subvols subvolgroups fails when their names uses 2024-09-21T07:50:44.156 INFO:tasks.cephfs_test_runner:test_names_can_only_be_goodchars (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:50:44.156 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 -- 172.21.15.88:0/2275180402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600b3550 msgr2=0x7f3e600b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 --2- 172.21.15.88:0/2275180402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600b3550 0x7f3e600b3960 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f3e58005fd0 tx=0x7f3e5802a540 comp rx=0 tx=0).stop 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 -- 172.21.15.88:0/2275180402 shutdown_connections 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 --2- 172.21.15.88:0/2275180402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600b3550 0x7f3e600b3960 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 --2- 172.21.15.88:0/2275180402 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e600ad0e0 0x7f3e600ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 -- 172.21.15.88:0/2275180402 >> 172.21.15.88:0/2275180402 conn(0x7f3e6006d730 msgr2=0x7f3e600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:44.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 -- 172.21.15.88:0/2275180402 shutdown_connections 2024-09-21T07:50:44.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.375+0000 7f3e66600640 1 -- 172.21.15.88:0/2275180402 wait complete. 2024-09-21T07:50:44.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.377+0000 7f3e66600640 1 Processor -- start 2024-09-21T07:50:44.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.377+0000 7f3e66600640 1 -- start start 2024-09-21T07:50:44.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e66600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 0x7f3e60156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e66600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e600b3550 0x7f3e60074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e66600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e6015d760 0x7f3e601f9020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e66600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600b4da0 con 0x7f3e600ad0e0 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e66600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e600b4c20 con 0x7f3e600b3550 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e66600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3e600b3ea0 con 0x7f3e6015d760 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 0x7f3e60156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e5f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 0x7f3e60156910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56584/0 (socket says 172.21.15.88:56584) 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e5f400640 1 -- 172.21.15.88:0/271787060 learned_addr learned my addr 172.21.15.88:0/271787060 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e56a00640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e600b3550 0x7f3e60074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.379+0000 7f3e5fe00640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e6015d760 0x7f3e601f9020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e5f400640 1 -- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e6015d760 msgr2=0x7f3e601f9020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e5f400640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e6015d760 0x7f3e601f9020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e5f400640 1 -- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e600b3550 msgr2=0x7f3e60074800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e5f400640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e600b3550 0x7f3e60074800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e5f400640 1 -- 172.21.15.88:0/271787060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e600cf2f0 con 0x7f3e600ad0e0 2024-09-21T07:50:44.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e5f400640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 0x7f3e60156910 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c730 tx=0x7f3e5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:44.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e50003b70 con 0x7f3e600ad0e0 2024-09-21T07:50:44.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e50003d30 con 0x7f3e600ad0e0 2024-09-21T07:50:44.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e60154980 con 0x7f3e600ad0e0 2024-09-21T07:50:44.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e5000aab0 con 0x7f3e600ad0e0 2024-09-21T07:50:44.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.380+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e60154c20 con 0x7f3e600ad0e0 2024-09-21T07:50:44.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.382+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e6015c590 con 0x7f3e600ad0e0 2024-09-21T07:50:44.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.383+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f3e5000ac50 con 0x7f3e600ad0e0 2024-09-21T07:50:44.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.384+0000 7f3e57e00640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3e3c078ce0 0x7f3e3c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.384+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7f3e5009ab10 con 0x7f3e600ad0e0 2024-09-21T07:50:44.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.384+0000 7f3e56a00640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3e3c078ce0 0x7f3e3c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.385+0000 7f3e56a00640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3e3c078ce0 0x7f3e3c07b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3e60072ff0 tx=0x7f3e4800b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:44.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.388+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e600ada30 con 0x7f3e600ad0e0 2024-09-21T07:50:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.580+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3e600b4330 con 0x7f3e600ad0e0 2024-09-21T07:50:44.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.581+0000 7f3e57e00640 1 -- 172.21.15.88:0/271787060 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+13868 (secure 0 0 0) 0x7f3e50062830 con 0x7f3e600ad0e0 2024-09-21T07:50:44.582 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:44.583 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":77,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:43.122634+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/2332018233":"2024-09-22T07:50:40.111076+0000","172.21.15.88:6834/2332018233":"2024-09-22T07:50:40.111076+0000","172.21.15.175:0/2931692545":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3845646698":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2524320784":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/691722558":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1647382363":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/569875066":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1554442378":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/2139739944":"2024-09-22T07:50:35.674965+0000","172.21.15.175:6801/3294411971":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3468185230":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/3741301681":"2024-09-22T07:50:35.674965+0000","172.21.15.175:0/1671168534":"2024-09-22T07:50:35.674965+0000","172.21.15.175:6800/3294411971":"2024-09-22T07:50:35.674965+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.583+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3e3c078ce0 msgr2=0x7f3e3c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.583+0000 7f3e66600640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3e3c078ce0 0x7f3e3c07b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3e60072ff0 tx=0x7f3e4800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.583+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 msgr2=0x7f3e60156910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.583+0000 7f3e66600640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 0x7f3e60156910 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f3e5000c730 tx=0x7f3e5000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 shutdown_connections 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3e3c078ce0 0x7f3e3c07b1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e6015d760 0x7f3e601f9020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e600b3550 0x7f3e60074800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 --2- 172.21.15.88:0/271787060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e600ad0e0 0x7f3e60156910 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 >> 172.21.15.88:0/271787060 conn(0x7f3e6006d730 msgr2=0x7f3e600cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:44.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 shutdown_connections 2024-09-21T07:50:44.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.584+0000 7f3e66600640 1 -- 172.21.15.88:0/271787060 wait complete. 2024-09-21T07:50:44.610 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:50:44.610 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.832+0000 7fbcde400640 1 -- 172.21.15.88:0/3541544943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd806f4b0 msgr2=0x7fbcd8073460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.832+0000 7fbcde400640 1 --2- 172.21.15.88:0/3541544943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd806f4b0 0x7fbcd8073460 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fbcd8070fb0 tx=0x7fbcc402a560 comp rx=0 tx=0).stop 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.832+0000 7fbcde400640 1 -- 172.21.15.88:0/3541544943 shutdown_connections 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.832+0000 7fbcde400640 1 --2- 172.21.15.88:0/3541544943 >> v2:172.21.15.175:3300/0 conn(0x7fbcd80739a0 0x7fbcd8155090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.832+0000 7fbcde400640 1 --2- 172.21.15.88:0/3541544943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd806f4b0 0x7fbcd8073460 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.832+0000 7fbcde400640 1 --2- 172.21.15.88:0/3541544943 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80b5000 0x7fbcd806ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.833+0000 7fbcde400640 1 -- 172.21.15.88:0/3541544943 >> 172.21.15.88:0/3541544943 conn(0x7fbcd806d710 msgr2=0x7fbcd80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.833+0000 7fbcde400640 1 -- 172.21.15.88:0/3541544943 shutdown_connections 2024-09-21T07:50:44.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.833+0000 7fbcde400640 1 -- 172.21.15.88:0/3541544943 wait complete. 2024-09-21T07:50:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.835+0000 7fbcde400640 1 Processor -- start 2024-09-21T07:50:44.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.835+0000 7fbcde400640 1 -- start start 2024-09-21T07:50:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcde400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd806f4b0 0x7fbcd81f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcde400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80739a0 0x7fbcd814da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcde400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 0x7fbcd81532c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcde400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd81557e0 con 0x7fbcd80b5000 2024-09-21T07:50:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcde400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbcd815f3c0 con 0x7fbcd806f4b0 2024-09-21T07:50:44.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcde400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbcd8155960 con 0x7fbcd80739a0 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.836+0000 7fbcd7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 0x7fbcd81532c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 0x7fbcd81532c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56596/0 (socket says 172.21.15.88:56596) 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 -- 172.21.15.88:0/933013647 learned_addr learned my addr 172.21.15.88:0/933013647 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd806f4b0 0x7fbcd81f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd6a00640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80739a0 0x7fbcd814da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 -- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80739a0 msgr2=0x7fbcd814da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80739a0 0x7fbcd814da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 -- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd806f4b0 msgr2=0x7fbcd81f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd806f4b0 0x7fbcd81f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 -- 172.21.15.88:0/933013647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbcd815b440 con 0x7fbcd80b5000 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd6a00640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80739a0 0x7fbcd814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd806f4b0 0x7fbcd81f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:44.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbcd7e00640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 0x7fbcd81532c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000e9c0 tx=0x7fbcd000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:44.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.837+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0004d20 con 0x7fbcd80b5000 2024-09-21T07:50:44.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.838+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbcd000cb20 con 0x7fbcd80b5000 2024-09-21T07:50:44.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.838+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbcd0005020 con 0x7fbcd80b5000 2024-09-21T07:50:44.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.838+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbcd80cec70 con 0x7fbcd80b5000 2024-09-21T07:50:44.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.838+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbcd81598e0 con 0x7fbcd80b5000 2024-09-21T07:50:44.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.839+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbcd8155090 con 0x7fbcd80b5000 2024-09-21T07:50:44.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.840+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fbcd00051c0 con 0x7fbcd80b5000 2024-09-21T07:50:44.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.841+0000 7fbccf400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbcb8078cb0 0x7fbcb807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:44.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.841+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7fbcd009b790 con 0x7fbcd80b5000 2024-09-21T07:50:44.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.841+0000 7fbcd7400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbcb8078cb0 0x7fbcb807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:44.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.842+0000 7fbcd7400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbcb8078cb0 0x7fbcb807b1c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005fd0 tx=0x7fbcc800f360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:44.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:44.845+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbcd0016030 con 0x7fbcd80b5000 2024-09-21T07:50:45.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.029+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbcd8159390 con 0x7fbcd80b5000 2024-09-21T07:50:45.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.030+0000 7fbccf400640 1 -- 172.21.15.88:0/933013647 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v77) ==== 93+0+10 (secure 0 0 0) 0x7fbcd0063a70 con 0x7fbcd80b5000 2024-09-21T07:50:45.031 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:50:45.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbcb8078cb0 msgr2=0x7fbcb807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbcb8078cb0 0x7fbcb807b1c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fbcc8005fd0 tx=0x7fbcc800f360 comp rx=0 tx=0).stop 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 msgr2=0x7fbcd81532c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 0x7fbcd81532c0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fbcd000e9c0 tx=0x7fbcd000ee80 comp rx=0 tx=0).stop 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 shutdown_connections 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbcb8078cb0 0x7fbcb807b1c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbcd80739a0 0x7fbcd814da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbcd80b5000 0x7fbcd81532c0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 --2- 172.21.15.88:0/933013647 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbcd806f4b0 0x7fbcd81f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 >> 172.21.15.88:0/933013647 conn(0x7fbcd806d710 msgr2=0x7fbcd80af1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.032+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 shutdown_connections 2024-09-21T07:50:45.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.033+0000 7fbcde400640 1 -- 172.21.15.88:0/933013647 wait complete. 2024-09-21T07:50:45.058 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-21T07:50:45.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 -- 172.21.15.88:0/2946823200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a4152920 msgr2=0x7fe3a415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:45.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/2946823200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a4152920 0x7fe3a415f3d0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fe3a4160420 tx=0x7fe39002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 -- 172.21.15.88:0/2946823200 shutdown_connections 2024-09-21T07:50:45.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/2946823200 >> v2:172.21.15.175:3300/0 conn(0x7fe3a415f910 0x7fe3a40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/2946823200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a4152920 0x7fe3a415f3d0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 -- 172.21.15.88:0/2946823200 >> 172.21.15.88:0/2946823200 conn(0x7fe3a406d730 msgr2=0x7fe3a40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:45.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 -- 172.21.15.88:0/2946823200 shutdown_connections 2024-09-21T07:50:45.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.279+0000 7fe3aaa00640 1 -- 172.21.15.88:0/2946823200 wait complete. 2024-09-21T07:50:45.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.281+0000 7fe3aaa00640 1 Processor -- start 2024-09-21T07:50:45.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.281+0000 7fe3aaa00640 1 -- start start 2024-09-21T07:50:45.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3aaa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a40aebc0 0x7fe3a40cc440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3aaa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a4152920 0x7fe3a40ce800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3aaa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 0x7fe3a409db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3aaa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40b40d0 con 0x7fe3a415f910 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3aaa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a40b3f50 con 0x7fe3a4152920 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3aaa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe3a40b4250 con 0x7fe3a40aebc0 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 0x7fe3a409db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 0x7fe3a409db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:56612/0 (socket says 172.21.15.88:56612) 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 -- 172.21.15.88:0/4100242783 learned_addr learned my addr 172.21.15.88:0/4100242783 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a3400640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a4152920 0x7fe3a40ce800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a3e00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a40aebc0 0x7fe3a40cc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 -- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a40aebc0 msgr2=0x7fe3a40cc440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a40aebc0 0x7fe3a40cc440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 -- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a4152920 msgr2=0x7fe3a40ce800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:45.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a4152920 0x7fe3a40ce800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:45.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.283+0000 7fe3a8a00640 1 -- 172.21.15.88:0/4100242783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a4152fa0 con 0x7fe3a415f910 2024-09-21T07:50:45.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.284+0000 7fe3a8a00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 0x7fe3a409db20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00cb00 tx=0x7fe39c00a250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:45.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.284+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00f450 con 0x7fe3a415f910 2024-09-21T07:50:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.284+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe39c00f700 con 0x7fe3a415f910 2024-09-21T07:50:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.284+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe39c00fb50 con 0x7fe3a415f910 2024-09-21T07:50:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.284+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a4153280 con 0x7fe3a415f910 2024-09-21T07:50:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.284+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a4155c30 con 0x7fe3a415f910 2024-09-21T07:50:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.285+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a41559a0 con 0x7fe3a415f910 2024-09-21T07:50:45.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.286+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe39c00fcf0 con 0x7fe3a415f910 2024-09-21T07:50:45.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.287+0000 7fe39be00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe384078ce0 0x7fe38407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:45.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.287+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7fe39c09b9b0 con 0x7fe3a415f910 2024-09-21T07:50:45.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.288+0000 7fe3a3e00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe384078ce0 0x7fe38407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:45.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.288+0000 7fe3a3e00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe384078ce0 0x7fe38407b1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe394006fd0 tx=0x7fe394009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:45.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.292+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe39c014030 con 0x7fe3a415f910 2024-09-21T07:50:45.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:45.550+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7fe3a40b47e0 con 0x7fe3a415f910 2024-09-21T07:50:46.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.137+0000 7fe39be00640 1 -- 172.21.15.88:0/4100242783 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v265) ==== 149+0+0 (secure 0 0 0) 0x7fe39c0635b0 con 0x7fe3a415f910 2024-09-21T07:50:46.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe384078ce0 msgr2=0x7fe38407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe384078ce0 0x7fe38407b1f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fe394006fd0 tx=0x7fe394009060 comp rx=0 tx=0).stop 2024-09-21T07:50:46.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 msgr2=0x7fe3a409db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 0x7fe3a409db20 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe39c00cb00 tx=0x7fe39c00a250 comp rx=0 tx=0).stop 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 shutdown_connections 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe384078ce0 0x7fe38407b1f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe3a415f910 0x7fe3a409db20 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe3a4152920 0x7fe3a40ce800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 --2- 172.21.15.88:0/4100242783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe3a40aebc0 0x7fe3a40cc440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 >> 172.21.15.88:0/4100242783 conn(0x7fe3a406d730 msgr2=0x7fe3a4068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 shutdown_connections 2024-09-21T07:50:46.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.140+0000 7fe3aaa00640 1 -- 172.21.15.88:0/4100242783 wait complete. 2024-09-21T07:50:46.163 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:50:46.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 -- 172.21.15.88:0/1896221062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 msgr2=0x7f0bb815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/1896221062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb815a0f0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f0bb8051590 tx=0x7f0ba002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 -- 172.21.15.88:0/1896221062 shutdown_connections 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/1896221062 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb815d670 0x7f0bb8070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/1896221062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb815a0f0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/1896221062 >> v2:172.21.15.175:3300/0 conn(0x7f0bb80ad9c0 0x7f0bb80af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 -- 172.21.15.88:0/1896221062 >> 172.21.15.88:0/1896221062 conn(0x7f0bb80b1e40 msgr2=0x7f0bb80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 -- 172.21.15.88:0/1896221062 shutdown_connections 2024-09-21T07:50:46.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.382+0000 7f0bb7e00640 1 -- 172.21.15.88:0/1896221062 wait complete. 2024-09-21T07:50:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.384+0000 7f0bb7e00640 1 Processor -- start 2024-09-21T07:50:46.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.384+0000 7f0bb7e00640 1 -- start start 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb80ad9c0 0x7f0bb8150850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb80ceec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb815d670 0x7f0bb8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb8070c80 con 0x7f0bb80b2f60 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0bb806f290 con 0x7f0bb80ad9c0 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0bb8157030 con 0x7f0bb815d670 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bae000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb80ceec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bae000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb80ceec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57994/0 (socket says 172.21.15.88:57994) 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bae000640 1 -- 172.21.15.88:0/2135258954 learned_addr learned my addr 172.21.15.88:0/2135258954 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb6a00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb80ad9c0 0x7f0bb8150850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bb7400640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb815d670 0x7f0bb8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.386+0000 7f0bae000640 1 -- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb815d670 msgr2=0x7f0bb8156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bae000640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb815d670 0x7f0bb8156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bae000640 1 -- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb80ad9c0 msgr2=0x7f0bb8150850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bae000640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb80ad9c0 0x7f0bb8150850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bae000640 1 -- 172.21.15.88:0/2135258954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0bb81f97a0 con 0x7f0bb80b2f60 2024-09-21T07:50:46.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bae000640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb80ceec0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0000c00 tx=0x7f0ba0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba003e030 con 0x7f0bb80b2f60 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0ba0003e00 con 0x7f0bb80b2f60 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0bb8156b60 con 0x7f0bb80b2f60 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ba0035040 con 0x7f0bb80b2f60 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.387+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0bb81f5270 con 0x7f0bb80b2f60 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.388+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0bb81f9510 con 0x7f0bb80b2f60 2024-09-21T07:50:46.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.389+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f0ba0041030 con 0x7f0bb80b2f60 2024-09-21T07:50:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.390+0000 7f0baf400640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0b94078cb0 0x7f0b9407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.391+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7f0ba00bb670 con 0x7f0bb80b2f60 2024-09-21T07:50:46.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.391+0000 7f0bb6a00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0b94078cb0 0x7f0b9407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.391+0000 7f0bb6a00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0b94078cb0 0x7f0b9407b1c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8005fd0 tx=0x7f0ba80096c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:46.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.395+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0ba0083390 con 0x7f0bb80b2f60 2024-09-21T07:50:46.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.578+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f0bb815a0f0 con 0x7f0bb80b2f60 2024-09-21T07:50:46.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.579+0000 7f0baf400640 1 -- 172.21.15.88:0/2135258954 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v39) ==== 59+0+2294 (secure 0 0 0) 0x7f0ba0087dc0 con 0x7f0bb80b2f60 2024-09-21T07:50:46.580 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:50:46.580 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:50:46.580 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:50:46.580 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:50:46.580 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:50:46.581 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:50:46.582 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:50:46.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0b94078cb0 msgr2=0x7f0b9407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0b94078cb0 0x7f0b9407b1c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0ba8005fd0 tx=0x7f0ba80096c0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 msgr2=0x7f0bb80ceec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb80ceec0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f0ba0000c00 tx=0x7f0ba0003910 comp rx=0 tx=0).stop 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 shutdown_connections 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0b94078cb0 0x7f0b9407b1c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0bb815d670 0x7f0bb8156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0bb80b2f60 0x7f0bb80ceec0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 --2- 172.21.15.88:0/2135258954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0bb80ad9c0 0x7f0bb8150850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.581+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 >> 172.21.15.88:0/2135258954 conn(0x7f0bb80b1e40 msgr2=0x7f0bb80b3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.582+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 shutdown_connections 2024-09-21T07:50:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.582+0000 7f0bb7e00640 1 -- 172.21.15.88:0/2135258954 wait complete. 2024-09-21T07:50:46.604 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:50:46.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 -- 172.21.15.88:0/919526018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800d0fc0 msgr2=0x7f5f800cd0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 --2- 172.21.15.88:0/919526018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800d0fc0 0x7f5f800cd0e0 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f5f800cfd70 tx=0x7f5f7802a9a0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 -- 172.21.15.88:0/919526018 shutdown_connections 2024-09-21T07:50:46.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 --2- 172.21.15.88:0/919526018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800d0fc0 0x7f5f800cd0e0 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 --2- 172.21.15.88:0/919526018 >> v2:172.21.15.175:3300/0 conn(0x7f5f800b7d70 0x7f5f800d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 -- 172.21.15.88:0/919526018 >> 172.21.15.88:0/919526018 conn(0x7f5f800cef20 msgr2=0x7f5f800cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:46.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 -- 172.21.15.88:0/919526018 shutdown_connections 2024-09-21T07:50:46.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.828+0000 7f5f85800640 1 -- 172.21.15.88:0/919526018 wait complete. 2024-09-21T07:50:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.830+0000 7f5f85800640 1 Processor -- start 2024-09-21T07:50:46.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.830+0000 7f5f85800640 1 -- start start 2024-09-21T07:50:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f85800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 0x7f5f80154bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f85800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f800b7d70 0x7f5f80157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f85800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f800d0fc0 0x7f5f80158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f85800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f800af900 con 0x7f5f800b7960 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f85800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f800d14e0 con 0x7f5f800d0fc0 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f85800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f8015ecf0 con 0x7f5f800b7d70 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 0x7f5f80154bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 0x7f5f80154bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58004/0 (socket says 172.21.15.88:58004) 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 -- 172.21.15.88:0/211642854 learned_addr learned my addr 172.21.15.88:0/211642854 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7e000640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f800b7d70 0x7f5f80157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7f400640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f800d0fc0 0x7f5f80158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 -- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f800b7d70 msgr2=0x7f5f80157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f800b7d70 0x7f5f80157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 -- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f800d0fc0 msgr2=0x7f5f80158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f800d0fc0 0x7f5f80158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:46.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.832+0000 7f5f7ea00640 1 -- 172.21.15.88:0/211642854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f801567b0 con 0x7f5f800b7960 2024-09-21T07:50:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f7e000640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f800b7d70 0x7f5f80157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f7f400640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f800d0fc0 0x7f5f80158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:46.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f7ea00640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 0x7f5f80154bf0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000c3c0 tx=0x7f5f7000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f70017070 con 0x7f5f800b7960 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f700053c0 con 0x7f5f800b7960 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f700038a0 con 0x7f5f800b7960 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f80156a90 con 0x7f5f800b7960 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.833+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f80157050 con 0x7f5f800b7960 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.834+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f800d0a80 con 0x7f5f800b7960 2024-09-21T07:50:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.835+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5f7001a070 con 0x7f5f800b7960 2024-09-21T07:50:46.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.837+0000 7f5f76a00640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f60078cb0 0x7f5f6007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:46.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.837+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7f5f7009a2d0 con 0x7f5f800b7960 2024-09-21T07:50:46.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.837+0000 7f5f7e000640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f60078cb0 0x7f5f6007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:46.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.838+0000 7f5f7e000640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f60078cb0 0x7f5f6007b1c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ce1d0 tx=0x7f5f6c005f10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:46.842+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f700625b0 con 0x7f5f800b7960 2024-09-21T07:50:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.027+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f5f800b72e0 con 0x7f5f800b7960 2024-09-21T07:50:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.031+0000 7f5f76a00640 1 -- 172.21.15.88:0/211642854 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7f5f70061eb0 con 0x7f5f800b7960 2024-09-21T07:50:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f60078cb0 msgr2=0x7f5f6007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f60078cb0 0x7f5f6007b1c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f5f800ce1d0 tx=0x7f5f6c005f10 comp rx=0 tx=0).stop 2024-09-21T07:50:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 msgr2=0x7f5f80154bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 0x7f5f80154bf0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000c3c0 tx=0x7f5f7000c880 comp rx=0 tx=0).stop 2024-09-21T07:50:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 shutdown_connections 2024-09-21T07:50:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f60078cb0 0x7f5f6007b1c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f800b7d70 0x7f5f80157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f800d0fc0 0x7f5f80158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 --2- 172.21.15.88:0/211642854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f800b7960 0x7f5f80154bf0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 >> 172.21.15.88:0/211642854 conn(0x7f5f800cef20 msgr2=0x7f5f800af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 shutdown_connections 2024-09-21T07:50:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.034+0000 7f5f85800640 1 -- 172.21.15.88:0/211642854 wait complete. 2024-09-21T07:50:47.061 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:50:47.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:47.144 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:50:47.226 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:50:47.226 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:47.226 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:47.261 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:47.261 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:47.261 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:47.298 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:47.298 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:50:47.298 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:47.336 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:50:47.336 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:50:47.336 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:50:47.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:50:47.374 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:50:47.375 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 -- 172.21.15.88:0/603042000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0ad3c0 msgr2=0x7f477c0ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 --2- 172.21.15.88:0/603042000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0ad3c0 0x7f477c0ad7d0 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f477c0afd40 tx=0x7f476802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 -- 172.21.15.88:0/603042000 shutdown_connections 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 --2- 172.21.15.88:0/603042000 >> v2:172.21.15.175:3300/0 conn(0x7f477c0add10 0x7f477c0b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 --2- 172.21.15.88:0/603042000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0ad3c0 0x7f477c0ad7d0 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 --2- 172.21.15.88:0/603042000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477c0704b0 0x7f477c073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 -- 172.21.15.88:0/603042000 >> 172.21.15.88:0/603042000 conn(0x7f477c06d730 msgr2=0x7f477c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:47.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 -- 172.21.15.88:0/603042000 shutdown_connections 2024-09-21T07:50:47.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.593+0000 7f4783200640 1 -- 172.21.15.88:0/603042000 wait complete. 2024-09-21T07:50:47.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.596+0000 7f4783200640 1 Processor -- start 2024-09-21T07:50:47.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.596+0000 7f4783200640 1 -- start start 2024-09-21T07:50:47.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f4783200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 0x7f477c1f9780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:47.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f4783200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477c0ad3c0 0x7f477c1e5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f4783200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477c0add10 0x7f477c1e4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f4783200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f477c0712b0 con 0x7f477c0704b0 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f4783200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f477c071130 con 0x7f477c0ad3c0 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f4783200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f477c0b4350 con 0x7f477c0add10 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f477be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 0x7f477c1f9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f477be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 0x7f477c1f9780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58010/0 (socket says 172.21.15.88:58010) 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f477be00640 1 -- 172.21.15.88:0/2043399209 learned_addr learned my addr 172.21.15.88:0/2043399209 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f4781200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477c0add10 0x7f477c1e4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.597+0000 7f477b400640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477c0ad3c0 0x7f477c1e5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f477be00640 1 -- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477c0add10 msgr2=0x7f477c1e4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f477be00640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477c0add10 0x7f477c1e4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f477be00640 1 -- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477c0ad3c0 msgr2=0x7f477c1e5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f477be00640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477c0ad3c0 0x7f477c1e5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f477be00640 1 -- 172.21.15.88:0/2043399209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f477c15a2c0 con 0x7f477c0704b0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f477be00640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 0x7f477c1f9780 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e890 tx=0x7f476c00ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c00cb60 con 0x7f477c0704b0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f476c00cd00 con 0x7f477c0704b0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f477c15a5a0 con 0x7f477c0704b0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f476c017680 con 0x7f477c0704b0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.598+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f477c1f9260 con 0x7f477c0704b0 2024-09-21T07:50:47.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.599+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f477c0ad7d0 con 0x7f477c0704b0 2024-09-21T07:50:47.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.601+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f476c020440 con 0x7f477c0704b0 2024-09-21T07:50:47.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.602+0000 7f4773e00640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f475c078ce0 0x7f475c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:47.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.602+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 5502+0+0 (secure 0 0 0) 0x7f476c09b270 con 0x7f477c0704b0 2024-09-21T07:50:47.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.602+0000 7f477b400640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f475c078ce0 0x7f475c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:47.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.603+0000 7f477b400640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f475c078ce0 0x7f475c07b1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4768033000 tx=0x7f4768004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:47.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.606+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f476c063030 con 0x7f477c0704b0 2024-09-21T07:50:47.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.829+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f477c15e420 con 0x7f477c0704b0 2024-09-21T07:50:47.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.830+0000 7f4773e00640 1 -- 172.21.15.88:0/2043399209 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 49 v49) ==== 94+0+3917 (secure 0 0 0) 0x7f476c067a60 con 0x7f477c0704b0 2024-09-21T07:50:47.831 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:47.831 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":49,"btime":"2024-09-21T07:50:41:114698+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5564,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6836/2951468846","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":2951468846},{"type":"v1","addr":"172.21.15.175:6836","nonce":2951468846}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":5567,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6837/3878221925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6835","nonce":3878221925},{"type":"v1","addr":"172.21.15.175:6837","nonce":3878221925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":41},{"gid":6177,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/3455638018","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3455638018},{"type":"v1","addr":"172.21.15.88:6833","nonce":3455638018}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":40},{"gid":6507,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3432886976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":3432886976},{"type":"v1","addr":"172.21.15.88:6835","nonce":3432886976}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":49}],"filesystems":[]} 2024-09-21T07:50:47.832 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 49 2024-09-21T07:50:47.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.832+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f475c078ce0 msgr2=0x7f475c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.832+0000 7f4783200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f475c078ce0 0x7f475c07b1f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f4768033000 tx=0x7f4768004730 comp rx=0 tx=0).stop 2024-09-21T07:50:47.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 msgr2=0x7f477c1f9780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:47.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 0x7f477c1f9780 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f476c00e890 tx=0x7f476c00ed50 comp rx=0 tx=0).stop 2024-09-21T07:50:47.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 shutdown_connections 2024-09-21T07:50:47.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f475c078ce0 0x7f475c07b1f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477c0add10 0x7f477c1e4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f477c0ad3c0 0x7f477c1e5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 --2- 172.21.15.88:0/2043399209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477c0704b0 0x7f477c1f9780 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 >> 172.21.15.88:0/2043399209 conn(0x7f477c06d730 msgr2=0x7f477c1e8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 shutdown_connections 2024-09-21T07:50:47.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:47.833+0000 7f4783200640 1 -- 172.21.15.88:0/2043399209 wait complete. 2024-09-21T07:50:47.857 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:50:47.857 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:50:47.857 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:50:47.857 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:47.858 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:50:47.858 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:50:47.858 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:50:47.858 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:47.858 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:50:47.858 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:50:47.858 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:50:47.858 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:47.859 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:50:47.859 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:50:47.859 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:50:47.859 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:50:47.877 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:50:47.878 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:50:47.881 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:50:47.881 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:50:47.885 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:50:47.885 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:50:47.887 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:50:47.887 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:50:47.924 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:50:47.925 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:47.928 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:50:47.928 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:47.931 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:50:47.931 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:47.934 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:50:47.934 DEBUG:teuthology.parallel:result is None 2024-09-21T07:50:47.934 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:50:48.124 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:50:48.124 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:50:48.130 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:50:48.130 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/1717716434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d11d0 msgr2=0x7f7ad01638c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/1717716434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d11d0 0x7f7ad01638c0 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f7ad006a0d0 tx=0x7f7ac802aab0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/1717716434 shutdown_connections 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/1717716434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d11d0 0x7f7ad01638c0 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/1717716434 >> v2:172.21.15.175:3300/0 conn(0x7f7ad00d03b0 0x7f7ad00d07c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/1717716434 >> 172.21.15.88:0/1717716434 conn(0x7f7ad00cef30 msgr2=0x7f7ad00cf340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/1717716434 shutdown_connections 2024-09-21T07:50:48.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/1717716434 wait complete. 2024-09-21T07:50:48.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.172+0000 7f7ad7600640 1 Processor -- start 2024-09-21T07:50:48.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.173+0000 7f7ad7600640 1 -- start start 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad7600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 0x7f7ad015b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00d0dc0 0x7f7ad015c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00d11d0 0x7f7ad015d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad7600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad00b1a20 con 0x7f7ad00d03b0 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ad0070ff0 con 0x7f7ad00d0dc0 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ad00b78e0 con 0x7f7ad00d11d0 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 0x7f7ad015b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 0x7f7ad015b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58074/0 (socket says 172.21.15.88:58074) 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad4c00640 1 -- 172.21.15.88:0/2235665519 learned_addr learned my addr 172.21.15.88:0/2235665519 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:48.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7acfe00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00d0dc0 0x7f7ad015c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.174+0000 7f7ad5600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00d11d0 0x7f7ad015d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad4c00640 1 -- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00d11d0 msgr2=0x7f7ad015d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad4c00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00d11d0 0x7f7ad015d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad4c00640 1 -- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00d0dc0 msgr2=0x7f7ad015c8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad4c00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00d0dc0 0x7f7ad015c8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad4c00640 1 -- 172.21.15.88:0/2235665519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ad015c0e0 con 0x7f7ad00d03b0 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7acfe00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00d0dc0 0x7f7ad015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:48.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad4c00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 0x7f7ad015b9a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9f0 tx=0x7f7ac400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:48.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac400ccc0 con 0x7f7ad00d03b0 2024-09-21T07:50:48.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7ac400ce60 con 0x7f7ad00d03b0 2024-09-21T07:50:48.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ad0160240 con 0x7f7ad00d03b0 2024-09-21T07:50:48.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ac4004280 con 0x7f7ad00d03b0 2024-09-21T07:50:48.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.175+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ad0160800 con 0x7f7ad00d03b0 2024-09-21T07:50:48.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.177+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ad01eab60 con 0x7f7ad00d03b0 2024-09-21T07:50:48.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.178+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7ac40035f0 con 0x7f7ad00d03b0 2024-09-21T07:50:48.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.179+0000 7f7accc00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ab0078cb0 0x7f7ab007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:48.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.179+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 5502+0+0 (secure 0 0 0) 0x7f7ac409b8a0 con 0x7f7ad00d03b0 2024-09-21T07:50:48.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.179+0000 7f7acfe00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ab0078cb0 0x7f7ab007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:48.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.180+0000 7f7acfe00640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ab0078cb0 0x7f7ab007b1c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00b7560 tx=0x7f7abc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:48.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.183+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7ac4063c20 con 0x7f7ad00d03b0 2024-09-21T07:50:48.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:48.379+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f7ad00d17b0 con 0x7f7ad00d03b0 2024-09-21T07:50:49.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.166+0000 7f7accc00640 1 -- 172.21.15.88:0/2235665519 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v82) ==== 97+0+0 (secure 0 0 0) 0x7f7ac4063520 con 0x7f7ad00d03b0 2024-09-21T07:50:49.168 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.169+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ab0078cb0 msgr2=0x7f7ab007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.169+0000 7f7ad7600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ab0078cb0 0x7f7ab007b1c0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f7ad00b7560 tx=0x7f7abc009040 comp rx=0 tx=0).stop 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.169+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 msgr2=0x7f7ad015b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.169+0000 7f7ad7600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 0x7f7ad015b9a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f7ac400e9f0 tx=0x7f7ac400eeb0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 shutdown_connections 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ab0078cb0 0x7f7ab007b1c0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ad00d11d0 0x7f7ad015d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ad00d0dc0 0x7f7ad015c8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 --2- 172.21.15.88:0/2235665519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ad00d03b0 0x7f7ad015b9a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 >> 172.21.15.88:0/2235665519 conn(0x7f7ad00cef30 msgr2=0x7f7ad00b3070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 shutdown_connections 2024-09-21T07:50:49.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.170+0000 7f7ad7600640 1 -- 172.21.15.88:0/2235665519 wait complete. 2024-09-21T07:50:49.195 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:50:49.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 -- 172.21.15.88:0/2820752966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 msgr2=0x7f2fb40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2820752966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb40ad4f0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005fd0 tx=0x7f2fa002a540 comp rx=0 tx=0).stop 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 -- 172.21.15.88:0/2820752966 shutdown_connections 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2820752966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb40ad4f0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2820752966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 0x7f2fb41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 -- 172.21.15.88:0/2820752966 >> 172.21.15.88:0/2820752966 conn(0x7f2fb406d730 msgr2=0x7f2fb40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 -- 172.21.15.88:0/2820752966 shutdown_connections 2024-09-21T07:50:49.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.416+0000 7f2fbac00640 1 -- 172.21.15.88:0/2820752966 wait complete. 2024-09-21T07:50:49.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.418+0000 7f2fbac00640 1 Processor -- start 2024-09-21T07:50:49.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.419+0000 7f2fbac00640 1 -- start start 2024-09-21T07:50:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fbac00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb41f4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fbac00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 0x7f2fb40b0f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fbac00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4160310 0x7f2fb41f6d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fbac00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb406f220 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fbac00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2fb406f0a0 con 0x7f2fb4160310 2024-09-21T07:50:49.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fbac00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2fb40b4c20 con 0x7f2fb415d760 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb41f4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb41f4370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58100/0 (socket says 172.21.15.88:58100) 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.420+0000 7f2fb3e00640 1 -- 172.21.15.88:0/2519843731 learned_addr learned my addr 172.21.15.88:0/2519843731 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3400640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 0x7f2fb40b0f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb8c00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4160310 0x7f2fb41f6d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3e00640 1 -- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 msgr2=0x7f2fb40b0f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3e00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 0x7f2fb40b0f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3e00640 1 -- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4160310 msgr2=0x7f2fb41f6d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3e00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4160310 0x7f2fb41f6d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3e00640 1 -- 172.21.15.88:0/2519843731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fb41f7c60 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3400640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 0x7f2fb40b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fb3e00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb41f4370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e790 tx=0x7f2fa400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa400eea0 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2fa400cdb0 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4010690 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fb41f7980 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.421+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fb41f3b70 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.423+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fb4160fa0 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.424+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2fa4010830 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.425+0000 7f2fabe00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f94078cb0 0x7f2f9407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.425+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4857+0+0 (secure 0 0 0) 0x7f2fa409b630 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.425+0000 7f2fb3400640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f94078cb0 0x7f2f9407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.426+0000 7f2fb3400640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f94078cb0 0x7f2f9407b1c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0033000 tx=0x7f2fa0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:49.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.429+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2fa4063c30 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.681+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f2fb40d0cc0 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.681+0000 7f2fabe00640 1 -- 172.21.15.88:0/2519843731 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v14) ==== 80+0+5304 (secure 0 0 0) 0x7f2fa4063530 con 0x7f2fb40ad0e0 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:49.683 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.684 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:49.685 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:50:49.686 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.687 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.688 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.689 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.690 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:50:49.691 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:50:49.692 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.693 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:50:49.694 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:50:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:50:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:49.695 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:50:49.695 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f94078cb0 msgr2=0x7f2f9407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f94078cb0 0x7f2f9407b1c0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0033000 tx=0x7f2fa0004730 comp rx=0 tx=0).stop 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 msgr2=0x7f2fb41f4370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb41f4370 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f2fa400e790 tx=0x7f2fa400ec50 comp rx=0 tx=0).stop 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 shutdown_connections 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f94078cb0 0x7f2f9407b1c0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2fb415d760 0x7f2fb40b0f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2fb4160310 0x7f2fb41f6d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 --2- 172.21.15.88:0/2519843731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fb40ad0e0 0x7f2fb41f4370 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 >> 172.21.15.88:0/2519843731 conn(0x7f2fb406d730 msgr2=0x7f2fb4160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 shutdown_connections 2024-09-21T07:50:49.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.684+0000 7f2fbac00640 1 -- 172.21.15.88:0/2519843731 wait complete. 2024-09-21T07:50:49.707 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.925+0000 7f08d6e00640 1 -- 172.21.15.88:0/2341673724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d00ad0e0 msgr2=0x7f08d00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.925+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2341673724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d00ad0e0 0x7f08d00ad4f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009860 tx=0x7f08b802a510 comp rx=0 tx=0).stop 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.925+0000 7f08d6e00640 1 -- 172.21.15.88:0/2341673724 shutdown_connections 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.925+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2341673724 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d00ad0e0 0x7f08d00ad4f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.925+0000 7f08d6e00640 1 -- 172.21.15.88:0/2341673724 >> 172.21.15.88:0/2341673724 conn(0x7f08d006d730 msgr2=0x7f08d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.926+0000 7f08d6e00640 1 -- 172.21.15.88:0/2341673724 shutdown_connections 2024-09-21T07:50:49.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.926+0000 7f08d6e00640 1 -- 172.21.15.88:0/2341673724 wait complete. 2024-09-21T07:50:49.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.928+0000 7f08d6e00640 1 Processor -- start 2024-09-21T07:50:49.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.928+0000 7f08d6e00640 1 -- start start 2024-09-21T07:50:49.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.929+0000 7f08d6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08d00ad0e0 0x7f08d00ceb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.929+0000 7f08d6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 0x7f08d01f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.929+0000 7f08d6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d015d760 0x7f08d0156370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.929+0000 7f08d6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f08d00b4da0 con 0x7f08d00b3550 2024-09-21T07:50:49.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.929+0000 7f08d6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f08d00b4c20 con 0x7f08d00ad0e0 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.929+0000 7f08d6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f08d00b4070 con 0x7f08d015d760 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 0x7f08d01f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 0x7f08d01f6200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58102/0 (socket says 172.21.15.88:58102) 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 -- 172.21.15.88:0/2978570071 learned_addr learned my addr 172.21.15.88:0/2978570071 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08d4e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d015d760 0x7f08d0156370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08cfe00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08d00ad0e0 0x7f08d00ceb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 -- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d015d760 msgr2=0x7f08d0156370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d015d760 0x7f08d0156370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 -- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08d00ad0e0 msgr2=0x7f08d00ceb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08d00ad0e0 0x7f08d00ceb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:49.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.930+0000 7f08c7e00640 1 -- 172.21.15.88:0/2978570071 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08d00cf080 con 0x7f08d00b3550 2024-09-21T07:50:49.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.931+0000 7f08c7e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 0x7f08d01f6200 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f08b802ab00 tx=0x7f08b8033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:49.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.931+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8031b50 con 0x7f08d00b3550 2024-09-21T07:50:49.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.931+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08b8031de0 con 0x7f08d00b3550 2024-09-21T07:50:49.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.931+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08d01f6740 con 0x7f08d00b3550 2024-09-21T07:50:49.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.931+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08b8033330 con 0x7f08d00b3550 2024-09-21T07:50:49.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.931+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08d0153f50 con 0x7f08d00b3550 2024-09-21T07:50:49.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.932+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f08d00ad4f0 con 0x7f08d00b3550 2024-09-21T07:50:49.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.933+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f08b80334d0 con 0x7f08d00b3550 2024-09-21T07:50:49.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.934+0000 7f08ccc00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08ac078cb0 0x7f08ac07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:49.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.934+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4857+0+0 (secure 0 0 0) 0x7f08b80bbe50 con 0x7f08d00b3550 2024-09-21T07:50:49.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.934+0000 7f08cfe00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08ac078cb0 0x7f08ac07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:49.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.935+0000 7f08cfe00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08ac078cb0 0x7f08ac07b1c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08c0001120 tx=0x7f08c0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:49.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:49.939+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f08d01f6740 con 0x7f08d00b3550 2024-09-21T07:50:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.128+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f08d0154b10 con 0x7f08d00b3550 2024-09-21T07:50:50.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.128+0000 7f08ccc00640 1 -- 172.21.15.88:0/2978570071 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v82) ==== 84+0+65 (secure 0 0 0) 0x7f08b803f030 con 0x7f08d00b3550 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:50:50.130 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:50:50.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08ac078cb0 msgr2=0x7f08ac07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08ac078cb0 0x7f08ac07b1c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f08c0001120 tx=0x7f08c0009040 comp rx=0 tx=0).stop 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 msgr2=0x7f08d01f6200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 0x7f08d01f6200 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f08b802ab00 tx=0x7f08b8033ba0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 shutdown_connections 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08ac078cb0 0x7f08ac07b1c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08d015d760 0x7f08d0156370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08d00b3550 0x7f08d01f6200 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 --2- 172.21.15.88:0/2978570071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f08d00ad0e0 0x7f08d00ceb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 >> 172.21.15.88:0/2978570071 conn(0x7f08d006d730 msgr2=0x7f08d0152960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 shutdown_connections 2024-09-21T07:50:50.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.131+0000 7f08d6e00640 1 -- 172.21.15.88:0/2978570071 wait complete. 2024-09-21T07:50:50.154 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:50:50.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:50:50.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 -- 172.21.15.88:0/3207881478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b6700 msgr2=0x7ff4840d2b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 --2- 172.21.15.88:0/3207881478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b6700 0x7ff4840d2b90 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7ff46c005fd0 tx=0x7ff46c02a530 comp rx=0 tx=0).stop 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 -- 172.21.15.88:0/3207881478 shutdown_connections 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 --2- 172.21.15.88:0/3207881478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b6700 0x7ff4840d2b90 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 --2- 172.21.15.88:0/3207881478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840b9bc0 0x7ff4840d2650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 -- 172.21.15.88:0/3207881478 >> 172.21.15.88:0/3207881478 conn(0x7ff48408bfe0 msgr2=0x7ff4840d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 -- 172.21.15.88:0/3207881478 shutdown_connections 2024-09-21T07:50:50.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.379+0000 7ff488a00640 1 -- 172.21.15.88:0/3207881478 wait complete. 2024-09-21T07:50:50.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.381+0000 7ff488a00640 1 Processor -- start 2024-09-21T07:50:50.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.381+0000 7ff488a00640 1 -- start start 2024-09-21T07:50:50.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff488a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4840b6700 0x7ff4841f9a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:50.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff488a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 0x7ff484156f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff488a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ceb70 0x7ff484161390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff488a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840b91b0 con 0x7ff4840b9bc0 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff488a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff4840b9030 con 0x7ff4840b6700 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff488a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff4840b9330 con 0x7ff4840ceb70 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff479e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 0x7ff484156f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff479e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 0x7ff484156f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58110/0 (socket says 172.21.15.88:58110) 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.383+0000 7ff479e00640 1 -- 172.21.15.88:0/1461159753 learned_addr learned my addr 172.21.15.88:0/1461159753 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff483200640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ceb70 0x7ff484161390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff482800640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4840b6700 0x7ff4841f9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff479e00640 1 -- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ceb70 msgr2=0x7ff484161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff479e00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ceb70 0x7ff484161390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff479e00640 1 -- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4840b6700 msgr2=0x7ff4841f9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff479e00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4840b6700 0x7ff4841f9a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:50.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff479e00640 1 -- 172.21.15.88:0/1461159753 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff4840cc890 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff483200640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ceb70 0x7ff484161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff482800640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4840b6700 0x7ff4841f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff479e00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 0x7ff484156f90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff46c034040 tx=0x7ff46c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c03e030 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff46c0047a0 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff4840ccb70 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff46c035040 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.384+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff4840cd130 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.385+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff4840cf940 con 0x7ff4840b9bc0 2024-09-21T07:50:50.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.387+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ff46c041030 con 0x7ff4840b9bc0 2024-09-21T07:50:50.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.387+0000 7ff47b400640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff460078cb0 0x7ff46007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:50.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.388+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 4857+0+0 (secure 0 0 0) 0x7ff46c0bb630 con 0x7ff4840b9bc0 2024-09-21T07:50:50.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.388+0000 7ff482800640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff460078cb0 0x7ff46007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:50.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.388+0000 7ff482800640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff460078cb0 0x7ff46007b1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff47400ba60 tx=0x7ff474005cf0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:50.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.392+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff46c083b00 con 0x7ff4840b9bc0 2024-09-21T07:50:50.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:50.577+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7ff48406f1f0 con 0x7ff4840b9bc0 2024-09-21T07:50:51.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.178+0000 7ff47b400640 1 -- 172.21.15.88:0/1461159753 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v83) ==== 138+0+0 (secure 0 0 0) 0x7ff46c083400 con 0x7ff4840b9bc0 2024-09-21T07:50:51.180 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff460078cb0 msgr2=0x7ff46007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff460078cb0 0x7ff46007b1c0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff47400ba60 tx=0x7ff474005cf0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 msgr2=0x7ff484156f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 0x7ff484156f90 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7ff46c034040 tx=0x7ff46c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 shutdown_connections 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff460078cb0 0x7ff46007b1c0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff4840ceb70 0x7ff484161390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff4840b9bc0 0x7ff484156f90 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 --2- 172.21.15.88:0/1461159753 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff4840b6700 0x7ff4841f9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.181+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 >> 172.21.15.88:0/1461159753 conn(0x7ff48408bfe0 msgr2=0x7ff4840d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:51.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.182+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 shutdown_connections 2024-09-21T07:50:51.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.182+0000 7ff488a00640 1 -- 172.21.15.88:0/1461159753 wait complete. 2024-09-21T07:50:51.213 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 -- 172.21.15.88:0/2449911176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e64073c50 msgr2=0x7f1e640cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2449911176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e64073c50 0x7f1e640cd610 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c005fd0 tx=0x7f1e5c02ab20 comp rx=0 tx=0).stop 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 -- 172.21.15.88:0/2449911176 shutdown_connections 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2449911176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e64073c50 0x7f1e640cd610 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2449911176 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e64145550 0x7f1e6406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 -- 172.21.15.88:0/2449911176 >> 172.21.15.88:0/2449911176 conn(0x7f1e6406d1c0 msgr2=0x7f1e6406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 -- 172.21.15.88:0/2449911176 shutdown_connections 2024-09-21T07:50:51.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.440+0000 7f1e69a00640 1 -- 172.21.15.88:0/2449911176 wait complete. 2024-09-21T07:50:51.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.442+0000 7f1e69a00640 1 Processor -- start 2024-09-21T07:50:51.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.443+0000 7f1e69a00640 1 -- start start 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e69a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e64145550 0x7f1e641f3ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e69a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e64145a40 0x7f1e640af4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e69a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 0x7f1e6415e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e69a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640ce1f0 con 0x7f1e6415fd10 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e69a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1e640ce070 con 0x7f1e64145550 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e69a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1e640ce370 con 0x7f1e64145a40 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e63400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 0x7f1e6415e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e63400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 0x7f1e6415e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58126/0 (socket says 172.21.15.88:58126) 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e63400640 1 -- 172.21.15.88:0/2483807351 learned_addr learned my addr 172.21.15.88:0/2483807351 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e62a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e64145550 0x7f1e641f3ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:51.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.444+0000 7f1e62000640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e64145a40 0x7f1e640af4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:51.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e63400640 1 -- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e64145a40 msgr2=0x7f1e640af4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:51.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e63400640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e64145a40 0x7f1e640af4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e63400640 1 -- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e64145550 msgr2=0x7f1e641f3ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:51.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e63400640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e64145550 0x7f1e641f3ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:51.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e63400640 1 -- 172.21.15.88:0/2483807351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e640ae420 con 0x7f1e6415fd10 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e63400640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 0x7f1e6415e940 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c006010 tx=0x7f1e5c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c044070 con 0x7f1e6415fd10 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1e5c02c820 con 0x7f1e6415fd10 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e5c0044b0 con 0x7f1e6415fd10 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.445+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e640ae700 con 0x7f1e6415fd10 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.446+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e640aecc0 con 0x7f1e6415fd10 2024-09-21T07:50:51.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.447+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e640ce8d0 con 0x7f1e6415fd10 2024-09-21T07:50:51.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.448+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1e5c004650 con 0x7f1e6415fd10 2024-09-21T07:50:51.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.449+0000 7f1e5aa00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1e44078cb0 0x7f1e4407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:51.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.449+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 5227+0+0 (secure 0 0 0) 0x7f1e5c0bb550 con 0x7f1e6415fd10 2024-09-21T07:50:51.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.449+0000 7f1e62a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1e44078cb0 0x7f1e4407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:51.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.450+0000 7f1e62a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1e44078cb0 0x7f1e4407b1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400bb50 tx=0x7f1e5400b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:51.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.453+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1e5c032030 con 0x7f1e6415fd10 2024-09-21T07:50:51.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:51.640+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f1e64141040 con 0x7f1e6415fd10 2024-09-21T07:50:52.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.186+0000 7f1e5aa00640 1 -- 172.21.15.88:0/2483807351 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v84) ==== 170+0+0 (secure 0 0 0) 0x7f1e5c0833e0 con 0x7f1e6415fd10 2024-09-21T07:50:52.187 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.189+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1e44078cb0 msgr2=0x7f1e4407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.189+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1e44078cb0 0x7f1e4407b1c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f1e5400bb50 tx=0x7f1e5400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 msgr2=0x7f1e6415e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 0x7f1e6415e940 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f1e5c006010 tx=0x7f1e5c003f10 comp rx=0 tx=0).stop 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 shutdown_connections 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1e44078cb0 0x7f1e4407b1c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1e64145a40 0x7f1e640af4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e6415fd10 0x7f1e6415e940 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 --2- 172.21.15.88:0/2483807351 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1e64145550 0x7f1e641f3ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 >> 172.21.15.88:0/2483807351 conn(0x7f1e6406d1c0 msgr2=0x7f1e6413edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:52.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 shutdown_connections 2024-09-21T07:50:52.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.190+0000 7f1e69a00640 1 -- 172.21.15.88:0/2483807351 wait complete. 2024-09-21T07:50:52.232 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.455+0000 7f994c600640 1 -- 172.21.15.88:0/4174609259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c003a50 msgr2=0x7f992c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.455+0000 7f994c600640 1 --2- 172.21.15.88:0/4174609259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c003a50 0x7f992c003ec0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f99380016c0 tx=0x7f992c005830 comp rx=0 tx=0).stop 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.456+0000 7f994c600640 1 -- 172.21.15.88:0/4174609259 shutdown_connections 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.456+0000 7f994c600640 1 --2- 172.21.15.88:0/4174609259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f992c003a50 0x7f992c003ec0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.456+0000 7f994c600640 1 -- 172.21.15.88:0/4174609259 >> 172.21.15.88:0/4174609259 conn(0x7f994406d1c0 msgr2=0x7f994406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.456+0000 7f994c600640 1 -- 172.21.15.88:0/4174609259 shutdown_connections 2024-09-21T07:50:52.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.456+0000 7f994c600640 1 -- 172.21.15.88:0/4174609259 wait complete. 2024-09-21T07:50:52.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.458+0000 7f994c600640 1 Processor -- start 2024-09-21T07:50:52.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.458+0000 7f994c600640 1 -- start start 2024-09-21T07:50:52.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.459+0000 7f994c600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c003a50 0x7f99441f6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:52.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.459+0000 7f994c600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f994415c150 0x7f994413b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.459+0000 7f994c600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 0x7f99440b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.459+0000 7f994c600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9944073540 con 0x7f994413b570 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.459+0000 7f994c600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9944070180 con 0x7f994415c150 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.459+0000 7f994c600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99440736c0 con 0x7f992c003a50 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 0x7f99440b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 0x7f99440b1610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58154/0 (socket says 172.21.15.88:58154) 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 -- 172.21.15.88:0/301123036 learned_addr learned my addr 172.21.15.88:0/301123036 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f9949c00640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c003a50 0x7f99441f6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f9949200640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f994415c150 0x7f994413b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 -- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c003a50 msgr2=0x7f99441f6190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c003a50 0x7f99441f6190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 -- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f994415c150 msgr2=0x7f994413b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:52.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f994415c150 0x7f994413b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:52.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.460+0000 7f994a600640 1 -- 172.21.15.88:0/301123036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f994413b980 con 0x7f994413b570 2024-09-21T07:50:52.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.461+0000 7f994a600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 0x7f99440b1610 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f993c005fd0 tx=0x7f993c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:52.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.461+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c017070 con 0x7f994413b570 2024-09-21T07:50:52.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.461+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f993c015070 con 0x7f994413b570 2024-09-21T07:50:52.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.461+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f993c003970 con 0x7f994413b570 2024-09-21T07:50:52.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.461+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f994413c5a0 con 0x7f994413b570 2024-09-21T07:50:52.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.461+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99440b0e10 con 0x7f994413b570 2024-09-21T07:50:52.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.462+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9944146350 con 0x7f994413b570 2024-09-21T07:50:52.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.464+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f993c011440 con 0x7f994413b570 2024-09-21T07:50:52.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.464+0000 7f9941600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9924078cb0 0x7f992407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:52.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.465+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 5609+0+0 (secure 0 0 0) 0x7f993c09b3c0 con 0x7f994413b570 2024-09-21T07:50:52.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.465+0000 7f9949c00640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9924078cb0 0x7f992407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:52.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.465+0000 7f9949c00640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9924078cb0 0x7f992407b1c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9938032000 tx=0x7f9938001190 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:52.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.469+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9944146350 con 0x7f994413b570 2024-09-21T07:50:52.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:52.692+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f994413c330 con 0x7f994413b570 2024-09-21T07:50:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.188+0000 7f9941600640 1 -- 172.21.15.88:0/301123036 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v52) ==== 174+0+0 (secure 0 0 0) 0x7f994413b980 con 0x7f994413b570 2024-09-21T07:50:53.189 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 12 and data pool 13 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9924078cb0 msgr2=0x7f992407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9924078cb0 0x7f992407b1c0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9938032000 tx=0x7f9938001190 comp rx=0 tx=0).stop 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 msgr2=0x7f99440b1610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 0x7f99440b1610 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f993c005fd0 tx=0x7f993c010f40 comp rx=0 tx=0).stop 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 shutdown_connections 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9924078cb0 0x7f992407b1c0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f994413b570 0x7f99440b1610 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f994415c150 0x7f994413b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 --2- 172.21.15.88:0/301123036 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f992c003a50 0x7f99441f6190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.193+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 >> 172.21.15.88:0/301123036 conn(0x7f994406d1c0 msgr2=0x7f99441420d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.195+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 shutdown_connections 2024-09-21T07:50:53.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.195+0000 7f994c600640 1 -- 172.21.15.88:0/301123036 wait complete. 2024-09-21T07:50:53.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.467+0000 7f63ee000640 1 -- 172.21.15.88:0/150528802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e815fae0 msgr2=0x7f63e80b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.467+0000 7f63ee000640 1 --2- 172.21.15.88:0/150528802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e815fae0 0x7f63e80b3750 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f63e0005fd0 tx=0x7f63e002aa10 comp rx=0 tx=0).stop 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.467+0000 7f63ee000640 1 -- 172.21.15.88:0/150528802 shutdown_connections 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.467+0000 7f63ee000640 1 --2- 172.21.15.88:0/150528802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e815fae0 0x7f63e80b3750 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.467+0000 7f63ee000640 1 --2- 172.21.15.88:0/150528802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63e80aebc0 0x7f63e8152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.467+0000 7f63ee000640 1 -- 172.21.15.88:0/150528802 >> 172.21.15.88:0/150528802 conn(0x7f63e806d730 msgr2=0x7f63e80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.468+0000 7f63ee000640 1 -- 172.21.15.88:0/150528802 shutdown_connections 2024-09-21T07:50:53.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.468+0000 7f63ee000640 1 -- 172.21.15.88:0/150528802 wait complete. 2024-09-21T07:50:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.470+0000 7f63ee000640 1 Processor -- start 2024-09-21T07:50:53.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.470+0000 7f63ee000640 1 -- start start 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63ee000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63e80aebc0 0x7f63e8155860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 0x7f63e80cc560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63ee000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e815b120 0x7f63e815b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63ee000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80b4210 con 0x7f63e80b2750 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63ee000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f63e80b4090 con 0x7f63e815b120 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63ee000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f63e80b4390 con 0x7f63e80aebc0 2024-09-21T07:50:53.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.471+0000 7f63e6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 0x7f63e80cc560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 0x7f63e80cc560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58168/0 (socket says 172.21.15.88:58168) 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 -- 172.21.15.88:0/2290354298 learned_addr learned my addr 172.21.15.88:0/2290354298 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e7400640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63e80aebc0 0x7f63e8155860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e7e00640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e815b120 0x7f63e815b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 -- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63e80aebc0 msgr2=0x7f63e8155860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63e80aebc0 0x7f63e8155860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 -- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e815b120 msgr2=0x7f63e815b530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e815b120 0x7f63e815b530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 -- 172.21.15.88:0/2290354298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63e815ba70 con 0x7f63e80b2750 2024-09-21T07:50:53.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.472+0000 7f63e6a00640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 0x7f63e80cc560 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c3a0 tx=0x7f63d400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:53.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.473+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4017070 con 0x7f63e80b2750 2024-09-21T07:50:53.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.473+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f63d40053c0 con 0x7f63e80b2750 2024-09-21T07:50:53.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.473+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d4003850 con 0x7f63e80b2750 2024-09-21T07:50:53.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.473+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63e81f34f0 con 0x7f63e80b2750 2024-09-21T07:50:53.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.473+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63e81f3ab0 con 0x7f63e80b2750 2024-09-21T07:50:53.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.474+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63e80b48b0 con 0x7f63e80b2750 2024-09-21T07:50:53.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.475+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f63d401a070 con 0x7f63e80b2750 2024-09-21T07:50:53.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.476+0000 7f63df400640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f63c8078cb0 0x7f63c807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.476+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7f63d409a380 con 0x7f63e80b2750 2024-09-21T07:50:53.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.477+0000 7f63e7400640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f63c8078cb0 0x7f63c807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.477+0000 7f63e7400640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f63c8078cb0 0x7f63c807b1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005fd0 tx=0x7f63d800a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:53.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.481+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f63e815ba70 con 0x7f63e80b2750 2024-09-21T07:50:53.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.667+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f63e815ac00 con 0x7f63e80b2750 2024-09-21T07:50:53.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.668+0000 7f63df400640 1 -- 172.21.15.88:0/2290354298 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16757 (secure 0 0 0) 0x7f63d4061eb0 con 0x7f63e80b2750 2024-09-21T07:50:53.670 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:53.670 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":85,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:52.693218+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:50.589543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:51.652329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:53.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f63c8078cb0 msgr2=0x7f63c807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f63c8078cb0 0x7f63c807b1c0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63d8005fd0 tx=0x7f63d800a250 comp rx=0 tx=0).stop 2024-09-21T07:50:53.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 msgr2=0x7f63e80cc560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 0x7f63e80cc560 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f63d400c3a0 tx=0x7f63d400c860 comp rx=0 tx=0).stop 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 shutdown_connections 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f63c8078cb0 0x7f63c807b1c0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f63e815b120 0x7f63e815b530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f63e80b2750 0x7f63e80cc560 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 --2- 172.21.15.88:0/2290354298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f63e80aebc0 0x7f63e8155860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 >> 172.21.15.88:0/2290354298 conn(0x7f63e806d730 msgr2=0x7f63e80b1da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 shutdown_connections 2024-09-21T07:50:53.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.671+0000 7f63ee000640 1 -- 172.21.15.88:0/2290354298 wait complete. 2024-09-21T07:50:53.694 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:53.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 -- 172.21.15.88:0/1360742479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c070000 msgr2=0x7fb60c070410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 --2- 172.21.15.88:0/1360742479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c070000 0x7fb60c070410 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7fb60c0d1190 tx=0x7fb60402a580 comp rx=0 tx=0).stop 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 -- 172.21.15.88:0/1360742479 shutdown_connections 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 --2- 172.21.15.88:0/1360742479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c070000 0x7fb60c070410 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 --2- 172.21.15.88:0/1360742479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 0x7fb60c073270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 --2- 172.21.15.88:0/1360742479 >> v2:172.21.15.175:3300/0 conn(0x7fb60c0af050 0x7fb60c0ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 -- 172.21.15.88:0/1360742479 >> 172.21.15.88:0/1360742479 conn(0x7fb60c0b1e40 msgr2=0x7fb60c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 -- 172.21.15.88:0/1360742479 shutdown_connections 2024-09-21T07:50:53.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.916+0000 7fb612c00640 1 -- 172.21.15.88:0/1360742479 wait complete. 2024-09-21T07:50:53.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.918+0000 7fb612c00640 1 Processor -- start 2024-09-21T07:50:53.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.918+0000 7fb612c00640 1 -- start start 2024-09-21T07:50:53.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c070000 0x7fb60c15d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 0x7fb60c13f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 0x7fb60c141ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c15c0b0 con 0x7fb60c0af050 2024-09-21T07:50:53.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb60c15bf30 con 0x7fb60c070000 2024-09-21T07:50:53.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb60c15c230 con 0x7fb60c072e60 2024-09-21T07:50:53.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 0x7fb60c141ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 0x7fb60c141ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58178/0 (socket says 172.21.15.88:58178) 2024-09-21T07:50:53.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.920+0000 7fb612200640 1 -- 172.21.15.88:0/958623921 learned_addr learned my addr 172.21.15.88:0/958623921 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:53.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb611800640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c070000 0x7fb60c15d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb610e00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 0x7fb60c13f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612200640 1 -- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 msgr2=0x7fb60c13f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612200640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 0x7fb60c13f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612200640 1 -- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c070000 msgr2=0x7fb60c15d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:53.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612200640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c070000 0x7fb60c15d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612200640 1 -- 172.21.15.88:0/958623921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb60c13fb60 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb610e00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 0x7fb60c13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612200640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 0x7fb60c141ef0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb604034040 tx=0x7fb604003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb60403e030 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb604003d70 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb60c15db10 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb604035040 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.921+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb60c15e0d0 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.922+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb60c15d880 con 0x7fb60c0af050 2024-09-21T07:50:53.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.924+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb604041030 con 0x7fb60c0af050 2024-09-21T07:50:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.925+0000 7fb609600640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5ec078cb0 0x7fb5ec07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.925+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 5709+0+0 (secure 0 0 0) 0x7fb6040bb780 con 0x7fb60c0af050 2024-09-21T07:50:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.925+0000 7fb611800640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5ec078cb0 0x7fb5ec07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.925+0000 7fb611800640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5ec078cb0 0x7fb5ec07b1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb600005fd0 tx=0x7fb6000097d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:53.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:53.929+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb604083990 con 0x7fb60c0af050 2024-09-21T07:50:54.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.117+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb60c0b2590 con 0x7fb60c0af050 2024-09-21T07:50:54.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.118+0000 7fb609600640 1 -- 172.21.15.88:0/958623921 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v85) ==== 74+0+16757 (secure 0 0 0) 0x7fb604083290 con 0x7fb60c0af050 2024-09-21T07:50:54.120 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:54.120 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":85,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:52.693218+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:50.589543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:51.652329+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5ec078cb0 msgr2=0x7fb5ec07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5ec078cb0 0x7fb5ec07b1c0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb600005fd0 tx=0x7fb6000097d0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 msgr2=0x7fb60c141ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 0x7fb60c141ef0 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7fb604034040 tx=0x7fb604003880 comp rx=0 tx=0).stop 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 shutdown_connections 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5ec078cb0 0x7fb5ec07b1c0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb60c072e60 0x7fb60c13f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb60c0af050 0x7fb60c141ef0 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 --2- 172.21.15.88:0/958623921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb60c070000 0x7fb60c15d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 >> 172.21.15.88:0/958623921 conn(0x7fb60c0b1e40 msgr2=0x7fb60c09b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 shutdown_connections 2024-09-21T07:50:54.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.121+0000 7fb612c00640 1 -- 172.21.15.88:0/958623921 wait complete. 2024-09-21T07:50:54.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:50:54.164 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-09-21T07:50:54.163+0000 7f32e6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:50:54.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 -- 172.21.15.88:0/3128589570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145550 msgr2=0x7f7fc806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 --2- 172.21.15.88:0/3128589570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145550 0x7f7fc806fb40 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8005fd0 tx=0x7f7fb802a670 comp rx=0 tx=0).stop 2024-09-21T07:50:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 -- 172.21.15.88:0/3128589570 shutdown_connections 2024-09-21T07:50:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 --2- 172.21.15.88:0/3128589570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fc8073c50 0x7f7fc80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 --2- 172.21.15.88:0/3128589570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145550 0x7f7fc806fb40 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 -- 172.21.15.88:0/3128589570 >> 172.21.15.88:0/3128589570 conn(0x7f7fc806d1c0 msgr2=0x7f7fc806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 -- 172.21.15.88:0/3128589570 shutdown_connections 2024-09-21T07:50:54.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.369+0000 7f7fce800640 1 -- 172.21.15.88:0/3128589570 wait complete. 2024-09-21T07:50:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.371+0000 7f7fce800640 1 Processor -- start 2024-09-21T07:50:54.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.371+0000 7f7fce800640 1 -- start start 2024-09-21T07:50:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fce800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8145550 0x7f7fc80b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fce800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 0x7f7fc80cbff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fce800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fc80c6f40 0x7f7fc81f5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fce800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc80ce150 con 0x7f7fc8145a40 2024-09-21T07:50:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fce800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fc80cdfd0 con 0x7f7fc8145550 2024-09-21T07:50:54.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fce800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7fc80ce2d0 con 0x7f7fc80c6f40 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 0x7f7fc80cbff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 0x7f7fc80cbff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58196/0 (socket says 172.21.15.88:58196) 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 -- 172.21.15.88:0/3697517219 learned_addr learned my addr 172.21.15.88:0/3697517219 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc7400640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8145550 0x7f7fc80b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc7e00640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fc80c6f40 0x7f7fc81f5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 -- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fc80c6f40 msgr2=0x7f7fc81f5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fc80c6f40 0x7f7fc81f5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 -- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8145550 msgr2=0x7f7fc80b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8145550 0x7f7fc80b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.373+0000 7f7fc6a00640 1 -- 172.21.15.88:0/3697517219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fc80c5ea0 con 0x7f7fc8145a40 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.374+0000 7f7fc6a00640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 0x7f7fc80cbff0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c370 tx=0x7f7fb400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.374+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4017070 con 0x7f7fc8145a40 2024-09-21T07:50:54.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.374+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7fb40053c0 con 0x7f7fc8145a40 2024-09-21T07:50:54.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.374+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fc80c6180 con 0x7f7fc8145a40 2024-09-21T07:50:54.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.374+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fb4003850 con 0x7f7fc8145a40 2024-09-21T07:50:54.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.374+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fc80c6740 con 0x7f7fc8145a40 2024-09-21T07:50:54.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.375+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fc80ce7f0 con 0x7f7fc8145a40 2024-09-21T07:50:54.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.376+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7fb401a070 con 0x7f7fc8145a40 2024-09-21T07:50:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.377+0000 7f7fbf400640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7fa8078ce0 0x7f7fa807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.377+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f7fb409a290 con 0x7f7fc8145a40 2024-09-21T07:50:54.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.378+0000 7f7fc7400640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7fa8078ce0 0x7f7fa807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:54.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.378+0000 7f7fc7400640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7fa8078ce0 0x7f7fa807b1f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8033000 tx=0x7f7fb8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:54.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.382+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7fb4061e50 con 0x7f7fc8145a40 2024-09-21T07:50:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:54.604+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f7fc8143e80 con 0x7f7fc8145a40 2024-09-21T07:50:55.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.217+0000 7f7fbf400640 1 -- 172.21.15.88:0/3697517219 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v54) ==== 118+0+0 (secure 0 0 0) 0x7f7fb4066880 con 0x7f7fc8145a40 2024-09-21T07:50:55.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7fa8078ce0 msgr2=0x7f7fa807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7fa8078ce0 0x7f7fa807b1f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7fb8033000 tx=0x7f7fb8004730 comp rx=0 tx=0).stop 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 msgr2=0x7f7fc80cbff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 0x7f7fc80cbff0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f7fb400c370 tx=0x7f7fb400c830 comp rx=0 tx=0).stop 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 shutdown_connections 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7fa8078ce0 0x7f7fa807b1f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fc80c6f40 0x7f7fc81f5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fc8145a40 0x7f7fc80cbff0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 --2- 172.21.15.88:0/3697517219 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fc8145550 0x7f7fc80b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 >> 172.21.15.88:0/3697517219 conn(0x7f7fc806d1c0 msgr2=0x7f7fc81411c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 shutdown_connections 2024-09-21T07:50:55.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.220+0000 7f7fce800640 1 -- 172.21.15.88:0/3697517219 wait complete. 2024-09-21T07:50:55.245 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:50:55.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:50:55.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.507+0000 7ffa53e00640 1 -- 172.21.15.88:0/2200936186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c0747e0 msgr2=0x7ffa4c153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.507+0000 7ffa53e00640 1 --2- 172.21.15.88:0/2200936186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c0747e0 0x7ffa4c153950 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7ffa4c073720 tx=0x7ffa3802a560 comp rx=0 tx=0).stop 2024-09-21T07:50:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.508+0000 7ffa53e00640 1 -- 172.21.15.88:0/2200936186 shutdown_connections 2024-09-21T07:50:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.508+0000 7ffa53e00640 1 --2- 172.21.15.88:0/2200936186 >> v2:172.21.15.175:3300/0 conn(0x7ffa4c06ddb0 0x7ffa4c06e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.508+0000 7ffa53e00640 1 --2- 172.21.15.88:0/2200936186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c0747e0 0x7ffa4c153950 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.508+0000 7ffa53e00640 1 -- 172.21.15.88:0/2200936186 >> 172.21.15.88:0/2200936186 conn(0x7ffa4c073920 msgr2=0x7ffa4c073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:55.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.508+0000 7ffa53e00640 1 -- 172.21.15.88:0/2200936186 shutdown_connections 2024-09-21T07:50:55.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.508+0000 7ffa53e00640 1 -- 172.21.15.88:0/2200936186 wait complete. 2024-09-21T07:50:55.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.510+0000 7ffa53e00640 1 Processor -- start 2024-09-21T07:50:55.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.510+0000 7ffa53e00640 1 -- start start 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 0x7ffa4c155a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa4c06f960 0x7ffa4c0a94b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa53e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa4c0747e0 0x7ffa4c1454e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa53e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c15d4d0 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa53e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ffa4c15d350 con 0x7ffa4c0747e0 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa53e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ffa4c15d650 con 0x7ffa4c06f960 2024-09-21T07:50:55.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 0x7ffa4c155a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 0x7ffa4c155a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:58218/0 (socket says 172.21.15.88:58218) 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 -- 172.21.15.88:0/3316117786 learned_addr learned my addr 172.21.15.88:0/3316117786 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 -- 172.21.15.88:0/3316117786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa4c06f960 msgr2=0x7ffa4c0a94b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa4c06f960 0x7ffa4c0a94b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 -- 172.21.15.88:0/3316117786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa4c0747e0 msgr2=0x7ffa4c1454e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa4c0747e0 0x7ffa4c1454e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.512+0000 7ffa51400640 1 -- 172.21.15.88:0/3316117786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4c0aa2a0 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.513+0000 7ffa51400640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 0x7ffa4c155a10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000c3a0 tx=0x7ffa4000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.513+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40017070 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.513+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ffa400053c0 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.513+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40003850 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.513+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4c0aa580 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.513+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa4c154300 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.514+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa4c15ec80 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.515+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ffa4001a070 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.516+0000 7ffa49600640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ffa2c078cb0 0x7ffa2c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:55.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.516+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7ffa4009a270 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.516+0000 7ffa50a00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ffa2c078cb0 0x7ffa2c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:55.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.517+0000 7ffa50a00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ffa2c078cb0 0x7ffa2c07b1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa38033000 tx=0x7ffa380046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:55.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.521+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ffa4c06e700 con 0x7ffa4c06ddb0 2024-09-21T07:50:55.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:55.743+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7ffa4c06f640 con 0x7ffa4c06ddb0 2024-09-21T07:50:56.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.223+0000 7ffa49600640 1 -- 172.21.15.88:0/3316117786 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v55) ==== 122+0+0 (secure 0 0 0) 0x7ffa400625b0 con 0x7ffa4c06ddb0 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ffa2c078cb0 msgr2=0x7ffa2c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ffa2c078cb0 0x7ffa2c07b1c0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ffa38033000 tx=0x7ffa380046c0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 msgr2=0x7ffa4c155a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 0x7ffa4c155a10 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000c3a0 tx=0x7ffa4000c860 comp rx=0 tx=0).stop 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 shutdown_connections 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ffa2c078cb0 0x7ffa2c07b1c0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ffa4c06f960 0x7ffa4c0a94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ffa4c0747e0 0x7ffa4c1454e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 --2- 172.21.15.88:0/3316117786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa4c06ddb0 0x7ffa4c155a10 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 >> 172.21.15.88:0/3316117786 conn(0x7ffa4c073920 msgr2=0x7ffa4c0b1650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:56.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 shutdown_connections 2024-09-21T07:50:56.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.226+0000 7ffa53e00640 1 -- 172.21.15.88:0/3316117786 wait complete. 2024-09-21T07:50:56.252 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:50:56.253 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 -- 172.21.15.88:0/1201066585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 msgr2=0x7f9f440b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 --2- 172.21.15.88:0/1201066585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b3960 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7f9f44072d60 tx=0x7f9f3c02a980 comp rx=0 tx=0).stop 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 -- 172.21.15.88:0/1201066585 shutdown_connections 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 --2- 172.21.15.88:0/1201066585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b3960 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 --2- 172.21.15.88:0/1201066585 >> v2:172.21.15.175:3300/0 conn(0x7f9f440ad0e0 0x7f9f440ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 -- 172.21.15.88:0/1201066585 >> 172.21.15.88:0/1201066585 conn(0x7f9f4406d730 msgr2=0x7f9f440b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 -- 172.21.15.88:0/1201066585 shutdown_connections 2024-09-21T07:50:56.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.478+0000 7f9f48c00640 1 -- 172.21.15.88:0/1201066585 wait complete. 2024-09-21T07:50:56.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.481+0000 7f9f48c00640 1 Processor -- start 2024-09-21T07:50:56.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.481+0000 7f9f48c00640 1 -- start start 2024-09-21T07:50:56.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f48c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f440ad0e0 0x7f9f441516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f48c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f48c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4415d760 0x7f9f440b12e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f48c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9f440b4da0 con 0x7f9f440b3550 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f48c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9f440b4c20 con 0x7f9f4415d760 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f48c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9f440b3ea0 con 0x7f9f440ad0e0 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.482+0000 7f9f39400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36602/0 (socket says 172.21.15.88:36602) 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 -- 172.21.15.88:0/959946159 learned_addr learned my addr 172.21.15.88:0/959946159 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f42800640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4415d760 0x7f9f440b12e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f41e00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f440ad0e0 0x7f9f441516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 -- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f440ad0e0 msgr2=0x7f9f441516e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f440ad0e0 0x7f9f441516e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 -- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4415d760 msgr2=0x7f9f440b12e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4415d760 0x7f9f440b12e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 -- 172.21.15.88:0/959946159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4409e040 con 0x7f9f440b3550 2024-09-21T07:50:56.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f41e00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f440ad0e0 0x7f9f441516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f42800640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4415d760 0x7f9f440b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f39400640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b0da0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00d6d0 tx=0x7f9f2c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c017070 con 0x7f9f440b3550 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.483+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f2c004310 con 0x7f9f440b3550 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.484+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f2c009a80 con 0x7f9f440b3550 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.484+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f44154090 con 0x7f9f440b3550 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.484+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f44154650 con 0x7f9f440b3550 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.485+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f440b4360 con 0x7f9f440b3550 2024-09-21T07:50:56.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.486+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f9f2c0035f0 con 0x7f9f440b3550 2024-09-21T07:50:56.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.487+0000 7f9f3aa00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f20078ce0 0x7f9f2007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.487+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f9f2c09b2d0 con 0x7f9f440b3550 2024-09-21T07:50:56.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.488+0000 7f9f41e00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f20078ce0 0x7f9f2007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.488+0000 7f9f41e00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f20078ce0 0x7f9f2007b1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400a830 tx=0x7f9f34008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:56.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.492+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f2c063450 con 0x7f9f440b3550 2024-09-21T07:50:56.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.717+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9f4415e380 con 0x7f9f440b3550 2024-09-21T07:50:56.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.718+0000 7f9f3aa00640 1 -- 172.21.15.88:0/959946159 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5228 (secure 0 0 0) 0x7f9f2c062d50 con 0x7f9f440b3550 2024-09-21T07:50:56.719 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:56.719 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":55,"btime":"2024-09-21T07:50:56:218406+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3755060210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3755060210},{"type":"v1","addr":"172.21.15.175:6835","nonce":3755060210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:52.693177+0000","modified":"2024-09-21T07:50:55.744683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1717882335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1717882335},{"type":"v1","addr":"172.21.15.88:6835","nonce":1717882335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6600,"qdb_cluster":[6600]},"id":6}]} 2024-09-21T07:50:56.720 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 55 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f20078ce0 msgr2=0x7f9f2007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f20078ce0 0x7f9f2007b1f0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400a830 tx=0x7f9f34008040 comp rx=0 tx=0).stop 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 msgr2=0x7f9f440b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b0da0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f9f2c00d6d0 tx=0x7f9f2c00db90 comp rx=0 tx=0).stop 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 shutdown_connections 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f20078ce0 0x7f9f2007b1f0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9f4415d760 0x7f9f440b12e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9f440b3550 0x7f9f440b0da0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 --2- 172.21.15.88:0/959946159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9f440ad0e0 0x7f9f441516e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 >> 172.21.15.88:0/959946159 conn(0x7f9f4406d730 msgr2=0x7f9f440cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:56.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 shutdown_connections 2024-09-21T07:50:56.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.721+0000 7f9f48c00640 1 -- 172.21.15.88:0/959946159 wait complete. 2024-09-21T07:50:56.747 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:50:56.827 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:56.831 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:50:56.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c3e00640 1 -- 172.21.15.88:0/585894266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00a4004d90 msgr2=0x7f00a4005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c3e00640 1 --2- 172.21.15.88:0/585894266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00a4004d90 0x7f00a4005220 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009ed0 tx=0x7f00a4005830 comp rx=0 tx=0).stop 2024-09-21T07:50:56.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c1e00640 1 -- 172.21.15.88:0/585894266 reap_dead start 2024-09-21T07:50:56.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c3e00640 1 -- 172.21.15.88:0/585894266 shutdown_connections 2024-09-21T07:50:56.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c3e00640 1 -- 172.21.15.88:0/585894266 >> 172.21.15.88:0/585894266 conn(0x7f00bc06d730 msgr2=0x7f00bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:56.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c3e00640 1 -- 172.21.15.88:0/585894266 shutdown_connections 2024-09-21T07:50:56.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.972+0000 7f00c3e00640 1 -- 172.21.15.88:0/585894266 wait complete. 2024-09-21T07:50:56.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.974+0000 7f00c3e00640 1 Processor -- start 2024-09-21T07:50:56.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.974+0000 7f00c3e00640 1 -- start start 2024-09-21T07:50:56.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a4004d90 0x7f00bc147440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00bc148c80 0x7f00bc14a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 0x7f00bc1f6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc0b4da0 con 0x7f00bc14a9e0 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00bc0b4c20 con 0x7f00a4004d90 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00bc165890 con 0x7f00bc148c80 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 0x7f00bc1f6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 0x7f00bc1f6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36626/0 (socket says 172.21.15.88:36626) 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 -- 172.21.15.88:0/1788219518 learned_addr learned my addr 172.21.15.88:0/1788219518 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 -- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00bc148c80 msgr2=0x7f00bc14a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c0a00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00bc148c80 0x7f00bc14a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1400640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a4004d90 0x7f00bc147440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00bc148c80 0x7f00bc14a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 -- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a4004d90 msgr2=0x7f00bc147440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a4004d90 0x7f00bc147440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.976+0000 7f00c1e00640 1 -- 172.21.15.88:0/1788219518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00bc147980 con 0x7f00bc14a9e0 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00c0a00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00bc148c80 0x7f00bc14a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00c1400640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a4004d90 0x7f00bc147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:56.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00c1e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 0x7f00bc1f6b40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009ed0 tx=0x7f00b4033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4043070 con 0x7f00bc14a9e0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00b4033d20 con 0x7f00bc14a9e0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00b4038040 con 0x7f00bc14a9e0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00bc156a90 con 0x7f00bc14a9e0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.977+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00bc0afdc0 con 0x7f00bc14a9e0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.978+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00bc1535c0 con 0x7f00bc14a9e0 2024-09-21T07:50:56.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.979+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f00b4046070 con 0x7f00bc14a9e0 2024-09-21T07:50:56.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.980+0000 7f00b9600640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f009c078ce0 0x7f009c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:56.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.980+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f00b40bada0 con 0x7f00bc14a9e0 2024-09-21T07:50:56.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.980+0000 7f00c1400640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f009c078ce0 0x7f009c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:56.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.981+0000 7f00c1400640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f009c078ce0 0x7f009c07b1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006c30 tx=0x7f00b00014f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:56.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:56.985+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00b40828d0 con 0x7f00bc14a9e0 2024-09-21T07:50:57.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.171+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f00bc1517b0 con 0x7f00bc14a9e0 2024-09-21T07:50:57.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.172+0000 7f00b9600640 1 -- 172.21.15.88:0/1788219518 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+16744 (secure 0 0 0) 0x7f00b4087300 con 0x7f00bc14a9e0 2024-09-21T07:50:57.174 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:57.174 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":86,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:50:54.203593+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:50.589543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:51.652329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":84,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f009c078ce0 msgr2=0x7f009c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f009c078ce0 0x7f009c07b1f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f00b0006c30 tx=0x7f00b00014f0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 msgr2=0x7f00bc1f6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 0x7f00bc1f6b40 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7f00b4009ed0 tx=0x7f00b4033c30 comp rx=0 tx=0).stop 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 shutdown_connections 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f009c078ce0 0x7f009c07b1f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00bc148c80 0x7f00bc14a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00bc14a9e0 0x7f00bc1f6b40 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 --2- 172.21.15.88:0/1788219518 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00a4004d90 0x7f00bc147440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 >> 172.21.15.88:0/1788219518 conn(0x7f00bc06d730 msgr2=0x7f00bc1524d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:57.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 shutdown_connections 2024-09-21T07:50:57.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.175+0000 7f00c3e00640 1 -- 172.21.15.88:0/1788219518 wait complete. 2024-09-21T07:50:57.201 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:50:57.201 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.422+0000 7fb145c00640 1 -- 172.21.15.88:0/3497702048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 msgr2=0x7fb124004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.422+0000 7fb145c00640 1 --2- 172.21.15.88:0/3497702048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb124004800 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009b20 tx=0x7fb124005830 comp rx=0 tx=0).stop 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.422+0000 7fb13f400640 1 -- 172.21.15.88:0/3497702048 reap_dead start 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.422+0000 7fb145c00640 1 -- 172.21.15.88:0/3497702048 shutdown_connections 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.422+0000 7fb145c00640 1 -- 172.21.15.88:0/3497702048 >> 172.21.15.88:0/3497702048 conn(0x7fb14006d730 msgr2=0x7fb1400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.423+0000 7fb145c00640 1 -- 172.21.15.88:0/3497702048 shutdown_connections 2024-09-21T07:50:57.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.423+0000 7fb145c00640 1 -- 172.21.15.88:0/3497702048 wait complete. 2024-09-21T07:50:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.425+0000 7fb145c00640 1 Processor -- start 2024-09-21T07:50:57.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.425+0000 7fb145c00640 1 -- start start 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.426+0000 7fb145c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb140156530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.426+0000 7fb145c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401f9d30 0x7fb140149a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.426+0000 7fb145c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1401f90e0 0x7fb1401f94f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.426+0000 7fb145c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1400b4da0 con 0x7fb1240041c0 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.426+0000 7fb145c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb1400b4c20 con 0x7fb1401f90e0 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.426+0000 7fb145c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb1400ad4f0 con 0x7fb1401f9d30 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb140156530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb140156530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36654/0 (socket says 172.21.15.88:36654) 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 -- 172.21.15.88:0/1448425841 learned_addr learned my addr 172.21.15.88:0/1448425841 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:57.429 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13f400640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1401f90e0 0x7fb1401f94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13e000640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401f9d30 0x7fb140149a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 -- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401f9d30 msgr2=0x7fb140149a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401f9d30 0x7fb140149a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 -- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1401f90e0 msgr2=0x7fb1401f94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1401f90e0 0x7fb1401f94f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 -- 172.21.15.88:0/1448425841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb1401f9a30 con 0x7fb1240041c0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.427+0000 7fb13ea00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb140156530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb130004d90 tx=0x7fb130001800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.428+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130010890 con 0x7fb1240041c0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.428+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb130016070 con 0x7fb1240041c0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.428+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb130010da0 con 0x7fb1240041c0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.428+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb14009b460 con 0x7fb1240041c0 2024-09-21T07:50:57.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.428+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb14009ba20 con 0x7fb1240041c0 2024-09-21T07:50:57.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.429+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1401535c0 con 0x7fb1240041c0 2024-09-21T07:50:57.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.430+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb13001b460 con 0x7fb1240041c0 2024-09-21T07:50:57.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.431+0000 7fb136a00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb120078ce0 0x7fb12007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.431+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7fb13009b7e0 con 0x7fb1240041c0 2024-09-21T07:50:57.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.431+0000 7fb13e000640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb120078ce0 0x7fb12007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.432+0000 7fb13e000640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb120078ce0 0x7fb12007b1f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb12c035000 tx=0x7fb12c0046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:57.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.435+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb1401f9a30 con 0x7fb1240041c0 2024-09-21T07:50:57.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.619+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fb1400700c0 con 0x7fb120078ce0 2024-09-21T07:50:57.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.624+0000 7fb136a00640 1 -- 172.21.15.88:0/1448425841 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240181 (secure 0 0 0) 0x7fb1400700c0 con 0x7fb120078ce0 2024-09-21T07:50:57.628 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:57.628 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb120078ce0 msgr2=0x7fb12007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb120078ce0 0x7fb12007b1f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb12c035000 tx=0x7fb12c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 msgr2=0x7fb140156530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb140156530 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7fb130004d90 tx=0x7fb130001800 comp rx=0 tx=0).stop 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 shutdown_connections 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb120078ce0 0x7fb12007b1f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb1401f9d30 0x7fb140149a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb1401f90e0 0x7fb1401f94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 --2- 172.21.15.88:0/1448425841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb1240041c0 0x7fb140156530 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 >> 172.21.15.88:0/1448425841 conn(0x7fb14006d730 msgr2=0x7fb1400cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:57.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 shutdown_connections 2024-09-21T07:50:57.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.628+0000 7fb145c00640 1 -- 172.21.15.88:0/1448425841 wait complete. 2024-09-21T07:50:57.654 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-21T07:50:55.697044+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":5,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258524,"kb_used_data":8996,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754716196,"statfs":{"total":773094113280,"available":772829384704,"internally_reserved":0,"allocated":9211904,"data_stored":4599437,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.517337"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.187476+0000","last_change":"2024-09-21T07:50:51.187476+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.187476+0000","last_undegraded":"2024-09-21T07:50:51.187476+0000","last_fullsized":"2024-09-21T07:50:51.187476+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186665+0000","last_change":"2024-09-21T07:50:51.186665+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186665+0000","last_undegraded":"2024-09-21T07:50:51.186665+0000","last_fullsized":"2024-09-21T07:50:51.186665+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186575+0000","last_change":"2024-09-21T07:50:51.186575+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186575+0000","last_undegraded":"2024-09-21T07:50:51.186575+0000","last_fullsized":"2024-09-21T07:50:51.186575+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.183203+0000","last_change":"2024-09-21T07:50:51.183203+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.183203+0000","last_undegraded":"2024-09-21T07:50:51.183203+0000","last_fullsized":"2024-09-21T07:50:51.183203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.188828+0000","last_change":"2024-09-21T07:50:51.188828+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.188828+0000","last_undegraded":"2024-09-21T07:50:51.188828+0000","last_fullsized":"2024-09-21T07:50:51.188828+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.185943+0000","last_change":"2024-09-21T07:50:51.185943+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.185943+0000","last_undegraded":"2024-09-21T07:50:51.185943+0000","last_fullsized":"2024-09-21T07:50:51.185943+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.189270+0000","last_change":"2024-09-21T07:50:51.189270+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.189270+0000","last_undegraded":"2024-09-21T07:50:51.189270+0000","last_fullsized":"2024-09-21T07:50:51.189270+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.189988+0000","last_change":"2024-09-21T07:50:51.189988+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.189988+0000","last_undegraded":"2024-09-21T07:50:51.189988+0000","last_fullsized":"2024-09-21T07:50:51.189988+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.183105+0000","last_change":"2024-09-21T07:50:51.183105+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.183105+0000","last_undegraded":"2024-09-21T07:50:51.183105+0000","last_fullsized":"2024-09-21T07:50:51.183105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186171+0000","last_change":"2024-09-21T07:50:51.186171+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186171+0000","last_undegraded":"2024-09-21T07:50:51.186171+0000","last_fullsized":"2024-09-21T07:50:51.186171+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.15","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.183060+0000","last_change":"2024-09-21T07:50:51.183060+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.183060+0000","last_undegraded":"2024-09-21T07:50:51.183060+0000","last_fullsized":"2024-09-21T07:50:51.183060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.14","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.189490+0000","last_change":"2024-09-21T07:50:51.189490+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.189490+0000","last_undegraded":"2024-09-21T07:50:51.189490+0000","last_fullsized":"2024-09-21T07:50:51.189490+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.13","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.183366+0000","last_change":"2024-09-21T07:50:51.183366+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.183366+0000","last_undegraded":"2024-09-21T07:50:51.183366+0000","last_fullsized":"2024-09-21T07:50:51.183366+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.184495+0000","last_change":"2024-09-21T07:50:51.184495+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.184495+0000","last_undegraded":"2024-09-21T07:50:51.184495+0000","last_fullsized":"2024-09-21T07:50:51.184495+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.185485+0000","last_change":"2024-09-21T07:50:51.185485+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.185485+0000","last_undegraded":"2024-09-21T07:50:51.185485+0000","last_fullsized":"2024-09-21T07:50:51.185485+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.10","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186507+0000","last_change":"2024-09-21T07:50:51.186507+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186507+0000","last_undegraded":"2024-09-21T07:50:51.186507+0000","last_fullsized":"2024-09-21T07:50:51.186507+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.f","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186952+0000","last_change":"2024-09-21T07:50:51.186952+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186952+0000","last_undegraded":"2024-09-21T07:50:51.186952+0000","last_fullsized":"2024-09-21T07:50:51.186952+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.e","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.190349+0000","last_change":"2024-09-21T07:50:51.190349+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.190349+0000","last_undegraded":"2024-09-21T07:50:51.190349+0000","last_fullsized":"2024-09-21T07:50:51.190349+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.190414+0000","last_change":"2024-09-21T07:50:51.190414+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.190414+0000","last_undegraded":"2024-09-21T07:50:51.190414+0000","last_fullsized":"2024-09-21T07:50:51.190414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.b","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.189394+0000","last_change":"2024-09-21T07:50:51.189394+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.189394+0000","last_undegraded":"2024-09-21T07:50:51.189394+0000","last_fullsized":"2024-09-21T07:50:51.189394+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.183701+0000","last_change":"2024-09-21T07:50:51.183701+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.183701+0000","last_undegraded":"2024-09-21T07:50:51.183701+0000","last_fullsized":"2024-09-21T07:50:51.183701+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186561+0000","last_change":"2024-09-21T07:50:51.186561+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186561+0000","last_undegraded":"2024-09-21T07:50:51.186561+0000","last_fullsized":"2024-09-21T07:50:51.186561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.189182+0000","last_change":"2024-09-21T07:50:51.189182+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.189182+0000","last_undegraded":"2024-09-21T07:50:51.189182+0000","last_fullsized":"2024-09-21T07:50:51.189182+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.7","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.190017+0000","last_change":"2024-09-21T07:50:51.190017+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.190017+0000","last_undegraded":"2024-09-21T07:50:51.190017+0000","last_fullsized":"2024-09-21T07:50:51.190017+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186592+0000","last_change":"2024-09-21T07:50:51.186592+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186592+0000","last_undegraded":"2024-09-21T07:50:51.186592+0000","last_fullsized":"2024-09-21T07:50:51.186592+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.188902+0000","last_change":"2024-09-21T07:50:51.188902+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.188902+0000","last_undegraded":"2024-09-21T07:50:51.188902+0000","last_fullsized":"2024-09-21T07:50:51.188902+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.185740+0000","last_change":"2024-09-21T07:50:51.185740+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.185740+0000","last_undegraded":"2024-09-21T07:50:51.185740+0000","last_fullsized":"2024-09-21T07:50:51.185740+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.182888+0000","last_change":"2024-09-21T07:50:51.182888+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.182888+0000","last_undegraded":"2024-09-21T07:50:51.182888+0000","last_fullsized":"2024-09-21T07:50:51.182888+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.185465+0000","last_change":"2024-09-21T07:50:51.185465+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.185465+0000","last_undegraded":"2024-09-21T07:50:51.185465+0000","last_fullsized":"2024-09-21T07:50:51.185465+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.183060+0000","last_change":"2024-09-21T07:50:51.183060+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.183060+0000","last_undegraded":"2024-09-21T07:50:51.183060+0000","last_fullsized":"2024-09-21T07:50:51.183060+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.188846+0000","last_change":"2024-09-21T07:50:51.188846+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.188846+0000","last_undegraded":"2024-09-21T07:50:51.188846+0000","last_fullsized":"2024-09-21T07:50:51.188846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:50:52.179615+0000","last_change":"2024-09-21T07:50:52.179615+0000","last_active":"2024-09-21T07:50:52.179615+0000","last_peered":"2024-09-21T07:50:52.179615+0000","last_clean":"2024-09-21T07:50:52.179615+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:52.179615+0000","last_undegraded":"2024-09-21T07:50:52.179615+0000","last_fullsized":"2024-09-21T07:50:52.179615+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"12.d","version":"0'0","reported_seq":1,"reported_epoch":83,"state":"creating+peering","last_fresh":"2024-09-21T07:50:51.186972+0000","last_change":"2024-09-21T07:50:51.186972+0000","last_active":"2024-09-21T07:50:51.170764+0000","last_peered":"2024-09-21T07:50:51.170764+0000","last_clean":"2024-09-21T07:50:51.170764+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:50:51.186972+0000","last_undegraded":"2024-09-21T07:50:51.186972+0000","last_fullsized":"2024-09-21T07:50:51.186972+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":253,"reported_epoch":83,"state":"active+clean","last_fresh":"2024-09-21T07:50:51.181329+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:50:51.181329+0000","last_peered":"2024-09-21T07:50:51.181329+0000","last_clean":"2024-09-21T07:50:51.181329+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:50:51.181329+0000","last_undegraded":"2024-09-21T07:50:51.181329+0000","last_fullsized":"2024-09-21T07:50:51.181329+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542207,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":429519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509502,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32176,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339664,"statfs":{"total":96636764160,"available":96603815936,"internally_reserved":0,"allocated":1007616,"data_stored":429519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509503,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":1019887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509503,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32172,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339668,"statfs":{"total":96636764160,"available":96603820032,"internally_reserved":0,"allocated":1007616,"data_stored":429519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476801,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":423702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476801,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32168,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339672,"statfs":{"total":96636764160,"available":96603824128,"internally_reserved":0,"allocated":995328,"data_stored":423702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476798,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32752,"kb_used_data":1564,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339088,"statfs":{"total":96636764160,"available":96603226112,"internally_reserved":0,"allocated":1601536,"data_stored":1019887,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32164,"kb_used_data":972,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339676,"statfs":{"total":96636764160,"available":96603828224,"internally_reserved":0,"allocated":995328,"data_stored":423702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:50:57.658 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 -- 172.21.15.88:0/453257840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f496015fae0 msgr2=0x7f49600b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 --2- 172.21.15.88:0/453257840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f496015fae0 0x7f49600b3750 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f4958005fd0 tx=0x7f495802aa10 comp rx=0 tx=0).stop 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 -- 172.21.15.88:0/453257840 shutdown_connections 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 --2- 172.21.15.88:0/453257840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f496015fae0 0x7f49600b3750 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 --2- 172.21.15.88:0/453257840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49600aebc0 0x7f4960152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 -- 172.21.15.88:0/453257840 >> 172.21.15.88:0/453257840 conn(0x7f496006d730 msgr2=0x7f49600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 -- 172.21.15.88:0/453257840 shutdown_connections 2024-09-21T07:50:57.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.913+0000 7f4967800640 1 -- 172.21.15.88:0/453257840 wait complete. 2024-09-21T07:50:57.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.915+0000 7f4967800640 1 Processor -- start 2024-09-21T07:50:57.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.916+0000 7f4967800640 1 -- start start 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4967800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49600aebc0 0x7f4960155420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4967800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49600b2750 0x7f49600c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4967800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 0x7f49601f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4967800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49600b4210 con 0x7f49600c9a00 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4967800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49600b4090 con 0x7f49600aebc0 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4967800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49600b4390 con 0x7f49600b2750 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4965800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 0x7f49601f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4965800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 0x7f49601f8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36664/0 (socket says 172.21.15.88:36664) 2024-09-21T07:50:57.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.917+0000 7f4965800640 1 -- 172.21.15.88:0/1448631476 learned_addr learned my addr 172.21.15.88:0/1448631476 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4964e00640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49600aebc0 0x7f4960155420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4957e00640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49600b2750 0x7f49600c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4965800640 1 -- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49600b2750 msgr2=0x7f49600c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4965800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49600b2750 0x7f49600c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4965800640 1 -- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49600aebc0 msgr2=0x7f4960155420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4965800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49600aebc0 0x7f4960155420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:57.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4965800640 1 -- 172.21.15.88:0/1448631476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4960153f10 con 0x7f49600c9a00 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.918+0000 7f4965800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 0x7f49601f8b60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f495802af30 tx=0x7f4958003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.919+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958033070 con 0x7f49600c9a00 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.919+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4958004450 con 0x7f49600c9a00 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.919+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4958038450 con 0x7f49600c9a00 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.919+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4960154380 con 0x7f49600c9a00 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.919+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4960154940 con 0x7f49600c9a00 2024-09-21T07:50:57.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.920+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49600b48b0 con 0x7f49600c9a00 2024-09-21T07:50:57.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.921+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f4958036070 con 0x7f49600c9a00 2024-09-21T07:50:57.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.922+0000 7f495d600640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f493c078ce0 0x7f493c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:57.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.922+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f49580bc1a0 con 0x7f49600c9a00 2024-09-21T07:50:57.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.922+0000 7f4964e00640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f493c078ce0 0x7f493c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:57.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.923+0000 7f4964e00640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f493c078ce0 0x7f493c07b1f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4950009b30 tx=0x7f4950005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:57.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:57.926+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4958083df0 con 0x7f49600c9a00 2024-09-21T07:50:58.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.110+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49601540e0 con 0x7f493c078ce0 2024-09-21T07:50:58.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.115+0000 7f495d600640 1 -- 172.21.15.88:0/1448631476 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252098 (secure 0 0 0) 0x7f49601540e0 con 0x7f493c078ce0 2024-09-21T07:50:58.119 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:58.120 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:50:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f493c078ce0 msgr2=0x7f493c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f493c078ce0 0x7f493c07b1f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f4950009b30 tx=0x7f4950005f90 comp rx=0 tx=0).stop 2024-09-21T07:50:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 msgr2=0x7f49601f8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 0x7f49601f8b60 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f495802af30 tx=0x7f4958003f90 comp rx=0 tx=0).stop 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 shutdown_connections 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f493c078ce0 0x7f493c07b1f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49600b2750 0x7f49600c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49600c9a00 0x7f49601f8b60 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 --2- 172.21.15.88:0/1448631476 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49600aebc0 0x7f4960155420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 >> 172.21.15.88:0/1448631476 conn(0x7f496006d730 msgr2=0x7f496015c4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 shutdown_connections 2024-09-21T07:50:58.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.119+0000 7f4967800640 1 -- 172.21.15.88:0/1448631476 wait complete. 2024-09-21T07:50:58.144 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":27,"stamp":"2024-09-21T07:50:57.697450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":190,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":258900,"kb_used_data":9384,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754715820,"statfs":{"total":773094113280,"available":772828999680,"internally_reserved":0,"allocated":9609216,"data_stored":4776508,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":33,"apply_latency_ms":33,"commit_latency_ns":33000000,"apply_latency_ns":33000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001302"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213836+0000","last_change":"2024-09-21T07:50:53.224156+0000","last_active":"2024-09-21T07:50:54.213836+0000","last_peered":"2024-09-21T07:50:54.213836+0000","last_clean":"2024-09-21T07:50:54.213836+0000","last_became_active":"2024-09-21T07:50:53.217253+0000","last_became_peered":"2024-09-21T07:50:53.217253+0000","last_unstale":"2024-09-21T07:50:54.213836+0000","last_undegraded":"2024-09-21T07:50:54.213836+0000","last_fullsized":"2024-09-21T07:50:54.213836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:02:40.503710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215105+0000","last_change":"2024-09-21T07:50:53.202879+0000","last_active":"2024-09-21T07:50:54.215105+0000","last_peered":"2024-09-21T07:50:54.215105+0000","last_clean":"2024-09-21T07:50:54.215105+0000","last_became_active":"2024-09-21T07:50:53.200221+0000","last_became_peered":"2024-09-21T07:50:53.200221+0000","last_unstale":"2024-09-21T07:50:54.215105+0000","last_undegraded":"2024-09-21T07:50:54.215105+0000","last_fullsized":"2024-09-21T07:50:54.215105+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:50:40.924081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007952+0000","last_change":"2024-09-21T07:50:53.211120+0000","last_active":"2024-09-21T07:50:55.007952+0000","last_peered":"2024-09-21T07:50:55.007952+0000","last_clean":"2024-09-21T07:50:55.007952+0000","last_became_active":"2024-09-21T07:50:53.210773+0000","last_became_peered":"2024-09-21T07:50:53.210773+0000","last_unstale":"2024-09-21T07:50:55.007952+0000","last_undegraded":"2024-09-21T07:50:55.007952+0000","last_fullsized":"2024-09-21T07:50:55.007952+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:12.986992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215955+0000","last_change":"2024-09-21T07:50:53.220040+0000","last_active":"2024-09-21T07:50:54.215955+0000","last_peered":"2024-09-21T07:50:54.215955+0000","last_clean":"2024-09-21T07:50:54.215955+0000","last_became_active":"2024-09-21T07:50:53.219730+0000","last_became_peered":"2024-09-21T07:50:53.219730+0000","last_unstale":"2024-09-21T07:50:54.215955+0000","last_undegraded":"2024-09-21T07:50:54.215955+0000","last_fullsized":"2024-09-21T07:50:54.215955+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:35:43.708511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007801+0000","last_change":"2024-09-21T07:50:53.218814+0000","last_active":"2024-09-21T07:50:55.007801+0000","last_peered":"2024-09-21T07:50:55.007801+0000","last_clean":"2024-09-21T07:50:55.007801+0000","last_became_active":"2024-09-21T07:50:53.218396+0000","last_became_peered":"2024-09-21T07:50:53.218396+0000","last_unstale":"2024-09-21T07:50:55.007801+0000","last_undegraded":"2024-09-21T07:50:55.007801+0000","last_fullsized":"2024-09-21T07:50:55.007801+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:21.667749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006565+0000","last_change":"2024-09-21T07:50:53.213835+0000","last_active":"2024-09-21T07:50:55.006565+0000","last_peered":"2024-09-21T07:50:55.006565+0000","last_clean":"2024-09-21T07:50:55.006565+0000","last_became_active":"2024-09-21T07:50:53.213550+0000","last_became_peered":"2024-09-21T07:50:53.213550+0000","last_unstale":"2024-09-21T07:50:55.006565+0000","last_undegraded":"2024-09-21T07:50:55.006565+0000","last_fullsized":"2024-09-21T07:50:55.006565+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:41.828518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.214978+0000","last_change":"2024-09-21T07:50:53.212366+0000","last_active":"2024-09-21T07:50:54.214978+0000","last_peered":"2024-09-21T07:50:54.214978+0000","last_clean":"2024-09-21T07:50:54.214978+0000","last_became_active":"2024-09-21T07:50:53.210401+0000","last_became_peered":"2024-09-21T07:50:53.210401+0000","last_unstale":"2024-09-21T07:50:54.214978+0000","last_undegraded":"2024-09-21T07:50:54.214978+0000","last_fullsized":"2024-09-21T07:50:54.214978+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:07.537040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006613+0000","last_change":"2024-09-21T07:50:53.229289+0000","last_active":"2024-09-21T07:50:55.006613+0000","last_peered":"2024-09-21T07:50:55.006613+0000","last_clean":"2024-09-21T07:50:55.006613+0000","last_became_active":"2024-09-21T07:50:53.229048+0000","last_became_peered":"2024-09-21T07:50:53.229048+0000","last_unstale":"2024-09-21T07:50:55.006613+0000","last_undegraded":"2024-09-21T07:50:55.006613+0000","last_fullsized":"2024-09-21T07:50:55.006613+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:06.968261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213841+0000","last_change":"2024-09-21T07:50:53.224237+0000","last_active":"2024-09-21T07:50:54.213841+0000","last_peered":"2024-09-21T07:50:54.213841+0000","last_clean":"2024-09-21T07:50:54.213841+0000","last_became_active":"2024-09-21T07:50:53.217586+0000","last_became_peered":"2024-09-21T07:50:53.217586+0000","last_unstale":"2024-09-21T07:50:54.213841+0000","last_undegraded":"2024-09-21T07:50:54.213841+0000","last_fullsized":"2024-09-21T07:50:54.213841+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:44.893797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006708+0000","last_change":"2024-09-21T07:50:53.211185+0000","last_active":"2024-09-21T07:50:55.006708+0000","last_peered":"2024-09-21T07:50:55.006708+0000","last_clean":"2024-09-21T07:50:55.006708+0000","last_became_active":"2024-09-21T07:50:53.210910+0000","last_became_peered":"2024-09-21T07:50:53.210910+0000","last_unstale":"2024-09-21T07:50:55.006708+0000","last_undegraded":"2024-09-21T07:50:55.006708+0000","last_fullsized":"2024-09-21T07:50:55.006708+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:47.629829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868538+0000","last_change":"2024-09-21T07:50:53.207779+0000","last_active":"2024-09-21T07:50:54.868538+0000","last_peered":"2024-09-21T07:50:54.868538+0000","last_clean":"2024-09-21T07:50:54.868538+0000","last_became_active":"2024-09-21T07:50:53.206699+0000","last_became_peered":"2024-09-21T07:50:53.206699+0000","last_unstale":"2024-09-21T07:50:54.868538+0000","last_undegraded":"2024-09-21T07:50:54.868538+0000","last_fullsized":"2024-09-21T07:50:54.868538+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:28.863920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006440+0000","last_change":"2024-09-21T07:50:53.219864+0000","last_active":"2024-09-21T07:50:55.006440+0000","last_peered":"2024-09-21T07:50:55.006440+0000","last_clean":"2024-09-21T07:50:55.006440+0000","last_became_active":"2024-09-21T07:50:53.218342+0000","last_became_peered":"2024-09-21T07:50:53.218342+0000","last_unstale":"2024-09-21T07:50:55.006440+0000","last_undegraded":"2024-09-21T07:50:55.006440+0000","last_fullsized":"2024-09-21T07:50:55.006440+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:34.959325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008311+0000","last_change":"2024-09-21T07:50:53.212583+0000","last_active":"2024-09-21T07:50:55.008311+0000","last_peered":"2024-09-21T07:50:55.008311+0000","last_clean":"2024-09-21T07:50:55.008311+0000","last_became_active":"2024-09-21T07:50:53.212273+0000","last_became_peered":"2024-09-21T07:50:53.212273+0000","last_unstale":"2024-09-21T07:50:55.008311+0000","last_undegraded":"2024-09-21T07:50:55.008311+0000","last_fullsized":"2024-09-21T07:50:55.008311+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:08.942144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006863+0000","last_change":"2024-09-21T07:50:53.220581+0000","last_active":"2024-09-21T07:50:55.006863+0000","last_peered":"2024-09-21T07:50:55.006863+0000","last_clean":"2024-09-21T07:50:55.006863+0000","last_became_active":"2024-09-21T07:50:53.219928+0000","last_became_peered":"2024-09-21T07:50:53.219928+0000","last_unstale":"2024-09-21T07:50:55.006863+0000","last_undegraded":"2024-09-21T07:50:55.006863+0000","last_fullsized":"2024-09-21T07:50:55.006863+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:45.227133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215052+0000","last_change":"2024-09-21T07:50:53.216661+0000","last_active":"2024-09-21T07:50:54.215052+0000","last_peered":"2024-09-21T07:50:54.215052+0000","last_clean":"2024-09-21T07:50:54.215052+0000","last_became_active":"2024-09-21T07:50:53.216034+0000","last_became_peered":"2024-09-21T07:50:53.216034+0000","last_unstale":"2024-09-21T07:50:54.215052+0000","last_undegraded":"2024-09-21T07:50:54.215052+0000","last_fullsized":"2024-09-21T07:50:54.215052+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:21.757511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007213+0000","last_change":"2024-09-21T07:50:53.211476+0000","last_active":"2024-09-21T07:50:55.007213+0000","last_peered":"2024-09-21T07:50:55.007213+0000","last_clean":"2024-09-21T07:50:55.007213+0000","last_became_active":"2024-09-21T07:50:53.211189+0000","last_became_peered":"2024-09-21T07:50:53.211189+0000","last_unstale":"2024-09-21T07:50:55.007213+0000","last_undegraded":"2024-09-21T07:50:55.007213+0000","last_fullsized":"2024-09-21T07:50:55.007213+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:44.551223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007489+0000","last_change":"2024-09-21T07:50:53.227402+0000","last_active":"2024-09-21T07:50:55.007489+0000","last_peered":"2024-09-21T07:50:55.007489+0000","last_clean":"2024-09-21T07:50:55.007489+0000","last_became_active":"2024-09-21T07:50:53.226794+0000","last_became_peered":"2024-09-21T07:50:53.226794+0000","last_unstale":"2024-09-21T07:50:55.007489+0000","last_undegraded":"2024-09-21T07:50:55.007489+0000","last_fullsized":"2024-09-21T07:50:55.007489+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:23.403279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868499+0000","last_change":"2024-09-21T07:50:53.217343+0000","last_active":"2024-09-21T07:50:54.868499+0000","last_peered":"2024-09-21T07:50:54.868499+0000","last_clean":"2024-09-21T07:50:54.868499+0000","last_became_active":"2024-09-21T07:50:53.216734+0000","last_became_peered":"2024-09-21T07:50:53.216734+0000","last_unstale":"2024-09-21T07:50:54.868499+0000","last_undegraded":"2024-09-21T07:50:54.868499+0000","last_fullsized":"2024-09-21T07:50:54.868499+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:11.481631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007175+0000","last_change":"2024-09-21T07:50:53.227512+0000","last_active":"2024-09-21T07:50:55.007175+0000","last_peered":"2024-09-21T07:50:55.007175+0000","last_clean":"2024-09-21T07:50:55.007175+0000","last_became_active":"2024-09-21T07:50:53.226666+0000","last_became_peered":"2024-09-21T07:50:53.226666+0000","last_unstale":"2024-09-21T07:50:55.007175+0000","last_undegraded":"2024-09-21T07:50:55.007175+0000","last_fullsized":"2024-09-21T07:50:55.007175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:08:40.452933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007299+0000","last_change":"2024-09-21T07:50:53.226704+0000","last_active":"2024-09-21T07:50:55.007299+0000","last_peered":"2024-09-21T07:50:55.007299+0000","last_clean":"2024-09-21T07:50:55.007299+0000","last_became_active":"2024-09-21T07:50:53.225663+0000","last_became_peered":"2024-09-21T07:50:53.225663+0000","last_unstale":"2024-09-21T07:50:55.007299+0000","last_undegraded":"2024-09-21T07:50:55.007299+0000","last_fullsized":"2024-09-21T07:50:55.007299+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:13.298422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215374+0000","last_change":"2024-09-21T07:50:53.218954+0000","last_active":"2024-09-21T07:50:54.215374+0000","last_peered":"2024-09-21T07:50:54.215374+0000","last_clean":"2024-09-21T07:50:54.215374+0000","last_became_active":"2024-09-21T07:50:53.217841+0000","last_became_peered":"2024-09-21T07:50:53.217841+0000","last_unstale":"2024-09-21T07:50:54.215374+0000","last_undegraded":"2024-09-21T07:50:54.215374+0000","last_fullsized":"2024-09-21T07:50:54.215374+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:12.986992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007450+0000","last_change":"2024-09-21T07:50:53.221838+0000","last_active":"2024-09-21T07:50:55.007450+0000","last_peered":"2024-09-21T07:50:55.007450+0000","last_clean":"2024-09-21T07:50:55.007450+0000","last_became_active":"2024-09-21T07:50:53.220269+0000","last_became_peered":"2024-09-21T07:50:53.220269+0000","last_unstale":"2024-09-21T07:50:55.007450+0000","last_undegraded":"2024-09-21T07:50:55.007450+0000","last_fullsized":"2024-09-21T07:50:55.007450+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:03.884764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006904+0000","last_change":"2024-09-21T07:50:53.216177+0000","last_active":"2024-09-21T07:50:55.006904+0000","last_peered":"2024-09-21T07:50:55.006904+0000","last_clean":"2024-09-21T07:50:55.006904+0000","last_became_active":"2024-09-21T07:50:53.215877+0000","last_became_peered":"2024-09-21T07:50:53.215877+0000","last_unstale":"2024-09-21T07:50:55.006904+0000","last_undegraded":"2024-09-21T07:50:55.006904+0000","last_fullsized":"2024-09-21T07:50:55.006904+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:19:24.985233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.867992+0000","last_change":"2024-09-21T07:50:53.203936+0000","last_active":"2024-09-21T07:50:54.867992+0000","last_peered":"2024-09-21T07:50:54.867992+0000","last_clean":"2024-09-21T07:50:54.867992+0000","last_became_active":"2024-09-21T07:50:53.203508+0000","last_became_peered":"2024-09-21T07:50:53.203508+0000","last_unstale":"2024-09-21T07:50:54.867992+0000","last_undegraded":"2024-09-21T07:50:54.867992+0000","last_fullsized":"2024-09-21T07:50:54.867992+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:08.740436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868667+0000","last_change":"2024-09-21T07:50:53.214286+0000","last_active":"2024-09-21T07:50:54.868667+0000","last_peered":"2024-09-21T07:50:54.868667+0000","last_clean":"2024-09-21T07:50:54.868667+0000","last_became_active":"2024-09-21T07:50:53.213746+0000","last_became_peered":"2024-09-21T07:50:53.213746+0000","last_unstale":"2024-09-21T07:50:54.868667+0000","last_undegraded":"2024-09-21T07:50:54.868667+0000","last_fullsized":"2024-09-21T07:50:54.868667+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:25.864905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008040+0000","last_change":"2024-09-21T07:50:53.219310+0000","last_active":"2024-09-21T07:50:55.008040+0000","last_peered":"2024-09-21T07:50:55.008040+0000","last_clean":"2024-09-21T07:50:55.008040+0000","last_became_active":"2024-09-21T07:50:53.216618+0000","last_became_peered":"2024-09-21T07:50:53.216618+0000","last_unstale":"2024-09-21T07:50:55.008040+0000","last_undegraded":"2024-09-21T07:50:55.008040+0000","last_fullsized":"2024-09-21T07:50:55.008040+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:40.753302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008177+0000","last_change":"2024-09-21T07:50:53.217783+0000","last_active":"2024-09-21T07:50:55.008177+0000","last_peered":"2024-09-21T07:50:55.008177+0000","last_clean":"2024-09-21T07:50:55.008177+0000","last_became_active":"2024-09-21T07:50:53.217476+0000","last_became_peered":"2024-09-21T07:50:53.217476+0000","last_unstale":"2024-09-21T07:50:55.008177+0000","last_undegraded":"2024-09-21T07:50:55.008177+0000","last_fullsized":"2024-09-21T07:50:55.008177+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:26:42.092735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213262+0000","last_change":"2024-09-21T07:50:53.212936+0000","last_active":"2024-09-21T07:50:54.213262+0000","last_peered":"2024-09-21T07:50:54.213262+0000","last_clean":"2024-09-21T07:50:54.213262+0000","last_became_active":"2024-09-21T07:50:53.212625+0000","last_became_peered":"2024-09-21T07:50:53.212625+0000","last_unstale":"2024-09-21T07:50:54.213262+0000","last_undegraded":"2024-09-21T07:50:54.213262+0000","last_fullsized":"2024-09-21T07:50:54.213262+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:27:20.171759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215604+0000","last_change":"2024-09-21T07:50:53.216079+0000","last_active":"2024-09-21T07:50:54.215604+0000","last_peered":"2024-09-21T07:50:54.215604+0000","last_clean":"2024-09-21T07:50:54.215604+0000","last_became_active":"2024-09-21T07:50:53.215772+0000","last_became_peered":"2024-09-21T07:50:53.215772+0000","last_unstale":"2024-09-21T07:50:54.215604+0000","last_undegraded":"2024-09-21T07:50:54.215604+0000","last_fullsized":"2024-09-21T07:50:54.215604+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:48.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007429+0000","last_change":"2024-09-21T07:50:53.222549+0000","last_active":"2024-09-21T07:50:55.007429+0000","last_peered":"2024-09-21T07:50:55.007429+0000","last_clean":"2024-09-21T07:50:55.007429+0000","last_became_active":"2024-09-21T07:50:53.222247+0000","last_became_peered":"2024-09-21T07:50:53.222247+0000","last_unstale":"2024-09-21T07:50:55.007429+0000","last_undegraded":"2024-09-21T07:50:55.007429+0000","last_fullsized":"2024-09-21T07:50:55.007429+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:19:17.506768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006449+0000","last_change":"2024-09-21T07:50:53.230256+0000","last_active":"2024-09-21T07:50:55.006449+0000","last_peered":"2024-09-21T07:50:55.006449+0000","last_clean":"2024-09-21T07:50:55.006449+0000","last_became_active":"2024-09-21T07:50:53.229671+0000","last_became_peered":"2024-09-21T07:50:53.229671+0000","last_unstale":"2024-09-21T07:50:55.006449+0000","last_undegraded":"2024-09-21T07:50:55.006449+0000","last_fullsized":"2024-09-21T07:50:55.006449+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:24.397960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007974+0000","last_change":"2024-09-21T07:50:53.230426+0000","last_active":"2024-09-21T07:50:55.007974+0000","last_peered":"2024-09-21T07:50:55.007974+0000","last_clean":"2024-09-21T07:50:55.007974+0000","last_became_active":"2024-09-21T07:50:53.229741+0000","last_became_peered":"2024-09-21T07:50:53.229741+0000","last_unstale":"2024-09-21T07:50:55.007974+0000","last_undegraded":"2024-09-21T07:50:55.007974+0000","last_fullsized":"2024-09-21T07:50:55.007974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:48.006967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006933+0000","last_change":"2024-09-21T07:50:53.214113+0000","last_active":"2024-09-21T07:50:55.006933+0000","last_peered":"2024-09-21T07:50:55.006933+0000","last_clean":"2024-09-21T07:50:55.006933+0000","last_became_active":"2024-09-21T07:50:53.213823+0000","last_became_peered":"2024-09-21T07:50:53.213823+0000","last_unstale":"2024-09-21T07:50:55.006933+0000","last_undegraded":"2024-09-21T07:50:55.006933+0000","last_fullsized":"2024-09-21T07:50:55.006933+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:24.397960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1f","version":"85'4","reported_seq":19,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008247+0000","last_change":"2024-09-21T07:50:52.212768+0000","last_active":"2024-09-21T07:50:55.008247+0000","last_peered":"2024-09-21T07:50:55.008247+0000","last_clean":"2024-09-21T07:50:55.008247+0000","last_became_active":"2024-09-21T07:50:52.212470+0000","last_became_peered":"2024-09-21T07:50:52.212470+0000","last_unstale":"2024-09-21T07:50:55.008247+0000","last_undegraded":"2024-09-21T07:50:55.008247+0000","last_fullsized":"2024-09-21T07:50:55.008247+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:28.900062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213974+0000","last_change":"2024-09-21T07:50:53.216897+0000","last_active":"2024-09-21T07:50:54.213974+0000","last_peered":"2024-09-21T07:50:54.213974+0000","last_clean":"2024-09-21T07:50:54.213974+0000","last_became_active":"2024-09-21T07:50:53.210738+0000","last_became_peered":"2024-09-21T07:50:53.210738+0000","last_unstale":"2024-09-21T07:50:54.213974+0000","last_undegraded":"2024-09-21T07:50:54.213974+0000","last_fullsized":"2024-09-21T07:50:54.213974+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:15:21.222229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1e","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006778+0000","last_change":"2024-09-21T07:50:52.210476+0000","last_active":"2024-09-21T07:50:55.006778+0000","last_peered":"2024-09-21T07:50:55.006778+0000","last_clean":"2024-09-21T07:50:55.006778+0000","last_became_active":"2024-09-21T07:50:52.210105+0000","last_became_peered":"2024-09-21T07:50:52.210105+0000","last_unstale":"2024-09-21T07:50:55.006778+0000","last_undegraded":"2024-09-21T07:50:55.006778+0000","last_fullsized":"2024-09-21T07:50:55.006778+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:12:24.856055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006698+0000","last_change":"2024-09-21T07:50:53.230595+0000","last_active":"2024-09-21T07:50:55.006698+0000","last_peered":"2024-09-21T07:50:55.006698+0000","last_clean":"2024-09-21T07:50:55.006698+0000","last_became_active":"2024-09-21T07:50:53.222435+0000","last_became_peered":"2024-09-21T07:50:53.222435+0000","last_unstale":"2024-09-21T07:50:55.006698+0000","last_undegraded":"2024-09-21T07:50:55.006698+0000","last_fullsized":"2024-09-21T07:50:55.006698+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:45.777997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.1d","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868414+0000","last_change":"2024-09-21T07:50:52.210198+0000","last_active":"2024-09-21T07:50:54.868414+0000","last_peered":"2024-09-21T07:50:54.868414+0000","last_clean":"2024-09-21T07:50:54.868414+0000","last_became_active":"2024-09-21T07:50:52.209899+0000","last_became_peered":"2024-09-21T07:50:52.209899+0000","last_unstale":"2024-09-21T07:50:54.868414+0000","last_undegraded":"2024-09-21T07:50:54.868414+0000","last_fullsized":"2024-09-21T07:50:54.868414+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:58.361230+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213895+0000","last_change":"2024-09-21T07:50:53.213735+0000","last_active":"2024-09-21T07:50:54.213895+0000","last_peered":"2024-09-21T07:50:54.213895+0000","last_clean":"2024-09-21T07:50:54.213895+0000","last_became_active":"2024-09-21T07:50:53.212950+0000","last_became_peered":"2024-09-21T07:50:53.212950+0000","last_unstale":"2024-09-21T07:50:54.213895+0000","last_undegraded":"2024-09-21T07:50:54.213895+0000","last_fullsized":"2024-09-21T07:50:54.213895+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:13.497186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1c","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215891+0000","last_change":"2024-09-21T07:50:52.217435+0000","last_active":"2024-09-21T07:50:54.215891+0000","last_peered":"2024-09-21T07:50:54.215891+0000","last_clean":"2024-09-21T07:50:54.215891+0000","last_became_active":"2024-09-21T07:50:52.216917+0000","last_became_peered":"2024-09-21T07:50:52.216917+0000","last_unstale":"2024-09-21T07:50:54.215891+0000","last_undegraded":"2024-09-21T07:50:54.215891+0000","last_fullsized":"2024-09-21T07:50:54.215891+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:56:23.389109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.216244+0000","last_change":"2024-09-21T07:50:53.221304+0000","last_active":"2024-09-21T07:50:54.216244+0000","last_peered":"2024-09-21T07:50:54.216244+0000","last_clean":"2024-09-21T07:50:54.216244+0000","last_became_active":"2024-09-21T07:50:53.219371+0000","last_became_peered":"2024-09-21T07:50:53.219371+0000","last_unstale":"2024-09-21T07:50:54.216244+0000","last_undegraded":"2024-09-21T07:50:54.216244+0000","last_fullsized":"2024-09-21T07:50:54.216244+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:16:24.136343+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.1b","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007447+0000","last_change":"2024-09-21T07:50:52.216924+0000","last_active":"2024-09-21T07:50:55.007447+0000","last_peered":"2024-09-21T07:50:55.007447+0000","last_clean":"2024-09-21T07:50:55.007447+0000","last_became_active":"2024-09-21T07:50:52.216409+0000","last_became_peered":"2024-09-21T07:50:52.216409+0000","last_unstale":"2024-09-21T07:50:55.007447+0000","last_undegraded":"2024-09-21T07:50:55.007447+0000","last_fullsized":"2024-09-21T07:50:55.007447+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:31.821481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007279+0000","last_change":"2024-09-21T07:50:53.228882+0000","last_active":"2024-09-21T07:50:55.007279+0000","last_peered":"2024-09-21T07:50:55.007279+0000","last_clean":"2024-09-21T07:50:55.007279+0000","last_became_active":"2024-09-21T07:50:53.228599+0000","last_became_peered":"2024-09-21T07:50:53.228599+0000","last_unstale":"2024-09-21T07:50:55.007279+0000","last_undegraded":"2024-09-21T07:50:55.007279+0000","last_fullsized":"2024-09-21T07:50:55.007279+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:44:59.002716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.1a","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006639+0000","last_change":"2024-09-21T07:50:52.212520+0000","last_active":"2024-09-21T07:50:55.006639+0000","last_peered":"2024-09-21T07:50:55.006639+0000","last_clean":"2024-09-21T07:50:55.006639+0000","last_became_active":"2024-09-21T07:50:52.211900+0000","last_became_peered":"2024-09-21T07:50:52.211900+0000","last_unstale":"2024-09-21T07:50:55.006639+0000","last_undegraded":"2024-09-21T07:50:55.006639+0000","last_fullsized":"2024-09-21T07:50:55.006639+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:27.855069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213521+0000","last_change":"2024-09-21T07:50:53.223335+0000","last_active":"2024-09-21T07:50:54.213521+0000","last_peered":"2024-09-21T07:50:54.213521+0000","last_clean":"2024-09-21T07:50:54.213521+0000","last_became_active":"2024-09-21T07:50:53.222994+0000","last_became_peered":"2024-09-21T07:50:53.222994+0000","last_unstale":"2024-09-21T07:50:54.213521+0000","last_undegraded":"2024-09-21T07:50:54.213521+0000","last_fullsized":"2024-09-21T07:50:54.213521+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:24.683003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006734+0000","last_change":"2024-09-21T07:50:52.214609+0000","last_active":"2024-09-21T07:50:55.006734+0000","last_peered":"2024-09-21T07:50:55.006734+0000","last_clean":"2024-09-21T07:50:55.006734+0000","last_became_active":"2024-09-21T07:50:52.212515+0000","last_became_peered":"2024-09-21T07:50:52.212515+0000","last_unstale":"2024-09-21T07:50:55.006734+0000","last_undegraded":"2024-09-21T07:50:55.006734+0000","last_fullsized":"2024-09-21T07:50:55.006734+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:48:35.280625+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006836+0000","last_change":"2024-09-21T07:50:53.220109+0000","last_active":"2024-09-21T07:50:55.006836+0000","last_peered":"2024-09-21T07:50:55.006836+0000","last_clean":"2024-09-21T07:50:55.006836+0000","last_became_active":"2024-09-21T07:50:53.219780+0000","last_became_peered":"2024-09-21T07:50:53.219780+0000","last_unstale":"2024-09-21T07:50:55.006836+0000","last_undegraded":"2024-09-21T07:50:55.006836+0000","last_fullsized":"2024-09-21T07:50:55.006836+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:51:08.881908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006576+0000","last_change":"2024-09-21T07:50:52.213997+0000","last_active":"2024-09-21T07:50:55.006576+0000","last_peered":"2024-09-21T07:50:55.006576+0000","last_clean":"2024-09-21T07:50:55.006576+0000","last_became_active":"2024-09-21T07:50:52.212284+0000","last_became_peered":"2024-09-21T07:50:52.212284+0000","last_unstale":"2024-09-21T07:50:55.006576+0000","last_undegraded":"2024-09-21T07:50:55.006576+0000","last_fullsized":"2024-09-21T07:50:55.006576+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:05.494214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007519+0000","last_change":"2024-09-21T07:50:53.227929+0000","last_active":"2024-09-21T07:50:55.007519+0000","last_peered":"2024-09-21T07:50:55.007519+0000","last_clean":"2024-09-21T07:50:55.007519+0000","last_became_active":"2024-09-21T07:50:53.227445+0000","last_became_peered":"2024-09-21T07:50:53.227445+0000","last_unstale":"2024-09-21T07:50:55.007519+0000","last_undegraded":"2024-09-21T07:50:55.007519+0000","last_fullsized":"2024-09-21T07:50:55.007519+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:06.519861+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.214031+0000","last_change":"2024-09-21T07:50:52.208097+0000","last_active":"2024-09-21T07:50:54.214031+0000","last_peered":"2024-09-21T07:50:54.214031+0000","last_clean":"2024-09-21T07:50:54.214031+0000","last_became_active":"2024-09-21T07:50:52.207682+0000","last_became_peered":"2024-09-21T07:50:52.207682+0000","last_unstale":"2024-09-21T07:50:54.214031+0000","last_undegraded":"2024-09-21T07:50:54.214031+0000","last_fullsized":"2024-09-21T07:50:54.214031+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:38.252097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213986+0000","last_change":"2024-09-21T07:50:53.212161+0000","last_active":"2024-09-21T07:50:54.213986+0000","last_peered":"2024-09-21T07:50:54.213986+0000","last_clean":"2024-09-21T07:50:54.213986+0000","last_became_active":"2024-09-21T07:50:53.211874+0000","last_became_peered":"2024-09-21T07:50:53.211874+0000","last_unstale":"2024-09-21T07:50:54.213986+0000","last_undegraded":"2024-09-21T07:50:54.213986+0000","last_fullsized":"2024-09-21T07:50:54.213986+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:36:12.680504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006892+0000","last_change":"2024-09-21T07:50:52.214980+0000","last_active":"2024-09-21T07:50:55.006892+0000","last_peered":"2024-09-21T07:50:55.006892+0000","last_clean":"2024-09-21T07:50:55.006892+0000","last_became_active":"2024-09-21T07:50:52.214678+0000","last_became_peered":"2024-09-21T07:50:52.214678+0000","last_unstale":"2024-09-21T07:50:55.006892+0000","last_undegraded":"2024-09-21T07:50:55.006892+0000","last_fullsized":"2024-09-21T07:50:55.006892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:20.748661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868093+0000","last_change":"2024-09-21T07:50:53.211116+0000","last_active":"2024-09-21T07:50:54.868093+0000","last_peered":"2024-09-21T07:50:54.868093+0000","last_clean":"2024-09-21T07:50:54.868093+0000","last_became_active":"2024-09-21T07:50:53.210433+0000","last_became_peered":"2024-09-21T07:50:53.210433+0000","last_unstale":"2024-09-21T07:50:54.868093+0000","last_undegraded":"2024-09-21T07:50:54.868093+0000","last_fullsized":"2024-09-21T07:50:54.868093+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:13.502496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.15","version":"85'3","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213971+0000","last_change":"2024-09-21T07:50:52.213569+0000","last_active":"2024-09-21T07:50:54.213971+0000","last_peered":"2024-09-21T07:50:54.213971+0000","last_clean":"2024-09-21T07:50:54.213971+0000","last_became_active":"2024-09-21T07:50:52.213204+0000","last_became_peered":"2024-09-21T07:50:52.213204+0000","last_unstale":"2024-09-21T07:50:54.213971+0000","last_undegraded":"2024-09-21T07:50:54.213971+0000","last_fullsized":"2024-09-21T07:50:54.213971+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:57:05.226321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007883+0000","last_change":"2024-09-21T07:50:53.215771+0000","last_active":"2024-09-21T07:50:55.007883+0000","last_peered":"2024-09-21T07:50:55.007883+0000","last_clean":"2024-09-21T07:50:55.007883+0000","last_became_active":"2024-09-21T07:50:53.213537+0000","last_became_peered":"2024-09-21T07:50:53.213537+0000","last_unstale":"2024-09-21T07:50:55.007883+0000","last_undegraded":"2024-09-21T07:50:55.007883+0000","last_fullsized":"2024-09-21T07:50:55.007883+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:37.214008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.14","version":"85'6","reported_seq":21,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006770+0000","last_change":"2024-09-21T07:50:52.215730+0000","last_active":"2024-09-21T07:50:55.006770+0000","last_peered":"2024-09-21T07:50:55.006770+0000","last_clean":"2024-09-21T07:50:55.006770+0000","last_became_active":"2024-09-21T07:50:52.215442+0000","last_became_peered":"2024-09-21T07:50:52.215442+0000","last_unstale":"2024-09-21T07:50:55.006770+0000","last_undegraded":"2024-09-21T07:50:55.006770+0000","last_fullsized":"2024-09-21T07:50:55.006770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:36:09.825718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868006+0000","last_change":"2024-09-21T07:50:53.208374+0000","last_active":"2024-09-21T07:50:54.868006+0000","last_peered":"2024-09-21T07:50:54.868006+0000","last_clean":"2024-09-21T07:50:54.868006+0000","last_became_active":"2024-09-21T07:50:53.207490+0000","last_became_peered":"2024-09-21T07:50:53.207490+0000","last_unstale":"2024-09-21T07:50:54.868006+0000","last_undegraded":"2024-09-21T07:50:54.868006+0000","last_fullsized":"2024-09-21T07:50:54.868006+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:42.205523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.13","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215466+0000","last_change":"2024-09-21T07:50:52.214394+0000","last_active":"2024-09-21T07:50:54.215466+0000","last_peered":"2024-09-21T07:50:54.215466+0000","last_clean":"2024-09-21T07:50:54.215466+0000","last_became_active":"2024-09-21T07:50:52.213942+0000","last_became_peered":"2024-09-21T07:50:52.213942+0000","last_unstale":"2024-09-21T07:50:54.215466+0000","last_undegraded":"2024-09-21T07:50:54.215466+0000","last_fullsized":"2024-09-21T07:50:54.215466+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:51.281739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213516+0000","last_change":"2024-09-21T07:50:53.212100+0000","last_active":"2024-09-21T07:50:54.213516+0000","last_peered":"2024-09-21T07:50:54.213516+0000","last_clean":"2024-09-21T07:50:54.213516+0000","last_became_active":"2024-09-21T07:50:53.211782+0000","last_became_peered":"2024-09-21T07:50:53.211782+0000","last_unstale":"2024-09-21T07:50:54.213516+0000","last_undegraded":"2024-09-21T07:50:54.213516+0000","last_fullsized":"2024-09-21T07:50:54.213516+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:39.962541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215477+0000","last_change":"2024-09-21T07:50:52.206692+0000","last_active":"2024-09-21T07:50:54.215477+0000","last_peered":"2024-09-21T07:50:54.215477+0000","last_clean":"2024-09-21T07:50:54.215477+0000","last_became_active":"2024-09-21T07:50:52.206371+0000","last_became_peered":"2024-09-21T07:50:52.206371+0000","last_unstale":"2024-09-21T07:50:54.215477+0000","last_undegraded":"2024-09-21T07:50:54.215477+0000","last_fullsized":"2024-09-21T07:50:54.215477+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:47:12.493646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006821+0000","last_change":"2024-09-21T07:50:53.220642+0000","last_active":"2024-09-21T07:50:55.006821+0000","last_peered":"2024-09-21T07:50:55.006821+0000","last_clean":"2024-09-21T07:50:55.006821+0000","last_became_active":"2024-09-21T07:50:53.220296+0000","last_became_peered":"2024-09-21T07:50:53.220296+0000","last_unstale":"2024-09-21T07:50:55.006821+0000","last_undegraded":"2024-09-21T07:50:55.006821+0000","last_fullsized":"2024-09-21T07:50:55.006821+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:32:11.065025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006523+0000","last_change":"2024-09-21T07:50:52.206510+0000","last_active":"2024-09-21T07:50:55.006523+0000","last_peered":"2024-09-21T07:50:55.006523+0000","last_clean":"2024-09-21T07:50:55.006523+0000","last_became_active":"2024-09-21T07:50:52.205927+0000","last_became_peered":"2024-09-21T07:50:52.205927+0000","last_unstale":"2024-09-21T07:50:55.006523+0000","last_undegraded":"2024-09-21T07:50:55.006523+0000","last_fullsized":"2024-09-21T07:50:55.006523+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:29.790747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.867938+0000","last_change":"2024-09-21T07:50:53.202703+0000","last_active":"2024-09-21T07:50:54.867938+0000","last_peered":"2024-09-21T07:50:54.867938+0000","last_clean":"2024-09-21T07:50:54.867938+0000","last_became_active":"2024-09-21T07:50:53.202344+0000","last_became_peered":"2024-09-21T07:50:53.202344+0000","last_unstale":"2024-09-21T07:50:54.867938+0000","last_undegraded":"2024-09-21T07:50:54.867938+0000","last_fullsized":"2024-09-21T07:50:54.867938+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:20.721931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.10","version":"85'8","reported_seq":23,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.867953+0000","last_change":"2024-09-21T07:50:52.207818+0000","last_active":"2024-09-21T07:50:54.867953+0000","last_peered":"2024-09-21T07:50:54.867953+0000","last_clean":"2024-09-21T07:50:54.867953+0000","last_became_active":"2024-09-21T07:50:52.207518+0000","last_became_peered":"2024-09-21T07:50:52.207518+0000","last_unstale":"2024-09-21T07:50:54.867953+0000","last_undegraded":"2024-09-21T07:50:54.867953+0000","last_fullsized":"2024-09-21T07:50:54.867953+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:08.997983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007233+0000","last_change":"2024-09-21T07:50:53.222319+0000","last_active":"2024-09-21T07:50:55.007233+0000","last_peered":"2024-09-21T07:50:55.007233+0000","last_clean":"2024-09-21T07:50:55.007233+0000","last_became_active":"2024-09-21T07:50:53.222018+0000","last_became_peered":"2024-09-21T07:50:53.222018+0000","last_unstale":"2024-09-21T07:50:55.007233+0000","last_undegraded":"2024-09-21T07:50:55.007233+0000","last_fullsized":"2024-09-21T07:50:55.007233+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:47.629829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.f","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008324+0000","last_change":"2024-09-21T07:50:52.216501+0000","last_active":"2024-09-21T07:50:55.008324+0000","last_peered":"2024-09-21T07:50:55.008324+0000","last_clean":"2024-09-21T07:50:55.008324+0000","last_became_active":"2024-09-21T07:50:52.215516+0000","last_became_peered":"2024-09-21T07:50:52.215516+0000","last_unstale":"2024-09-21T07:50:55.008324+0000","last_undegraded":"2024-09-21T07:50:55.008324+0000","last_fullsized":"2024-09-21T07:50:55.008324+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:19.441012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007076+0000","last_change":"2024-09-21T07:50:53.215001+0000","last_active":"2024-09-21T07:50:55.007076+0000","last_peered":"2024-09-21T07:50:55.007076+0000","last_clean":"2024-09-21T07:50:55.007076+0000","last_became_active":"2024-09-21T07:50:53.214682+0000","last_became_peered":"2024-09-21T07:50:53.214682+0000","last_unstale":"2024-09-21T07:50:55.007076+0000","last_undegraded":"2024-09-21T07:50:55.007076+0000","last_fullsized":"2024-09-21T07:50:55.007076+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:23.403279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.e","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007204+0000","last_change":"2024-09-21T07:50:52.207244+0000","last_active":"2024-09-21T07:50:55.007204+0000","last_peered":"2024-09-21T07:50:55.007204+0000","last_clean":"2024-09-21T07:50:55.007204+0000","last_became_active":"2024-09-21T07:50:52.206938+0000","last_became_peered":"2024-09-21T07:50:52.206938+0000","last_unstale":"2024-09-21T07:50:55.007204+0000","last_undegraded":"2024-09-21T07:50:55.007204+0000","last_fullsized":"2024-09-21T07:50:55.007204+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:37:17.085507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007211+0000","last_change":"2024-09-21T07:50:53.206764+0000","last_active":"2024-09-21T07:50:55.007211+0000","last_peered":"2024-09-21T07:50:55.007211+0000","last_clean":"2024-09-21T07:50:55.007211+0000","last_became_active":"2024-09-21T07:50:53.206356+0000","last_became_peered":"2024-09-21T07:50:53.206356+0000","last_unstale":"2024-09-21T07:50:55.007211+0000","last_undegraded":"2024-09-21T07:50:55.007211+0000","last_fullsized":"2024-09-21T07:50:55.007211+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:41.828518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006992+0000","last_change":"2024-09-21T07:50:52.211926+0000","last_active":"2024-09-21T07:50:55.006992+0000","last_peered":"2024-09-21T07:50:55.006992+0000","last_clean":"2024-09-21T07:50:55.006992+0000","last_became_active":"2024-09-21T07:50:52.211613+0000","last_became_peered":"2024-09-21T07:50:52.211613+0000","last_unstale":"2024-09-21T07:50:55.006992+0000","last_undegraded":"2024-09-21T07:50:55.006992+0000","last_fullsized":"2024-09-21T07:50:55.006992+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:46.979912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215404+0000","last_change":"2024-09-21T07:50:53.207296+0000","last_active":"2024-09-21T07:50:54.215404+0000","last_peered":"2024-09-21T07:50:54.215404+0000","last_clean":"2024-09-21T07:50:54.215404+0000","last_became_active":"2024-09-21T07:50:53.206950+0000","last_became_peered":"2024-09-21T07:50:53.206950+0000","last_unstale":"2024-09-21T07:50:54.215404+0000","last_undegraded":"2024-09-21T07:50:54.215404+0000","last_fullsized":"2024-09-21T07:50:54.215404+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:32.754590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.b","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007079+0000","last_change":"2024-09-21T07:50:52.210055+0000","last_active":"2024-09-21T07:50:55.007079+0000","last_peered":"2024-09-21T07:50:55.007079+0000","last_clean":"2024-09-21T07:50:55.007079+0000","last_became_active":"2024-09-21T07:50:52.209537+0000","last_became_peered":"2024-09-21T07:50:52.209537+0000","last_unstale":"2024-09-21T07:50:55.007079+0000","last_undegraded":"2024-09-21T07:50:55.007079+0000","last_fullsized":"2024-09-21T07:50:55.007079+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:35:02.176677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008131+0000","last_change":"2024-09-21T07:50:53.214768+0000","last_active":"2024-09-21T07:50:55.008131+0000","last_peered":"2024-09-21T07:50:55.008131+0000","last_clean":"2024-09-21T07:50:55.008131+0000","last_became_active":"2024-09-21T07:50:53.214481+0000","last_became_peered":"2024-09-21T07:50:53.214481+0000","last_unstale":"2024-09-21T07:50:55.008131+0000","last_undegraded":"2024-09-21T07:50:55.008131+0000","last_fullsized":"2024-09-21T07:50:55.008131+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:07.273516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215585+0000","last_change":"2024-09-21T07:50:52.209708+0000","last_active":"2024-09-21T07:50:54.215585+0000","last_peered":"2024-09-21T07:50:54.215585+0000","last_clean":"2024-09-21T07:50:54.215585+0000","last_became_active":"2024-09-21T07:50:52.208927+0000","last_became_peered":"2024-09-21T07:50:52.208927+0000","last_unstale":"2024-09-21T07:50:54.215585+0000","last_undegraded":"2024-09-21T07:50:54.215585+0000","last_fullsized":"2024-09-21T07:50:54.215585+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:36.493093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.214651+0000","last_change":"2024-09-21T07:50:53.217587+0000","last_active":"2024-09-21T07:50:54.214651+0000","last_peered":"2024-09-21T07:50:54.214651+0000","last_clean":"2024-09-21T07:50:54.214651+0000","last_became_active":"2024-09-21T07:50:53.217282+0000","last_became_peered":"2024-09-21T07:50:53.217282+0000","last_unstale":"2024-09-21T07:50:54.214651+0000","last_undegraded":"2024-09-21T07:50:54.214651+0000","last_fullsized":"2024-09-21T07:50:54.214651+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.253795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.867783+0000","last_change":"2024-09-21T07:50:52.213545+0000","last_active":"2024-09-21T07:50:54.867783+0000","last_peered":"2024-09-21T07:50:54.867783+0000","last_clean":"2024-09-21T07:50:54.867783+0000","last_became_active":"2024-09-21T07:50:52.212910+0000","last_became_peered":"2024-09-21T07:50:52.212910+0000","last_unstale":"2024-09-21T07:50:54.867783+0000","last_undegraded":"2024-09-21T07:50:54.867783+0000","last_fullsized":"2024-09-21T07:50:54.867783+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:10:20.695783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006438+0000","last_change":"2024-09-21T07:50:53.226200+0000","last_active":"2024-09-21T07:50:55.006438+0000","last_peered":"2024-09-21T07:50:55.006438+0000","last_clean":"2024-09-21T07:50:55.006438+0000","last_became_active":"2024-09-21T07:50:53.222557+0000","last_became_peered":"2024-09-21T07:50:53.222557+0000","last_unstale":"2024-09-21T07:50:55.006438+0000","last_undegraded":"2024-09-21T07:50:55.006438+0000","last_fullsized":"2024-09-21T07:50:55.006438+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:15:19.108085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006864+0000","last_change":"2024-09-21T07:50:52.208866+0000","last_active":"2024-09-21T07:50:55.006864+0000","last_peered":"2024-09-21T07:50:55.006864+0000","last_clean":"2024-09-21T07:50:55.006864+0000","last_became_active":"2024-09-21T07:50:52.208528+0000","last_became_peered":"2024-09-21T07:50:52.208528+0000","last_unstale":"2024-09-21T07:50:55.006864+0000","last_undegraded":"2024-09-21T07:50:55.006864+0000","last_fullsized":"2024-09-21T07:50:55.006864+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:51.610726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008175+0000","last_change":"2024-09-21T07:50:53.214975+0000","last_active":"2024-09-21T07:50:55.008175+0000","last_peered":"2024-09-21T07:50:55.008175+0000","last_clean":"2024-09-21T07:50:55.008175+0000","last_became_active":"2024-09-21T07:50:53.213427+0000","last_became_peered":"2024-09-21T07:50:53.213427+0000","last_unstale":"2024-09-21T07:50:55.008175+0000","last_undegraded":"2024-09-21T07:50:55.008175+0000","last_fullsized":"2024-09-21T07:50:55.008175+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:33.457104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.7","version":"85'2","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007384+0000","last_change":"2024-09-21T07:50:52.211874+0000","last_active":"2024-09-21T07:50:55.007384+0000","last_peered":"2024-09-21T07:50:55.007384+0000","last_clean":"2024-09-21T07:50:55.007384+0000","last_became_active":"2024-09-21T07:50:52.211526+0000","last_became_peered":"2024-09-21T07:50:52.211526+0000","last_unstale":"2024-09-21T07:50:55.007384+0000","last_undegraded":"2024-09-21T07:50:55.007384+0000","last_fullsized":"2024-09-21T07:50:55.007384+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:45.691075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008249+0000","last_change":"2024-09-21T07:50:53.219976+0000","last_active":"2024-09-21T07:50:55.008249+0000","last_peered":"2024-09-21T07:50:55.008249+0000","last_clean":"2024-09-21T07:50:55.008249+0000","last_became_active":"2024-09-21T07:50:53.219632+0000","last_became_peered":"2024-09-21T07:50:53.219632+0000","last_unstale":"2024-09-21T07:50:55.008249+0000","last_undegraded":"2024-09-21T07:50:55.008249+0000","last_fullsized":"2024-09-21T07:50:55.008249+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:24:39.322887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.6","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868317+0000","last_change":"2024-09-21T07:50:52.211977+0000","last_active":"2024-09-21T07:50:54.868317+0000","last_peered":"2024-09-21T07:50:54.868317+0000","last_clean":"2024-09-21T07:50:54.868317+0000","last_became_active":"2024-09-21T07:50:52.211666+0000","last_became_peered":"2024-09-21T07:50:52.211666+0000","last_unstale":"2024-09-21T07:50:54.868317+0000","last_undegraded":"2024-09-21T07:50:54.868317+0000","last_fullsized":"2024-09-21T07:50:54.868317+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:28:11.048789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008073+0000","last_change":"2024-09-21T07:50:53.217409+0000","last_active":"2024-09-21T07:50:55.008073+0000","last_peered":"2024-09-21T07:50:55.008073+0000","last_clean":"2024-09-21T07:50:55.008073+0000","last_became_active":"2024-09-21T07:50:53.217130+0000","last_became_peered":"2024-09-21T07:50:53.217130+0000","last_unstale":"2024-09-21T07:50:55.008073+0000","last_undegraded":"2024-09-21T07:50:55.008073+0000","last_fullsized":"2024-09-21T07:50:55.008073+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:18.222544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007284+0000","last_change":"2024-09-21T07:50:52.214686+0000","last_active":"2024-09-21T07:50:55.007284+0000","last_peered":"2024-09-21T07:50:55.007284+0000","last_clean":"2024-09-21T07:50:55.007284+0000","last_became_active":"2024-09-21T07:50:52.214343+0000","last_became_peered":"2024-09-21T07:50:52.214343+0000","last_unstale":"2024-09-21T07:50:55.007284+0000","last_undegraded":"2024-09-21T07:50:55.007284+0000","last_fullsized":"2024-09-21T07:50:55.007284+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:16.490813+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.007813+0000","last_change":"2024-09-21T07:50:53.227685+0000","last_active":"2024-09-21T07:50:55.007813+0000","last_peered":"2024-09-21T07:50:55.007813+0000","last_clean":"2024-09-21T07:50:55.007813+0000","last_became_active":"2024-09-21T07:50:53.227408+0000","last_became_peered":"2024-09-21T07:50:53.227408+0000","last_unstale":"2024-09-21T07:50:55.007813+0000","last_undegraded":"2024-09-21T07:50:55.007813+0000","last_fullsized":"2024-09-21T07:50:55.007813+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:07:08.253795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006358+0000","last_change":"2024-09-21T07:50:52.217865+0000","last_active":"2024-09-21T07:50:55.006358+0000","last_peered":"2024-09-21T07:50:55.006358+0000","last_clean":"2024-09-21T07:50:55.006358+0000","last_became_active":"2024-09-21T07:50:52.217565+0000","last_became_peered":"2024-09-21T07:50:52.217565+0000","last_unstale":"2024-09-21T07:50:55.006358+0000","last_undegraded":"2024-09-21T07:50:55.006358+0000","last_fullsized":"2024-09-21T07:50:55.006358+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:10.472781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213247+0000","last_change":"2024-09-21T07:50:53.208529+0000","last_active":"2024-09-21T07:50:54.213247+0000","last_peered":"2024-09-21T07:50:54.213247+0000","last_clean":"2024-09-21T07:50:54.213247+0000","last_became_active":"2024-09-21T07:50:53.208218+0000","last_became_peered":"2024-09-21T07:50:53.208218+0000","last_unstale":"2024-09-21T07:50:54.213247+0000","last_undegraded":"2024-09-21T07:50:54.213247+0000","last_fullsized":"2024-09-21T07:50:54.213247+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:00.544836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.215717+0000","last_change":"2024-09-21T07:50:52.211531+0000","last_active":"2024-09-21T07:50:54.215717+0000","last_peered":"2024-09-21T07:50:54.215717+0000","last_clean":"2024-09-21T07:50:54.215717+0000","last_became_active":"2024-09-21T07:50:52.211025+0000","last_became_peered":"2024-09-21T07:50:52.211025+0000","last_unstale":"2024-09-21T07:50:54.215717+0000","last_undegraded":"2024-09-21T07:50:54.215717+0000","last_fullsized":"2024-09-21T07:50:54.215717+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:41.196672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868142+0000","last_change":"2024-09-21T07:50:53.219206+0000","last_active":"2024-09-21T07:50:54.868142+0000","last_peered":"2024-09-21T07:50:54.868142+0000","last_clean":"2024-09-21T07:50:54.868142+0000","last_became_active":"2024-09-21T07:50:53.218854+0000","last_became_peered":"2024-09-21T07:50:53.218854+0000","last_unstale":"2024-09-21T07:50:54.868142+0000","last_undegraded":"2024-09-21T07:50:54.868142+0000","last_fullsized":"2024-09-21T07:50:54.868142+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:37.501943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006760+0000","last_change":"2024-09-21T07:50:52.212217+0000","last_active":"2024-09-21T07:50:55.006760+0000","last_peered":"2024-09-21T07:50:55.006760+0000","last_clean":"2024-09-21T07:50:55.006760+0000","last_became_active":"2024-09-21T07:50:52.211503+0000","last_became_peered":"2024-09-21T07:50:52.211503+0000","last_unstale":"2024-09-21T07:50:55.006760+0000","last_undegraded":"2024-09-21T07:50:55.006760+0000","last_fullsized":"2024-09-21T07:50:55.006760+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:06.528189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213218+0000","last_change":"2024-09-21T07:50:53.210012+0000","last_active":"2024-09-21T07:50:54.213218+0000","last_peered":"2024-09-21T07:50:54.213218+0000","last_clean":"2024-09-21T07:50:54.213218+0000","last_became_active":"2024-09-21T07:50:53.209727+0000","last_became_peered":"2024-09-21T07:50:53.209727+0000","last_unstale":"2024-09-21T07:50:54.213218+0000","last_undegraded":"2024-09-21T07:50:54.213218+0000","last_fullsized":"2024-09-21T07:50:54.213218+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:30.830914+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213281+0000","last_change":"2024-09-21T07:50:52.213532+0000","last_active":"2024-09-21T07:50:54.213281+0000","last_peered":"2024-09-21T07:50:54.213281+0000","last_clean":"2024-09-21T07:50:54.213281+0000","last_became_active":"2024-09-21T07:50:52.213238+0000","last_became_peered":"2024-09-21T07:50:52.213238+0000","last_unstale":"2024-09-21T07:50:54.213281+0000","last_undegraded":"2024-09-21T07:50:54.213281+0000","last_fullsized":"2024-09-21T07:50:54.213281+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:40.582748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213222+0000","last_change":"2024-09-21T07:50:53.215762+0000","last_active":"2024-09-21T07:50:54.213222+0000","last_peered":"2024-09-21T07:50:54.213222+0000","last_clean":"2024-09-21T07:50:54.213222+0000","last_became_active":"2024-09-21T07:50:53.215482+0000","last_became_peered":"2024-09-21T07:50:53.215482+0000","last_unstale":"2024-09-21T07:50:54.213222+0000","last_undegraded":"2024-09-21T07:50:54.213222+0000","last_fullsized":"2024-09-21T07:50:54.213222+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:38.879701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.006684+0000","last_change":"2024-09-21T07:50:52.212434+0000","last_active":"2024-09-21T07:50:55.006684+0000","last_peered":"2024-09-21T07:50:55.006684+0000","last_clean":"2024-09-21T07:50:55.006684+0000","last_became_active":"2024-09-21T07:50:52.212030+0000","last_became_peered":"2024-09-21T07:50:52.212030+0000","last_unstale":"2024-09-21T07:50:55.006684+0000","last_undegraded":"2024-09-21T07:50:55.006684+0000","last_fullsized":"2024-09-21T07:50:55.006684+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:19:11.963528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":13,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.868327+0000","last_change":"2024-09-21T07:50:53.214252+0000","last_active":"2024-09-21T07:50:54.868327+0000","last_peered":"2024-09-21T07:50:54.868327+0000","last_clean":"2024-09-21T07:50:54.868327+0000","last_became_active":"2024-09-21T07:50:53.213932+0000","last_became_peered":"2024-09-21T07:50:53.213932+0000","last_unstale":"2024-09-21T07:50:54.868327+0000","last_undegraded":"2024-09-21T07:50:54.868327+0000","last_fullsized":"2024-09-21T07:50:54.868327+0000","mapping_epoch":84,"log_start":"0'0","ondisk_log_start":"0'0","created":84,"last_epoch_clean":85,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:52.179615+0000","last_clean_scrub_stamp":"2024-09-21T07:50:52.179615+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:59:30.799597+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.d","version":"85'1","reported_seq":16,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:55.008103+0000","last_change":"2024-09-21T07:50:52.199423+0000","last_active":"2024-09-21T07:50:55.008103+0000","last_peered":"2024-09-21T07:50:55.008103+0000","last_clean":"2024-09-21T07:50:55.008103+0000","last_became_active":"2024-09-21T07:50:52.199031+0000","last_became_peered":"2024-09-21T07:50:52.199031+0000","last_unstale":"2024-09-21T07:50:55.008103+0000","last_undegraded":"2024-09-21T07:50:55.008103+0000","last_fullsized":"2024-09-21T07:50:55.008103+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:50:51.170764+0000","last_clean_scrub_stamp":"2024-09-21T07:50:51.170764+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:17.079248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":259,"reported_epoch":86,"state":"active+clean","last_fresh":"2024-09-21T07:50:54.213290+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:50:54.213290+0000","last_peered":"2024-09-21T07:50:54.213290+0000","last_clean":"2024-09-21T07:50:54.213290+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:50:54.213290+0000","last_undegraded":"2024-09-21T07:50:54.213290+0000","last_fullsized":"2024-09-21T07:50:54.213290+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542208,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32216,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339624,"statfs":{"total":96636764160,"available":96603774976,"internally_reserved":0,"allocated":1048576,"data_stored":448939,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509503,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1052672,"data_stored":449541,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509504,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32804,"kb_used_data":1616,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339036,"statfs":{"total":96636764160,"available":96603172864,"internally_reserved":0,"allocated":1654784,"data_stored":1040883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509504,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32212,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339628,"statfs":{"total":96636764160,"available":96603779072,"internally_reserved":0,"allocated":1048576,"data_stored":448903,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476802,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32220,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339620,"statfs":{"total":96636764160,"available":96603770880,"internally_reserved":0,"allocated":1056768,"data_stored":449543,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476802,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32208,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339632,"statfs":{"total":96636764160,"available":96603783168,"internally_reserved":0,"allocated":1044480,"data_stored":448869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,0,2],"upper_bound":32},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476799,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":1039327,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476799,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32228,"kb_used_data":1036,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339612,"statfs":{"total":96636764160,"available":96603762688,"internally_reserved":0,"allocated":1060864,"data_stored":450503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1646,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:50:58.150 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:50:58.151 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 -- 172.21.15.88:0/3966561534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e5015bca0 msgr2=0x7f5e500cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 --2- 172.21.15.88:0/3966561534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e5015bca0 0x7f5e500cf9a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f5e40005fd0 tx=0x7f5e4002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 -- 172.21.15.88:0/3966561534 shutdown_connections 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 --2- 172.21.15.88:0/3966561534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e5015bca0 0x7f5e500cf9a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 -- 172.21.15.88:0/3966561534 >> 172.21.15.88:0/3966561534 conn(0x7f5e500b1dd0 msgr2=0x7f5e500b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 -- 172.21.15.88:0/3966561534 shutdown_connections 2024-09-21T07:50:58.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.412+0000 7f5e56600640 1 -- 172.21.15.88:0/3966561534 wait complete. 2024-09-21T07:50:58.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.414+0000 7f5e56600640 1 Processor -- start 2024-09-21T07:50:58.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.415+0000 7f5e56600640 1 -- start start 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e56600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 0x7f5e501e85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e56600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e500cf1c0 0x7f5e5014c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e56600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5015bca0 0x7f5e5015a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e56600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e500cd680 con 0x7f5e50070220 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e56600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e500cd500 con 0x7f5e5015bca0 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e56600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5e500cd800 con 0x7f5e500cf1c0 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e55200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 0x7f5e501e85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e55200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 0x7f5e501e85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36686/0 (socket says 172.21.15.88:36686) 2024-09-21T07:50:58.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.416+0000 7f5e55200640 1 -- 172.21.15.88:0/3897274994 learned_addr learned my addr 172.21.15.88:0/3897274994 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e47e00640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e500cf1c0 0x7f5e5014c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55200640 1 -- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e500cf1c0 msgr2=0x7f5e5014c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55c00640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5015bca0 0x7f5e5015a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55200640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e500cf1c0 0x7f5e5014c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55200640 1 -- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5015bca0 msgr2=0x7f5e5015a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55200640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5015bca0 0x7f5e5015a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55200640 1 -- 172.21.15.88:0/3897274994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e501f6250 con 0x7f5e50070220 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e47e00640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e500cf1c0 0x7f5e5014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55c00640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5015bca0 0x7f5e5015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e55200640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 0x7f5e501e85b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5e40034040 tx=0x7f5e400039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:58.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e4003e030 con 0x7f5e50070220 2024-09-21T07:50:58.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e400047a0 con 0x7f5e50070220 2024-09-21T07:50:58.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.417+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e501f6530 con 0x7f5e50070220 2024-09-21T07:50:58.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.418+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e40035040 con 0x7f5e50070220 2024-09-21T07:50:58.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.418+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e501f6af0 con 0x7f5e50070220 2024-09-21T07:50:58.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.419+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e500b2a60 con 0x7f5e50070220 2024-09-21T07:50:58.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.420+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5e40041030 con 0x7f5e50070220 2024-09-21T07:50:58.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.421+0000 7f5e4d600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e2c078cb0 0x7f5e2c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.421+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f5e400bb870 con 0x7f5e50070220 2024-09-21T07:50:58.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.421+0000 7f5e47e00640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e2c078cb0 0x7f5e2c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.422+0000 7f5e47e00640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e2c078cb0 0x7f5e2c07b1c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e5015b460 tx=0x7f5e38009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:58.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.425+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e40083b10 con 0x7f5e50070220 2024-09-21T07:50:58.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.680+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f5e5015fd40 con 0x7f5e50070220 2024-09-21T07:50:58.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.683+0000 7f5e4d600640 1 -- 172.21.15.88:0/3897274994 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v15) ==== 190+0+147 (secure 0 0 0) 0x7f5e40083410 con 0x7f5e50070220 2024-09-21T07:50:58.685 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:50:58.685 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:50:58.685 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:50:58.685 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:50:58.685 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:50:58.685 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:50:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e2c078cb0 msgr2=0x7f5e2c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e2c078cb0 0x7f5e2c07b1c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5e5015b460 tx=0x7f5e38009040 comp rx=0 tx=0).stop 2024-09-21T07:50:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 msgr2=0x7f5e501e85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 0x7f5e501e85b0 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f5e40034040 tx=0x7f5e400039e0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 shutdown_connections 2024-09-21T07:50:58.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e2c078cb0 0x7f5e2c07b1c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e500cf1c0 0x7f5e5014c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e5015bca0 0x7f5e5015a6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 --2- 172.21.15.88:0/3897274994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e50070220 0x7f5e501e85b0 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 >> 172.21.15.88:0/3897274994 conn(0x7f5e500b1dd0 msgr2=0x7f5e5015d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:58.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 shutdown_connections 2024-09-21T07:50:58.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.686+0000 7f5e56600640 1 -- 172.21.15.88:0/3897274994 wait complete. 2024-09-21T07:50:58.709 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:50:58.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557d800640 1 -- 172.21.15.88:0/952806157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c003a50 msgr2=0x7f555c003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557d800640 1 --2- 172.21.15.88:0/952806157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f555c003a50 0x7f555c003ec0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f556800c560 tx=0x7f555c005830 comp rx=0 tx=0).stop 2024-09-21T07:50:58.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557ce00640 1 -- 172.21.15.88:0/952806157 reap_dead start 2024-09-21T07:50:58.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557d800640 1 -- 172.21.15.88:0/952806157 shutdown_connections 2024-09-21T07:50:58.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557d800640 1 -- 172.21.15.88:0/952806157 >> 172.21.15.88:0/952806157 conn(0x7f5578071070 msgr2=0x7f5578071480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:58.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557d800640 1 -- 172.21.15.88:0/952806157 shutdown_connections 2024-09-21T07:50:58.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.929+0000 7f557d800640 1 -- 172.21.15.88:0/952806157 wait complete. 2024-09-21T07:50:58.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.931+0000 7f557d800640 1 Processor -- start 2024-09-21T07:50:58.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.931+0000 7f557d800640 1 -- start start 2024-09-21T07:50:58.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557d800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555c003a50 0x7f55781f5e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557d800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55780aeb70 0x7f55780b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 0x7f557815f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557d800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55780cd3e0 con 0x7f5578161b20 2024-09-21T07:50:58.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557d800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f55780cd260 con 0x7f55780aeb70 2024-09-21T07:50:58.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557d800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f55780cd560 con 0x7f555c003a50 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 0x7f557815f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 0x7f557815f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36708/0 (socket says 172.21.15.88:36708) 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f557ce00640 1 -- 172.21.15.88:0/1012258949 learned_addr learned my addr 172.21.15.88:0/1012258949 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f5577e00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555c003a50 0x7f55781f5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.933+0000 7f5577400640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55780aeb70 0x7f55780b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557ce00640 1 -- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555c003a50 msgr2=0x7f55781f5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557ce00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555c003a50 0x7f55781f5e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557ce00640 1 -- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55780aeb70 msgr2=0x7f55780b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557ce00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55780aeb70 0x7f55780b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557ce00640 1 -- 172.21.15.88:0/1012258949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55781f66a0 con 0x7f5578161b20 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557ce00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 0x7f557815f3b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f55780d1110 tx=0x7f557000f960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:58.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570011910 con 0x7f5578161b20 2024-09-21T07:50:58.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5570011ba0 con 0x7f5578161b20 2024-09-21T07:50:58.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f557001b410 con 0x7f5578161b20 2024-09-21T07:50:58.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55781f6980 con 0x7f5578161b20 2024-09-21T07:50:58.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.934+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f557815ebb0 con 0x7f5578161b20 2024-09-21T07:50:58.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.935+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55780b5510 con 0x7f5578161b20 2024-09-21T07:50:58.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.937+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5570019020 con 0x7f5578161b20 2024-09-21T07:50:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.938+0000 7f556fe00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5558078cb0 0x7f555807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.938+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f557009b310 con 0x7f5578161b20 2024-09-21T07:50:58.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.938+0000 7f5577e00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5558078cb0 0x7f555807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:58.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.939+0000 7f5577e00640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5558078cb0 0x7f555807b1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5568033000 tx=0x7f556802d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:58.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:58.943+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55780b5510 con 0x7f5578161b20 2024-09-21T07:50:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.165+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f557815e9a0 con 0x7f5578161b20 2024-09-21T07:50:59.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.165+0000 7f556fe00640 1 -- 172.21.15.88:0/1012258949 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 55 v55) ==== 94+0+5228 (secure 0 0 0) 0x7f5570063000 con 0x7f5578161b20 2024-09-21T07:50:59.167 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:50:59.167 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":55,"btime":"2024-09-21T07:50:56:218406+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6367,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3755060210","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3755060210},{"type":"v1","addr":"172.21.15.175:6835","nonce":3755060210}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:52.693177+0000","modified":"2024-09-21T07:50:55.744683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1717882335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1717882335},{"type":"v1","addr":"172.21.15.88:6835","nonce":1717882335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6600,"qdb_cluster":[6600]},"id":6}]} 2024-09-21T07:50:59.167 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 55 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5558078cb0 msgr2=0x7f555807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5558078cb0 0x7f555807b1c0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5568033000 tx=0x7f556802d040 comp rx=0 tx=0).stop 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 msgr2=0x7f557815f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 0x7f557815f3b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f55780d1110 tx=0x7f557000f960 comp rx=0 tx=0).stop 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 shutdown_connections 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5558078cb0 0x7f555807b1c0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578161b20 0x7f557815f3b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55780aeb70 0x7f55780b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 --2- 172.21.15.88:0/1012258949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f555c003a50 0x7f55781f5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 >> 172.21.15.88:0/1012258949 conn(0x7f5578071070 msgr2=0x7f55780cdaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 shutdown_connections 2024-09-21T07:50:59.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.168+0000 7f557d800640 1 -- 172.21.15.88:0/1012258949 wait complete. 2024-09-21T07:50:59.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 55, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:50:52.693177+0000', 'modified': '2024-09-21T07:50:55.744683+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6600}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6600': {'gid': 6600, 'name': 'a', 'rank': 0, 'incarnation': 52, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/1717882335', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6833', 'nonce': 1717882335}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 1717882335}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6600, 'qdb_cluster': [6600]} 2024-09-21T07:50:59.597 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:50:59.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:50:59.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 -- 172.21.15.88:0/191927800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 msgr2=0x7f7fac0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/191927800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac0af170 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7fac0af6b0 tx=0x7f7f9c02a560 comp rx=0 tx=0).stop 2024-09-21T07:50:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 -- 172.21.15.88:0/191927800 shutdown_connections 2024-09-21T07:50:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/191927800 >> v2:172.21.15.175:3300/0 conn(0x7f7fac15a0f0 0x7f7fac15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/191927800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac0af170 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 -- 172.21.15.88:0/191927800 >> 172.21.15.88:0/191927800 conn(0x7f7fac0b1e40 msgr2=0x7f7fac0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 -- 172.21.15.88:0/191927800 shutdown_connections 2024-09-21T07:50:59.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.817+0000 7f7fb1e00640 1 -- 172.21.15.88:0/191927800 wait complete. 2024-09-21T07:50:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.819+0000 7f7fb1e00640 1 Processor -- start 2024-09-21T07:50:59.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.820+0000 7f7fb1e00640 1 -- start start 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac09d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fac0b2f60 0x7f7fac151040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb1e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fac15a0f0 0x7f7fac0747e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb1e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac06f410 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb1e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac06f290 con 0x7f7fac0b2f60 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb1e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac06f590 con 0x7f7fac15a0f0 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac09d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac09d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36730/0 (socket says 172.21.15.88:36730) 2024-09-21T07:50:59.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.821+0000 7f7fb0a00640 1 -- 172.21.15.88:0/2961717735 learned_addr learned my addr 172.21.15.88:0/2961717735 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fac0b2f60 0x7f7fac151040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb1400640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fac15a0f0 0x7f7fac0747e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb0a00640 1 -- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fac15a0f0 msgr2=0x7f7fac0747e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fac15a0f0 0x7f7fac0747e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb0a00640 1 -- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fac0b2f60 msgr2=0x7f7fac151040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fac0b2f60 0x7f7fac151040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb0a00640 1 -- 172.21.15.88:0/2961717735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fac09c550 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac09d5f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c034040 tx=0x7f7f9c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fa9600640 1 -- 172.21.15.88:0/2961717735 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c03e030 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fa9600640 1 -- 172.21.15.88:0/2961717735 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f9c003e00 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fac09c830 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.822+0000 7f7fa9600640 1 -- 172.21.15.88:0/2961717735 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c035040 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.823+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fac09cdf0 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.826+0000 7f7fa9600640 1 -- 172.21.15.88:0/2961717735 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7f9c041030 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.827+0000 7f7fa9600640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f88078cb0 0x7f7f8807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.827+0000 7f7fa9600640 1 -- 172.21.15.88:0/2961717735 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f7f9c0bb7f0 con 0x7f7fac0ad9c0 2024-09-21T07:50:59.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.827+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f88078cb0 0x7f7f8807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.827+0000 7f7fba460740 1 Processor -- start 2024-09-21T07:50:59.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.827+0000 7f7fba460740 1 -- start start 2024-09-21T07:50:59.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.828+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f88078cb0 0x7f7f8807b1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f940015e0 tx=0x7f7f94009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fba460740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba58cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba58d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba55d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fba460740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560eba4e6430 con 0x560eba1bc270 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560eba4d65d0 con 0x560eba1933b0 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x560eba4c5a00 con 0x560eba19b590 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba58cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba58cb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36744/0 (socket says 172.21.15.88:36744) 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 -- 172.21.15.88:0/810853262 learned_addr learned my addr 172.21.15.88:0/810853262 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 -- 172.21.15.88:0/810853262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 msgr2=0x560eba55d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba55d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 -- 172.21.15.88:0/810853262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 msgr2=0x560eba58d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba58d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 -- 172.21.15.88:0/810853262 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eba48e520 con 0x560eba1bc270 2024-09-21T07:50:59.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.829+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba58cb60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005fd0 tx=0x7f7f9c082c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd0030efbb3fab4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.830+0000 7f7fa1200640 1 -- 172.21.15.88:0/810853262 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c0a9070 con 0x560eba1bc270 2024-09-21T07:50:59.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.830+0000 7f7fa1200640 1 -- 172.21.15.88:0/810853262 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f9c083a10 con 0x560eba1bc270 2024-09-21T07:50:59.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.830+0000 7f7fa1200640 1 -- 172.21.15.88:0/810853262 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9c048040 con 0x560eba1bc270 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.830+0000 7f7fba460740 1 -- 172.21.15.88:0/810853262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 msgr2=0x560eba58cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.830+0000 7f7fba460740 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba58cb60 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c005fd0 tx=0x7f7f9c082c60 comp rx=0 tx=0).stop 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 -- 172.21.15.88:0/810853262 shutdown_connections 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba55d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba58d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 --2- 172.21.15.88:0/810853262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba58cb60 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 -- 172.21.15.88:0/810853262 >> 172.21.15.88:0/810853262 conn(0x560eba1b03c0 msgr2=0x560eba562b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 -- 172.21.15.88:0/810853262 shutdown_connections 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.831+0000 7f7fba460740 1 -- 172.21.15.88:0/810853262 wait complete. 2024-09-21T07:50:59.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.832+0000 7f7fba460740 1 -- start start 2024-09-21T07:50:59.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.832+0000 7f7fba460740 1 Processor -- start 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5b77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba586b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560eba46c330 con 0x560eba1bc270 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560eba4c9810 con 0x560eba1933b0 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x560eba236620 con 0x560eba19b590 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.833+0000 7f7fba460740 10 client.0 _ll_register_callbacks cb 0x560eba471420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba586b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba586b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36754/0 (socket says 172.21.15.88:36754) 2024-09-21T07:50:59.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 -- 172.21.15.88:0/1520266352 learned_addr learned my addr 172.21.15.88:0/1520266352 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5b77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 -- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 msgr2=0x560eba5b77b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5b77b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 -- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 msgr2=0x560eba5a11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 -- 172.21.15.88:0/1520266352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eba227850 con 0x560eba1bc270 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7fb1400640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba586b10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40106d0 tx=0x7f7fa4010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d359aa02f9ee1726 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.834+0000 7f7f99600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7f99600640 1 -- 172.21.15.88:0/1520266352 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4004d20 con 0x560eba1bc270 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7f99600640 1 -- 172.21.15.88:0/1520266352 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f7fa400e8d0 con 0x560eba1bc270 2024-09-21T07:50:59.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9c083a10 con 0x560eba1bc270 2024-09-21T07:50:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7f99600640 1 -- 172.21.15.88:0/1520266352 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7fa4023050 con 0x560eba1bc270 2024-09-21T07:50:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 10 client.6723 fetch_fsmap learned FSMap version 55 2024-09-21T07:50:59.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x560eba29e6b0 con 0x560eba1bc270 2024-09-21T07:50:59.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7f99600640 1 -- 172.21.15.88:0/1520266352 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f9c083a10 con 0x560eba1bc270 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7f99600640 1 -- 172.21.15.88:0/1520266352 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f7fa4026020 con 0x560eba1bc270 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 10 client.6723 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 10 client.6723 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 20 client.6723 populate_metadata read hostname 'smithi088' 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba17ae40 0x560eba5ba8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 4 client.6723 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.835+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 --> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560eba4d8520 con 0x560eba17ae40 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.836+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba17ae40 0x560eba5ba8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.836+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba17ae40 0x560eba5ba8e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=97ad002d5f1bbcc7 server_cookie=6908e4ede9b9693 in_seq=0 out_seq=0 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.836+0000 7f7f99600640 10 client.6723 ms_handle_connect on v2:172.21.15.88:6833/1717882335 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7f99600640 1 -- 172.21.15.88:0/1520266352 <== mds.0 v2:172.21.15.88:6833/1717882335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560eba4d8520 con 0x560eba17ae40 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7f99600640 10 client.6723 handle_command_reply: tid=0 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 client.6723 shutdown 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 msgr2=0x560eba586b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba586b10 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f7fa40106d0 tx=0x7f7fa4010b90 comp rx=0 tx=0).stop 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 shutdown_connections 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba17ae40 0x560eba5ba8e0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5b77b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba586b10 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 --2- 172.21.15.88:0/1520266352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7f99600640 0 client.6723 ms_handle_reset on v2:172.21.15.88:6833/1717882335 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 >> 172.21.15.88:0/1520266352 conn(0x560eba1b03c0 msgr2=0x560eba5a9ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 shutdown_connections 2024-09-21T07:50:59.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.837+0000 7f7fba460740 1 -- 172.21.15.88:0/1520266352 wait complete. 2024-09-21T07:50:59.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.838+0000 7f7fba460740 20 client.6723 trim_cache size 0 max 16384 2024-09-21T07:50:59.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.856+0000 7f7fba460740 1 Processor -- start 2024-09-21T07:50:59.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.857+0000 7f7fba460740 1 -- start start 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fba460740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 0x560eba586300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba1bc270 0x560eba586840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fba460740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac200f40 con 0x560eba19b590 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7fac1f8f20 con 0x560eba1933b0 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7fac1f9570 con 0x560eba1bc270 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 0x560eba586300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fa3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 0x560eba586300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36756/0 (socket says 172.21.15.88:36756) 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fa3e00640 1 -- 172.21.15.88:0/888133208 learned_addr learned my addr 172.21.15.88:0/888133208 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.858+0000 7f7fb1400640 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba1bc270 0x560eba586840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7fa3e00640 1 -- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba1bc270 msgr2=0x560eba586840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba1bc270 0x560eba586840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7fa3e00640 1 -- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 msgr2=0x560eba5a9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7fa3e00640 1 -- 172.21.15.88:0/888133208 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560eba563e50 con 0x560eba19b590 2024-09-21T07:50:59.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 0x560eba586300 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7f94001e70 tx=0x7f7f94011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f28b66be71e2f97e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7f9be00640 1 -- 172.21.15.88:0/888133208 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f94027070 con 0x560eba19b590 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7f9be00640 1 -- 172.21.15.88:0/888133208 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f9400e5a0 con 0x560eba19b590 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.859+0000 7f7f9be00640 1 -- 172.21.15.88:0/888133208 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f9400e790 con 0x560eba19b590 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 -- 172.21.15.88:0/888133208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 msgr2=0x560eba586300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 0x560eba586300 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7f94001e70 tx=0x7f7f94011000 comp rx=0 tx=0).stop 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 -- 172.21.15.88:0/888133208 shutdown_connections 2024-09-21T07:50:59.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba1bc270 0x560eba586840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba19b590 0x560eba586300 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 --2- 172.21.15.88:0/888133208 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba5a9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 -- 172.21.15.88:0/888133208 >> 172.21.15.88:0/888133208 conn(0x560eba1b03c0 msgr2=0x560eba55e210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 -- 172.21.15.88:0/888133208 shutdown_connections 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.860+0000 7f7fba460740 1 -- 172.21.15.88:0/888133208 wait complete. 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.861+0000 7f7fba460740 1 -- start start 2024-09-21T07:50:59.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.861+0000 7f7fba460740 1 Processor -- start 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba62ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5c0910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba5c0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560eba5b7d70 con 0x560eba1bc270 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x560eba585620 con 0x560eba1933b0 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x560eba588fd0 con 0x560eba19b590 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fba460740 10 client.0 _ll_register_callbacks cb 0x7f7fa4009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba5c0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba5c0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36770/0 (socket says 172.21.15.88:36770) 2024-09-21T07:50:59.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 -- 172.21.15.88:0/2206202606 learned_addr learned my addr 172.21.15.88:0/2206202606 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba62ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fa3e00640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5c0910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 -- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 msgr2=0x560eba5c0910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5c0910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 -- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 msgr2=0x560eba62ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba62ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.863+0000 7f7fb1400640 1 -- 172.21.15.88:0/2206202606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560eba62b340 con 0x560eba1bc270 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7fb1400640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba5c0e50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000f80 tx=0x7f7fa4009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd8c0919735e29d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7f99600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7f99600640 1 -- 172.21.15.88:0/2206202606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4005200 con 0x560eba1bc270 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7f99600640 1 -- 172.21.15.88:0/2206202606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f7fa40053a0 con 0x560eba1bc270 2024-09-21T07:50:59.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7f9400e5a0 con 0x560eba1bc270 2024-09-21T07:50:59.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7f99600640 1 -- 172.21.15.88:0/2206202606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=55) ==== 24+0+0 (secure 0 0 0) 0x7f7fa4039050 con 0x560eba1bc270 2024-09-21T07:50:59.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7fba460740 10 client.6729 fetch_fsmap learned FSMap version 55 2024-09-21T07:50:59.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=55,monmap=2+}) -- 0x560eba5b0680 con 0x560eba1bc270 2024-09-21T07:50:59.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.864+0000 7f7f99600640 1 -- 172.21.15.88:0/2206202606 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f9400e5a0 con 0x560eba1bc270 2024-09-21T07:50:59.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7f99600640 1 -- 172.21.15.88:0/2206202606 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 55) ==== 3502+0+0 (secure 0 0 0) 0x7f7fa403c020 con 0x560eba1bc270 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fba460740 10 client.6729 fetch_fsmap finished waiting for FSMap version 55 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fba460740 10 client.6729 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fba460740 20 client.6729 populate_metadata read hostname 'smithi088' 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fba460740 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba62c1f0 0x560eba5aaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fba460740 4 client.6729 mds_command: new command op to 6600 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 --> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] -- command(tid 0: {"prefix": "status"}) -- 0x560eba4d3fd0 con 0x560eba62c1f0 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba62c1f0 0x560eba5aaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7fb0a00640 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba62c1f0 0x560eba5aaeb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80db725f8af83dd3 server_cookie=4f948b38842f800f in_seq=0 out_seq=0 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.865+0000 7f7f99600640 10 client.6729 ms_handle_connect on v2:172.21.15.88:6833/1717882335 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7f99600640 1 -- 172.21.15.88:0/2206202606 <== mds.0 v2:172.21.15.88:6833/1717882335 1 ==== command_reply(tid 0: 0 ) ==== 8+0+331 (crc 0 0 0) 0x560eba4d3fd0 con 0x560eba62c1f0 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7f99600640 10 client.6729 handle_command_reply: tid=0 2024-09-21T07:50:59.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 client.6729 shutdown 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 msgr2=0x560eba5c0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba5c0e50 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4000f80 tx=0x7f7fa4009240 comp rx=0 tx=0).stop 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 shutdown_connections 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:6833/1717882335,v1:172.21.15.88:6835/1717882335] conn(0x560eba62c1f0 0x560eba5aaeb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x560eba19b590 0x560eba5c0910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560eba1bc270 0x560eba5c0e50 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 --2- 172.21.15.88:0/2206202606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x560eba1933b0 0x560eba62ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7f99600640 0 client.6729 ms_handle_reset on v2:172.21.15.88:6833/1717882335 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 >> 172.21.15.88:0/2206202606 conn(0x560eba1b03c0 msgr2=0x560eba587f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 shutdown_connections 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 1 -- 172.21.15.88:0/2206202606 wait complete. 2024-09-21T07:50:59.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.866+0000 7f7fba460740 20 client.6729 trim_cache size 0 max 16384 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6600, 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6767009310000001, 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 55, 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 86, 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 86, 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.738520062999999 2024-09-21T07:50:59.871 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:50:59.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f88078cb0 msgr2=0x7f7f8807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f88078cb0 0x7f7f8807b1c0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f7f940015e0 tx=0x7f7f94009040 comp rx=0 tx=0).stop 2024-09-21T07:50:59.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 msgr2=0x7f7fac09d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:50:59.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac09d5f0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7f9c034040 tx=0x7f7f9c003910 comp rx=0 tx=0).stop 2024-09-21T07:50:59.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 shutdown_connections 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f88078cb0 0x7f7f8807b1c0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7fac15a0f0 0x7f7fac0747e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7fac0b2f60 0x7f7fac151040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 --2- 172.21.15.88:0/2961717735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fac0ad9c0 0x7f7fac09d5f0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 >> 172.21.15.88:0/2961717735 conn(0x7f7fac0b1e40 msgr2=0x7f7fac15c0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 shutdown_connections 2024-09-21T07:50:59.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:50:59.867+0000 7f7fb1e00640 1 -- 172.21.15.88:0/2961717735 wait complete. 2024-09-21T07:50:59.889 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.107+0000 7ff9ef200640 1 -- 172.21.15.88:0/1122513882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e81518d0 msgr2=0x7ff9e806f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.107+0000 7ff9ef200640 1 --2- 172.21.15.88:0/1122513882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e81518d0 0x7ff9e806f9f0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7ff9e806af60 tx=0x7ff9dc02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.107+0000 7ff9ef200640 1 -- 172.21.15.88:0/1122513882 shutdown_connections 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.107+0000 7ff9ef200640 1 --2- 172.21.15.88:0/1122513882 >> v2:172.21.15.175:3300/0 conn(0x7ff9e80b3520 0x7ff9e80b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.107+0000 7ff9ef200640 1 --2- 172.21.15.88:0/1122513882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e81518d0 0x7ff9e806f9f0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.107+0000 7ff9ef200640 1 -- 172.21.15.88:0/1122513882 >> 172.21.15.88:0/1122513882 conn(0x7ff9e80b1e40 msgr2=0x7ff9e80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.108+0000 7ff9ef200640 1 -- 172.21.15.88:0/1122513882 shutdown_connections 2024-09-21T07:51:00.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.108+0000 7ff9ef200640 1 -- 172.21.15.88:0/1122513882 wait complete. 2024-09-21T07:51:00.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.110+0000 7ff9ef200640 1 Processor -- start 2024-09-21T07:51:00.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.110+0000 7ff9ef200640 1 -- start start 2024-09-21T07:51:00.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.111+0000 7ff9ef200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e80735c0 0x7ff9e813b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.111+0000 7ff9ef200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 0x7ff9e81f7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.111+0000 7ff9ef200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9e81518d0 0x7ff9e80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.111+0000 7ff9ef200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e80710d0 con 0x7ff9e80b3520 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.111+0000 7ff9ef200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9e8070f50 con 0x7ff9e80735c0 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.111+0000 7ff9ef200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9e80b3e70 con 0x7ff9e81518d0 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 0x7ff9e81f7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 0x7ff9e81f7860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36788/0 (socket says 172.21.15.88:36788) 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ede00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e80735c0 0x7ff9e813b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 -- 172.21.15.88:0/4125217933 learned_addr learned my addr 172.21.15.88:0/4125217933 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ee800640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9e81518d0 0x7ff9e80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 -- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9e81518d0 msgr2=0x7ff9e80b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9e81518d0 0x7ff9e80b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 -- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e80735c0 msgr2=0x7ff9e813b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e80735c0 0x7ff9e813b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 -- 172.21.15.88:0/4125217933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9e8155790 con 0x7ff9e80b3520 2024-09-21T07:51:00.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ee800640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9e81518d0 0x7ff9e80b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:00.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.112+0000 7ff9ed400640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 0x7ff9e81f7860 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c3d0 tx=0x7ff9d800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:00.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.113+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8017070 con 0x7ff9e80b3520 2024-09-21T07:51:00.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.113+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9d80053c0 con 0x7ff9e80b3520 2024-09-21T07:51:00.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.113+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9e8155a70 con 0x7ff9e80b3520 2024-09-21T07:51:00.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.113+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9d8003850 con 0x7ff9e80b3520 2024-09-21T07:51:00.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.113+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9e8156030 con 0x7ff9e80b3520 2024-09-21T07:51:00.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.114+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9e80b3930 con 0x7ff9e80b3520 2024-09-21T07:51:00.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.115+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ff9d801a070 con 0x7ff9e80b3520 2024-09-21T07:51:00.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.116+0000 7ff9e2000640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9cc078ce0 0x7ff9cc07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:00.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.116+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7ff9d809a2a0 con 0x7ff9e80b3520 2024-09-21T07:51:00.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.116+0000 7ff9ede00640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9cc078ce0 0x7ff9cc07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:00.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.117+0000 7ff9ede00640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9cc078ce0 0x7ff9cc07b1f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc033000 tx=0x7ff9dc004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:00.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.121+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9d8061e70 con 0x7ff9e80b3520 2024-09-21T07:51:00.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.347+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff9e80b2530 con 0x7ff9e80b3520 2024-09-21T07:51:00.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.347+0000 7ff9e2000640 1 -- 172.21.15.88:0/4125217933 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7ff9d80668a0 con 0x7ff9e80b3520 2024-09-21T07:51:00.349 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.349+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9cc078ce0 msgr2=0x7ff9cc07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.349+0000 7ff9ef200640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9cc078ce0 0x7ff9cc07b1f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc033000 tx=0x7ff9dc004730 comp rx=0 tx=0).stop 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.349+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 msgr2=0x7ff9e81f7860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.349+0000 7ff9ef200640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 0x7ff9e81f7860 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7ff9d800c3d0 tx=0x7ff9d800c890 comp rx=0 tx=0).stop 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 shutdown_connections 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9cc078ce0 0x7ff9cc07b1f0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9e81518d0 0x7ff9e80b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9e80b3520 0x7ff9e81f7860 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 --2- 172.21.15.88:0/4125217933 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9e80735c0 0x7ff9e813b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:00.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 >> 172.21.15.88:0/4125217933 conn(0x7ff9e80b1e40 msgr2=0x7ff9e815cd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:00.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 shutdown_connections 2024-09-21T07:51:00.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00.350+0000 7ff9ef200640 1 -- 172.21.15.88:0/4125217933 wait complete. 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:51:00.374 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:00.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:00.375 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.410 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42434sec preferred_lft 42434sec 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:51:00.411 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:51:00.412 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:00.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:00.413 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:51:00.502 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:51:00.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:00.506 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:51:00.541 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:51:00.541 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:51:00.541 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:51:00.541 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:51:00.541 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:51:00.542 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:51:00.542 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:51:00.542 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:51:00.542 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:51:00.542 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:51:00.542 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:51:00.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:51:00.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:00 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:00.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:00.739 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:51:00.774 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:51:00.774 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:51:00.775 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:00.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:00.775 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:51:00.871 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:00.871 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:00.906 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:00.906 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:00.906 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:00.982 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:51:00.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:00.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:01.177 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:01.178 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:51:01.178 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:51:01.198 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:01.198 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:51:01.198 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:51:01.198 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:01.198 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:01.230 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:01.230 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:51:01.230 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:51:01.231 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:01.231 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:01.231 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:51:01.231 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:51:01.231 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:51:01.232 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:51:01.232 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:51:01.232 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:51:01.232 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:51:01.233 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:51:01.235 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:51:01.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:01.235 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:01.291 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:51:01.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:51:01.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:01.386 INFO:teuthology.orchestra.run.smithi088.stdout:client6738 2024-09-21T07:51:01.386 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:01.387 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:01.421 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:51:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 -- 172.21.15.88:0/860791108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff06415d760 msgr2=0x7ff0641535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 --2- 172.21.15.88:0/860791108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff06415d760 0x7ff0641535c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7ff054005fd0 tx=0x7ff05402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 -- 172.21.15.88:0/860791108 shutdown_connections 2024-09-21T07:51:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 --2- 172.21.15.88:0/860791108 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 0x7ff0640b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 --2- 172.21.15.88:0/860791108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff06415d760 0x7ff0641535c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 -- 172.21.15.88:0/860791108 >> 172.21.15.88:0/860791108 conn(0x7ff06406d730 msgr2=0x7ff0640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 -- 172.21.15.88:0/860791108 shutdown_connections 2024-09-21T07:51:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.682+0000 7ff069c00640 1 -- 172.21.15.88:0/860791108 wait complete. 2024-09-21T07:51:01.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.684+0000 7ff069c00640 1 Processor -- start 2024-09-21T07:51:01.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.685+0000 7ff069c00640 1 -- start start 2024-09-21T07:51:01.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff069c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 0x7ff064074800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff069c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 0x7ff064148cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff069c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06415d760 0x7ff06415b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff069c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff06406f240 con 0x7ff0640ad0e0 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff069c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff0640b4e30 con 0x7ff06415d760 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff069c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff0640b3ea0 con 0x7ff0640b3550 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff062a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 0x7ff064074800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff062a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 0x7ff064074800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36814/0 (socket says 172.21.15.88:36814) 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.686+0000 7ff062a00640 1 -- 172.21.15.88:0/410815989 learned_addr learned my addr 172.21.15.88:0/410815989 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff063400640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06415d760 0x7ff06415b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062000640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 0x7ff064148cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062a00640 1 -- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 msgr2=0x7ff064148cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062a00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 0x7ff064148cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062a00640 1 -- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06415d760 msgr2=0x7ff06415b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062a00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06415d760 0x7ff06415b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062a00640 1 -- 172.21.15.88:0/410815989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff064074d40 con 0x7ff0640ad0e0 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062000640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 0x7ff064148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff062a00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 0x7ff064074800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff054006010 tx=0x7ff0540039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff05403e030 con 0x7ff0640ad0e0 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff0540047a0 con 0x7ff0640ad0e0 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff064149210 con 0x7ff0640ad0e0 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.687+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff054035040 con 0x7ff0640ad0e0 2024-09-21T07:51:01.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.688+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0641f9ad0 con 0x7ff0640ad0e0 2024-09-21T07:51:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.688+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0640b4370 con 0x7ff0640ad0e0 2024-09-21T07:51:01.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.690+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7ff054041030 con 0x7ff0640ad0e0 2024-09-21T07:51:01.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.691+0000 7ff05aa00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff044078ce0 0x7ff04407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:01.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.691+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7ff0540bb870 con 0x7ff0640ad0e0 2024-09-21T07:51:01.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.691+0000 7ff062000640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff044078ce0 0x7ff04407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:01.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.692+0000 7ff062000640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff044078ce0 0x7ff04407b1f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0641538e0 tx=0x7ff050008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:01.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.695+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff054083b00 con 0x7ff0640ad0e0 2024-09-21T07:51:01.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.920+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7ff064149560 con 0x7ff0640ad0e0 2024-09-21T07:51:01.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.920+0000 7ff05aa00640 1 -- 172.21.15.88:0/410815989 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v55) ==== 53+0+73 (secure 0 0 0) 0x7ff054083400 con 0x7ff0640ad0e0 2024-09-21T07:51:01.922 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:51:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff044078ce0 msgr2=0x7ff04407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff044078ce0 0x7ff04407b1f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ff0641538e0 tx=0x7ff050008040 comp rx=0 tx=0).stop 2024-09-21T07:51:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 msgr2=0x7ff064074800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:01.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 0x7ff064074800 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7ff054006010 tx=0x7ff0540039e0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 shutdown_connections 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff044078ce0 0x7ff04407b1f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff0640b3550 0x7ff064148cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff06415d760 0x7ff06415b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 --2- 172.21.15.88:0/410815989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0640ad0e0 0x7ff064074800 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 >> 172.21.15.88:0/410815989 conn(0x7ff06406d730 msgr2=0x7ff0640cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 shutdown_connections 2024-09-21T07:51:01.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:01.923+0000 7ff069c00640 1 -- 172.21.15.88:0/410815989 wait complete. 2024-09-21T07:51:01.946 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:51:01.946 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:51:01.946 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:51:01.947 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:51:01.947 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:01.947 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:51:01.947 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:51:01.947 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:51:01.947 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:51:01.947 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:01.947 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:51:01.988 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:51:01.988 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:51:01.988 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:51:01.988 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.988 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42383sec preferred_lft 42383sec 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:51:01.989 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:01.990 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:51:01.991 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:51:01.991 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:01.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:01.991 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:51:02.085 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:51:02.088 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:02.088 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:51:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:51:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:51:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:51:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:51:02.125 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:51:02.126 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:51:02.126 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:51:02.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:51:02.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:51:02.126 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:51:02.126 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:51:02.243 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:51:02.315 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:02.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:02.324 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:51:02.358 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:51:02.358 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:51:02.359 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:51:02.359 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:02.359 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:51:02.455 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:02.455 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:02.489 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:51:02.490 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:02.490 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:02.564 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:51:02.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:02.564 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:02.758 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:02.759 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:51:02.759 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:51:02.778 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:02.778 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:51:02.778 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:51:02.778 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:02.779 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:02.807 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:02.807 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:51:02.808 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:51:02.808 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:02.808 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:02.808 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:51:02.808 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:51:02.808 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:51:02.809 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:51:02.810 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:51:02.810 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:51:02.810 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:51:02.810 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:51:02.812 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:51:02.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:02.812 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:02.868 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:51:02.945 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:02 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:51:02.949 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:02.956 INFO:teuthology.orchestra.run.smithi175.stdout:client6469 2024-09-21T07:51:02.957 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:02.957 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:02.992 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:51:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 -- 172.21.15.88:0/3563785494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b806cda0 msgr2=0x7fb2b806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 --2- 172.21.15.88:0/3563785494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b806cda0 0x7fb2b806d1b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0005fd0 tx=0x7fb2b002aaa0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 -- 172.21.15.88:0/3563785494 shutdown_connections 2024-09-21T07:51:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 --2- 172.21.15.88:0/3563785494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b806cda0 0x7fb2b806d1b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 --2- 172.21.15.88:0/3563785494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2b80b4620 0x7fb2b8162ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 -- 172.21.15.88:0/3563785494 >> 172.21.15.88:0/3563785494 conn(0x7fb2b80b2540 msgr2=0x7fb2b80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 -- 172.21.15.88:0/3563785494 shutdown_connections 2024-09-21T07:51:03.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.216+0000 7fb2bee00640 1 -- 172.21.15.88:0/3563785494 wait complete. 2024-09-21T07:51:03.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.218+0000 7fb2bee00640 1 Processor -- start 2024-09-21T07:51:03.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.218+0000 7fb2bee00640 1 -- start start 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2b806cda0 0x7fb2b81f6230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2b80b4620 0x7fb2b815c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 0x7fb2b81572a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80af490 con 0x7fb2b815fd30 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2b80af310 con 0x7fb2b80b4620 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2b80af610 con 0x7fb2b806cda0 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 0x7fb2b81572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 0x7fb2b81572a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36828/0 (socket says 172.21.15.88:36828) 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2bce00640 1 -- 172.21.15.88:0/774364973 learned_addr learned my addr 172.21.15.88:0/774364973 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2b7e00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2b806cda0 0x7fb2b81f6230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.220+0000 7fb2b7400640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2b80b4620 0x7fb2b815c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bce00640 1 -- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2b806cda0 msgr2=0x7fb2b81f6230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bce00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2b806cda0 0x7fb2b81f6230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bce00640 1 -- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2b80b4620 msgr2=0x7fb2b815c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bce00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2b80b4620 0x7fb2b815c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bce00640 1 -- 172.21.15.88:0/774364973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2b80afcf0 con 0x7fb2b815fd30 2024-09-21T07:51:03.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bce00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 0x7fb2b81572a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009d90 tx=0x7fb2b0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0044070 con 0x7fb2b815fd30 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2b0004250 con 0x7fb2b815fd30 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2b0004680 con 0x7fb2b815fd30 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2b80affd0 con 0x7fb2b815fd30 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.221+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2b80736b0 con 0x7fb2b815fd30 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.222+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2b8073910 con 0x7fb2b815fd30 2024-09-21T07:51:03.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.224+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb2b0047070 con 0x7fb2b815fd30 2024-09-21T07:51:03.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.225+0000 7fb2afe00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb298078cb0 0x7fb29807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.225+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7fb2b00bc060 con 0x7fb2b815fd30 2024-09-21T07:51:03.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.225+0000 7fb2b7e00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb298078cb0 0x7fb29807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.226+0000 7fb2b7e00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb298078cb0 0x7fb29807b1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8001720 tx=0x7fb2a8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:03.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.229+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2b806d6f0 con 0x7fb2b815fd30 2024-09-21T07:51:03.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.451+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fb2b806d1b0 con 0x7fb2b815fd30 2024-09-21T07:51:03.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.504+0000 7fb2afe00640 1 -- 172.21.15.88:0/774364973 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v56) ==== 136+0+0 (secure 0 0 0) 0x7fb2b0032030 con 0x7fb2b815fd30 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb298078cb0 msgr2=0x7fb29807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb298078cb0 0x7fb29807b1c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb2a8001720 tx=0x7fb2a8009040 comp rx=0 tx=0).stop 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 msgr2=0x7fb2b81572a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 0x7fb2b81572a0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7fb2b0009d90 tx=0x7fb2b0003f10 comp rx=0 tx=0).stop 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 shutdown_connections 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb298078cb0 0x7fb29807b1c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2b815fd30 0x7fb2b81572a0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2b80b4620 0x7fb2b815c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 --2- 172.21.15.88:0/774364973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2b806cda0 0x7fb2b81f6230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.506+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 >> 172.21.15.88:0/774364973 conn(0x7fb2b80b2540 msgr2=0x7fb2b806fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.507+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 shutdown_connections 2024-09-21T07:51:03.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.507+0000 7fb2bee00640 1 -- 172.21.15.88:0/774364973 wait complete. 2024-09-21T07:51:03.535 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:51:03.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 -- 172.21.15.88:0/770854442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 msgr2=0x7fe5c815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 --2- 172.21.15.88:0/770854442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c815d240 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7fe5c8071220 tx=0x7fe5c002a980 comp rx=0 tx=0).stop 2024-09-21T07:51:03.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 -- 172.21.15.88:0/770854442 shutdown_connections 2024-09-21T07:51:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 --2- 172.21.15.88:0/770854442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c815d240 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 --2- 172.21.15.88:0/770854442 >> v2:172.21.15.175:3300/0 conn(0x7fe5c80705b0 0x7fe5c80709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 --2- 172.21.15.88:0/770854442 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c80cfc80 0x7fe5c8156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 -- 172.21.15.88:0/770854442 >> 172.21.15.88:0/770854442 conn(0x7fe5c806d3d0 msgr2=0x7fe5c80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 -- 172.21.15.88:0/770854442 shutdown_connections 2024-09-21T07:51:03.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.753+0000 7fe5cd200640 1 -- 172.21.15.88:0/770854442 wait complete. 2024-09-21T07:51:03.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.755+0000 7fe5cd200640 1 Processor -- start 2024-09-21T07:51:03.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.755+0000 7fe5cd200640 1 -- start start 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5cd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c80705b0 0x7fe5c8074f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5cd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c80cfc80 0x7fe5c814d260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5cd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c80c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5cd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80cce40 con 0x7fe5c815ce30 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5cd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5c80cccc0 con 0x7fe5c80cfc80 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5cd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5c80ccfc0 con 0x7fe5c80705b0 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5c6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c80c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5c6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c80c7e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36832/0 (socket says 172.21.15.88:36832) 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5c6800640 1 -- 172.21.15.88:0/2603683607 learned_addr learned my addr 172.21.15.88:0/2603683607 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5c5e00640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c80705b0 0x7fe5c8074f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.757+0000 7fe5bfe00640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c80cfc80 0x7fe5c814d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5c6800640 1 -- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c80705b0 msgr2=0x7fe5c8074f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5c6800640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c80705b0 0x7fe5c8074f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5c6800640 1 -- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c80cfc80 msgr2=0x7fe5c814d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5c6800640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c80cfc80 0x7fe5c814d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5c6800640 1 -- 172.21.15.88:0/2603683607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5c8074180 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5c6800640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c80c7e80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009d60 tx=0x7fe5c0003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0034040 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5c0004540 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5c0045480 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5c8074460 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.758+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5c8074a20 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.760+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5c80cd550 con 0x7fe5c815ce30 2024-09-21T07:51:03.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.761+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe5c0037020 con 0x7fe5c815ce30 2024-09-21T07:51:03.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.761+0000 7fe5be000640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe5a4078cb0 0x7fe5a407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:03.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.762+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7fe5c00bc930 con 0x7fe5c815ce30 2024-09-21T07:51:03.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.762+0000 7fe5c5e00640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe5a4078cb0 0x7fe5a407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:03.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.762+0000 7fe5c5e00640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe5a4078cb0 0x7fe5a407b1c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80058c0 tx=0x7fe5b8005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:03.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.766+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe5c8070f00 con 0x7fe5c815ce30 2024-09-21T07:51:03.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.976+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fe5c80afa70 con 0x7fe5a4078cb0 2024-09-21T07:51:03.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.978+0000 7fe5be000640 1 -- 172.21.15.88:0/2603683607 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fe5c80afa70 con 0x7fe5a4078cb0 2024-09-21T07:51:03.980 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:51:03.980 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:03.980 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:51:03.980 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:03.980 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:51:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.980+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe5a4078cb0 msgr2=0x7fe5a407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.980+0000 7fe5cd200640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe5a4078cb0 0x7fe5a407b1c0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fe5b80058c0 tx=0x7fe5b8005850 comp rx=0 tx=0).stop 2024-09-21T07:51:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 msgr2=0x7fe5c80c7e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c80c7e80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0009d60 tx=0x7fe5c0003f10 comp rx=0 tx=0).stop 2024-09-21T07:51:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 shutdown_connections 2024-09-21T07:51:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe5a4078cb0 0x7fe5a407b1c0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5c815ce30 0x7fe5c80c7e80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe5c80cfc80 0x7fe5c814d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 --2- 172.21.15.88:0/2603683607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5c80705b0 0x7fe5c8074f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 >> 172.21.15.88:0/2603683607 conn(0x7fe5c806d3d0 msgr2=0x7fe5c815e990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 shutdown_connections 2024-09-21T07:51:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:03.981+0000 7fe5cd200640 1 -- 172.21.15.88:0/2603683607 wait complete. 2024-09-21T07:51:04.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:51:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 -- 172.21.15.88:0/96532598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effe4005250 msgr2=0x7effe400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 --2- 172.21.15.88:0/96532598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effe4005250 0x7effe400b9c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efff800c610 tx=0x7effe4005830 comp rx=0 tx=0).stop 2024-09-21T07:51:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 -- 172.21.15.88:0/96532598 shutdown_connections 2024-09-21T07:51:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 --2- 172.21.15.88:0/96532598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effe4005250 0x7effe400b9c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 -- 172.21.15.88:0/96532598 >> 172.21.15.88:0/96532598 conn(0x7f000006d620 msgr2=0x7f00000b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 -- 172.21.15.88:0/96532598 shutdown_connections 2024-09-21T07:51:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.230+0000 7f0004a00640 1 -- 172.21.15.88:0/96532598 wait complete. 2024-09-21T07:51:04.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.232+0000 7f0004a00640 1 Processor -- start 2024-09-21T07:51:04.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.232+0000 7f0004a00640 1 -- start start 2024-09-21T07:51:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7f0004a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe4005250 0x7f00001e5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7f0004a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 0x7f00001e6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7f0004a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0000149020 0x7f00001e7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7f0004a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0000157030 con 0x7f0000148c10 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7f0004a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0000156eb0 con 0x7effe4005250 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7f0004a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00000ccac0 con 0x7f0000149020 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7efff6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 0x7f00001e6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7efff6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 0x7f00001e6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36858/0 (socket says 172.21.15.88:36858) 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7efff6a00640 1 -- 172.21.15.88:0/29346913 learned_addr learned my addr 172.21.15.88:0/29346913 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7effff400640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe4005250 0x7f00001e5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.234+0000 7effffe00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0000149020 0x7f00001e7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff6a00640 1 -- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0000149020 msgr2=0x7f00001e7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff6a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0000149020 0x7f00001e7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff6a00640 1 -- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe4005250 msgr2=0x7f00001e5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff6a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe4005250 0x7f00001e5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff6a00640 1 -- 172.21.15.88:0/29346913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00001562d0 con 0x7f0000148c10 2024-09-21T07:51:04.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff6a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 0x7f00001e6ee0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7effe8003ea0 tx=0x7effe8001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:04.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8015070 con 0x7f0000148c10 2024-09-21T07:51:04.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effe8011840 con 0x7f0000148c10 2024-09-21T07:51:04.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effe8011c50 con 0x7f0000148c10 2024-09-21T07:51:04.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00001e64c0 con 0x7f0000148c10 2024-09-21T07:51:04.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.235+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00001e8430 con 0x7f0000148c10 2024-09-21T07:51:04.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.236+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00000b0250 con 0x7f0000148c10 2024-09-21T07:51:04.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.237+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7effe8018070 con 0x7f0000148c10 2024-09-21T07:51:04.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.238+0000 7efff7e00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7effdc078cb0 0x7effdc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.238+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7effe809bc20 con 0x7f0000148c10 2024-09-21T07:51:04.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.239+0000 7effff400640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7effdc078cb0 0x7effdc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.239+0000 7effff400640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7effdc078cb0 0x7effdc07b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efff0009bd0 tx=0x7efff0005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:04.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.243+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effe8063750 con 0x7f0000148c10 2024-09-21T07:51:04.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.428+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f00001e6250 con 0x7f0000148c10 2024-09-21T07:51:04.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.428+0000 7efff7e00640 1 -- 172.21.15.88:0/29346913 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) ==== 122+0+0 (secure 0 0 0) 0x7effe8068180 con 0x7f0000148c10 2024-09-21T07:51:04.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7effdc078cb0 msgr2=0x7effdc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7effdc078cb0 0x7effdc07b1c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7efff0009bd0 tx=0x7efff0005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:04.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 msgr2=0x7f00001e6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 0x7f00001e6ee0 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7effe8003ea0 tx=0x7effe8001630 comp rx=0 tx=0).stop 2024-09-21T07:51:04.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 shutdown_connections 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7effdc078cb0 0x7effdc07b1c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0000149020 0x7f00001e7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0000148c10 0x7f00001e6ee0 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 --2- 172.21.15.88:0/29346913 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effe4005250 0x7f00001e5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 >> 172.21.15.88:0/29346913 conn(0x7f000006d620 msgr2=0x7f00000ae020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 shutdown_connections 2024-09-21T07:51:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.431+0000 7f0004a00640 1 -- 172.21.15.88:0/29346913 wait complete. 2024-09-21T07:51:04.459 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create 'abcd@#' 2024-09-21T07:51:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 -- 172.21.15.88:0/4278510572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145550 msgr2=0x7f89b406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 --2- 172.21.15.88:0/4278510572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145550 0x7f89b406fb40 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f89a4005fd0 tx=0x7f89a402a670 comp rx=0 tx=0).stop 2024-09-21T07:51:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 -- 172.21.15.88:0/4278510572 shutdown_connections 2024-09-21T07:51:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 --2- 172.21.15.88:0/4278510572 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89b4073c50 0x7f89b40cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 --2- 172.21.15.88:0/4278510572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145550 0x7f89b406fb40 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 -- 172.21.15.88:0/4278510572 >> 172.21.15.88:0/4278510572 conn(0x7f89b406d1c0 msgr2=0x7f89b406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:04.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 -- 172.21.15.88:0/4278510572 shutdown_connections 2024-09-21T07:51:04.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.681+0000 7f89bb000640 1 -- 172.21.15.88:0/4278510572 wait complete. 2024-09-21T07:51:04.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.684+0000 7f89bb000640 1 Processor -- start 2024-09-21T07:51:04.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.684+0000 7f89bb000640 1 -- start start 2024-09-21T07:51:04.688 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.685+0000 7f89bb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89b4145550 0x7f89b40b2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.685+0000 7f89bb000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145a40 0x7f89b40b0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.685+0000 7f89bb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89b415f8e0 0x7f89b4143720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.685+0000 7f89bb000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40ce150 con 0x7f89b4145a40 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.685+0000 7f89bb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f89b40cdfd0 con 0x7f89b4145550 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.685+0000 7f89bb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f89b40ce2d0 con 0x7f89b415f8e0 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145a40 0x7f89b40b0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145a40 0x7f89b40b0d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36864/0 (socket says 172.21.15.88:36864) 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 -- 172.21.15.88:0/3051707480 learned_addr learned my addr 172.21.15.88:0/3051707480 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b9000640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89b415f8e0 0x7f89b4143720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3e00640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89b4145550 0x7f89b40b2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 -- 172.21.15.88:0/3051707480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89b415f8e0 msgr2=0x7f89b4143720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89b415f8e0 0x7f89b4143720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 -- 172.21.15.88:0/3051707480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89b4145550 msgr2=0x7f89b40b2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f89b4145550 0x7f89b40b2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.686+0000 7f89b3400640 1 -- 172.21.15.88:0/3051707480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89b41f8a30 con 0x7f89b4145a40 2024-09-21T07:51:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.687+0000 7f89b3400640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89b4145a40 0x7f89b40b0d10 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f89a000c390 tx=0x7f89a000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.687+0000 7f89abe00640 1 -- 172.21.15.88:0/3051707480 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0017070 con 0x7f89b4145a40 2024-09-21T07:51:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.687+0000 7f89abe00640 1 -- 172.21.15.88:0/3051707480 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f89a00035f0 con 0x7f89b4145a40 2024-09-21T07:51:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.687+0000 7f89bb000640 1 -- 172.21.15.88:0/3051707480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89b41f8d10 con 0x7f89b4145a40 2024-09-21T07:51:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.687+0000 7f89abe00640 1 -- 172.21.15.88:0/3051707480 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89a0003a20 con 0x7f89b4145a40 2024-09-21T07:51:04.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.687+0000 7f89bb000640 1 -- 172.21.15.88:0/3051707480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89b41f92d0 con 0x7f89b4145a40 2024-09-21T07:51:04.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.689+0000 7f89bb000640 1 -- 172.21.15.88:0/3051707480 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f89b415c850 con 0x7f89b4145a40 2024-09-21T07:51:04.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.710+0000 7f89abe00640 1 -- 172.21.15.88:0/3051707480 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f89a001a070 con 0x7f89b4145a40 2024-09-21T07:51:04.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.711+0000 7f89abe00640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8994078cb0 0x7f899407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:04.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.711+0000 7f89abe00640 1 -- 172.21.15.88:0/3051707480 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f89a009a160 con 0x7f89b4145a40 2024-09-21T07:51:04.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.711+0000 7f89b3e00640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8994078cb0 0x7f899407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:04.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.712+0000 7f89b3e00640 1 --2- 172.21.15.88:0/3051707480 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8994078cb0 0x7f899407b1c0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f89a4033000 tx=0x7f89a4004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:04.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:04.716+0000 7f89abe00640 1 -- 172.21.15.88:0/3051707480 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89b415c850 con 0x7f89b4145a40 2024-09-21T07:51:04.926 INFO:teuthology.orchestra.run.smithi088.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-21T07:51:04.926 INFO:teuthology.orchestra.run.smithi088.stderr:fs volume create [] : Create a CephFS volume 2024-09-21T07:51:04.926 INFO:teuthology.orchestra.run.smithi088.stderr:Error EINVAL: invalid command 2024-09-21T07:51:04.942 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T07:51:04.943 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume create testvol 2024-09-21T07:51:05.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 -- 172.21.15.88:0/2569224961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600b3f30 msgr2=0x7f7c60156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:05.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 --2- 172.21.15.88:0/2569224961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600b3f30 0x7f7c60156ca0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009860 tx=0x7f7c5002a530 comp rx=0 tx=0).stop 2024-09-21T07:51:05.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 -- 172.21.15.88:0/2569224961 shutdown_connections 2024-09-21T07:51:05.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 --2- 172.21.15.88:0/2569224961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c600b3f30 0x7f7c60156ca0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:05.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 -- 172.21.15.88:0/2569224961 >> 172.21.15.88:0/2569224961 conn(0x7f7c6006d620 msgr2=0x7f7c600b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:05.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 -- 172.21.15.88:0/2569224961 shutdown_connections 2024-09-21T07:51:05.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.163+0000 7f7c66000640 1 -- 172.21.15.88:0/2569224961 wait complete. 2024-09-21T07:51:05.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.166+0000 7f7c66000640 1 Processor -- start 2024-09-21T07:51:05.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.166+0000 7f7c66000640 1 -- start start 2024-09-21T07:51:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c66000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c600b3f30 0x7f7c60153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c66000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 0x7f7c60155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c66000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c6015d6e0 0x7f7c601453a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c66000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c600af7e0 con 0x7f7c600cd1d0 2024-09-21T07:51:05.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c66000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7c60070370 con 0x7f7c600b3f30 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c66000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7c600af960 con 0x7f7c6015d6e0 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 0x7f7c60155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c57e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 0x7f7c60155180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36880/0 (socket says 172.21.15.88:36880) 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.167+0000 7f7c57e00640 1 -- 172.21.15.88:0/2440117242 learned_addr learned my addr 172.21.15.88:0/2440117242 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c64c00640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c600b3f30 0x7f7c60153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c65600640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c6015d6e0 0x7f7c601453a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c57e00640 1 -- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c6015d6e0 msgr2=0x7f7c601453a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c57e00640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c6015d6e0 0x7f7c601453a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c57e00640 1 -- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c600b3f30 msgr2=0x7f7c60153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c57e00640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c600b3f30 0x7f7c60153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c57e00640 1 -- 172.21.15.88:0/2440117242 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c60156170 con 0x7f7c600cd1d0 2024-09-21T07:51:05.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.168+0000 7f7c57e00640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 0x7f7c60155180 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800c4d0 tx=0x7f7c4800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.169+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48016020 con 0x7f7c600cd1d0 2024-09-21T07:51:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.169+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7c48012920 con 0x7f7c600cd1d0 2024-09-21T07:51:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.169+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c60155bb0 con 0x7f7c600cd1d0 2024-09-21T07:51:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.169+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48012d70 con 0x7f7c600cd1d0 2024-09-21T07:51:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.169+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c60143060 con 0x7f7c600cd1d0 2024-09-21T07:51:05.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.170+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c60156ca0 con 0x7f7c600cd1d0 2024-09-21T07:51:05.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.171+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7c48019020 con 0x7f7c600cd1d0 2024-09-21T07:51:05.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.172+0000 7f7c5d600640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7c3c078ce0 0x7f7c3c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:05.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.172+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 5709+0+0 (secure 0 0 0) 0x7f7c4809ae40 con 0x7f7c600cd1d0 2024-09-21T07:51:05.174 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.172+0000 7f7c64c00640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7c3c078ce0 0x7f7c3c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:05.175 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.173+0000 7f7c64c00640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7c3c078ce0 0x7f7c3c07b1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034000 tx=0x7f7c50004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:05.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.177+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7c48020020 con 0x7f7c600cd1d0 2024-09-21T07:51:05.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:05.384+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume create", "name": "testvol", "target": ["mon-mgr", ""]}) -- 0x7f7c600b03a0 con 0x7f7c3c078ce0 2024-09-21T07:51:07.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.538+0000 7f7c5d600640 1 -- 172.21.15.88:0/2440117242 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 Volume created successfully (no MDS daemons created)) ==== 60+0+0 (secure 0 0 0) 0x7f7c600b03a0 con 0x7f7c3c078ce0 2024-09-21T07:51:07.540 INFO:teuthology.orchestra.run.smithi088.stderr:Volume created successfully (no MDS daemons created) 2024-09-21T07:51:07.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7c3c078ce0 msgr2=0x7f7c3c07b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7c3c078ce0 0x7f7c3c07b1f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7c50034000 tx=0x7f7c50004730 comp rx=0 tx=0).stop 2024-09-21T07:51:07.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 msgr2=0x7f7c60155180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 0x7f7c60155180 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800c4d0 tx=0x7f7c4800c990 comp rx=0 tx=0).stop 2024-09-21T07:51:07.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 shutdown_connections 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7c3c078ce0 0x7f7c3c07b1f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7c6015d6e0 0x7f7c601453a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c600cd1d0 0x7f7c60155180 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 --2- 172.21.15.88:0/2440117242 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7c600b3f30 0x7f7c60153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 >> 172.21.15.88:0/2440117242 conn(0x7f7c6006d620 msgr2=0x7f7c60068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 shutdown_connections 2024-09-21T07:51:07.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.541+0000 7f7c66000640 1 -- 172.21.15.88:0/2440117242 wait complete. 2024-09-21T07:51:07.573 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create testvol 'abcd@#' 2024-09-21T07:51:07.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 -- 172.21.15.88:0/1742838025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 msgr2=0x7f8de80cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 --2- 172.21.15.88:0/1742838025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 0x7f8de80cda20 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f8de0005fd0 tx=0x7f8de002a8a0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 -- 172.21.15.88:0/1742838025 shutdown_connections 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 --2- 172.21.15.88:0/1742838025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 0x7f8de80cda20 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 --2- 172.21.15.88:0/1742838025 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de80735e0 0x7f8de80739f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 -- 172.21.15.88:0/1742838025 >> 172.21.15.88:0/1742838025 conn(0x7f8de806d1c0 msgr2=0x7f8de806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 -- 172.21.15.88:0/1742838025 shutdown_connections 2024-09-21T07:51:07.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.798+0000 7f8decc00640 1 -- 172.21.15.88:0/1742838025 wait complete. 2024-09-21T07:51:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.800+0000 7f8decc00640 1 Processor -- start 2024-09-21T07:51:07.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.800+0000 7f8decc00640 1 -- start start 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8decc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80735e0 0x7f8de814eaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8decc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 0x7f8de8151380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8decc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de8145450 0x7f8de81518c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8decc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80ce2f0 con 0x7f8de80cd610 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8decc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8de80ce170 con 0x7f8de80735e0 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8decc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8de80ce470 con 0x7f8de8145450 2024-09-21T07:51:07.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 0x7f8de8151380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 0x7f8de8151380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36482/0 (socket says 172.21.15.88:36482) 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 -- 172.21.15.88:0/2415152709 learned_addr learned my addr 172.21.15.88:0/2415152709 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8de6800640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de8145450 0x7f8de81518c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8de5e00640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80735e0 0x7f8de814eaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 -- 172.21.15.88:0/2415152709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de8145450 msgr2=0x7f8de81518c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8de8145450 0x7f8de81518c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 -- 172.21.15.88:0/2415152709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80735e0 msgr2=0x7f8de814eaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8de80735e0 0x7f8de814eaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.802+0000 7f8ddd400640 1 -- 172.21.15.88:0/2415152709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8de814efe0 con 0x7f8de80cd610 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.803+0000 7f8ddd400640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8de80cd610 0x7f8de8151380 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f8dd000c970 tx=0x7f8dd000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.803+0000 7f8ddea00640 1 -- 172.21.15.88:0/2415152709 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd0007b50 con 0x7f8de80cd610 2024-09-21T07:51:07.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.803+0000 7f8ddea00640 1 -- 172.21.15.88:0/2415152709 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8dd0007de0 con 0x7f8de80cd610 2024-09-21T07:51:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.803+0000 7f8ddea00640 1 -- 172.21.15.88:0/2415152709 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dd000f6b0 con 0x7f8de80cd610 2024-09-21T07:51:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.803+0000 7f8decc00640 1 -- 172.21.15.88:0/2415152709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8de81f9670 con 0x7f8de80cd610 2024-09-21T07:51:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.803+0000 7f8decc00640 1 -- 172.21.15.88:0/2415152709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8de81f9c30 con 0x7f8de80cd610 2024-09-21T07:51:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.804+0000 7f8decc00640 1 -- 172.21.15.88:0/2415152709 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8de80739f0 con 0x7f8de80cd610 2024-09-21T07:51:07.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.806+0000 7f8ddea00640 1 -- 172.21.15.88:0/2415152709 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f8dd000f850 con 0x7f8de80cd610 2024-09-21T07:51:07.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.806+0000 7f8ddea00640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8dc4078cb0 0x7f8dc407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:07.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.807+0000 7f8ddea00640 1 -- 172.21.15.88:0/2415152709 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6543+0+0 (secure 0 0 0) 0x7f8dd009b970 con 0x7f8de80cd610 2024-09-21T07:51:07.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.807+0000 7f8de5e00640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8dc4078cb0 0x7f8dc407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:07.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.807+0000 7f8de5e00640 1 --2- 172.21.15.88:0/2415152709 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8dc4078cb0 0x7f8dc407b1c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8dd8000f60 tx=0x7f8dd8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:07.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:07.812+0000 7f8ddea00640 1 -- 172.21.15.88:0/2415152709 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8dd0014030 con 0x7f8de80cd610 2024-09-21T07:51:08.023 INFO:teuthology.orchestra.run.smithi088.stderr:Invalid command: invalid chars @# in abcd@# 2024-09-21T07:51:08.023 INFO:teuthology.orchestra.run.smithi088.stderr:fs subvolumegroup create [] [] [] [] [] : Create a CephFS subvolume group in a volume, and optionally, with a specific data pool layout, and a specific numeric mode 2024-09-21T07:51:08.023 INFO:teuthology.orchestra.run.smithi088.stderr:Error EINVAL: invalid command 2024-09-21T07:51:08.039 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T07:51:08.040 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create testvol 'abcd@#' 2024-09-21T07:51:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 -- 172.21.15.88:0/3048842050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc0cd1d0 msgr2=0x7f3abc0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 --2- 172.21.15.88:0/3048842050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc0cd1d0 0x7f3abc0af0d0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f3ab4005fd0 tx=0x7f3ab402a830 comp rx=0 tx=0).stop 2024-09-21T07:51:08.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 -- 172.21.15.88:0/3048842050 shutdown_connections 2024-09-21T07:51:08.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 --2- 172.21.15.88:0/3048842050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc0cd1d0 0x7f3abc0af0d0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 --2- 172.21.15.88:0/3048842050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3abc0b3f30 0x7f3abc156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 -- 172.21.15.88:0/3048842050 >> 172.21.15.88:0/3048842050 conn(0x7f3abc06d620 msgr2=0x7f3abc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 -- 172.21.15.88:0/3048842050 shutdown_connections 2024-09-21T07:51:08.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.264+0000 7f3abbe00640 1 -- 172.21.15.88:0/3048842050 wait complete. 2024-09-21T07:51:08.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.266+0000 7f3abbe00640 1 Processor -- start 2024-09-21T07:51:08.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.266+0000 7f3abbe00640 1 -- start start 2024-09-21T07:51:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3abc0b3f30 0x7f3abc15b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3abc0cd1d0 0x7f3abc158ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc15d6e0 0x7f3abc15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc0af9a0 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3abc0af820 con 0x7f3abc15d6e0 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3abc0afb20 con 0x7f3abc0b3f30 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3aba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3abc0cd1d0 0x7f3abc158ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3aba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3abc0cd1d0 0x7f3abc158ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36484/0 (socket says 172.21.15.88:36484) 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3aba000640 1 -- 172.21.15.88:0/2004764238 learned_addr learned my addr 172.21.15.88:0/2004764238 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abaa00640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3abc0b3f30 0x7f3abc15b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.268+0000 7f3abb400640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc15d6e0 0x7f3abc15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3aba000640 1 -- 172.21.15.88:0/2004764238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3abc0b3f30 msgr2=0x7f3abc15b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3aba000640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3abc0b3f30 0x7f3abc15b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3aba000640 1 -- 172.21.15.88:0/2004764238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc15d6e0 msgr2=0x7f3abc15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3aba000640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3abc15d6e0 0x7f3abc15bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3aba000640 1 -- 172.21.15.88:0/2004764238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3abc15a4e0 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3aba000640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3abc0cd1d0 0x7f3abc158ad0 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800c530 tx=0x7f3aa800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3ab3400640 1 -- 172.21.15.88:0/2004764238 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8012830 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3ab3400640 1 -- 172.21.15.88:0/2004764238 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3aa8017070 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3ab3400640 1 -- 172.21.15.88:0/2004764238 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8012d60 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3abbe00640 1 -- 172.21.15.88:0/2004764238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3abc15a7c0 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.269+0000 7f3abbe00640 1 -- 172.21.15.88:0/2004764238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3abc15ad80 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.270+0000 7f3abbe00640 1 -- 172.21.15.88:0/2004764238 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3abc156ca0 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.272+0000 7f3ab3400640 1 -- 172.21.15.88:0/2004764238 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f3aa801b410 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.273+0000 7f3ab3400640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3a9c078ce0 0x7f3a9c07b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.273+0000 7f3ab3400640 1 -- 172.21.15.88:0/2004764238 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 6543+0+0 (secure 0 0 0) 0x7f3aa809a930 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.273+0000 7f3abaa00640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3a9c078ce0 0x7f3a9c07b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.274+0000 7f3abaa00640 1 --2- 172.21.15.88:0/2004764238 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3a9c078ce0 0x7f3a9c07b1f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3aac0095f0 tx=0x7f3aac005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.277+0000 7f3ab3400640 1 -- 172.21.15.88:0/2004764238 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3aa8020020 con 0x7f3abc0cd1d0 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi088.stderr:Invalid command: invalid chars #@ in abcd@# 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi088.stderr:fs subvolume create [] [] [] [] [] [] [--namespace-isolated] [] : Create a CephFS subvolume in a volume, and optionally, with a specific size (in bytes), a specific data pool layout, a specific mode, in a specific subvolume group and in separate RADOS namespace 2024-09-21T07:51:08.489 INFO:teuthology.orchestra.run.smithi088.stderr:Error EINVAL: invalid command 2024-09-21T07:51:08.505 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-09-21T07:51:08.506 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume rm testvol --yes-i-really-mean-it 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.739+0000 7f1eed200640 1 -- 172.21.15.88:0/513609862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 msgr2=0x7f1ee80ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.739+0000 7f1eed200640 1 --2- 172.21.15.88:0/513609862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 0x7f1ee80ccac0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009860 tx=0x7f1ed002a530 comp rx=0 tx=0).stop 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.740+0000 7f1eed200640 1 -- 172.21.15.88:0/513609862 shutdown_connections 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.740+0000 7f1eed200640 1 --2- 172.21.15.88:0/513609862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 0x7f1ee80ccac0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.740+0000 7f1eed200640 1 -- 172.21.15.88:0/513609862 >> 172.21.15.88:0/513609862 conn(0x7f1ee806d620 msgr2=0x7f1ee80b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.740+0000 7f1eed200640 1 -- 172.21.15.88:0/513609862 shutdown_connections 2024-09-21T07:51:08.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.740+0000 7f1eed200640 1 -- 172.21.15.88:0/513609862 wait complete. 2024-09-21T07:51:08.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.742+0000 7f1eed200640 1 Processor -- start 2024-09-21T07:51:08.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.742+0000 7f1eed200640 1 -- start start 2024-09-21T07:51:08.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1eed200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee80b3f30 0x7f1ee81f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1eed200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 0x7f1ee80ae590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1eed200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 0x7f1ee80aead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1eed200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee80af7e0 con 0x7f1ee80cd000 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1eed200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1ee80703b0 con 0x7f1ee80b3f30 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1eed200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1ee80af960 con 0x7f1ee8156ca0 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1edea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 0x7f1ee80ae590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1edea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 0x7f1ee80ae590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36492/0 (socket says 172.21.15.88:36492) 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.743+0000 7f1edea00640 1 -- 172.21.15.88:0/3180487584 learned_addr learned my addr 172.21.15.88:0/3180487584 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1ee7400640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee80b3f30 0x7f1ee81f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1ee7e00640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 0x7f1ee80aead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edea00640 1 -- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 msgr2=0x7f1ee80aead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edea00640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 0x7f1ee80aead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edea00640 1 -- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee80b3f30 msgr2=0x7f1ee81f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edea00640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee80b3f30 0x7f1ee81f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edea00640 1 -- 172.21.15.88:0/3180487584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ee815c720 con 0x7f1ee80cd000 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edea00640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 0x7f1ee80ae590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009980 tx=0x7f1ed0003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed0005c90 con 0x7f1ee80cd000 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1ed0005310 con 0x7f1ee80cd000 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ee81f5fa0 con 0x7f1ee80cd000 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.744+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ed00047a0 con 0x7f1ee80cd000 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.745+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ee80b0ff0 con 0x7f1ee80cd000 2024-09-21T07:51:08.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.746+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1ee8145df0 con 0x7f1ee80cd000 2024-09-21T07:51:08.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.747+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1ed0004940 con 0x7f1ee80cd000 2024-09-21T07:51:08.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.748+0000 7f1edfe00640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ec4078cb0 0x7f1ec407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:08.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.748+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 6543+0+0 (secure 0 0 0) 0x7f1ed00bc630 con 0x7f1ee80cd000 2024-09-21T07:51:08.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.748+0000 7f1ee7400640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ec4078cb0 0x7f1ec407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:08.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.749+0000 7f1ee7400640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ec4078cb0 0x7f1ec407b1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001240 tx=0x7f1ed8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:08.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.752+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1ed003f030 con 0x7f1ee80cd000 2024-09-21T07:51:08.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:08.966+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume rm", "vol_name": "testvol", "yes-i-really-mean-it": "--yes-i-really-mean-it", "target": ["mon-mgr", ""]}) -- 0x7f1ee80ccaa0 con 0x7f1ec4078cb0 2024-09-21T07:51:08.972 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:51:08.970+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:51:09.100 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:51:11.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.991+0000 7f1edfe00640 1 -- 172.21.15.88:0/3180487584 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+78 (secure 0 0 0) 0x7f1ee80ccaa0 con 0x7f1ec4078cb0 2024-09-21T07:51:11.993 INFO:teuthology.orchestra.run.smithi088.stdout:metadata pool: cephfs.testvol.meta data pool: ['cephfs.testvol.data'] removed 2024-09-21T07:51:11.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.995+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ec4078cb0 msgr2=0x7f1ec407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.995+0000 7f1eed200640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ec4078cb0 0x7f1ec407b1c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1ed8001240 tx=0x7f1ed8009040 comp rx=0 tx=0).stop 2024-09-21T07:51:11.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 msgr2=0x7f1ee80ae590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:11.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 0x7f1ee80ae590 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f1ed0009980 tx=0x7f1ed0003b90 comp rx=0 tx=0).stop 2024-09-21T07:51:11.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 shutdown_connections 2024-09-21T07:51:11.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ec4078cb0 0x7f1ec407b1c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1ee8156ca0 0x7f1ee80aead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ee80cd000 0x7f1ee80ae590 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 --2- 172.21.15.88:0/3180487584 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1ee80b3f30 0x7f1ee81f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 >> 172.21.15.88:0/3180487584 conn(0x7f1ee806d620 msgr2=0x7f1ee80b0440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 shutdown_connections 2024-09-21T07:51:11.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:11.996+0000 7f1eed200640 1 -- 172.21.15.88:0/3180487584 wait complete. 2024-09-21T07:51:12.025 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:51:12.025 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:51:12.104 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:51:12.198 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:51:12.198 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:51:12.198 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:12.198 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:12.234 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:12.234 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:12.310 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:51:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 -- 172.21.15.88:0/1683845095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc0af170 msgr2=0x7f76bc0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 --2- 172.21.15.88:0/1683845095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc0af170 0x7f76bc0b2f60 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f76a4009830 tx=0x7f76a402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 -- 172.21.15.88:0/1683845095 shutdown_connections 2024-09-21T07:51:12.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 --2- 172.21.15.88:0/1683845095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc0af170 0x7f76bc0b2f60 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 -- 172.21.15.88:0/1683845095 >> 172.21.15.88:0/1683845095 conn(0x7f76bc0b1e40 msgr2=0x7f76bc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:12.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 -- 172.21.15.88:0/1683845095 shutdown_connections 2024-09-21T07:51:12.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.572+0000 7f76bbe00640 1 -- 172.21.15.88:0/1683845095 wait complete. 2024-09-21T07:51:12.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.574+0000 7f76bbe00640 1 Processor -- start 2024-09-21T07:51:12.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.574+0000 7f76bbe00640 1 -- start start 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc0ad9c0 0x7f76bc14be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76bc0af170 0x7f76bc1f2df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 0x7f76bc157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc06fb20 con 0x7f76bc15a0f0 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76bc06f9a0 con 0x7f76bc0ad9c0 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76bc15d670 con 0x7f76bc0af170 2024-09-21T07:51:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 0x7f76bc157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 0x7f76bc157fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36504/0 (socket says 172.21.15.88:36504) 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76bb400640 1 -- 172.21.15.88:0/355428434 learned_addr learned my addr 172.21.15.88:0/355428434 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76baa00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc0ad9c0 0x7f76bc14be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.576+0000 7f76b2000640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76bc0af170 0x7f76bc1f2df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bb400640 1 -- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76bc0af170 msgr2=0x7f76bc1f2df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bb400640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76bc0af170 0x7f76bc1f2df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bb400640 1 -- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc0ad9c0 msgr2=0x7f76bc14be90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bb400640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc0ad9c0 0x7f76bc14be90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bb400640 1 -- 172.21.15.88:0/355428434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76bc1f6bb0 con 0x7f76bc15a0f0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76b2000640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76bc0af170 0x7f76bc1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bb400640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 0x7f76bc157fd0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f76b400cb10 tx=0x7f76b400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4016020 con 0x7f76bc15a0f0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f76b400ad60 con 0x7f76bc15a0f0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76bc1f6e20 con 0x7f76bc15a0f0 2024-09-21T07:51:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4012680 con 0x7f76bc15a0f0 2024-09-21T07:51:12.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.577+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76bc1f3330 con 0x7f76bc15a0f0 2024-09-21T07:51:12.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.578+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76bc0b2f60 con 0x7f76bc15a0f0 2024-09-21T07:51:12.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.580+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f76b4019020 con 0x7f76bc15a0f0 2024-09-21T07:51:12.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.581+0000 7f76b3e00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7698078cb0 0x7f769807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:12.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.581+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f76b409b100 con 0x7f76bc15a0f0 2024-09-21T07:51:12.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.581+0000 7f76baa00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7698078cb0 0x7f769807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:12.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.582+0000 7f76baa00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7698078cb0 0x7f769807b1c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f76ac000c00 tx=0x7f76ac009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:12.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.586+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76b4020030 con 0x7f76bc15a0f0 2024-09-21T07:51:12.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.782+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f76bc0c5360 con 0x7f76bc15a0f0 2024-09-21T07:51:12.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.783+0000 7f76b3e00640 1 -- 172.21.15.88:0/355428434 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) ==== 80+0+124 (secure 0 0 0) 0x7f76b40633e0 con 0x7f76bc15a0f0 2024-09-21T07:51:12.785 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.175:6835/3755060210 2024-09-22T07:51:08.971204+0000 2024-09-21T07:51:12.785 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.175:6834/3755060210 2024-09-22T07:51:08.971204+0000 2024-09-21T07:51:12.785 INFO:teuthology.orchestra.run.smithi088.stderr:listed 2 entries 2024-09-21T07:51:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7698078cb0 msgr2=0x7f769807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7698078cb0 0x7f769807b1c0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f76ac000c00 tx=0x7f76ac009040 comp rx=0 tx=0).stop 2024-09-21T07:51:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 msgr2=0x7f76bc157fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 0x7f76bc157fd0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f76b400cb10 tx=0x7f76b400a060 comp rx=0 tx=0).stop 2024-09-21T07:51:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 shutdown_connections 2024-09-21T07:51:12.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7698078cb0 0x7f769807b1c0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76bc0af170 0x7f76bc1f2df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76bc15a0f0 0x7f76bc157fd0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.785+0000 7f76bbe00640 1 --2- 172.21.15.88:0/355428434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76bc0ad9c0 0x7f76bc14be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.786+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 >> 172.21.15.88:0/355428434 conn(0x7f76bc0b1e40 msgr2=0x7f76bc070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.786+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 shutdown_connections 2024-09-21T07:51:12.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:12.786+0000 7f76bbe00640 1 -- 172.21.15.88:0/355428434 wait complete. 2024-09-21T07:51:12.811 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:51:12.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:12.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:12.877 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:51:12.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:12.877 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:51:12.913 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:51:12.913 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:12.913 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:12.953 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:12.953 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:13.032 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:51:13.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd9400640 1 -- 172.21.15.88:0/3217990971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 msgr2=0x7fdfb8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd9400640 1 --2- 172.21.15.88:0/3217990971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfb8005150 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7fdfcc009ed0 tx=0x7fdfb8005830 comp rx=0 tx=0).stop 2024-09-21T07:51:13.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd8a00640 1 -- 172.21.15.88:0/3217990971 reap_dead start 2024-09-21T07:51:13.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd9400640 1 -- 172.21.15.88:0/3217990971 shutdown_connections 2024-09-21T07:51:13.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd9400640 1 -- 172.21.15.88:0/3217990971 >> 172.21.15.88:0/3217990971 conn(0x7fdfd40b1e40 msgr2=0x7fdfd40b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:13.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd9400640 1 -- 172.21.15.88:0/3217990971 shutdown_connections 2024-09-21T07:51:13.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.252+0000 7fdfd9400640 1 -- 172.21.15.88:0/3217990971 wait complete. 2024-09-21T07:51:13.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.254+0000 7fdfd9400640 1 Processor -- start 2024-09-21T07:51:13.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.254+0000 7fdfd9400640 1 -- start start 2024-09-21T07:51:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfd4154ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd9400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40ce8b0 0x7fdfd40743c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd9400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4155b50 0x7fdfd4074900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd9400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415d7f0 con 0x7fdfb8004d40 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd9400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdfd415d670 con 0x7fdfd40ce8b0 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd9400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdfd415d970 con 0x7fdfd4155b50 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfd4154ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfd4154ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36530/0 (socket says 172.21.15.88:36530) 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd3e00640 1 -- 172.21.15.88:0/373502325 learned_addr learned my addr 172.21.15.88:0/373502325 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd8a00640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4155b50 0x7fdfd4074900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.256+0000 7fdfd3400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40ce8b0 0x7fdfd40743c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3e00640 1 -- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4155b50 msgr2=0x7fdfd4074900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3e00640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4155b50 0x7fdfd4074900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3e00640 1 -- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40ce8b0 msgr2=0x7fdfd40743c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3e00640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40ce8b0 0x7fdfd40743c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3e00640 1 -- 172.21.15.88:0/373502325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfd40cee60 con 0x7fdfb8004d40 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40ce8b0 0x7fdfd40743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd3e00640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfd4154ee0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006660 tx=0x7fdfc4003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400f760 con 0x7fdfb8004d40 2024-09-21T07:51:13.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdfc4016020 con 0x7fdfb8004d40 2024-09-21T07:51:13.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfd41f9060 con 0x7fdfb8004d40 2024-09-21T07:51:13.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfc400fbc0 con 0x7fdfb8004d40 2024-09-21T07:51:13.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.257+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfd4155600 con 0x7fdfb8004d40 2024-09-21T07:51:13.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.258+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfd4155980 con 0x7fdfb8004d40 2024-09-21T07:51:13.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.260+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fdfc4011430 con 0x7fdfb8004d40 2024-09-21T07:51:13.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.260+0000 7fdfcbe00640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdfb4078cb0 0x7fdfb407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.261+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fdfc409bbb0 con 0x7fdfb8004d40 2024-09-21T07:51:13.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.261+0000 7fdfd3400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdfb4078cb0 0x7fdfb407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.261+0000 7fdfd3400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdfb4078cb0 0x7fdfb407b1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40b1550 tx=0x7fdfc0005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:13.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.265+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdfc4020020 con 0x7fdfb8004d40 2024-09-21T07:51:13.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.461+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdfd41f8e70 con 0x7fdfb8004d40 2024-09-21T07:51:13.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.462+0000 7fdfcbe00640 1 -- 172.21.15.88:0/373502325 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 2 entries v93) ==== 80+0+124 (secure 0 0 0) 0x7fdfc4063e10 con 0x7fdfb8004d40 2024-09-21T07:51:13.464 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.175:6835/3755060210 2024-09-22T07:51:08.971204+0000 2024-09-21T07:51:13.464 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.15.175:6834/3755060210 2024-09-22T07:51:08.971204+0000 2024-09-21T07:51:13.464 INFO:teuthology.orchestra.run.smithi088.stderr:listed 2 entries 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdfb4078cb0 msgr2=0x7fdfb407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdfb4078cb0 0x7fdfb407b1c0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fdfd40b1550 tx=0x7fdfc0005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 msgr2=0x7fdfd4154ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfd4154ee0 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fdfc4006660 tx=0x7fdfc4003a80 comp rx=0 tx=0).stop 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 shutdown_connections 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdfb4078cb0 0x7fdfb407b1c0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdfd4155b50 0x7fdfd4074900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdfd40ce8b0 0x7fdfd40743c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 --2- 172.21.15.88:0/373502325 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdfb8004d40 0x7fdfd4154ee0 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 >> 172.21.15.88:0/373502325 conn(0x7fdfd40b1e40 msgr2=0x7fdfd4068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.464+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 shutdown_connections 2024-09-21T07:51:13.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.465+0000 7fdfd9400640 1 -- 172.21.15.88:0/373502325 wait complete. 2024-09-21T07:51:13.488 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:51:13.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:13.488 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:13.556 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:51:13.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:13.556 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:51:13.594 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:51:13.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:13.594 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:13.629 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:51:13.629 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:51:13.629 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:13.629 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:13.670 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:51:13.670 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 -- 172.21.15.88:0/2985481588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 msgr2=0x7f1b800d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 --2- 172.21.15.88:0/2985481588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800d0ef0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f1b800ba580 tx=0x7f1b6c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 -- 172.21.15.88:0/2985481588 shutdown_connections 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 --2- 172.21.15.88:0/2985481588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800d0ef0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 --2- 172.21.15.88:0/2985481588 >> v2:172.21.15.175:3300/0 conn(0x7f1b800b9b80 0x7f1b800cfd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 -- 172.21.15.88:0/2985481588 >> 172.21.15.88:0/2985481588 conn(0x7f1b8009d820 msgr2=0x7f1b8008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:13.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 -- 172.21.15.88:0/2985481588 shutdown_connections 2024-09-21T07:51:13.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.889+0000 7f1b86e00640 1 -- 172.21.15.88:0/2985481588 wait complete. 2024-09-21T07:51:13.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.891+0000 7f1b86e00640 1 Processor -- start 2024-09-21T07:51:13.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.891+0000 7f1b86e00640 1 -- start start 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b800b9610 0x7f1b800b2670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 0x7f1b800ca890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800b4360 con 0x7f1b801620e0 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b800b41e0 con 0x7f1b800b9610 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b800b44e0 con 0x7f1b800b9b80 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b85000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 0x7f1b800ca890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b86400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b85a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b800b9610 0x7f1b800b2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b85000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 0x7f1b800ca890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:41028/0 (socket says 172.21.15.88:41028) 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.893+0000 7f1b85000640 1 -- 172.21.15.88:0/1352066314 learned_addr learned my addr 172.21.15.88:0/1352066314 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85a00640 1 -- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 msgr2=0x7f1b800ca890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85a00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 0x7f1b800ca890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85a00640 1 -- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 msgr2=0x7f1b800cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85a00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85a00640 1 -- 172.21.15.88:0/1352066314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b800b2fc0 con 0x7f1b800b9610 2024-09-21T07:51:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85000640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 0x7f1b800ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b86400640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:51:13.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b85a00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b800b9610 0x7f1b800b2670 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e700 tx=0x7f1b7400ebc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7400cab0 con 0x7f1b800b9610 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b7400cd40 con 0x7f1b800b9610 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b801f8ec0 con 0x7f1b800b9610 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74010690 con 0x7f1b800b9610 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.894+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b800cadd0 con 0x7f1b800b9610 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.895+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b800d1660 con 0x7f1b800b9610 2024-09-21T07:51:13.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.897+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1b74010830 con 0x7f1b800b9610 2024-09-21T07:51:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.898+0000 7f1b7d600640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b60078cb0 0x7f1b6007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.898+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f1b7409bca0 con 0x7f1b800b9610 2024-09-21T07:51:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.898+0000 7f1b85000640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b60078cb0 0x7f1b6007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:13.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.898+0000 7f1b85000640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b60078cb0 0x7f1b6007b1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1b800b57d0 tx=0x7f1b6c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:13.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:13.902+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b740637f0 con 0x7f1b800b9610 2024-09-21T07:51:14.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.125+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1b8015f0d0 con 0x7f1b800b9610 2024-09-21T07:51:14.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.126+0000 7f1b7d600640 1 -- 172.21.15.88:0/1352066314 <== mon.1 v2:172.21.15.175:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7f1b74068220 con 0x7f1b800b9610 2024-09-21T07:51:14.128 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:14.128 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":61,"btime":"2024-09-21T07:51:09:974978+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3665288036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3665288036},{"type":"v1","addr":"172.21.15.175:6835","nonce":3665288036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:52.693177+0000","modified":"2024-09-21T07:50:55.744683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1717882335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1717882335},{"type":"v1","addr":"172.21.15.88:6835","nonce":1717882335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6600,"qdb_cluster":[6600]},"id":6}]} 2024-09-21T07:51:14.129 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 61 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b60078cb0 msgr2=0x7f1b6007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b60078cb0 0x7f1b6007b1c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f1b800b57d0 tx=0x7f1b6c004730 comp rx=0 tx=0).stop 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b800b9610 msgr2=0x7f1b800b2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b800b9610 0x7f1b800b2670 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400e700 tx=0x7f1b7400ebc0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 shutdown_connections 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b60078cb0 0x7f1b6007b1c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b800b9b80 0x7f1b800ca890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b801620e0 0x7f1b800cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 --2- 172.21.15.88:0/1352066314 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b800b9610 0x7f1b800b2670 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 >> 172.21.15.88:0/1352066314 conn(0x7f1b8009d820 msgr2=0x7f1b801612f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 shutdown_connections 2024-09-21T07:51:14.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.128+0000 7f1b86e00640 1 -- 172.21.15.88:0/1352066314 wait complete. 2024-09-21T07:51:14.152 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 -- 172.21.15.88:0/2021282564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd415f910 msgr2=0x7f2bd40b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 --2- 172.21.15.88:0/2021282564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd415f910 0x7f2bd40b3750 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc005fd0 tx=0x7f2bcc02aa10 comp rx=0 tx=0).stop 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 -- 172.21.15.88:0/2021282564 shutdown_connections 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 --2- 172.21.15.88:0/2021282564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd415f910 0x7f2bd40b3750 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 --2- 172.21.15.88:0/2021282564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd4152920 0x7f2bd415f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 -- 172.21.15.88:0/2021282564 >> 172.21.15.88:0/2021282564 conn(0x7f2bd406d730 msgr2=0x7f2bd40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 -- 172.21.15.88:0/2021282564 shutdown_connections 2024-09-21T07:51:14.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.369+0000 7f2bdba00640 1 -- 172.21.15.88:0/2021282564 wait complete. 2024-09-21T07:51:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.372+0000 7f2bdba00640 1 Processor -- start 2024-09-21T07:51:14.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.372+0000 7f2bdba00640 1 -- start start 2024-09-21T07:51:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bdba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd40aebc0 0x7f2bd40d0290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bdba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 0x7f2bd40d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bdba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd415f910 0x7f2bd41f9240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bdba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd40b41e0 con 0x7f2bd4152920 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bdba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2bd40b4060 con 0x7f2bd40aebc0 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bdba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2bd40b4360 con 0x7f2bd415f910 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bcbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 0x7f2bd40d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bcbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 0x7f2bd40d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36572/0 (socket says 172.21.15.88:36572) 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.373+0000 7f2bcbe00640 1 -- 172.21.15.88:0/3188564259 learned_addr learned my addr 172.21.15.88:0/3188564259 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bd9a00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd415f910 0x7f2bd41f9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bd9000640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd40aebc0 0x7f2bd40d0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bcbe00640 1 -- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd415f910 msgr2=0x7f2bd41f9240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bcbe00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd415f910 0x7f2bd41f9240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bcbe00640 1 -- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd40aebc0 msgr2=0x7f2bd40d0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bcbe00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd40aebc0 0x7f2bd40d0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bcbe00640 1 -- 172.21.15.88:0/3188564259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bd41f66d0 con 0x7f2bd4152920 2024-09-21T07:51:14.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bcbe00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 0x7f2bd40d07d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00e7c0 tx=0x7f2bbc00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc00ccc0 con 0x7f2bd4152920 2024-09-21T07:51:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bbc010430 con 0x7f2bd4152920 2024-09-21T07:51:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc010860 con 0x7f2bd4152920 2024-09-21T07:51:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.374+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bd41510f0 con 0x7f2bd4152920 2024-09-21T07:51:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.375+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bd41f5bf0 con 0x7f2bd4152920 2024-09-21T07:51:14.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.376+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bd40b48f0 con 0x7f2bd4152920 2024-09-21T07:51:14.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.377+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2bbc010a00 con 0x7f2bd4152920 2024-09-21T07:51:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.378+0000 7f2bd1600640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bb0078cb0 0x7f2bb007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.378+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f2bbc09ba60 con 0x7f2bd4152920 2024-09-21T07:51:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.378+0000 7f2bd9000640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bb0078cb0 0x7f2bb007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.378+0000 7f2bd9000640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bb0078cb0 0x7f2bb007b1c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4001720 tx=0x7f2bc4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:14.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.382+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2bbc0635d0 con 0x7f2bd4152920 2024-09-21T07:51:14.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.603+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2bd40cf250 con 0x7f2bd4152920 2024-09-21T07:51:14.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.604+0000 7f2bd1600640 1 -- 172.21.15.88:0/3188564259 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7f2bbc068000 con 0x7f2bd4152920 2024-09-21T07:51:14.606 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:14.606 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":61,"btime":"2024-09-21T07:51:09:974978+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3665288036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3665288036},{"type":"v1","addr":"172.21.15.175:6835","nonce":3665288036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:52.693177+0000","modified":"2024-09-21T07:50:55.744683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1717882335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1717882335},{"type":"v1","addr":"172.21.15.88:6835","nonce":1717882335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6600,"qdb_cluster":[6600]},"id":6}]} 2024-09-21T07:51:14.606 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 61 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bb0078cb0 msgr2=0x7f2bb007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bb0078cb0 0x7f2bb007b1c0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2bc4001720 tx=0x7f2bc4009040 comp rx=0 tx=0).stop 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 msgr2=0x7f2bd40d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 0x7f2bd40d07d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc00e7c0 tx=0x7f2bbc00ec80 comp rx=0 tx=0).stop 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 shutdown_connections 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bb0078cb0 0x7f2bb007b1c0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2bd415f910 0x7f2bd41f9240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bd4152920 0x7f2bd40d07d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 --2- 172.21.15.88:0/3188564259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2bd40aebc0 0x7f2bd40d0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 >> 172.21.15.88:0/3188564259 conn(0x7f2bd406d730 msgr2=0x7f2bd415f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 shutdown_connections 2024-09-21T07:51:14.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.606+0000 7f2bdba00640 1 -- 172.21.15.88:0/3188564259 wait complete. 2024-09-21T07:51:14.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 -- 172.21.15.88:0/1807325347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86500af050 msgr2=0x7f86500ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 --2- 172.21.15.88:0/1807325347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86500af050 0x7f86500ae080 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f8640005fd0 tx=0x7f864002a670 comp rx=0 tx=0).stop 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 -- 172.21.15.88:0/1807325347 shutdown_connections 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 --2- 172.21.15.88:0/1807325347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f86500af050 0x7f86500ae080 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 -- 172.21.15.88:0/1807325347 >> 172.21.15.88:0/1807325347 conn(0x7f86500b1e40 msgr2=0x7f86500b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 -- 172.21.15.88:0/1807325347 shutdown_connections 2024-09-21T07:51:14.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.852+0000 7f864fe00640 1 -- 172.21.15.88:0/1807325347 wait complete. 2024-09-21T07:51:14.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.854+0000 7f864fe00640 1 Processor -- start 2024-09-21T07:51:14.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.854+0000 7f864fe00640 1 -- start start 2024-09-21T07:51:14.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.855+0000 7f864fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 0x7f8650146690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.855+0000 7f864fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8650072e60 0x7f8650165650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.855+0000 7f864fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86500af050 0x7f8650151dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.855+0000 7f864fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8650166c60 con 0x7f8650070710 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8650166ae0 con 0x7f86500af050 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8650166de0 con 0x7f8650072e60 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 0x7f8650146690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 0x7f8650146690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36600/0 (socket says 172.21.15.88:36600) 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 -- 172.21.15.88:0/3430719585 learned_addr learned my addr 172.21.15.88:0/3430719585 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864f400640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86500af050 0x7f8650151dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864e000640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8650072e60 0x7f8650165650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 -- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8650072e60 msgr2=0x7f8650165650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8650072e60 0x7f8650165650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 -- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86500af050 msgr2=0x7f8650151dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:14.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86500af050 0x7f8650151dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:14.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 -- 172.21.15.88:0/3430719585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f865009b090 con 0x7f8650070710 2024-09-21T07:51:14.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864f400640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86500af050 0x7f8650151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:14.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.856+0000 7f864ea00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 0x7f8650146690 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8640034040 tx=0x7f86400039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:14.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.857+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f864003e030 con 0x7f8650070710 2024-09-21T07:51:14.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.857+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86400047a0 con 0x7f8650070710 2024-09-21T07:51:14.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.857+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8640035040 con 0x7f8650070710 2024-09-21T07:51:14.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.857+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f865009b370 con 0x7f8650070710 2024-09-21T07:51:14.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.857+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f865009b930 con 0x7f8650070710 2024-09-21T07:51:14.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.858+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86500ae080 con 0x7f8650070710 2024-09-21T07:51:14.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.860+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f8640041030 con 0x7f8650070710 2024-09-21T07:51:14.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.860+0000 7f8646a00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8630078cb0 0x7f863007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:14.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.861+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f86400bb900 con 0x7f8650070710 2024-09-21T07:51:14.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.861+0000 7f864e000640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8630078cb0 0x7f863007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:14.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.861+0000 7f864e000640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8630078cb0 0x7f863007b1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f865009bdb0 tx=0x7f863c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:14.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:14.865+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8640083ac0 con 0x7f8650070710 2024-09-21T07:51:15.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.054+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f865009bb70 con 0x7f8650070710 2024-09-21T07:51:15.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.055+0000 7f8646a00640 1 -- 172.21.15.88:0/3430719585 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16875 (secure 0 0 0) 0x7f86400833c0 con 0x7f8650070710 2024-09-21T07:51:15.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:15.058 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":93,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:51:11.980092+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:50.589543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:51.652329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/3755060210":"2024-09-22T07:51:08.971204+0000","172.21.15.175:6834/3755060210":"2024-09-22T07:51:08.971204+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8630078cb0 msgr2=0x7f863007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8630078cb0 0x7f863007b1c0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f865009bdb0 tx=0x7f863c009040 comp rx=0 tx=0).stop 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 msgr2=0x7f8650146690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 0x7f8650146690 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f8640034040 tx=0x7f86400039e0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 shutdown_connections 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8630078cb0 0x7f863007b1c0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8650072e60 0x7f8650165650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f86500af050 0x7f8650151dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 --2- 172.21.15.88:0/3430719585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8650070710 0x7f8650146690 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.057+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 >> 172.21.15.88:0/3430719585 conn(0x7f86500b1e40 msgr2=0x7f865014bc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.058+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 shutdown_connections 2024-09-21T07:51:15.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.058+0000 7f864fe00640 1 -- 172.21.15.88:0/3430719585 wait complete. 2024-09-21T07:51:15.083 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:51:15.083 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.300+0000 7f3ba6200640 1 -- 172.21.15.88:0/582590239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 msgr2=0x7f3b84003e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.300+0000 7f3ba6200640 1 --2- 172.21.15.88:0/582590239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3b84003e90 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f3b90009b50 tx=0x7f3b84005830 comp rx=0 tx=0).stop 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.300+0000 7f3ba5800640 1 -- 172.21.15.88:0/582590239 reap_dead start 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.301+0000 7f3ba6200640 1 -- 172.21.15.88:0/582590239 shutdown_connections 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.301+0000 7f3ba6200640 1 -- 172.21.15.88:0/582590239 >> 172.21.15.88:0/582590239 conn(0x7f3ba008bfe0 msgr2=0x7f3ba009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.301+0000 7f3ba6200640 1 -- 172.21.15.88:0/582590239 shutdown_connections 2024-09-21T07:51:15.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.301+0000 7f3ba6200640 1 -- 172.21.15.88:0/582590239 wait complete. 2024-09-21T07:51:15.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.303+0000 7f3ba6200640 1 Processor -- start 2024-09-21T07:51:15.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.303+0000 7f3ba6200640 1 -- start start 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.304+0000 7f3ba6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3ba015cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.304+0000 7f3ba6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba015fb20 0x7f3ba0161110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.304+0000 7f3ba6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba015ff30 0x7f3ba0161650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.304+0000 7f3ba6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00d1b10 con 0x7f3b84003a00 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.304+0000 7f3ba6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3ba00d1990 con 0x7f3ba015fb20 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.304+0000 7f3ba6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3ba00d1c90 con 0x7f3ba015ff30 2024-09-21T07:51:15.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3ba015cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3ba015cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:36608/0 (socket says 172.21.15.88:36608) 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 -- 172.21.15.88:0/3680023498 learned_addr learned my addr 172.21.15.88:0/3680023498 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3b97e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba015fb20 0x7f3ba0161110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba5800640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba015ff30 0x7f3ba0161650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 -- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba015ff30 msgr2=0x7f3ba0161650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba015ff30 0x7f3ba0161650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 -- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba015fb20 msgr2=0x7f3ba0161110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba015fb20 0x7f3ba0161110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 -- 172.21.15.88:0/3680023498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ba015e640 con 0x7f3b84003a00 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3b97e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba015fb20 0x7f3ba0161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba5800640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba015ff30 0x7f3ba0161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:15.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.305+0000 7f3ba4e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3ba015cd40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3b9002b4f0 tx=0x7f3b90033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.306+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90038040 con 0x7f3b84003a00 2024-09-21T07:51:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.306+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3b90004020 con 0x7f3b84003a00 2024-09-21T07:51:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.306+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3b90004450 con 0x7f3b84003a00 2024-09-21T07:51:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.306+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ba01f8ae0 con 0x7f3b84003a00 2024-09-21T07:51:15.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.306+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ba00a1c50 con 0x7f3b84003a00 2024-09-21T07:51:15.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.307+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ba0162a50 con 0x7f3b84003a00 2024-09-21T07:51:15.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.308+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f3b90045020 con 0x7f3b84003a00 2024-09-21T07:51:15.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.309+0000 7f3b9d600640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3b7c078cb0 0x7f3b7c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.309+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f3b900bbae0 con 0x7f3b84003a00 2024-09-21T07:51:15.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.309+0000 7f3b97e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3b7c078cb0 0x7f3b7c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.310+0000 7f3b97e00640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3b7c078cb0 0x7f3b7c07b1c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b88001860 tx=0x7f3b88001d10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:15.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.313+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3b90083ca0 con 0x7f3b84003a00 2024-09-21T07:51:15.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.532+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ba00b4620 con 0x7f3b84003a00 2024-09-21T07:51:15.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.532+0000 7f3b9d600640 1 -- 172.21.15.88:0/3680023498 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7f3b900835a0 con 0x7f3b84003a00 2024-09-21T07:51:15.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:15.535 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":61,"btime":"2024-09-21T07:51:09:974978+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3665288036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3665288036},{"type":"v1","addr":"172.21.15.175:6835","nonce":3665288036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:52.693177+0000","modified":"2024-09-21T07:50:55.744683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1717882335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1717882335},{"type":"v1","addr":"172.21.15.88:6835","nonce":1717882335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6600,"qdb_cluster":[6600]},"id":6}]} 2024-09-21T07:51:15.535 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 61 2024-09-21T07:51:15.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3b7c078cb0 msgr2=0x7f3b7c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3b7c078cb0 0x7f3b7c07b1c0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3b88001860 tx=0x7f3b88001d10 comp rx=0 tx=0).stop 2024-09-21T07:51:15.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 msgr2=0x7f3ba015cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3ba015cd40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f3b9002b4f0 tx=0x7f3b90033670 comp rx=0 tx=0).stop 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 shutdown_connections 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3b7c078cb0 0x7f3b7c07b1c0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3ba015ff30 0x7f3ba0161650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3ba015fb20 0x7f3ba0161110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 --2- 172.21.15.88:0/3680023498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3b84003a00 0x7f3ba015cd40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 >> 172.21.15.88:0/3680023498 conn(0x7f3ba008bfe0 msgr2=0x7f3ba00b5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 shutdown_connections 2024-09-21T07:51:15.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.535+0000 7f3ba6200640 1 -- 172.21.15.88:0/3680023498 wait complete. 2024-09-21T07:51:15.562 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 -- 172.21.15.88:0/417220379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c0b3550 msgr2=0x7f2f9c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 --2- 172.21.15.88:0/417220379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c0b3550 0x7f2f9c0b3960 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c075250 tx=0x7f2f9402aa30 comp rx=0 tx=0).stop 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 -- 172.21.15.88:0/417220379 shutdown_connections 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 --2- 172.21.15.88:0/417220379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c0b3550 0x7f2f9c0b3960 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 --2- 172.21.15.88:0/417220379 >> v2:172.21.15.175:3300/0 conn(0x7f2f9c15d760 0x7f2f9c1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 -- 172.21.15.88:0/417220379 >> 172.21.15.88:0/417220379 conn(0x7f2f9c06d730 msgr2=0x7f2f9c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 -- 172.21.15.88:0/417220379 shutdown_connections 2024-09-21T07:51:15.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.779+0000 7f2fa1200640 1 -- 172.21.15.88:0/417220379 wait complete. 2024-09-21T07:51:15.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.782+0000 7f2fa1200640 1 Processor -- start 2024-09-21T07:51:15.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.782+0000 7f2fa1200640 1 -- start start 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.783+0000 7f2fa1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f9c0ad0e0 0x7f2f9c1f6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.783+0000 7f2fa1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f9c0b3550 0x7f2f9c1599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.783+0000 7f2fa1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 0x7f2f9c0b0da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.783+0000 7f2fa1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c0b4da0 con 0x7f2f9c15d760 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.783+0000 7f2fa1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f9c0b4c20 con 0x7f2f9c0b3550 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.783+0000 7f2fa1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f9c0b3ea0 con 0x7f2f9c0ad0e0 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 0x7f2f9c0b0da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 0x7f2f9c0b0da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38680/0 (socket says 172.21.15.88:38680) 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 -- 172.21.15.88:0/215137011 learned_addr learned my addr 172.21.15.88:0/215137011 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f99e00640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f9c0ad0e0 0x7f2f9c1f6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f99400640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f9c0b3550 0x7f2f9c1599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 -- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f9c0ad0e0 msgr2=0x7f2f9c1f6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f9c0ad0e0 0x7f2f9c1f6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 -- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f9c0b3550 msgr2=0x7f2f9c1599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f9c0b3550 0x7f2f9c1599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:15.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 -- 172.21.15.88:0/215137011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f9c153e40 con 0x7f2f9c15d760 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.784+0000 7f2f9a800640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 0x7f2f9c0b0da0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2f9402b020 tx=0x7f2f94003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.785+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94033050 con 0x7f2f9c15d760 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.785+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f94004540 con 0x7f2f9c15d760 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.785+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94038450 con 0x7f2f9c15d760 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.785+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f9c154120 con 0x7f2f9c15d760 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.785+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f9c1546e0 con 0x7f2f9c15d760 2024-09-21T07:51:15.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.786+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f9c0b4360 con 0x7f2f9c15d760 2024-09-21T07:51:15.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.787+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2f94036070 con 0x7f2f9c15d760 2024-09-21T07:51:15.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.788+0000 7f2f92000640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f7c078cb0 0x7f2f7c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:15.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.788+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f2f940bc2e0 con 0x7f2f9c15d760 2024-09-21T07:51:15.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.788+0000 7f2f99e00640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f7c078cb0 0x7f2f7c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:15.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.789+0000 7f2f99e00640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f7c078cb0 0x7f2f7c07b1c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009b30 tx=0x7f2f8c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:15.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:15.793+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f94083e50 con 0x7f2f9c15d760 2024-09-21T07:51:16.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.018+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2f9c06e060 con 0x7f2f9c15d760 2024-09-21T07:51:16.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.019+0000 7f2f92000640 1 -- 172.21.15.88:0/215137011 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 61 v61) ==== 94+0+5228 (secure 0 0 0) 0x7f2f94088880 con 0x7f2f9c15d760 2024-09-21T07:51:16.021 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:16.021 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":61,"btime":"2024-09-21T07:51:09:974978+0000","default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3665288036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3665288036},{"type":"v1","addr":"172.21.15.175:6835","nonce":3665288036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61}],"filesystems":[{"mdsmap":{"epoch":55,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:50:52.693177+0000","modified":"2024-09-21T07:50:55.744683+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6600},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6600":{"gid":6600,"name":"a","rank":0,"incarnation":52,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/1717882335","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1717882335},{"type":"v1","addr":"172.21.15.88:6835","nonce":1717882335}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6600,"qdb_cluster":[6600]},"id":6}]} 2024-09-21T07:51:16.022 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 61 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f7c078cb0 msgr2=0x7f2f7c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f7c078cb0 0x7f2f7c07b1c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f2f8c009b30 tx=0x7f2f8c005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 msgr2=0x7f2f9c0b0da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 0x7f2f9c0b0da0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f2f9402b020 tx=0x7f2f94003f10 comp rx=0 tx=0).stop 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 shutdown_connections 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f7c078cb0 0x7f2f7c07b1c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f9c15d760 0x7f2f9c0b0da0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f9c0b3550 0x7f2f9c1599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 --2- 172.21.15.88:0/215137011 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f9c0ad0e0 0x7f2f9c1f6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.021+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 >> 172.21.15.88:0/215137011 conn(0x7f2f9c06d730 msgr2=0x7f2f9c15dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:16.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.022+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 shutdown_connections 2024-09-21T07:51:16.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.022+0000 7f2fa1200640 1 -- 172.21.15.88:0/215137011 wait complete. 2024-09-21T07:51:16.046 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 -- 172.21.15.88:0/1437779286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3550 msgr2=0x7fd2680b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 --2- 172.21.15.88:0/1437779286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3550 0x7fd2680b3960 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7fd268075250 tx=0x7fd26002aa30 comp rx=0 tx=0).stop 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 -- 172.21.15.88:0/1437779286 shutdown_connections 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 --2- 172.21.15.88:0/1437779286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3550 0x7fd2680b3960 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 --2- 172.21.15.88:0/1437779286 >> v2:172.21.15.175:3300/0 conn(0x7fd26815d760 0x7fd2681535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 -- 172.21.15.88:0/1437779286 >> 172.21.15.88:0/1437779286 conn(0x7fd26806d730 msgr2=0x7fd2680b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.266+0000 7fd270400640 1 -- 172.21.15.88:0/1437779286 shutdown_connections 2024-09-21T07:51:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.267+0000 7fd270400640 1 -- 172.21.15.88:0/1437779286 wait complete. 2024-09-21T07:51:16.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.269+0000 7fd270400640 1 Processor -- start 2024-09-21T07:51:16.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.269+0000 7fd270400640 1 -- start start 2024-09-21T07:51:16.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd270400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 0x7fd268154e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd270400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680b3550 0x7fd26806ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd270400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815d760 0x7fd2681581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd270400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680b4da0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd270400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2680b4c20 con 0x7fd2680b3550 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd270400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2680b3ea0 con 0x7fd26815d760 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd26da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 0x7fd268154e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd26da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 0x7fd268154e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38700/0 (socket says 172.21.15.88:38700) 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.270+0000 7fd26da00640 1 -- 172.21.15.88:0/4053954151 learned_addr learned my addr 172.21.15.88:0/4053954151 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26d000640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680b3550 0x7fd26806ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26e400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815d760 0x7fd2681581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26da00640 1 -- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815d760 msgr2=0x7fd2681581c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26da00640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815d760 0x7fd2681581c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26da00640 1 -- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680b3550 msgr2=0x7fd26806ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26da00640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680b3550 0x7fd26806ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26da00640 1 -- 172.21.15.88:0/4053954151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd268157cc0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26d000640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680b3550 0x7fd26806ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd26da00640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 0x7fd268154e90 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d740 tx=0x7fd25c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c017070 con 0x7fd2680ad0e0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.271+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd25c009d20 con 0x7fd2680ad0e0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.272+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2681f6420 con 0x7fd2680ad0e0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.272+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c0106c0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.272+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2681f69e0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.273+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2680b4360 con 0x7fd2680ad0e0 2024-09-21T07:51:16.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.274+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fd25c01a070 con 0x7fd2680ad0e0 2024-09-21T07:51:16.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.275+0000 7fd265600640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd248078cb0 0x7fd24807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.275+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7fd25c09b1f0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.275+0000 7fd26d000640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd248078cb0 0x7fd24807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.276+0000 7fd26d000640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd248078cb0 0x7fd24807b1c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd254007fd0 tx=0x7fd254009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.279+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd25c0633b0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.465+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd26815afe0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.467+0000 7fd265600640 1 -- 172.21.15.88:0/4053954151 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+16875 (secure 0 0 0) 0x7fd25c062cb0 con 0x7fd2680ad0e0 2024-09-21T07:51:16.469 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:16.469 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":93,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:51:11.980092+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:50:50.589543+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2024-09-21T07:50:51.652329+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/3755060210":"2024-09-22T07:51:08.971204+0000","172.21.15.175:6834/3755060210":"2024-09-22T07:51:08.971204+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:51:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd248078cb0 msgr2=0x7fd24807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd248078cb0 0x7fd24807b1c0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd254007fd0 tx=0x7fd254009040 comp rx=0 tx=0).stop 2024-09-21T07:51:16.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 msgr2=0x7fd268154e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 0x7fd268154e90 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00d740 tx=0x7fd25c00dc00 comp rx=0 tx=0).stop 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 shutdown_connections 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd248078cb0 0x7fd24807b1c0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815d760 0x7fd2681581c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680b3550 0x7fd26806ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 --2- 172.21.15.88:0/4053954151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680ad0e0 0x7fd268154e90 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 >> 172.21.15.88:0/4053954151 conn(0x7fd26806d730 msgr2=0x7fd2680cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 shutdown_connections 2024-09-21T07:51:16.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.469+0000 7fd270400640 1 -- 172.21.15.88:0/4053954151 wait complete. 2024-09-21T07:51:16.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:51:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eb1e00640 1 -- 172.21.15.88:0/2153342536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e90005150 msgr2=0x7f4e9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/2153342536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e90005150 0x7f4e9000b9c0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f4ea4005fd0 tx=0x7f4e90005830 comp rx=0 tx=0).stop 2024-09-21T07:51:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eab400640 1 -- 172.21.15.88:0/2153342536 reap_dead start 2024-09-21T07:51:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eb1e00640 1 -- 172.21.15.88:0/2153342536 shutdown_connections 2024-09-21T07:51:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eb1e00640 1 -- 172.21.15.88:0/2153342536 >> 172.21.15.88:0/2153342536 conn(0x7f4eac06d3d0 msgr2=0x7f4eac0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eb1e00640 1 -- 172.21.15.88:0/2153342536 shutdown_connections 2024-09-21T07:51:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.717+0000 7f4eb1e00640 1 -- 172.21.15.88:0/2153342536 wait complete. 2024-09-21T07:51:16.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.719+0000 7f4eb1e00640 1 Processor -- start 2024-09-21T07:51:16.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.720+0000 7f4eb1e00640 1 -- start start 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eb1e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e90005150 0x7f4eac0c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eb1e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eac0c7c20 0x7f4eac0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eb1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 0x7f4eac0b0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eb1e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac156e40 con 0x7f4eac156210 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eb1e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4eac070ed0 con 0x7f4eac0c7c20 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eb1e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4eac156fc0 con 0x7f4e90005150 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eab400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 0x7f4eac0b0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eab400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 0x7f4eac0b0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38716/0 (socket says 172.21.15.88:38716) 2024-09-21T07:51:16.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.721+0000 7f4eab400640 1 -- 172.21.15.88:0/904762097 learned_addr learned my addr 172.21.15.88:0/904762097 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4ea3e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eac0c7c20 0x7f4eac0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eaaa00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e90005150 0x7f4eac0c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eab400640 1 -- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e90005150 msgr2=0x7f4eac0c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eab400640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e90005150 0x7f4eac0c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eab400640 1 -- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eac0c7c20 msgr2=0x7f4eac0ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eab400640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eac0c7c20 0x7f4eac0ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eab400640 1 -- 172.21.15.88:0/904762097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4eac0c7540 con 0x7f4eac156210 2024-09-21T07:51:16.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4eab400640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 0x7f4eac0b0040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4ea402b860 tx=0x7f4ea4033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4042070 con 0x7f4eac156210 2024-09-21T07:51:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.722+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ea4003970 con 0x7f4eac156210 2024-09-21T07:51:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.723+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4eac0c7820 con 0x7f4eac156210 2024-09-21T07:51:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.723+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ea4038040 con 0x7f4eac156210 2024-09-21T07:51:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.723+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4eac0adf50 con 0x7f4eac156210 2024-09-21T07:51:16.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.724+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4eac15d240 con 0x7f4eac156210 2024-09-21T07:51:16.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.725+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f4ea4033d10 con 0x7f4eac156210 2024-09-21T07:51:16.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.726+0000 7f4ea3400640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e88078cb0 0x7f4e8807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:16.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.726+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 5795+0+0 (secure 0 0 0) 0x7f4ea40bb0f0 con 0x7f4eac156210 2024-09-21T07:51:16.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.726+0000 7f4eaaa00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e88078cb0 0x7f4e8807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:16.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.727+0000 7f4eaaa00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e88078cb0 0x7f4e8807b1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c001220 tx=0x7f4e9c001110 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:16.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.731+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ea4082c60 con 0x7f4eac156210 2024-09-21T07:51:16.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.954+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f4eac0afa70 con 0x7f4eac156210 2024-09-21T07:51:16.956 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:51:16.954+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:51:16.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.956+0000 7f4ea3400640 1 -- 172.21.15.88:0/904762097 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v62) ==== 161+0+0 (secure 0 0 0) 0x7f4ea4087690 con 0x7f4eac156210 2024-09-21T07:51:16.959 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:51:16.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e88078cb0 msgr2=0x7f4e8807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e88078cb0 0x7f4e8807b1c0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4e9c001220 tx=0x7f4e9c001110 comp rx=0 tx=0).stop 2024-09-21T07:51:16.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 msgr2=0x7f4eac0b0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:16.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 0x7f4eac0b0040 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f4ea402b860 tx=0x7f4ea4033ba0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 shutdown_connections 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e88078cb0 0x7f4e8807b1c0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4eac156210 0x7f4eac0b0040 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4eac0c7c20 0x7f4eac0ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 --2- 172.21.15.88:0/904762097 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e90005150 0x7f4eac0c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 >> 172.21.15.88:0/904762097 conn(0x7f4eac06d3d0 msgr2=0x7f4eac15ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.960+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 shutdown_connections 2024-09-21T07:51:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:16.961+0000 7f4eb1e00640 1 -- 172.21.15.88:0/904762097 wait complete. 2024-09-21T07:51:16.968 INFO:tasks.ceph.mds.a.smithi088.stderr: -2873> 2024-09-21T07:50:54.163+0000 7f32e6e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:51:16.986 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:51:17.092 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:51:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 -- 172.21.15.88:0/3327422668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d5150 msgr2=0x7fe1980b9b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 --2- 172.21.15.88:0/3327422668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d5150 0x7fe1980b9b10 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7fe1980cfd10 tx=0x7fe18c02a9f0 comp rx=0 tx=0).stop 2024-09-21T07:51:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 -- 172.21.15.88:0/3327422668 shutdown_connections 2024-09-21T07:51:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 --2- 172.21.15.88:0/3327422668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806dea0 0x7fe1980d0020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 --2- 172.21.15.88:0/3327422668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1980d5150 0x7fe1980b9b10 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 -- 172.21.15.88:0/3327422668 >> 172.21.15.88:0/3327422668 conn(0x7fe19808bfe0 msgr2=0x7fe1980d46e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 -- 172.21.15.88:0/3327422668 shutdown_connections 2024-09-21T07:51:17.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.215+0000 7fe1a0400640 1 -- 172.21.15.88:0/3327422668 wait complete. 2024-09-21T07:51:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.217+0000 7fe1a0400640 1 Processor -- start 2024-09-21T07:51:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.218+0000 7fe1a0400640 1 -- start start 2024-09-21T07:51:17.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe1a0400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 0x7fe1980cc4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe1a0400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806e2b0 0x7fe198158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe1a0400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1980d5150 0x7fe1980a0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe1a0400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980ba1a0 con 0x7fe19806dea0 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe1a0400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1980d31e0 con 0x7fe1980d5150 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe1a0400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1980b6870 con 0x7fe19806e2b0 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe19da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 0x7fe1980cc4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe19da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 0x7fe1980cc4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38750/0 (socket says 172.21.15.88:38750) 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe19da00640 1 -- 172.21.15.88:0/1287556352 learned_addr learned my addr 172.21.15.88:0/1287556352 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.219+0000 7fe19d000640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806e2b0 0x7fe198158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19e400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1980d5150 0x7fe1980a0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19da00640 1 -- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806e2b0 msgr2=0x7fe198158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:17.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19da00640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806e2b0 0x7fe198158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19da00640 1 -- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1980d5150 msgr2=0x7fe1980a0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19da00640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1980d5150 0x7fe1980a0cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19da00640 1 -- 172.21.15.88:0/1287556352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1981f69a0 con 0x7fe19806dea0 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19e400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1980d5150 0x7fe1980a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe19da00640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 0x7fe1980cc4e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009d50 tx=0x7fe18c003f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c033050 con 0x7fe19806dea0 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe18c004570 con 0x7fe19806dea0 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1980cb720 con 0x7fe19806dea0 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.220+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c038450 con 0x7fe19806dea0 2024-09-21T07:51:17.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.221+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1980cbce0 con 0x7fe19806dea0 2024-09-21T07:51:17.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.222+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1980d0020 con 0x7fe19806dea0 2024-09-21T07:51:17.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.223+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe18c036070 con 0x7fe19806dea0 2024-09-21T07:51:17.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.224+0000 7fe195600640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe178078cb0 0x7fe17807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:17.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.224+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7fe18c0bc3f0 con 0x7fe19806dea0 2024-09-21T07:51:17.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.224+0000 7fe19d000640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe178078cb0 0x7fe17807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:17.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.225+0000 7fe19d000640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe178078cb0 0x7fe17807b1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1980b3580 tx=0x7fe18400b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:17.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.228+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe18c0845f0 con 0x7fe19806dea0 2024-09-21T07:51:17.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:17.451+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fe1980d4f30 con 0x7fe19806dea0 2024-09-21T07:51:18.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.009+0000 7fe195600640 1 -- 172.21.15.88:0/1287556352 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v63) ==== 104+0+0 (secure 0 0 0) 0x7fe18c083ef0 con 0x7fe19806dea0 2024-09-21T07:51:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe178078cb0 msgr2=0x7fe17807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe178078cb0 0x7fe17807b1c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fe1980b3580 tx=0x7fe18400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 msgr2=0x7fe1980cc4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 0x7fe1980cc4e0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fe18c009d50 tx=0x7fe18c003f90 comp rx=0 tx=0).stop 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 shutdown_connections 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe178078cb0 0x7fe17807b1c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe19806e2b0 0x7fe198158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1980d5150 0x7fe1980a0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 --2- 172.21.15.88:0/1287556352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe19806dea0 0x7fe1980cc4e0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 >> 172.21.15.88:0/1287556352 conn(0x7fe19808bfe0 msgr2=0x7fe1980d2080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 shutdown_connections 2024-09-21T07:51:18.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.012+0000 7fe1a0400640 1 -- 172.21.15.88:0/1287556352 wait complete. 2024-09-21T07:51:18.038 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:51:18.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.260+0000 7f785fe00640 1 -- 172.21.15.88:0/45806808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 msgr2=0x7f7840004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.260+0000 7f785fe00640 1 --2- 172.21.15.88:0/45806808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f7840004c10 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7f7844009b20 tx=0x7f7840005830 comp rx=0 tx=0).stop 2024-09-21T07:51:18.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.261+0000 7f785de00640 1 -- 172.21.15.88:0/45806808 reap_dead start 2024-09-21T07:51:18.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.261+0000 7f785fe00640 1 -- 172.21.15.88:0/45806808 shutdown_connections 2024-09-21T07:51:18.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.261+0000 7f785fe00640 1 -- 172.21.15.88:0/45806808 >> 172.21.15.88:0/45806808 conn(0x7f78580b2540 msgr2=0x7f78580d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:18.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.261+0000 7f785fe00640 1 -- 172.21.15.88:0/45806808 shutdown_connections 2024-09-21T07:51:18.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.261+0000 7f785fe00640 1 -- 172.21.15.88:0/45806808 wait complete. 2024-09-21T07:51:18.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.263+0000 7f785fe00640 1 Processor -- start 2024-09-21T07:51:18.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.263+0000 7f785fe00640 1 -- start start 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f78581f5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7858153f90 0x7f7858153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7858155810 0x7f78580cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78580b3120 con 0x7f7840004800 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f785806d550 con 0x7f7858153f90 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78580b32a0 con 0x7f7858155810 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f78581f5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f78581f5fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38778/0 (socket says 172.21.15.88:38778) 2024-09-21T07:51:18.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 -- 172.21.15.88:0/2912473658 learned_addr learned my addr 172.21.15.88:0/2912473658 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785ca00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7858153f90 0x7f7858153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785de00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7858155810 0x7f78580cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 -- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7858155810 msgr2=0x7f78580cdfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7858155810 0x7f78580cdfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 -- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7858153f90 msgr2=0x7f7858153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7858153f90 0x7f7858153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.265+0000 7f785d400640 1 -- 172.21.15.88:0/2912473658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78581538a0 con 0x7f7840004800 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.266+0000 7f785d400640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f78581f5fe0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f784c011d40 tx=0x7f784c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.266+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c003d80 con 0x7f7840004800 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.266+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f784c004b20 con 0x7f7840004800 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.266+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7858154770 con 0x7f7840004800 2024-09-21T07:51:18.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.266+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f784c004050 con 0x7f7840004800 2024-09-21T07:51:18.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.266+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7858154d30 con 0x7f7840004800 2024-09-21T07:51:18.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.267+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78580d1030 con 0x7f7840004800 2024-09-21T07:51:18.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.269+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f784c0041f0 con 0x7f7840004800 2024-09-21T07:51:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.270+0000 7f7855600640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7838078cb0 0x7f783807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.270+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 5881+0+0 (secure 0 0 0) 0x7f784c09c430 con 0x7f7840004800 2024-09-21T07:51:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.270+0000 7f785ca00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7838078cb0 0x7f783807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:18.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.271+0000 7f785ca00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7838078cb0 0x7f783807b1c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7844035000 tx=0x7f784402d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:18.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.274+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78581538a0 con 0x7f7840004800 2024-09-21T07:51:18.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:18.458+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f7858070aa0 con 0x7f7840004800 2024-09-21T07:51:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.019+0000 7f7855600640 1 -- 172.21.15.88:0/2912473658 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v95) ==== 181+0+0 (secure 0 0 0) 0x7f784c0645a0 con 0x7f7840004800 2024-09-21T07:51:19.022 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:51:19.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7838078cb0 msgr2=0x7f783807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:19.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7838078cb0 0x7f783807b1c0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f7844035000 tx=0x7f784402d040 comp rx=0 tx=0).stop 2024-09-21T07:51:19.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 msgr2=0x7f78581f5fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f78581f5fe0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f784c011d40 tx=0x7f784c00f790 comp rx=0 tx=0).stop 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 shutdown_connections 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7838078cb0 0x7f783807b1c0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7858155810 0x7f78580cdfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7858153f90 0x7f7858153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 --2- 172.21.15.88:0/2912473658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7840004800 0x7f78581f5fe0 secure :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f784c011d40 tx=0x7f784c00f790 comp rx=0 tx=0).stop 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 >> 172.21.15.88:0/2912473658 conn(0x7f78580b2540 msgr2=0x7f78580981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.023+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 shutdown_connections 2024-09-21T07:51:19.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.024+0000 7f785fe00640 1 -- 172.21.15.88:0/2912473658 wait complete. 2024-09-21T07:51:19.053 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:51:19.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3c000640 1 -- 172.21.15.88:0/517773260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf1c004db0 msgr2=0x7fdf1c005280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:19.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3c000640 1 --2- 172.21.15.88:0/517773260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf1c004db0 0x7fdf1c005280 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fdf340d0d30 tx=0x7fdf1c005830 comp rx=0 tx=0).stop 2024-09-21T07:51:19.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3a000640 1 -- 172.21.15.88:0/517773260 reap_dead start 2024-09-21T07:51:19.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3c000640 1 -- 172.21.15.88:0/517773260 shutdown_connections 2024-09-21T07:51:19.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3c000640 1 -- 172.21.15.88:0/517773260 >> 172.21.15.88:0/517773260 conn(0x7fdf340b2540 msgr2=0x7fdf340d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:19.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3c000640 1 -- 172.21.15.88:0/517773260 shutdown_connections 2024-09-21T07:51:19.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.276+0000 7fdf3c000640 1 -- 172.21.15.88:0/517773260 wait complete. 2024-09-21T07:51:19.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.278+0000 7fdf3c000640 1 Processor -- start 2024-09-21T07:51:19.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.278+0000 7fdf3c000640 1 -- start start 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c004db0 0x7fdf34156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf341549a0 0x7fdf340cf510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 0x7fdf341555e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf340b32a0 con 0x7fdf340cfa50 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf340b3120 con 0x7fdf341549a0 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf340b3420 con 0x7fdf1c004db0 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 0x7fdf341555e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 0x7fdf341555e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38794/0 (socket says 172.21.15.88:38794) 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf3a000640 1 -- 172.21.15.88:0/2165566425 learned_addr learned my addr 172.21.15.88:0/2165566425 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf38c00640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf341549a0 0x7fdf340cf510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.280+0000 7fdf39600640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c004db0 0x7fdf34156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:19.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3a000640 1 -- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c004db0 msgr2=0x7fdf34156dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:19.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3a000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c004db0 0x7fdf34156dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:19.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3a000640 1 -- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf341549a0 msgr2=0x7fdf340cf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:19.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3a000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf341549a0 0x7fdf340cf510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:19.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3a000640 1 -- 172.21.15.88:0/2165566425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf340cfe60 con 0x7fdf340cfa50 2024-09-21T07:51:19.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf39600640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c004db0 0x7fdf34156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:19.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3a000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 0x7fdf341555e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c02b330 tx=0x7fdf2c001aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c02ba90 con 0x7fdf340cfa50 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf2c001250 con 0x7fdf340cfa50 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf3409cf40 con 0x7fdf340cfa50 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf2c006070 con 0x7fdf340cfa50 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.281+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf3409d500 con 0x7fdf340cfa50 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.282+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf340af510 con 0x7fdf340cfa50 2024-09-21T07:51:19.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.284+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fdf2c006210 con 0x7fdf340cfa50 2024-09-21T07:51:19.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.285+0000 7fdf31600640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf14078ce0 0x7fdf1407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:19.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.285+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 5443+0+0 (secure 0 0 0) 0x7fdf2c0baf50 con 0x7fdf340cfa50 2024-09-21T07:51:19.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.285+0000 7fdf39600640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf14078ce0 0x7fdf1407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:19.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.286+0000 7fdf39600640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf14078ce0 0x7fdf1407b1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009c50 tx=0x7fdf28005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:19.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.290+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf340cfe60 con 0x7fdf340cfa50 2024-09-21T07:51:19.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:19.474+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf340d1030 con 0x7fdf340cfa50 2024-09-21T07:51:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.029+0000 7fdf31600640 1 -- 172.21.15.88:0/2165566425 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v96) ==== 169+0+0 (secure 0 0 0) 0x7fdf2c082c20 con 0x7fdf340cfa50 2024-09-21T07:51:20.031 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:51:20.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.032+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf14078ce0 msgr2=0x7fdf1407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.032+0000 7fdf3c000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf14078ce0 0x7fdf1407b1f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fdf28009c50 tx=0x7fdf28005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:20.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.032+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 msgr2=0x7fdf341555e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.032+0000 7fdf3c000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 0x7fdf341555e0 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c02b330 tx=0x7fdf2c001aa0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 shutdown_connections 2024-09-21T07:51:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf14078ce0 0x7fdf1407b1f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf340cfa50 0x7fdf341555e0 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf341549a0 0x7fdf340cf510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 --2- 172.21.15.88:0/2165566425 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf1c004db0 0x7fdf34156dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 >> 172.21.15.88:0/2165566425 conn(0x7fdf340b2540 msgr2=0x7fdf340b1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 shutdown_connections 2024-09-21T07:51:20.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.035+0000 7fdf3c000640 1 -- 172.21.15.88:0/2165566425 wait complete. 2024-09-21T07:51:20.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:51:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ec800640 1 -- 172.21.15.88:0/977229417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc003fd0 msgr2=0x7fb5cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ec800640 1 --2- 172.21.15.88:0/977229417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5cc003fd0 0x7fb5cc004800 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009b20 tx=0x7fb5cc005830 comp rx=0 tx=0).stop 2024-09-21T07:51:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ea800640 1 -- 172.21.15.88:0/977229417 reap_dead start 2024-09-21T07:51:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ec800640 1 -- 172.21.15.88:0/977229417 shutdown_connections 2024-09-21T07:51:20.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ec800640 1 -- 172.21.15.88:0/977229417 >> 172.21.15.88:0/977229417 conn(0x7fb5e406d3d0 msgr2=0x7fb5e40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:20.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ec800640 1 -- 172.21.15.88:0/977229417 shutdown_connections 2024-09-21T07:51:20.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.289+0000 7fb5ec800640 1 -- 172.21.15.88:0/977229417 wait complete. 2024-09-21T07:51:20.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.291+0000 7fb5ec800640 1 Processor -- start 2024-09-21T07:51:20.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.291+0000 7fb5ec800640 1 -- start start 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ec800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5cc003fd0 0x7fb5e4157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ec800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 0x7fb5e40cb630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ec800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e40cbb70 0x7fb5e41f5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ec800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4070d50 con 0x7fb5e4155ed0 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ec800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e4070bd0 con 0x7fb5e40cbb70 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ec800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5e40d0110 con 0x7fb5cc003fd0 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 0x7fb5e40cb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5e9e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5cc003fd0 0x7fb5e4157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5e9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 0x7fb5e40cb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38806/0 (socket says 172.21.15.88:38806) 2024-09-21T07:51:20.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5e9400640 1 -- 172.21.15.88:0/480893566 learned_addr learned my addr 172.21.15.88:0/480893566 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5e9e00640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5cc003fd0 0x7fb5e4157de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:53004/0 (socket says 172.21.15.88:53004) 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.293+0000 7fb5ea800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e40cbb70 0x7fb5e41f5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5e9400640 1 -- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5cc003fd0 msgr2=0x7fb5e4157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5e9400640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5cc003fd0 0x7fb5e4157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5e9400640 1 -- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e40cbb70 msgr2=0x7fb5e41f5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5e9400640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e40cbb70 0x7fb5e41f5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5e9400640 1 -- 172.21.15.88:0/480893566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e41f6d50 con 0x7fb5e4155ed0 2024-09-21T07:51:20.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5e9400640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 0x7fb5e40cb630 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009a70 tx=0x7fb5d4033650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4038040 con 0x7fb5e4155ed0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb5d4041070 con 0x7fb5e4155ed0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e4158320 con 0x7fb5e4155ed0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d4033b30 con 0x7fb5e4155ed0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.294+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e40cab50 con 0x7fb5e4155ed0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.296+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e4156e40 con 0x7fb5e4155ed0 2024-09-21T07:51:20.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.297+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb5d4033cd0 con 0x7fb5e4155ed0 2024-09-21T07:51:20.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.298+0000 7fb5de000640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5c8078ce0 0x7fb5c807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.298+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7fb5d40bb580 con 0x7fb5e4155ed0 2024-09-21T07:51:20.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.298+0000 7fb5e9e00640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5c8078ce0 0x7fb5c807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.299+0000 7fb5e9e00640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5c8078ce0 0x7fb5c807b1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5e409e510 tx=0x7fb5d8005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:20.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.302+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb5d40833f0 con 0x7fb5e4155ed0 2024-09-21T07:51:20.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.501+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7fb5e40afa70 con 0x7fb5e4155ed0 2024-09-21T07:51:20.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.506+0000 7fb5de000640 1 -- 172.21.15.88:0/480893566 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v41) ==== 71+0+0 (secure 0 0 0) 0x7fb5d4087e20 con 0x7fb5e4155ed0 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5c8078ce0 msgr2=0x7fb5c807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5c8078ce0 0x7fb5c807b1f0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb5e409e510 tx=0x7fb5d8005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 msgr2=0x7fb5e40cb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 0x7fb5e40cb630 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7fb5d4009a70 tx=0x7fb5d4033650 comp rx=0 tx=0).stop 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 shutdown_connections 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5c8078ce0 0x7fb5c807b1f0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb5e40cbb70 0x7fb5e41f5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e4155ed0 0x7fb5e40cb630 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 --2- 172.21.15.88:0/480893566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb5cc003fd0 0x7fb5e4157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.508+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 >> 172.21.15.88:0/480893566 conn(0x7fb5e406d3d0 msgr2=0x7fb5e406f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.509+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 shutdown_connections 2024-09-21T07:51:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.509+0000 7fb5ec800640 1 -- 172.21.15.88:0/480893566 wait complete. 2024-09-21T07:51:20.536 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars' 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.753+0000 7efcdc000640 1 -- 172.21.15.88:0/2089054627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd40ad0e0 msgr2=0x7efcd40ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.753+0000 7efcdc000640 1 --2- 172.21.15.88:0/2089054627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd40ad0e0 0x7efcd40ad4f0 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7efcc0005fd0 tx=0x7efcc002a540 comp rx=0 tx=0).stop 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.754+0000 7efcdc000640 1 -- 172.21.15.88:0/2089054627 shutdown_connections 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.754+0000 7efcdc000640 1 --2- 172.21.15.88:0/2089054627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd40ad0e0 0x7efcd40ad4f0 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.754+0000 7efcdc000640 1 --2- 172.21.15.88:0/2089054627 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd415d760 0x7efcd41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.754+0000 7efcdc000640 1 -- 172.21.15.88:0/2089054627 >> 172.21.15.88:0/2089054627 conn(0x7efcd406d730 msgr2=0x7efcd40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.754+0000 7efcdc000640 1 -- 172.21.15.88:0/2089054627 shutdown_connections 2024-09-21T07:51:20.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.754+0000 7efcdc000640 1 -- 172.21.15.88:0/2089054627 wait complete. 2024-09-21T07:51:20.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.756+0000 7efcdc000640 1 Processor -- start 2024-09-21T07:51:20.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.756+0000 7efcdc000640 1 -- start start 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.757+0000 7efcdc000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40ad0e0 0x7efcd40b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.757+0000 7efcdc000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 0x7efcd4154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.757+0000 7efcdc000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4160310 0x7efcd4154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.757+0000 7efcdc000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efcd406f220 con 0x7efcd415d760 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.757+0000 7efcdc000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efcd406f0a0 con 0x7efcd4160310 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.757+0000 7efcdc000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efcd40b4c20 con 0x7efcd40ad0e0 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 0x7efcd4154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 0x7efcd4154550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38824/0 (socket says 172.21.15.88:38824) 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 -- 172.21.15.88:0/538039130 learned_addr learned my addr 172.21.15.88:0/538039130 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd9600640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40ad0e0 0x7efcd40b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcda000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4160310 0x7efcd4154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 -- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40ad0e0 msgr2=0x7efcd40b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40ad0e0 0x7efcd40b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 -- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4160310 msgr2=0x7efcd4154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4160310 0x7efcd4154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 -- 172.21.15.88:0/538039130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efcd41f4e90 con 0x7efcd415d760 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcda000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4160310 0x7efcd4154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.758+0000 7efcd8c00640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 0x7efcd4154550 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efcc0034040 tx=0x7efcc00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.759+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc003e030 con 0x7efcd415d760 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.759+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efcc0003d90 con 0x7efcd415d760 2024-09-21T07:51:20.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.759+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efcd41f5170 con 0x7efcd415d760 2024-09-21T07:51:20.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.759+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efcc0035040 con 0x7efcd415d760 2024-09-21T07:51:20.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.759+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efcd41f5730 con 0x7efcd415d760 2024-09-21T07:51:20.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.760+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efcd4160fa0 con 0x7efcd415d760 2024-09-21T07:51:20.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.761+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7efcc0041030 con 0x7efcd415d760 2024-09-21T07:51:20.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.762+0000 7efcd1600640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efcb4078cb0 0x7efcb407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:20.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.762+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7efcc00bb900 con 0x7efcd415d760 2024-09-21T07:51:20.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.762+0000 7efcd9600640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efcb4078cb0 0x7efcb407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:20.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.763+0000 7efcd9600640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efcb4078cb0 0x7efcb407b1c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005fd0 tx=0x7efcc8005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:20.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:20.767+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efcc0083d40 con 0x7efcd415d760 2024-09-21T07:51:21.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.027+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]} v 0) -- 0x7efcd415e7f0 con 0x7efcd415d760 2024-09-21T07:51:21.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.031+0000 7efcd1600640 1 -- 172.21.15.88:0/538039130 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_names_can_only_be_goodchars"]}]=0 v299) ==== 146+0+0 (secure 0 0 0) 0x7efcc0083640 con 0x7efcd415d760 2024-09-21T07:51:21.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efcb4078cb0 msgr2=0x7efcb407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efcb4078cb0 0x7efcb407b1c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efcc8005fd0 tx=0x7efcc8005850 comp rx=0 tx=0).stop 2024-09-21T07:51:21.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 msgr2=0x7efcd4154550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 0x7efcd4154550 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7efcc0034040 tx=0x7efcc00038a0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 shutdown_connections 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efcb4078cb0 0x7efcb407b1c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efcd4160310 0x7efcd4154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efcd415d760 0x7efcd4154550 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 --2- 172.21.15.88:0/538039130 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efcd40ad0e0 0x7efcd40b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 >> 172.21.15.88:0/538039130 conn(0x7efcd406d730 msgr2=0x7efcd415edd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 shutdown_connections 2024-09-21T07:51:21.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.033+0000 7efcdc000640 1 -- 172.21.15.88:0/538039130 wait complete. 2024-09-21T07:51:21.060 INFO:tasks.cephfs_test_runner:Test the creating vols, subvols subvolgroups fails when their names uses ... ok 2024-09-21T07:51:21.060 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-21T07:51:21.060 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_sanity (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:51:21.061 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:51:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d0b200640 1 -- 172.21.15.88:0/3694713842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce8004800 msgr2=0x7f5ce8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d0b200640 1 --2- 172.21.15.88:0/3694713842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ce8004800 0x7f5ce8004c10 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f5cec001830 tx=0x7f5ce8005830 comp rx=0 tx=0).stop 2024-09-21T07:51:21.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d09200640 1 -- 172.21.15.88:0/3694713842 reap_dead start 2024-09-21T07:51:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d0b200640 1 -- 172.21.15.88:0/3694713842 shutdown_connections 2024-09-21T07:51:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d0b200640 1 -- 172.21.15.88:0/3694713842 >> 172.21.15.88:0/3694713842 conn(0x7f5d0406d730 msgr2=0x7f5d040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d0b200640 1 -- 172.21.15.88:0/3694713842 shutdown_connections 2024-09-21T07:51:21.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.278+0000 7f5d0b200640 1 -- 172.21.15.88:0/3694713842 wait complete. 2024-09-21T07:51:21.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.281+0000 7f5d0b200640 1 Processor -- start 2024-09-21T07:51:21.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.281+0000 7f5d0b200640 1 -- start start 2024-09-21T07:51:21.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce8004800 0x7f5d04157160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 0x7f5d04158e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5d0b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d04157f90 0x7f5d04073700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5d0b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5d040b3fe0 con 0x7f5d041f52e0 2024-09-21T07:51:21.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5d0b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d040b3e60 con 0x7f5ce8004800 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5d0b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d040b4160 con 0x7f5d04157f90 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5cfb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 0x7f5d04158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5cfb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 0x7f5d04158e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38840/0 (socket says 172.21.15.88:38840) 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.282+0000 7f5cfb400640 1 -- 172.21.15.88:0/1935894699 learned_addr learned my addr 172.21.15.88:0/1935894699 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5d03e00640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce8004800 0x7f5d04157160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5d09200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d04157f90 0x7f5d04073700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5cfb400640 1 -- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d04157f90 msgr2=0x7f5d04073700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5cfb400640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d04157f90 0x7f5d04073700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5cfb400640 1 -- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce8004800 msgr2=0x7f5d04157160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5cfb400640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce8004800 0x7f5d04157160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5cfb400640 1 -- 172.21.15.88:0/1935894699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d041f5a90 con 0x7f5d041f52e0 2024-09-21T07:51:21.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5cfb400640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 0x7f5d04158e30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec0069d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.283+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec043070 con 0x7f5d041f52e0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.284+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cec003900 con 0x7f5d041f52e0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.284+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d041f5d70 con 0x7f5d041f52e0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.284+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cec003d50 con 0x7f5d041f52e0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.284+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d040731e0 con 0x7f5d041f52e0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.285+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d040b46f0 con 0x7f5d041f52e0 2024-09-21T07:51:21.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.286+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5cec0303c0 con 0x7f5d041f52e0 2024-09-21T07:51:21.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.287+0000 7f5d00c00640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ce0078ce0 0x7f5ce007b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.287+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7f5cec0ba940 con 0x7f5d041f52e0 2024-09-21T07:51:21.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.287+0000 7f5d03e00640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ce0078ce0 0x7f5ce007b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.288+0000 7f5d03e00640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ce0078ce0 0x7f5ce007b1f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009b30 tx=0x7f5cf4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:21.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.291+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cec082900 con 0x7f5d041f52e0 2024-09-21T07:51:21.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.477+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d041f6290 con 0x7f5d041f52e0 2024-09-21T07:51:21.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.478+0000 7f5d00c00640 1 -- 172.21.15.88:0/1935894699 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) ==== 74+0+13177 (secure 0 0 0) 0x7f5cec087330 con 0x7f5d041f52e0 2024-09-21T07:51:21.480 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:21.481 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":96,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:51:20.021837+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":88,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/1717882335":"2024-09-22T07:51:16.955881+0000","172.21.15.88:6833/1717882335":"2024-09-22T07:51:16.955881+0000","172.21.15.175:6835/3755060210":"2024-09-22T07:51:08.971204+0000","172.21.15.175:6834/3755060210":"2024-09-22T07:51:08.971204+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.480+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ce0078ce0 msgr2=0x7f5ce007b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.480+0000 7f5d0b200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ce0078ce0 0x7f5ce007b1f0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5cf4009b30 tx=0x7f5cf4005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.480+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 msgr2=0x7f5d04158e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.480+0000 7f5d0b200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 0x7f5d04158e30 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f5cec000c00 tx=0x7f5cec0069d0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 shutdown_connections 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ce0078ce0 0x7f5ce007b1f0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d04157f90 0x7f5d04073700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d041f52e0 0x7f5d04158e30 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 --2- 172.21.15.88:0/1935894699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ce8004800 0x7f5d04157160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 >> 172.21.15.88:0/1935894699 conn(0x7f5d0406d730 msgr2=0x7f5d040b1d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 shutdown_connections 2024-09-21T07:51:21.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.481+0000 7f5d0b200640 1 -- 172.21.15.88:0/1935894699 wait complete. 2024-09-21T07:51:21.505 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:51:21.505 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:51:21.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 -- 172.21.15.88:0/289972616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 msgr2=0x7fe63406ec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 --2- 172.21.15.88:0/289972616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe63406ec90 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7fe61c005fd0 tx=0x7fe61c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 -- 172.21.15.88:0/289972616 shutdown_connections 2024-09-21T07:51:21.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 --2- 172.21.15.88:0/289972616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6340d02f0 0x7fe6340d0700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 --2- 172.21.15.88:0/289972616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe63406ec90 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 -- 172.21.15.88:0/289972616 >> 172.21.15.88:0/289972616 conn(0x7fe63406cf80 msgr2=0x7fe63406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:21.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 -- 172.21.15.88:0/289972616 shutdown_connections 2024-09-21T07:51:21.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.728+0000 7fe639e00640 1 -- 172.21.15.88:0/289972616 wait complete. 2024-09-21T07:51:21.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.730+0000 7fe639e00640 1 Processor -- start 2024-09-21T07:51:21.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.730+0000 7fe639e00640 1 -- start start 2024-09-21T07:51:21.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe634074c00 0x7fe6341f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6340d02f0 0x7fe63415b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe6340ae850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe63415d010 con 0x7fe634160be0 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe63415ce90 con 0x7fe634074c00 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe63415d190 con 0x7fe6340d02f0 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe6340ae850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe6340ae850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38862/0 (socket says 172.21.15.88:38862) 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe639400640 1 -- 172.21.15.88:0/2972897820 learned_addr learned my addr 172.21.15.88:0/2972897820 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe62be00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6340d02f0 0x7fe63415b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.732+0000 7fe638a00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe634074c00 0x7fe6341f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639400640 1 -- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6340d02f0 msgr2=0x7fe63415b8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639400640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6340d02f0 0x7fe63415b8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639400640 1 -- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe634074c00 msgr2=0x7fe6341f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639400640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe634074c00 0x7fe6341f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639400640 1 -- 172.21.15.88:0/2972897820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6341f5b90 con 0x7fe634160be0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639400640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe6340ae850 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e960 tx=0x7fe62c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c004d20 con 0x7fe634160be0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe62c00cb20 con 0x7fe634160be0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6341f5e70 con 0x7fe634160be0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62c005020 con 0x7fe634160be0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.733+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6341f6430 con 0x7fe634160be0 2024-09-21T07:51:21.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.734+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe63415d6b0 con 0x7fe634160be0 2024-09-21T07:51:21.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.736+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe62c0051c0 con 0x7fe634160be0 2024-09-21T07:51:21.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.737+0000 7fe631600640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe610078cb0 0x7fe61007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:21.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.737+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7fe62c09b5e0 con 0x7fe634160be0 2024-09-21T07:51:21.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.737+0000 7fe638a00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe610078cb0 0x7fe61007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:21.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.738+0000 7fe638a00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe610078cb0 0x7fe61007b1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe624001720 tx=0x7fe624009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:21.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.741+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe62c016030 con 0x7fe634160be0 2024-09-21T07:51:21.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.927+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe6340cd6c0 con 0x7fe634160be0 2024-09-21T07:51:21.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.928+0000 7fe631600640 1 -- 172.21.15.88:0/2972897820 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) ==== 93+0+10 (secure 0 0 0) 0x7fe62c063450 con 0x7fe634160be0 2024-09-21T07:51:21.930 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe610078cb0 msgr2=0x7fe61007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe610078cb0 0x7fe61007b1c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fe624001720 tx=0x7fe624009040 comp rx=0 tx=0).stop 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 msgr2=0x7fe6340ae850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe6340ae850 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fe62c00e960 tx=0x7fe62c00ee20 comp rx=0 tx=0).stop 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 shutdown_connections 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe610078cb0 0x7fe61007b1c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6340d02f0 0x7fe63415b8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe634160be0 0x7fe6340ae850 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 --2- 172.21.15.88:0/2972897820 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe634074c00 0x7fe6341f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:21.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.930+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 >> 172.21.15.88:0/2972897820 conn(0x7fe63406cf80 msgr2=0x7fe63406d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:21.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.931+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 shutdown_connections 2024-09-21T07:51:21.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:21.931+0000 7fe639e00640 1 -- 172.21.15.88:0/2972897820 wait complete. 2024-09-21T07:51:21.955 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-21T07:51:22.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.176+0000 7fc7cf400640 1 -- 172.21.15.88:0/2590105427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80b3550 msgr2=0x7fc7c80b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:22.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.176+0000 7fc7cf400640 1 --2- 172.21.15.88:0/2590105427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80b3550 0x7fc7c80b3960 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fc7c0005fd0 tx=0x7fc7c002a820 comp rx=0 tx=0).stop 2024-09-21T07:51:22.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.177+0000 7fc7cf400640 1 -- 172.21.15.88:0/2590105427 shutdown_connections 2024-09-21T07:51:22.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.177+0000 7fc7cf400640 1 --2- 172.21.15.88:0/2590105427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80b3550 0x7fc7c80b3960 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:22.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.177+0000 7fc7cf400640 1 --2- 172.21.15.88:0/2590105427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80ad0e0 0x7fc7c80ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:22.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.177+0000 7fc7cf400640 1 -- 172.21.15.88:0/2590105427 >> 172.21.15.88:0/2590105427 conn(0x7fc7c806d730 msgr2=0x7fc7c80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:22.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.177+0000 7fc7cf400640 1 -- 172.21.15.88:0/2590105427 shutdown_connections 2024-09-21T07:51:22.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.177+0000 7fc7cf400640 1 -- 172.21.15.88:0/2590105427 wait complete. 2024-09-21T07:51:22.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.179+0000 7fc7cf400640 1 Processor -- start 2024-09-21T07:51:22.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.179+0000 7fc7cf400640 1 -- start start 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.180+0000 7fc7cf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 0x7fc7c81f5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.180+0000 7fc7cf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80b3550 0x7fc7c8155d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.180+0000 7fc7cf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c815d760 0x7fc7c815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.180+0000 7fc7cf400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c80b4da0 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.180+0000 7fc7cf400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7c80b4c20 con 0x7fc7c815d760 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.180+0000 7fc7cf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7c80b3ea0 con 0x7fc7c80b3550 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 0x7fc7c81f5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 0x7fc7c81f5070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38874/0 (socket says 172.21.15.88:38874) 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 -- 172.21.15.88:0/330298420 learned_addr learned my addr 172.21.15.88:0/330298420 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7c7e00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80b3550 0x7fc7c8155d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cd400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c815d760 0x7fc7c815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 -- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80b3550 msgr2=0x7fc7c8155d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80b3550 0x7fc7c8155d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:22.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 -- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c815d760 msgr2=0x7fc7c815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c815d760 0x7fc7c815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 -- 172.21.15.88:0/330298420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7c81f8fb0 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7c7e00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80b3550 0x7fc7c8155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.181+0000 7fc7cca00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 0x7fc7c81f5070 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c3f0 tx=0x7fc7bc00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.182+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc017070 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.182+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7bc0035f0 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.182+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc003a40 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.182+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7c81f9270 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.182+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7c81f9830 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.183+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7c80b43c0 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.184+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fc7bc01a070 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.185+0000 7fc7c4c00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7a8078cb0 0x7fc7a807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:22.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.185+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7fc7bc09a090 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.185+0000 7fc7c7e00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7a8078cb0 0x7fc7a807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:22.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.186+0000 7fc7c7e00640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7a8078cb0 0x7fc7a807b1c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7c814f790 tx=0x7fc7b4005f10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:22.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.189+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7c80ada30 con 0x7fc7c80ad0e0 2024-09-21T07:51:22.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:22.443+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7fc7c806fed0 con 0x7fc7c80ad0e0 2024-09-21T07:51:23.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.039+0000 7fc7c4c00640 1 -- 172.21.15.88:0/330298420 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v301) ==== 152+0+0 (secure 0 0 0) 0x7fc7bc062550 con 0x7fc7c80ad0e0 2024-09-21T07:51:23.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7a8078cb0 msgr2=0x7fc7a807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7a8078cb0 0x7fc7a807b1c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc7c814f790 tx=0x7fc7b4005f10 comp rx=0 tx=0).stop 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 msgr2=0x7fc7c81f5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 0x7fc7c81f5070 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c3f0 tx=0x7fc7bc00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 shutdown_connections 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7a8078cb0 0x7fc7a807b1c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7c80b3550 0x7fc7c8155d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7c815d760 0x7fc7c815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 --2- 172.21.15.88:0/330298420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c80ad0e0 0x7fc7c81f5070 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 >> 172.21.15.88:0/330298420 conn(0x7fc7c806d730 msgr2=0x7fc7c80cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 shutdown_connections 2024-09-21T07:51:23.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.043+0000 7fc7cf400640 1 -- 172.21.15.88:0/330298420 wait complete. 2024-09-21T07:51:23.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 -- 172.21.15.88:0/2702696324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc15f910 msgr2=0x7f1fdc0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 --2- 172.21.15.88:0/2702696324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc15f910 0x7f1fdc0b3750 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005fd0 tx=0x7f1fd402a810 comp rx=0 tx=0).stop 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 -- 172.21.15.88:0/2702696324 shutdown_connections 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 --2- 172.21.15.88:0/2702696324 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc15f910 0x7f1fdc0b3750 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 --2- 172.21.15.88:0/2702696324 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fdc152920 0x7f1fdc15f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 -- 172.21.15.88:0/2702696324 >> 172.21.15.88:0/2702696324 conn(0x7f1fdc06d730 msgr2=0x7f1fdc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 -- 172.21.15.88:0/2702696324 shutdown_connections 2024-09-21T07:51:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.285+0000 7f1fe2800640 1 -- 172.21.15.88:0/2702696324 wait complete. 2024-09-21T07:51:23.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.287+0000 7f1fe2800640 1 Processor -- start 2024-09-21T07:51:23.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.287+0000 7f1fe2800640 1 -- start start 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fdc0aebc0 0x7f1fdc1448f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 0x7f1fdc144e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fe2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fdc15f910 0x7f1fdc15e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fe2800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b4190 con 0x7f1fdc152920 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fe2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b4010 con 0x7f1fdc0aebc0 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fe2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1fdc0b4310 con 0x7f1fdc15f910 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fd2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 0x7f1fdc144e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fd2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 0x7f1fdc144e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38882/0 (socket says 172.21.15.88:38882) 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fd2a00640 1 -- 172.21.15.88:0/911684006 learned_addr learned my addr 172.21.15.88:0/911684006 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fdb400640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fdc0aebc0 0x7f1fdc1448f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.289+0000 7f1fdbe00640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fdc15f910 0x7f1fdc15e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd2a00640 1 -- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fdc15f910 msgr2=0x7f1fdc15e370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd2a00640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fdc15f910 0x7f1fdc15e370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd2a00640 1 -- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fdc0aebc0 msgr2=0x7f1fdc1448f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd2a00640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fdc0aebc0 0x7f1fdc1448f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd2a00640 1 -- 172.21.15.88:0/911684006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fdc145670 con 0x7f1fdc152920 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd2a00640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 0x7f1fdc144e30 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400d7a0 tx=0x7f1fc400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc4017070 con 0x7f1fdc152920 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1fc4009cc0 con 0x7f1fdc152920 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fc40106c0 con 0x7f1fdc152920 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fdc154990 con 0x7f1fdc152920 2024-09-21T07:51:23.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.290+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fdc154f50 con 0x7f1fdc152920 2024-09-21T07:51:23.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.291+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fdc0b48a0 con 0x7f1fdc152920 2024-09-21T07:51:23.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.293+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1fc401a070 con 0x7f1fdc152920 2024-09-21T07:51:23.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.294+0000 7f1fd3e00640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1fb8078ce0 0x7f1fb807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.294+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7f1fc409ae40 con 0x7f1fdc152920 2024-09-21T07:51:23.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.294+0000 7f1fdb400640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1fb8078ce0 0x7f1fb807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.294+0000 7f1fdb400640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1fb8078ce0 0x7f1fb807b1f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc007fd0 tx=0x7f1fcc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:23.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.298+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1fc40632f0 con 0x7f1fdc152920 2024-09-21T07:51:23.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.482+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f1fdc15b610 con 0x7f1fdc152920 2024-09-21T07:51:23.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.482+0000 7f1fd3e00640 1 -- 172.21.15.88:0/911684006 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v41) ==== 59+0+2294 (secure 0 0 0) 0x7f1fc4062bf0 con 0x7f1fdc152920 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:51:23.485 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:51:23.486 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:51:23.487 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:51:23.487 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:51:23.487 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:51:23.487 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:51:23.487 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:51:23.487 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:51:23.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1fb8078ce0 msgr2=0x7f1fb807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1fb8078ce0 0x7f1fb807b1f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f1fcc007fd0 tx=0x7f1fcc009040 comp rx=0 tx=0).stop 2024-09-21T07:51:23.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 msgr2=0x7f1fdc144e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 0x7f1fdc144e30 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f1fc400d7a0 tx=0x7f1fc400dc60 comp rx=0 tx=0).stop 2024-09-21T07:51:23.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 shutdown_connections 2024-09-21T07:51:23.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1fb8078ce0 0x7f1fb807b1f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1fdc15f910 0x7f1fdc15e370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fdc152920 0x7f1fdc144e30 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 --2- 172.21.15.88:0/911684006 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1fdc0aebc0 0x7f1fdc1448f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 >> 172.21.15.88:0/911684006 conn(0x7f1fdc06d730 msgr2=0x7f1fdc06f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 shutdown_connections 2024-09-21T07:51:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.485+0000 7f1fe2800640 1 -- 172.21.15.88:0/911684006 wait complete. 2024-09-21T07:51:23.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:51:23.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.728+0000 7f6220e00640 1 -- 172.21.15.88:0/2740380297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c15f9e0 msgr2=0x7f621c147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.728+0000 7f6220e00640 1 --2- 172.21.15.88:0/2740380297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c15f9e0 0x7f621c147b90 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f620c005fd0 tx=0x7f620c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.728+0000 7f6220e00640 1 -- 172.21.15.88:0/2740380297 shutdown_connections 2024-09-21T07:51:23.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.728+0000 7f6220e00640 1 --2- 172.21.15.88:0/2740380297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c073c50 0x7f621c0af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.728+0000 7f6220e00640 1 --2- 172.21.15.88:0/2740380297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c15f9e0 0x7f621c147b90 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.728+0000 7f6220e00640 1 -- 172.21.15.88:0/2740380297 >> 172.21.15.88:0/2740380297 conn(0x7f621c06d1c0 msgr2=0x7f621c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.729+0000 7f6220e00640 1 -- 172.21.15.88:0/2740380297 shutdown_connections 2024-09-21T07:51:23.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.729+0000 7f6220e00640 1 -- 172.21.15.88:0/2740380297 wait complete. 2024-09-21T07:51:23.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.731+0000 7f6220e00640 1 Processor -- start 2024-09-21T07:51:23.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.731+0000 7f6220e00640 1 -- start start 2024-09-21T07:51:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.732+0000 7f6220e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 0x7f621c0c7110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.732+0000 7f6220e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c15f9e0 0x7f621c134720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.732+0000 7f6220e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c135000 0x7f621c135410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.732+0000 7f6220e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f621c06fb60 con 0x7f621c0aee00 2024-09-21T07:51:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.732+0000 7f6220e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f621c06f9e0 con 0x7f621c15f9e0 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.732+0000 7f6220e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f621c06fce0 con 0x7f621c135000 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 0x7f621c0c7110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 0x7f621c0c7110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38890/0 (socket says 172.21.15.88:38890) 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 -- 172.21.15.88:0/1493687890 learned_addr learned my addr 172.21.15.88:0/1493687890 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6211400640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c15f9e0 0x7f621c134720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f621a800640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c135000 0x7f621c135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 -- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c135000 msgr2=0x7f621c135410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c135000 0x7f621c135410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 -- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c15f9e0 msgr2=0x7f621c134720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c15f9e0 0x7f621c134720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 -- 172.21.15.88:0/1493687890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f621c13b340 con 0x7f621c0aee00 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f621a800640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c135000 0x7f621c135410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:23.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.733+0000 7f6219e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 0x7f621c0c7110 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f620c034040 tx=0x7f620c0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:23.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.734+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c03e030 con 0x7f621c0aee00 2024-09-21T07:51:23.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.734+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f620c003d90 con 0x7f621c0aee00 2024-09-21T07:51:23.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.734+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f620c035040 con 0x7f621c0aee00 2024-09-21T07:51:23.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.734+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f621c0c6070 con 0x7f621c0aee00 2024-09-21T07:51:23.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.734+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f621c0c6630 con 0x7f621c0aee00 2024-09-21T07:51:23.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.735+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f621c0af210 con 0x7f621c0aee00 2024-09-21T07:51:23.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.736+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f620c041030 con 0x7f621c0aee00 2024-09-21T07:51:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.737+0000 7f6212a00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61f8078ce0 0x7f61f807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.737+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7f620c0bb7a0 con 0x7f621c0aee00 2024-09-21T07:51:23.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.737+0000 7f6211400640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61f8078ce0 0x7f61f807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:23.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.738+0000 7f6211400640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61f8078ce0 0x7f61f807b1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f621c147ca0 tx=0x7f6204009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:23.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.742+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f620c083600 con 0x7f621c0aee00 2024-09-21T07:51:23.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.929+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f621c069a10 con 0x7f621c0aee00 2024-09-21T07:51:23.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.934+0000 7f6212a00640 1 -- 172.21.15.88:0/1493687890 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f620c088030 con 0x7f621c0aee00 2024-09-21T07:51:23.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61f8078ce0 msgr2=0x7f61f807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61f8078ce0 0x7f61f807b1f0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f621c147ca0 tx=0x7f6204009040 comp rx=0 tx=0).stop 2024-09-21T07:51:23.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 msgr2=0x7f621c0c7110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 0x7f621c0c7110 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f620c034040 tx=0x7f620c0038a0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 shutdown_connections 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61f8078ce0 0x7f61f807b1f0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f621c135000 0x7f621c135410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f621c15f9e0 0x7f621c134720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 --2- 172.21.15.88:0/1493687890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f621c0aee00 0x7f621c0c7110 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 >> 172.21.15.88:0/1493687890 conn(0x7f621c06d1c0 msgr2=0x7f621c1466a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 shutdown_connections 2024-09-21T07:51:23.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:23.937+0000 7f6220e00640 1 -- 172.21.15.88:0/1493687890 wait complete. 2024-09-21T07:51:23.962 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:51:23.962 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:51:24.051 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:51:24.138 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:51:24.138 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:24.138 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:24.174 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:51:24.174 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:24.174 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:24.210 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:51:24.211 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:24.211 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:24.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:51:24.249 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:24.249 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:24.286 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:51:24.286 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:51:24.286 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:24.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.503+0000 7f2a19800640 1 -- 172.21.15.88:0/757548753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a1406f7b0 msgr2=0x7f2a140b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:24.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.503+0000 7f2a19800640 1 --2- 172.21.15.88:0/757548753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a1406f7b0 0x7f2a140b31e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c005fd0 tx=0x7f2a0c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.503+0000 7f2a19800640 1 -- 172.21.15.88:0/757548753 shutdown_connections 2024-09-21T07:51:24.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.503+0000 7f2a19800640 1 --2- 172.21.15.88:0/757548753 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a1406f7b0 0x7f2a140b31e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.503+0000 7f2a19800640 1 --2- 172.21.15.88:0/757548753 >> v2:172.21.15.175:3300/0 conn(0x7f2a140af630 0x7f2a14073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.503+0000 7f2a19800640 1 -- 172.21.15.88:0/757548753 >> 172.21.15.88:0/757548753 conn(0x7f2a1406d730 msgr2=0x7f2a140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:24.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.504+0000 7f2a19800640 1 -- 172.21.15.88:0/757548753 shutdown_connections 2024-09-21T07:51:24.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.504+0000 7f2a19800640 1 -- 172.21.15.88:0/757548753 wait complete. 2024-09-21T07:51:24.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.506+0000 7f2a19800640 1 Processor -- start 2024-09-21T07:51:24.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.506+0000 7f2a19800640 1 -- start start 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.507+0000 7f2a19800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a1406f7b0 0x7f2a140c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.507+0000 7f2a19800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a14073a40 0x7f2a1414e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.507+0000 7f2a19800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 0x7f2a1415abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.507+0000 7f2a19800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a140b3b90 con 0x7f2a140af630 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.507+0000 7f2a19800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a140b3a10 con 0x7f2a1406f7b0 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.507+0000 7f2a19800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a140b3d10 con 0x7f2a14073a40 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 0x7f2a1415abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:24.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 0x7f2a1415abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38904/0 (socket says 172.21.15.88:38904) 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 -- 172.21.15.88:0/118924883 learned_addr learned my addr 172.21.15.88:0/118924883 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a12a00640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a1406f7b0 0x7f2a140c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a12000640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a14073a40 0x7f2a1414e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 -- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a14073a40 msgr2=0x7f2a1414e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a14073a40 0x7f2a1414e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 -- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a1406f7b0 msgr2=0x7f2a140c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a1406f7b0 0x7f2a140c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 -- 172.21.15.88:0/118924883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a140c9a00 con 0x7f2a140af630 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.508+0000 7f2a13400640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 0x7f2a1415abb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009cd0 tx=0x7f2a0c0316f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.509+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c031da0 con 0x7f2a140af630 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.509+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a0c003a60 con 0x7f2a140af630 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.509+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a141f5740 con 0x7f2a140af630 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.509+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a0c0330d0 con 0x7f2a140af630 2024-09-21T07:51:24.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.509+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a141f5d00 con 0x7f2a140af630 2024-09-21T07:51:24.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.510+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a140b4230 con 0x7f2a140af630 2024-09-21T07:51:24.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.511+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2a0c033270 con 0x7f2a140af630 2024-09-21T07:51:24.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.512+0000 7f2a0aa00640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29f4078cb0 0x7f29f407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:24.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.512+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 5029+0+0 (secure 0 0 0) 0x7f2a0c0bc090 con 0x7f2a140af630 2024-09-21T07:51:24.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.512+0000 7f2a12a00640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29f4078cb0 0x7f29f407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:24.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.513+0000 7f2a12a00640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29f4078cb0 0x7f29f407b1c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a040095f0 tx=0x7f2a04005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:24.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.516+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a0c03f030 con 0x7f2a140af630 2024-09-21T07:51:24.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.735+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2a1414ed20 con 0x7f2a140af630 2024-09-21T07:51:24.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.736+0000 7f2a0aa00640 1 -- 172.21.15.88:0/118924883 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 63 v63) ==== 94+0+3914 (secure 0 0 0) 0x7f2a0c0845e0 con 0x7f2a140af630 2024-09-21T07:51:24.738 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:24.739 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":63,"btime":"2024-09-21T07:51:18:003256+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5858,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3155548800","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3155548800},{"type":"v1","addr":"172.21.15.88:6834","nonce":3155548800}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":50},{"gid":6364,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/133923925","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":133923925},{"type":"v1","addr":"172.21.15.175:6801","nonce":133923925}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":51},{"gid":6520,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3665288036","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3665288036},{"type":"v1","addr":"172.21.15.175:6835","nonce":3665288036}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":61},{"gid":6571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/3868917178","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3868917178},{"type":"v1","addr":"172.21.15.88:6835","nonce":3868917178}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":63}],"filesystems":[]} 2024-09-21T07:51:24.739 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 63 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29f4078cb0 msgr2=0x7f29f407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29f4078cb0 0x7f29f407b1c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2a040095f0 tx=0x7f2a04005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 msgr2=0x7f2a1415abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 0x7f2a1415abb0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f2a0c009cd0 tx=0x7f2a0c0316f0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 shutdown_connections 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f29f4078cb0 0x7f29f407b1c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a14073a40 0x7f2a1414e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a140af630 0x7f2a1415abb0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.738+0000 7f2a19800640 1 --2- 172.21.15.88:0/118924883 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a1406f7b0 0x7f2a140c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.739+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 >> 172.21.15.88:0/118924883 conn(0x7f2a1406d730 msgr2=0x7f2a140affa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:24.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.739+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 shutdown_connections 2024-09-21T07:51:24.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:24.739+0000 7f2a19800640 1 -- 172.21.15.88:0/118924883 wait complete. 2024-09-21T07:51:24.762 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:51:24.763 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:51:24.763 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:51:24.763 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:51:24.763 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:51:24.763 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:51:24.763 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:51:24.764 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:51:24.764 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:51:24.764 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:51:24.764 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:51:24.764 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:51:24.764 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:51:24.764 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:51:24.764 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:51:24.765 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:51:24.788 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:51:24.788 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:51:24.789 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:51:24.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:51:24.790 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:51:24.790 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:51:24.791 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:51:24.791 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:51:24.834 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:51:24.835 DEBUG:teuthology.parallel:result is None 2024-09-21T07:51:24.835 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:51:24.835 DEBUG:teuthology.parallel:result is None 2024-09-21T07:51:24.836 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:51:24.837 DEBUG:teuthology.parallel:result is None 2024-09-21T07:51:24.837 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:51:24.837 DEBUG:teuthology.parallel:result is None 2024-09-21T07:51:24.838 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:51:25.030 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:51:25.030 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:51:25.031 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:51:25.031 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:51:25.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.072+0000 7f430e400640 1 -- 172.21.15.88:0/3510015191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080b3d60 msgr2=0x7f43080af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:25.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.072+0000 7f430e400640 1 --2- 172.21.15.88:0/3510015191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080b3d60 0x7f43080af510 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f43080ca8d0 tx=0x7f42f802ade0 comp rx=0 tx=0).stop 2024-09-21T07:51:25.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.073+0000 7f430e400640 1 -- 172.21.15.88:0/3510015191 shutdown_connections 2024-09-21T07:51:25.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.073+0000 7f430e400640 1 --2- 172.21.15.88:0/3510015191 >> v2:172.21.15.175:3300/0 conn(0x7f43080cd410 0x7f430806f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:25.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.073+0000 7f430e400640 1 --2- 172.21.15.88:0/3510015191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080b3d60 0x7f43080af510 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:25.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.073+0000 7f430e400640 1 -- 172.21.15.88:0/3510015191 >> 172.21.15.88:0/3510015191 conn(0x7f430806d3d0 msgr2=0x7f43080b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:25.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.073+0000 7f430e400640 1 -- 172.21.15.88:0/3510015191 shutdown_connections 2024-09-21T07:51:25.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.073+0000 7f430e400640 1 -- 172.21.15.88:0/3510015191 wait complete. 2024-09-21T07:51:25.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.076+0000 7f430e400640 1 Processor -- start 2024-09-21T07:51:25.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.076+0000 7f430e400640 1 -- start start 2024-09-21T07:51:25.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.077+0000 7f430e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43080b3d60 0x7f430815d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.077+0000 7f430e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 0x7f4308159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f430e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43080cd410 0x7f43080ae270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f430e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43080afb60 con 0x7f43080cd000 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f430e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f43080b4380 con 0x7f43080b3d60 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f430e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f43080711c0 con 0x7f43080cd410 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 0x7f4308159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 0x7f4308159390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:38958/0 (socket says 172.21.15.88:38958) 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 -- 172.21.15.88:0/3804018517 learned_addr learned my addr 172.21.15.88:0/3804018517 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4307400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43080b3d60 0x7f430815d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4307e00640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43080cd410 0x7f43080ae270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 -- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43080cd410 msgr2=0x7f43080ae270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43080cd410 0x7f43080ae270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 -- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43080b3d60 msgr2=0x7f430815d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:25.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43080b3d60 0x7f430815d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:25.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.078+0000 7f4306a00640 1 -- 172.21.15.88:0/3804018517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43080c9250 con 0x7f43080cd000 2024-09-21T07:51:25.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.079+0000 7f4306a00640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 0x7f4308159390 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f42f400b420 tx=0x7f42f400b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.079+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4011830 con 0x7f43080cd000 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.079+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f42f4016070 con 0x7f43080cd000 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.079+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4011d60 con 0x7f43080cd000 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.079+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43081f5ca0 con 0x7f43080cd000 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.079+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43081f3240 con 0x7f43080cd000 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.080+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43080cfdf0 con 0x7f43080cd000 2024-09-21T07:51:25.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.081+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f42f401b460 con 0x7f43080cd000 2024-09-21T07:51:25.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.083+0000 7f42ff400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f42e8078cb0 0x7f42e807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:25.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.083+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 5029+0+0 (secure 0 0 0) 0x7f42f409a740 con 0x7f43080cd000 2024-09-21T07:51:25.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.083+0000 7f4307400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f42e8078cb0 0x7f42e807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:25.086 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.084+0000 7f4307400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f42e8078cb0 0x7f42e807b1c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f42f8034000 tx=0x7f42f8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:25.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.087+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42f4020020 con 0x7f43080cd000 2024-09-21T07:51:25.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:25.286+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f43080c8fe0 con 0x7f43080cd000 2024-09-21T07:51:26.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.079+0000 7f42ff400640 1 -- 172.21.15.88:0/3804018517 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v101) ==== 97+0+0 (secure 0 0 0) 0x7f42f40625a0 con 0x7f43080cd000 2024-09-21T07:51:26.081 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:51:26.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f42e8078cb0 msgr2=0x7f42e807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f42e8078cb0 0x7f42e807b1c0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f42f8034000 tx=0x7f42f8004730 comp rx=0 tx=0).stop 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 msgr2=0x7f4308159390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 0x7f4308159390 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f42f400b420 tx=0x7f42f400b8e0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 shutdown_connections 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f42e8078cb0 0x7f42e807b1c0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f43080cd410 0x7f43080ae270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43080cd000 0x7f4308159390 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 --2- 172.21.15.88:0/3804018517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f43080b3d60 0x7f430815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 >> 172.21.15.88:0/3804018517 conn(0x7f430806d3d0 msgr2=0x7f43080af070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 shutdown_connections 2024-09-21T07:51:26.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.082+0000 7f430e400640 1 -- 172.21.15.88:0/3804018517 wait complete. 2024-09-21T07:51:26.109 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 -- 172.21.15.88:0/2029138858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0ad0e0 msgr2=0x7f1b5c0ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 --2- 172.21.15.88:0/2029138858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0ad0e0 0x7f1b5c0ad4f0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009830 tx=0x7f1b4802a520 comp rx=0 tx=0).stop 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 -- 172.21.15.88:0/2029138858 shutdown_connections 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 --2- 172.21.15.88:0/2029138858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0ad0e0 0x7f1b5c0ad4f0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 -- 172.21.15.88:0/2029138858 >> 172.21.15.88:0/2029138858 conn(0x7f1b5c06d730 msgr2=0x7f1b5c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 -- 172.21.15.88:0/2029138858 shutdown_connections 2024-09-21T07:51:26.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.326+0000 7f1b62800640 1 -- 172.21.15.88:0/2029138858 wait complete. 2024-09-21T07:51:26.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.328+0000 7f1b62800640 1 Processor -- start 2024-09-21T07:51:26.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.329+0000 7f1b62800640 1 -- start start 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b62800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0ad0e0 0x7f1b5c156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b62800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c0b3550 0x7f1b5c09d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b62800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 0x7f1b5c15b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b62800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b4da0 con 0x7f1b5c15d760 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b62800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b4c20 con 0x7f1b5c0b3550 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b62800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b4070 con 0x7f1b5c0ad0e0 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 0x7f1b5c15b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 0x7f1b5c15b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34226/0 (socket says 172.21.15.88:34226) 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b5be00640 1 -- 172.21.15.88:0/619150732 learned_addr learned my addr 172.21.15.88:0/619150732 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.330+0000 7f1b5aa00640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c0b3550 0x7f1b5c09d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5b400640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0ad0e0 0x7f1b5c156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5be00640 1 -- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0ad0e0 msgr2=0x7f1b5c156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5be00640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0ad0e0 0x7f1b5c156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5be00640 1 -- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c0b3550 msgr2=0x7f1b5c09d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5be00640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c0b3550 0x7f1b5c09d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5be00640 1 -- 172.21.15.88:0/619150732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c1566a0 con 0x7f1b5c15d760 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5b400640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0ad0e0 0x7f1b5c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b5be00640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 0x7f1b5c15b7f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400c700 tx=0x7f1b5400cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54003b70 con 0x7f1b5c15d760 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b54003d10 con 0x7f1b5c15d760 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c1f56f0 con 0x7f1b5c15d760 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b5400aab0 con 0x7f1b5c15d760 2024-09-21T07:51:26.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.331+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c1f5cb0 con 0x7f1b5c15d760 2024-09-21T07:51:26.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.332+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c09e170 con 0x7f1b5c15d760 2024-09-21T07:51:26.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.334+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1b5400ac50 con 0x7f1b5c15d760 2024-09-21T07:51:26.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.335+0000 7f1b53400640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c078cb0 0x7f1b3c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.335+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7f1b5409a910 con 0x7f1b5c15d760 2024-09-21T07:51:26.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.335+0000 7f1b5b400640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c078cb0 0x7f1b3c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.336+0000 7f1b5b400640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c078cb0 0x7f1b3c07b1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0046b0 tx=0x7f1b4c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:26.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.339+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b54062f10 con 0x7f1b5c15d760 2024-09-21T07:51:26.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.593+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f1b5c0ad4f0 con 0x7f1b5c15d760 2024-09-21T07:51:26.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.593+0000 7f1b53400640 1 -- 172.21.15.88:0/619150732 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) ==== 80+0+5304 (secure 0 0 0) 0x7f1b54062810 con 0x7f1b5c15d760 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:51:26.596 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.597 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.598 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.599 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.600 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:51:26.601 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.602 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:51:26.603 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.604 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:51:26.605 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:51:26.606 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.607 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:51:26.608 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c078cb0 msgr2=0x7f1b3c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c078cb0 0x7f1b3c07b1c0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1b4c0046b0 tx=0x7f1b4c009040 comp rx=0 tx=0).stop 2024-09-21T07:51:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 msgr2=0x7f1b5c15b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 0x7f1b5c15b7f0 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f1b5400c700 tx=0x7f1b5400cbc0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 shutdown_connections 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c078cb0 0x7f1b3c07b1c0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c15d760 0x7f1b5c15b7f0 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c0b3550 0x7f1b5c09d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 --2- 172.21.15.88:0/619150732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0ad0e0 0x7f1b5c156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 >> 172.21.15.88:0/619150732 conn(0x7f1b5c06d730 msgr2=0x7f1b5c0cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 shutdown_connections 2024-09-21T07:51:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.596+0000 7f1b62800640 1 -- 172.21.15.88:0/619150732 wait complete. 2024-09-21T07:51:26.623 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311a00640 1 -- 172.21.15.88:0/3114547361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 msgr2=0x7fb2f00052b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311a00640 1 --2- 172.21.15.88:0/3114547361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb2f00052b0 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7fb304009e20 tx=0x7fb2f0005830 comp rx=0 tx=0).stop 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311000640 1 -- 172.21.15.88:0/3114547361 reap_dead start 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311a00640 1 -- 172.21.15.88:0/3114547361 shutdown_connections 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311a00640 1 -- 172.21.15.88:0/3114547361 >> 172.21.15.88:0/3114547361 conn(0x7fb30c0d44a0 msgr2=0x7fb30c0d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311a00640 1 -- 172.21.15.88:0/3114547361 shutdown_connections 2024-09-21T07:51:26.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.843+0000 7fb311a00640 1 -- 172.21.15.88:0/3114547361 wait complete. 2024-09-21T07:51:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.845+0000 7fb311a00640 1 Processor -- start 2024-09-21T07:51:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.845+0000 7fb311a00640 1 -- start start 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb30c15be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c1f6080 0x7fb30c1f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c0a1880 0x7fb30c0a1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0d0530 con 0x7fb2f0004e60 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb30c0d03b0 con 0x7fb30c0a1880 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb30c0d0d10 con 0x7fb30c1f6080 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb30c15be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb30c15be90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34240/0 (socket says 172.21.15.88:34240) 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 -- 172.21.15.88:0/1334920843 learned_addr learned my addr 172.21.15.88:0/1334920843 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb303400640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c1f6080 0x7fb30c1f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb311000640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c0a1880 0x7fb30c0a1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 -- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c1f6080 msgr2=0x7fb30c1f6490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c1f6080 0x7fb30c1f6490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 -- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c0a1880 msgr2=0x7fb30c0a1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c0a1880 0x7fb30c0a1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.847+0000 7fb30be00640 1 -- 172.21.15.88:0/1334920843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb30c0a21d0 con 0x7fb2f0004e60 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.848+0000 7fb30be00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb30c15be90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc006d70 tx=0x7fb2fc0049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.848+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc015020 con 0x7fb2f0004e60 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.848+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2fc011960 con 0x7fb2f0004e60 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.848+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2fc011db0 con 0x7fb2f0004e60 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.848+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb30c15de00 con 0x7fb2f0004e60 2024-09-21T07:51:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.848+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb30c15e3c0 con 0x7fb2f0004e60 2024-09-21T07:51:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.849+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb30c08cc70 con 0x7fb2f0004e60 2024-09-21T07:51:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.850+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fb2fc018020 con 0x7fb2f0004e60 2024-09-21T07:51:26.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.852+0000 7fb308c00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2e8078cb0 0x7fb2e807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:26.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.852+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7fb2fc09b7e0 con 0x7fb2f0004e60 2024-09-21T07:51:26.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.852+0000 7fb303400640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2e8078cb0 0x7fb2e807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:26.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.853+0000 7fb303400640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2e8078cb0 0x7fb2e807b1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb30c1ec920 tx=0x7fb2f4006b70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:26.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:26.856+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb30c08cc70 con 0x7fb2f0004e60 2024-09-21T07:51:26.931 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:26.935 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:27.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.050+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb30c0d1210 con 0x7fb2f0004e60 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.050+0000 7fb308c00640 1 -- 172.21.15.88:0/1334920843 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v101) ==== 84+0+65 (secure 0 0 0) 0x7fb2fc01f020 con 0x7fb2f0004e60 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:51:27.052 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:27.053 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.052+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2e8078cb0 msgr2=0x7fb2e807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.052+0000 7fb311a00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2e8078cb0 0x7fb2e807b1c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fb30c1ec920 tx=0x7fb2f4006b70 comp rx=0 tx=0).stop 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.052+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 msgr2=0x7fb30c15be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.052+0000 7fb311a00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb30c15be90 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc006d70 tx=0x7fb2fc0049d0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 shutdown_connections 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2e8078cb0 0x7fb2e807b1c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb30c1f6080 0x7fb30c1f6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb30c0a1880 0x7fb30c0a1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 --2- 172.21.15.88:0/1334920843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f0004e60 0x7fb30c15be90 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 >> 172.21.15.88:0/1334920843 conn(0x7fb30c0d44a0 msgr2=0x7fb30c0d2070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 shutdown_connections 2024-09-21T07:51:27.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.053+0000 7fb311a00640 1 -- 172.21.15.88:0/1334920843 wait complete. 2024-09-21T07:51:27.077 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:51:27.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:51:27.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 -- 172.21.15.88:0/1730066133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0cc830 msgr2=0x7f299c0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 --2- 172.21.15.88:0/1730066133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0cc830 0x7f299c0b4620 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f298c005fd0 tx=0x7f298c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 -- 172.21.15.88:0/1730066133 shutdown_connections 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 --2- 172.21.15.88:0/1730066133 >> v2:172.21.15.175:3300/0 conn(0x7f299c06cda0 0x7f299c06d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 --2- 172.21.15.88:0/1730066133 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0cc830 0x7f299c0b4620 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 -- 172.21.15.88:0/1730066133 >> 172.21.15.88:0/1730066133 conn(0x7f299c0b2540 msgr2=0x7f299c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 -- 172.21.15.88:0/1730066133 shutdown_connections 2024-09-21T07:51:27.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.340+0000 7f29a2400640 1 -- 172.21.15.88:0/1730066133 wait complete. 2024-09-21T07:51:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.343+0000 7f29a2400640 1 Processor -- start 2024-09-21T07:51:27.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.343+0000 7f29a2400640 1 -- start start 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.344+0000 7f29a2400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299c06cda0 0x7f299c09cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.344+0000 7f29a2400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0af100 0x7f299c156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.344+0000 7f29a2400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 0x7f299c156c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.344+0000 7f29a2400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b3600 con 0x7f299c0cc830 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.344+0000 7f29a2400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f299c0b3480 con 0x7f299c06cda0 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.344+0000 7f29a2400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f299c0b3780 con 0x7f299c0af100 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 0x7f299c156c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f2992a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0af100 0x7f299c156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299c06cda0 0x7f299c09cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 0x7f299c156c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34256/0 (socket says 172.21.15.88:34256) 2024-09-21T07:51:27.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 -- 172.21.15.88:0/2718082288 learned_addr learned my addr 172.21.15.88:0/2718082288 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 -- 172.21.15.88:0/2718082288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0af100 msgr2=0x7f299c156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0af100 0x7f299c156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 -- 172.21.15.88:0/2718082288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299c06cda0 msgr2=0x7f299c09cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299c06cda0 0x7f299c09cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:27.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 -- 172.21.15.88:0/2718082288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f299c1f6e60 con 0x7f299c0cc830 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.345+0000 7f299be00640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 0x7f299c156c40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f299400c720 tx=0x7f299400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.346+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994003b70 con 0x7f299c0cc830 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.346+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f299c1f7140 con 0x7f299c0cc830 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.346+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2994003d30 con 0x7f299c0cc830 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.346+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400aab0 con 0x7f299c0cc830 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.346+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f299c1f7700 con 0x7f299c0cc830 2024-09-21T07:51:27.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.347+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f299c0b3ca0 con 0x7f299c0cc830 2024-09-21T07:51:27.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.348+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f299400ac50 con 0x7f299c0cc830 2024-09-21T07:51:27.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.349+0000 7f2993e00640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2978078ce0 0x7f297807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:27.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.349+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 4857+0+0 (secure 0 0 0) 0x7f299409a7c0 con 0x7f299c0cc830 2024-09-21T07:51:27.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.349+0000 7f299b400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2978078ce0 0x7f297807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:27.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.350+0000 7f299b400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2978078ce0 0x7f297807b1f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f298c033000 tx=0x7f298c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:27.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.354+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2994062780 con 0x7f299c0cc830 2024-09-21T07:51:27.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:27.538+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f299c0d1030 con 0x7f299c0cc830 2024-09-21T07:51:28.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.075+0000 7f2993e00640 1 -- 172.21.15.88:0/2718082288 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v102) ==== 138+0+0 (secure 0 0 0) 0x7f29940671b0 con 0x7f299c0cc830 2024-09-21T07:51:28.077 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2978078ce0 msgr2=0x7f297807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2978078ce0 0x7f297807b1f0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f298c033000 tx=0x7f298c004730 comp rx=0 tx=0).stop 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 msgr2=0x7f299c156c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 0x7f299c156c40 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f299400c720 tx=0x7f299400cbe0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 shutdown_connections 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2978078ce0 0x7f297807b1f0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f299c0af100 0x7f299c156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f299c0cc830 0x7f299c156c40 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 --2- 172.21.15.88:0/2718082288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299c06cda0 0x7f299c09cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.078+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 >> 172.21.15.88:0/2718082288 conn(0x7f299c0b2540 msgr2=0x7f299c0981c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.079+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 shutdown_connections 2024-09-21T07:51:28.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.079+0000 7f29a2400640 1 -- 172.21.15.88:0/2718082288 wait complete. 2024-09-21T07:51:28.105 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:51:28.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 -- 172.21.15.88:0/2557217727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 msgr2=0x7efc3c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 --2- 172.21.15.88:0/2557217727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c06fb40 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7efc3c1464e0 tx=0x7efc2c02a6e0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 -- 172.21.15.88:0/2557217727 shutdown_connections 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 --2- 172.21.15.88:0/2557217727 >> v2:172.21.15.175:3300/0 conn(0x7efc3c073c50 0x7efc3c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 --2- 172.21.15.88:0/2557217727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c06fb40 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 -- 172.21.15.88:0/2557217727 >> 172.21.15.88:0/2557217727 conn(0x7efc3c06d1c0 msgr2=0x7efc3c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 -- 172.21.15.88:0/2557217727 shutdown_connections 2024-09-21T07:51:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.329+0000 7efc41200640 1 -- 172.21.15.88:0/2557217727 wait complete. 2024-09-21T07:51:28.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.331+0000 7efc41200640 1 Processor -- start 2024-09-21T07:51:28.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.332+0000 7efc41200640 1 -- start start 2024-09-21T07:51:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc41200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c15f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc41200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc3c145a40 0x7efc3c1f3840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc41200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc3c1f6e30 0x7efc3c1f7240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc41200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c0ce130 con 0x7efc3c145550 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc41200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc3c0cdfb0 con 0x7efc3c1f6e30 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc41200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc3c0ce2b0 con 0x7efc3c145a40 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc39e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c15f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc39e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c15f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34276/0 (socket says 172.21.15.88:34276) 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.333+0000 7efc39e00640 1 -- 172.21.15.88:0/2573986118 learned_addr learned my addr 172.21.15.88:0/2573986118 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc3a800640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc3c1f6e30 0x7efc3c1f7240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc31400640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc3c145a40 0x7efc3c1f3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc39e00640 1 -- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc3c145a40 msgr2=0x7efc3c1f3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:28.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc39e00640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc3c145a40 0x7efc3c1f3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc39e00640 1 -- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc3c1f6e30 msgr2=0x7efc3c1f7240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc39e00640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc3c1f6e30 0x7efc3c1f7240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc39e00640 1 -- 172.21.15.88:0/2573986118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc3c1f4250 con 0x7efc3c145550 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc31400640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc3c145a40 0x7efc3c1f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc39e00640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c15f6b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efc2c034040 tx=0x7efc2c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c03e030 con 0x7efc3c145550 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc2c0047a0 con 0x7efc3c145550 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.334+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc3c1f4810 con 0x7efc3c145550 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.335+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2c035040 con 0x7efc3c145550 2024-09-21T07:51:28.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.335+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc3c1f7780 con 0x7efc3c145550 2024-09-21T07:51:28.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.336+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc3c0ce830 con 0x7efc3c145550 2024-09-21T07:51:28.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.337+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7efc2c041030 con 0x7efc3c145550 2024-09-21T07:51:28.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.338+0000 7efc32a00640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc18078cb0 0x7efc1807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:28.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.338+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 5227+0+0 (secure 0 0 0) 0x7efc2c0bba80 con 0x7efc3c145550 2024-09-21T07:51:28.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.338+0000 7efc31400640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc18078cb0 0x7efc1807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:28.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.339+0000 7efc31400640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc18078cb0 0x7efc1807b1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc3c15c040 tx=0x7efc24009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:28.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.342+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc2c083910 con 0x7efc3c145550 2024-09-21T07:51:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:28.530+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7efc3c143b10 con 0x7efc3c145550 2024-09-21T07:51:29.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.085+0000 7efc32a00640 1 -- 172.21.15.88:0/2573986118 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v103) ==== 170+0+0 (secure 0 0 0) 0x7efc2c088340 con 0x7efc3c145550 2024-09-21T07:51:29.088 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:51:29.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc18078cb0 msgr2=0x7efc1807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc18078cb0 0x7efc1807b1c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7efc3c15c040 tx=0x7efc24009040 comp rx=0 tx=0).stop 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 msgr2=0x7efc3c15f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c15f6b0 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7efc2c034040 tx=0x7efc2c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 shutdown_connections 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc18078cb0 0x7efc1807b1c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc3c145a40 0x7efc3c1f3840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc3c1f6e30 0x7efc3c1f7240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 --2- 172.21.15.88:0/2573986118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc3c145550 0x7efc3c15f6b0 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.088+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 >> 172.21.15.88:0/2573986118 conn(0x7efc3c06d1c0 msgr2=0x7efc3c0b2190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:29.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.089+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 shutdown_connections 2024-09-21T07:51:29.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.089+0000 7efc41200640 1 -- 172.21.15.88:0/2573986118 wait complete. 2024-09-21T07:51:29.126 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:51:29.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.345+0000 7f1573000640 1 -- 172.21.15.88:0/3822060827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c145550 msgr2=0x7f156c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.345+0000 7f1573000640 1 --2- 172.21.15.88:0/3822060827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c145550 0x7f156c06fb40 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f156c1464e0 tx=0x7f155c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 -- 172.21.15.88:0/3822060827 shutdown_connections 2024-09-21T07:51:29.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 --2- 172.21.15.88:0/3822060827 >> v2:172.21.15.175:3300/0 conn(0x7f156c073a80 0x7f156c160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 --2- 172.21.15.88:0/3822060827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c145a40 0x7f156c073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 --2- 172.21.15.88:0/3822060827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c145550 0x7f156c06fb40 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 -- 172.21.15.88:0/3822060827 >> 172.21.15.88:0/3822060827 conn(0x7f156c06d1c0 msgr2=0x7f156c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:29.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 -- 172.21.15.88:0/3822060827 shutdown_connections 2024-09-21T07:51:29.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.346+0000 7f1573000640 1 -- 172.21.15.88:0/3822060827 wait complete. 2024-09-21T07:51:29.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.348+0000 7f1573000640 1 Processor -- start 2024-09-21T07:51:29.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.348+0000 7f1573000640 1 -- start start 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.349+0000 7f1573000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 0x7f156c14fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.349+0000 7f1573000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c145550 0x7f156c15cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.349+0000 7f1573000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c145a40 0x7f156c15d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.349+0000 7f1573000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f156c160c20 con 0x7f156c073a80 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.349+0000 7f1573000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f156c160aa0 con 0x7f156c145a40 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.349+0000 7f1573000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f156c160da0 con 0x7f156c145550 2024-09-21T07:51:29.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 0x7f156c14fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 0x7f156c14fb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34302/0 (socket says 172.21.15.88:34302) 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 -- 172.21.15.88:0/1265957882 learned_addr learned my addr 172.21.15.88:0/1265957882 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f1571000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c145a40 0x7f156c15d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156b400640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c145550 0x7f156c15cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 -- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c145550 msgr2=0x7f156c15cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c145550 0x7f156c15cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 -- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c145a40 msgr2=0x7f156c15d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c145a40 0x7f156c15d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 -- 172.21.15.88:0/1265957882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f156c150060 con 0x7f156c073a80 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.350+0000 7f156be00640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 0x7f156c14fb20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:29.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.351+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c03e030 con 0x7f156c073a80 2024-09-21T07:51:29.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.351+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f155c003c20 con 0x7f156c073a80 2024-09-21T07:51:29.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.351+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f156c0b4c80 con 0x7f156c073a80 2024-09-21T07:51:29.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.351+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f155c035040 con 0x7f156c073a80 2024-09-21T07:51:29.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.351+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f156c0b4430 con 0x7f156c073a80 2024-09-21T07:51:29.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.352+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f156c0b4250 con 0x7f156c073a80 2024-09-21T07:51:29.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.353+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f155c041030 con 0x7f156c073a80 2024-09-21T07:51:29.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.354+0000 7f1563e00640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f154c078cb0 0x7f154c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:29.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.354+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 5609+0+0 (secure 0 0 0) 0x7f155c0bbc00 con 0x7f156c073a80 2024-09-21T07:51:29.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.354+0000 7f156b400640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f154c078cb0 0x7f154c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:29.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.355+0000 7f156b400640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f154c078cb0 0x7f154c07b1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f156c0af8f0 tx=0x7f1558005e30 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:29.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.358+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f156c0b4250 con 0x7f156c073a80 2024-09-21T07:51:29.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:29.579+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f156c06fb40 con 0x7f156c073a80 2024-09-21T07:51:30.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.102+0000 7f1563e00640 1 -- 172.21.15.88:0/1265957882 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v66) ==== 174+0+0 (secure 0 0 0) 0x7f155c083f60 con 0x7f156c073a80 2024-09-21T07:51:30.106 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 16 and data pool 17 2024-09-21T07:51:30.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.110+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f154c078cb0 msgr2=0x7f154c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.110+0000 7f1573000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f154c078cb0 0x7f154c07b1c0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f156c0af8f0 tx=0x7f1558005e30 comp rx=0 tx=0).stop 2024-09-21T07:51:30.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.110+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 msgr2=0x7f156c14fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.110+0000 7f1573000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 0x7f156c14fb20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f155c000c00 tx=0x7f155c003820 comp rx=0 tx=0).stop 2024-09-21T07:51:30.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.111+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 shutdown_connections 2024-09-21T07:51:30.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.111+0000 7f1573000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f154c078cb0 0x7f154c07b1c0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.111+0000 7f1573000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f156c145550 0x7f156c15cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.111+0000 7f1573000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f156c145a40 0x7f156c15d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.111+0000 7f1573000640 1 --2- 172.21.15.88:0/1265957882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f156c073a80 0x7f156c14fb20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.111+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 >> 172.21.15.88:0/1265957882 conn(0x7f156c06d1c0 msgr2=0x7f156c13ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:30.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.112+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 shutdown_connections 2024-09-21T07:51:30.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.112+0000 7f1573000640 1 -- 172.21.15.88:0/1265957882 wait complete. 2024-09-21T07:51:30.149 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:51:30.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.366+0000 7f4575000640 1 -- 172.21.15.88:0/4240461553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45700b2750 msgr2=0x7f457015f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.366+0000 7f4575000640 1 --2- 172.21.15.88:0/4240461553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45700b2750 0x7f457015f3d0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4558005fd0 tx=0x7f455802a540 comp rx=0 tx=0).stop 2024-09-21T07:51:30.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.367+0000 7f4575000640 1 -- 172.21.15.88:0/4240461553 shutdown_connections 2024-09-21T07:51:30.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.367+0000 7f4575000640 1 --2- 172.21.15.88:0/4240461553 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45700b2750 0x7f457015f3d0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.367+0000 7f4575000640 1 --2- 172.21.15.88:0/4240461553 >> v2:172.21.15.175:3300/0 conn(0x7f45700aebc0 0x7f4570152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.367+0000 7f4575000640 1 -- 172.21.15.88:0/4240461553 >> 172.21.15.88:0/4240461553 conn(0x7f457006d730 msgr2=0x7f45700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:30.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.367+0000 7f4575000640 1 -- 172.21.15.88:0/4240461553 shutdown_connections 2024-09-21T07:51:30.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.367+0000 7f4575000640 1 -- 172.21.15.88:0/4240461553 wait complete. 2024-09-21T07:51:30.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.369+0000 7f4575000640 1 Processor -- start 2024-09-21T07:51:30.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.369+0000 7f4575000640 1 -- start start 2024-09-21T07:51:30.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4575000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 0x7f45701582b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4575000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45700b2750 0x7f45701517f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4575000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f457015f910 0x7f45700ca640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4575000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f457015c620 con 0x7f45700aebc0 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4575000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f457015c4a0 con 0x7f45700b2750 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4575000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f457015c7a0 con 0x7f457015f910 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 0x7f45701582b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 0x7f45701582b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34314/0 (socket says 172.21.15.88:34314) 2024-09-21T07:51:30.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 -- 172.21.15.88:0/4193645233 learned_addr learned my addr 172.21.15.88:0/4193645233 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f4565400640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45700b2750 0x7f45701517f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456e800640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f457015f910 0x7f45700ca640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 -- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f457015f910 msgr2=0x7f45700ca640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f457015f910 0x7f45700ca640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 -- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45700b2750 msgr2=0x7f45701517f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45700b2750 0x7f45701517f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.371+0000 7f456de00640 1 -- 172.21.15.88:0/4193645233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45701f91b0 con 0x7f45700aebc0 2024-09-21T07:51:30.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.372+0000 7f456de00640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 0x7f45701582b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f456000e9f0 tx=0x7f456000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.372+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456000cd80 con 0x7f45700aebc0 2024-09-21T07:51:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.372+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4560004020 con 0x7f45700aebc0 2024-09-21T07:51:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.372+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4560004470 con 0x7f45700aebc0 2024-09-21T07:51:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.372+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45701f9490 con 0x7f45700aebc0 2024-09-21T07:51:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.372+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45701f9a50 con 0x7f45700aebc0 2024-09-21T07:51:30.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.373+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f457015cc50 con 0x7f45700aebc0 2024-09-21T07:51:30.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.375+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f45600035f0 con 0x7f45700aebc0 2024-09-21T07:51:30.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.376+0000 7f4566a00640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f454c078cb0 0x7f454c07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.376+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f456009bb10 con 0x7f45700aebc0 2024-09-21T07:51:30.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.376+0000 7f4565400640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f454c078cb0 0x7f454c07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.377+0000 7f4565400640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f454c078cb0 0x7f454c07b1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4558033000 tx=0x7f4558004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:30.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.381+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4560063c90 con 0x7f45700aebc0 2024-09-21T07:51:30.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.566+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f45701f57c0 con 0x7f45700aebc0 2024-09-21T07:51:30.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.567+0000 7f4566a00640 1 -- 172.21.15.88:0/4193645233 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16768 (secure 0 0 0) 0x7f4560063590 con 0x7f45700aebc0 2024-09-21T07:51:30.570 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:30.570 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":104,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:51:29.581152+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:51:27.552445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T07:51:28.544258+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f454c078cb0 msgr2=0x7f454c07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f454c078cb0 0x7f454c07b1c0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f4558033000 tx=0x7f4558004730 comp rx=0 tx=0).stop 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 msgr2=0x7f45701582b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 0x7f45701582b0 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f456000e9f0 tx=0x7f456000eeb0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 shutdown_connections 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f454c078cb0 0x7f454c07b1c0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f457015f910 0x7f45700ca640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45700b2750 0x7f45701517f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 --2- 172.21.15.88:0/4193645233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45700aebc0 0x7f45701582b0 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 >> 172.21.15.88:0/4193645233 conn(0x7f457006d730 msgr2=0x7f457015d900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:30.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 shutdown_connections 2024-09-21T07:51:30.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.570+0000 7f4575000640 1 -- 172.21.15.88:0/4193645233 wait complete. 2024-09-21T07:51:30.599 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:51:30.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 -- 172.21.15.88:0/808942237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 msgr2=0x7f946815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 --2- 172.21.15.88:0/808942237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 0x7f946815d670 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f9460005fd0 tx=0x7f946002aa10 comp rx=0 tx=0).stop 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 -- 172.21.15.88:0/808942237 shutdown_connections 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 --2- 172.21.15.88:0/808942237 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 0x7f946815d670 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 --2- 172.21.15.88:0/808942237 >> v2:172.21.15.175:3300/0 conn(0x7f94680af170 0x7f94680b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 -- 172.21.15.88:0/808942237 >> 172.21.15.88:0/808942237 conn(0x7f94680b1e40 msgr2=0x7f94680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 -- 172.21.15.88:0/808942237 shutdown_connections 2024-09-21T07:51:30.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.817+0000 7f946d000640 1 -- 172.21.15.88:0/808942237 wait complete. 2024-09-21T07:51:30.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.819+0000 7f946d000640 1 Processor -- start 2024-09-21T07:51:30.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.820+0000 7f946d000640 1 -- start start 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f946d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94680ad9c0 0x7f9468155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f946d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 0x7f9468158760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f946d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 0x7f9468158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f946d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f946806f290 con 0x7f94680af170 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f946d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f946815a5e0 con 0x7f94680ad9c0 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f946d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f946806f410 con 0x7f946815a0f0 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f945ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 0x7f9468158760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f945ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 0x7f9468158760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34330/0 (socket says 172.21.15.88:34330) 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.821+0000 7f945ea00640 1 -- 172.21.15.88:0/2228240041 learned_addr learned my addr 172.21.15.88:0/2228240041 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f9467e00640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 0x7f9468158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f9467400640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94680ad9c0 0x7f9468155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945ea00640 1 -- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 msgr2=0x7f9468158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945ea00640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 0x7f9468158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945ea00640 1 -- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94680ad9c0 msgr2=0x7f9468155080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945ea00640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94680ad9c0 0x7f9468155080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945ea00640 1 -- 172.21.15.88:0/2228240041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94681555c0 con 0x7f94680af170 2024-09-21T07:51:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945ea00640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 0x7f9468158760 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f945000e7c0 tx=0x7f945000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:30.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945000ccc0 con 0x7f94680af170 2024-09-21T07:51:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9450010430 con 0x7f94680af170 2024-09-21T07:51:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9450010860 con 0x7f94680af170 2024-09-21T07:51:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.822+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94681570d0 con 0x7f94680af170 2024-09-21T07:51:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.823+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9468157690 con 0x7f94680af170 2024-09-21T07:51:30.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.824+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94680ccac0 con 0x7f94680af170 2024-09-21T07:51:30.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.825+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f9450010a00 con 0x7f94680af170 2024-09-21T07:51:30.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.826+0000 7f945fe00640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9444078cb0 0x7f944407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:30.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.826+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 5709+0+0 (secure 0 0 0) 0x7f945009ba70 con 0x7f94680af170 2024-09-21T07:51:30.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.826+0000 7f9467400640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9444078cb0 0x7f944407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:30.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.827+0000 7f9467400640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9444078cb0 0x7f944407b1c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9458001610 tx=0x7f9458009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:30.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:30.831+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9450063bf0 con 0x7f94680af170 2024-09-21T07:51:31.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.016+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f946806f930 con 0x7f94680af170 2024-09-21T07:51:31.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.018+0000 7f945fe00640 1 -- 172.21.15.88:0/2228240041 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v104) ==== 74+0+16768 (secure 0 0 0) 0x7f94500634f0 con 0x7f94680af170 2024-09-21T07:51:31.020 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:31.020 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":104,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:51:29.581152+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:51:27.552445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T07:51:28.544258+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:51:31.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9444078cb0 msgr2=0x7f944407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:31.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9444078cb0 0x7f944407b1c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f9458001610 tx=0x7f9458009040 comp rx=0 tx=0).stop 2024-09-21T07:51:31.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 msgr2=0x7f9468158760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 0x7f9468158760 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f945000e7c0 tx=0x7f945000ec80 comp rx=0 tx=0).stop 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 shutdown_connections 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9444078cb0 0x7f944407b1c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f946815a0f0 0x7f9468158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94680af170 0x7f9468158760 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 --2- 172.21.15.88:0/2228240041 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f94680ad9c0 0x7f9468155080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 >> 172.21.15.88:0/2228240041 conn(0x7f94680b1e40 msgr2=0x7f94680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 shutdown_connections 2024-09-21T07:51:31.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.020+0000 7f946d000640 1 -- 172.21.15.88:0/2228240041 wait complete. 2024-09-21T07:51:31.045 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:51:31.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 -- 172.21.15.88:0/2895859939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 msgr2=0x7f2e480b2000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 --2- 172.21.15.88:0/2895859939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e480b2000 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009830 tx=0x7f2e3802a510 comp rx=0 tx=0).stop 2024-09-21T07:51:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 -- 172.21.15.88:0/2895859939 shutdown_connections 2024-09-21T07:51:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 --2- 172.21.15.88:0/2895859939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e480b2000 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 -- 172.21.15.88:0/2895859939 >> 172.21.15.88:0/2895859939 conn(0x7f2e480b6640 msgr2=0x7f2e480b2de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 -- 172.21.15.88:0/2895859939 shutdown_connections 2024-09-21T07:51:31.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.270+0000 7f2e47200640 1 -- 172.21.15.88:0/2895859939 wait complete. 2024-09-21T07:51:31.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.272+0000 7f2e47200640 1 Processor -- start 2024-09-21T07:51:31.275 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.272+0000 7f2e47200640 1 -- start start 2024-09-21T07:51:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e47200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e4814f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e47200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e480b2540 0x7f2e48142cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e47200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4815d8e0 0x7f2e48155d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e47200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2e480b0dc0 con 0x7f2e4806d7b0 2024-09-21T07:51:31.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e47200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4806dd00 con 0x7f2e4815d8e0 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e47200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2e480b0f40 con 0x7f2e480b2540 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e45e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e4814f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e45e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e4814f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34356/0 (socket says 172.21.15.88:34356) 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e45e00640 1 -- 172.21.15.88:0/4188767822 learned_addr learned my addr 172.21.15.88:0/4188767822 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e46800640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4815d8e0 0x7f2e48155d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.274+0000 7f2e3fe00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e480b2540 0x7f2e48142cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e45e00640 1 -- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e480b2540 msgr2=0x7f2e48142cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e45e00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e480b2540 0x7f2e48142cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e45e00640 1 -- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4815d8e0 msgr2=0x7f2e48155d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e45e00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4815d8e0 0x7f2e48155d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e45e00640 1 -- 172.21.15.88:0/4188767822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e481f7ed0 con 0x7f2e4806d7b0 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e46800640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4815d8e0 0x7f2e48155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e3fe00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e480b2540 0x7f2e48142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:31.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e45e00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e4814f6b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009950 tx=0x7f2e38033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38031d60 con 0x7f2e4806d7b0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2e38033070 con 0x7f2e4806d7b0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e481f3e40 con 0x7f2e4806d7b0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e380334c0 con 0x7f2e4806d7b0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.275+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e48151d00 con 0x7f2e4806d7b0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.276+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e48151f60 con 0x7f2e4806d7b0 2024-09-21T07:51:31.279 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.278+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f2e38033660 con 0x7f2e4806d7b0 2024-09-21T07:51:31.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.279+0000 7f2e3e000640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2e24078ce0 0x7f2e2407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:31.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.279+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f2e380bc570 con 0x7f2e4806d7b0 2024-09-21T07:51:31.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.279+0000 7f2e3fe00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2e24078ce0 0x7f2e2407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:31.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.280+0000 7f2e3fe00640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2e24078ce0 0x7f2e2407b1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e481f16e0 tx=0x7f2e30008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:31.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.284+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2e3803f030 con 0x7f2e4806d7b0 2024-09-21T07:51:31.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:31.508+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f2e480b2000 con 0x7f2e4806d7b0 2024-09-21T07:51:32.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.114+0000 7f2e3e000640 1 -- 172.21.15.88:0/4188767822 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v69) ==== 118+0+0 (secure 0 0 0) 0x7f2e380bae20 con 0x7f2e4806d7b0 2024-09-21T07:51:32.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2e24078ce0 msgr2=0x7f2e2407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:32.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2e24078ce0 0x7f2e2407b1f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2e481f16e0 tx=0x7f2e30008040 comp rx=0 tx=0).stop 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 msgr2=0x7f2e4814f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e4814f6b0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7f2e38009950 tx=0x7f2e38033d60 comp rx=0 tx=0).stop 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 shutdown_connections 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2e24078ce0 0x7f2e2407b1f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2e480b2540 0x7f2e48142cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2e4815d8e0 0x7f2e48155d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 --2- 172.21.15.88:0/4188767822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e4806d7b0 0x7f2e4814f6b0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 >> 172.21.15.88:0/4188767822 conn(0x7f2e480b6640 msgr2=0x7f2e4815b690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 shutdown_connections 2024-09-21T07:51:32.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.117+0000 7f2e47200640 1 -- 172.21.15.88:0/4188767822 wait complete. 2024-09-21T07:51:32.143 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:51:32.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.366+0000 7f192b200640 1 -- 172.21.15.88:0/2755391931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 msgr2=0x7f192406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.366+0000 7f192b200640 1 --2- 172.21.15.88:0/2755391931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f192406fb40 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7f1924160280 tx=0x7f191402a680 comp rx=0 tx=0).stop 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.366+0000 7f192b200640 1 -- 172.21.15.88:0/2755391931 shutdown_connections 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.366+0000 7f192b200640 1 --2- 172.21.15.88:0/2755391931 >> v2:172.21.15.175:3300/0 conn(0x7f1924145a40 0x7f1924073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.366+0000 7f192b200640 1 --2- 172.21.15.88:0/2755391931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f192406fb40 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.366+0000 7f192b200640 1 -- 172.21.15.88:0/2755391931 >> 172.21.15.88:0/2755391931 conn(0x7f192406d1c0 msgr2=0x7f192406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.367+0000 7f192b200640 1 -- 172.21.15.88:0/2755391931 shutdown_connections 2024-09-21T07:51:32.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.367+0000 7f192b200640 1 -- 172.21.15.88:0/2755391931 wait complete. 2024-09-21T07:51:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 Processor -- start 2024-09-21T07:51:32.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 -- start start 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1924073a80 0x7f192413ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f19241f9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1924145a40 0x7f19241f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1924160a30 con 0x7f1924145550 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19240ce870 con 0x7f1924073a80 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.370+0000 7f192b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1924160bb0 con 0x7f1924145a40 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f19241f9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f19241f9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34382/0 (socket says 172.21.15.88:34382) 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 -- 172.21.15.88:0/1041940212 learned_addr learned my addr 172.21.15.88:0/1041940212 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:32.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923e00640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1924073a80 0x7f192413ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:32.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1929200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1924145a40 0x7f19241f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:32.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 -- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1924145a40 msgr2=0x7f19241f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:32.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1924145a40 0x7f19241f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 -- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1924073a80 msgr2=0x7f192413ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1924073a80 0x7f192413ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 -- 172.21.15.88:0/1041940212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19241fa170 con 0x7f1924145550 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1929200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1924145a40 0x7f19241f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.371+0000 7f1923400640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f19241f9c30 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f191000c970 tx=0x7f191000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.372+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1910007bb0 con 0x7f1924145550 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.372+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1910007e40 con 0x7f1924145550 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.372+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f191000f6b0 con 0x7f1924145550 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.372+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19241f6200 con 0x7f1924145550 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.372+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19241f69f0 con 0x7f1924145550 2024-09-21T07:51:32.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.373+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19241f6480 con 0x7f1924145550 2024-09-21T07:51:32.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.374+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f191000f850 con 0x7f1924145550 2024-09-21T07:51:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.375+0000 7f191be00640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1904078ce0 0x7f190407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.375+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f191009b6e0 con 0x7f1924145550 2024-09-21T07:51:32.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.375+0000 7f1923e00640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1904078ce0 0x7f190407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:32.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.376+0000 7f1923e00640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1904078ce0 0x7f190407b1f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1914033000 tx=0x7f1914004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:32.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.379+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1910014030 con 0x7f1924145550 2024-09-21T07:51:32.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:32.601+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f19241610f0 con 0x7f1924145550 2024-09-21T07:51:33.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.122+0000 7f191be00640 1 -- 172.21.15.88:0/1041940212 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v70) ==== 122+0+0 (secure 0 0 0) 0x7f1910063980 con 0x7f1924145550 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1904078ce0 msgr2=0x7f190407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1904078ce0 0x7f190407b1f0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1914033000 tx=0x7f1914004730 comp rx=0 tx=0).stop 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 msgr2=0x7f19241f9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f19241f9c30 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f191000c970 tx=0x7f191000ce30 comp rx=0 tx=0).stop 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 shutdown_connections 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1904078ce0 0x7f190407b1f0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1924145a40 0x7f19241f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1924145550 0x7f19241f9c30 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 --2- 172.21.15.88:0/1041940212 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1924073a80 0x7f192413ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 >> 172.21.15.88:0/1041940212 conn(0x7f192406d1c0 msgr2=0x7f1924068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 shutdown_connections 2024-09-21T07:51:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.125+0000 7f192b200640 1 -- 172.21.15.88:0/1041940212 wait complete. 2024-09-21T07:51:33.150 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:51:33.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 -- 172.21.15.88:0/507999531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714071130 msgr2=0x7f97140704b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 --2- 172.21.15.88:0/507999531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714071130 0x7f97140704b0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7f9708005fd0 tx=0x7f970802a530 comp rx=0 tx=0).stop 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 -- 172.21.15.88:0/507999531 shutdown_connections 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 --2- 172.21.15.88:0/507999531 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97140ad3c0 0x7f97140ad7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 --2- 172.21.15.88:0/507999531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9714071130 0x7f97140704b0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 -- 172.21.15.88:0/507999531 >> 172.21.15.88:0/507999531 conn(0x7f971406d730 msgr2=0x7f97140b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.372+0000 7f971c200640 1 -- 172.21.15.88:0/507999531 shutdown_connections 2024-09-21T07:51:33.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.373+0000 7f971c200640 1 -- 172.21.15.88:0/507999531 wait complete. 2024-09-21T07:51:33.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.375+0000 7f971c200640 1 Processor -- start 2024-09-21T07:51:33.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.375+0000 7f971c200640 1 -- start start 2024-09-21T07:51:33.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9714071130 0x7f971415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97140ad3c0 0x7f9714153900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 0x7f97140999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97140b4960 con 0x7f97140add10 2024-09-21T07:51:33.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97140b47e0 con 0x7f97140ad3c0 2024-09-21T07:51:33.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f97140b4ae0 con 0x7f9714071130 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.376+0000 7f971a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 0x7f97140999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f9718e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97140ad3c0 0x7f9714153900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 0x7f97140999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34390/0 (socket says 172.21.15.88:34390) 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 -- 172.21.15.88:0/3212061885 learned_addr learned my addr 172.21.15.88:0/3212061885 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f9719800640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9714071130 0x7f971415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 -- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9714071130 msgr2=0x7f971415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9714071130 0x7f971415d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 -- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97140ad3c0 msgr2=0x7f9714153900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97140ad3c0 0x7f9714153900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 -- 172.21.15.88:0/3212061885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9714099ee0 con 0x7f97140add10 2024-09-21T07:51:33.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.377+0000 7f971a200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 0x7f97140999a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b9a0 tx=0x7f970c00be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:33.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.378+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c019070 con 0x7f97140add10 2024-09-21T07:51:33.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.378+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f970c00cc90 con 0x7f97140add10 2024-09-21T07:51:33.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.378+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9714148b50 con 0x7f97140add10 2024-09-21T07:51:33.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.378+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f970c0146a0 con 0x7f97140add10 2024-09-21T07:51:33.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.378+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9714163a50 con 0x7f97140add10 2024-09-21T07:51:33.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.379+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97141f6ac0 con 0x7f97140add10 2024-09-21T07:51:33.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.380+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f970c01c070 con 0x7f97140add10 2024-09-21T07:51:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.381+0000 7f9711600640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f96f4078ce0 0x7f96f407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.381+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f970c09abe0 con 0x7f97140add10 2024-09-21T07:51:33.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.381+0000 7f9719800640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f96f4078ce0 0x7f96f407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.382+0000 7f9719800640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f96f4078ce0 0x7f96f407b1f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9708033000 tx=0x7f9708004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:33.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.385+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9714099ee0 con 0x7f97140add10 2024-09-21T07:51:33.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.605+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97140704b0 con 0x7f97140add10 2024-09-21T07:51:33.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.606+0000 7f9711600640 1 -- 172.21.15.88:0/3212061885 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5225 (secure 0 0 0) 0x7f970c062710 con 0x7f97140add10 2024-09-21T07:51:33.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:33.607 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":70,"btime":"2024-09-21T07:51:33:116260+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:51:29.581113+0000","modified":"2024-09-21T07:51:32.603721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6975":{"gid":6975,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3744288878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3744288878},{"type":"v1","addr":"172.21.15.88:6835","nonce":3744288878}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6975,"qdb_cluster":[6975]},"id":8}]} 2024-09-21T07:51:33.608 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 70 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f96f4078ce0 msgr2=0x7f96f407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f96f4078ce0 0x7f96f407b1f0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9708033000 tx=0x7f9708004730 comp rx=0 tx=0).stop 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 msgr2=0x7f97140999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 0x7f97140999a0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f970c00b9a0 tx=0x7f970c00be60 comp rx=0 tx=0).stop 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 shutdown_connections 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f96f4078ce0 0x7f96f407b1f0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97140add10 0x7f97140999a0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97140ad3c0 0x7f9714153900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 --2- 172.21.15.88:0/3212061885 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9714071130 0x7f971415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.608+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 >> 172.21.15.88:0/3212061885 conn(0x7f971406d730 msgr2=0x7f9714156e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:33.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.609+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 shutdown_connections 2024-09-21T07:51:33.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.609+0000 7f971c200640 1 -- 172.21.15.88:0/3212061885 wait complete. 2024-09-21T07:51:33.631 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:51:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 -- 172.21.15.88:0/2842325862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0d14c0 msgr2=0x7f5aec0d51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 --2- 172.21.15.88:0/2842325862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0d14c0 0x7f5aec0d51a0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f5aec069bd0 tx=0x7f5ad802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 -- 172.21.15.88:0/2842325862 shutdown_connections 2024-09-21T07:51:33.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 --2- 172.21.15.88:0/2842325862 >> v2:172.21.15.175:3300/0 conn(0x7f5aec0d31e0 0x7f5aec0cf290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 --2- 172.21.15.88:0/2842325862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0d14c0 0x7f5aec0d51a0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 -- 172.21.15.88:0/2842325862 >> 172.21.15.88:0/2842325862 conn(0x7f5aec0d4680 msgr2=0x7f5aec0d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 -- 172.21.15.88:0/2842325862 shutdown_connections 2024-09-21T07:51:33.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.852+0000 7f5af3c00640 1 -- 172.21.15.88:0/2842325862 wait complete. 2024-09-21T07:51:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.854+0000 7f5af3c00640 1 Processor -- start 2024-09-21T07:51:33.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.854+0000 7f5af3c00640 1 -- start start 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af3c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 0x7f5aec0b5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af3c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aec0d31e0 0x7f5aec1f6200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af3c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aec0ccb00 0x7f5aec0cd6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af3c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec0cf7d0 con 0x7f5aec0b9b40 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af3c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5aec0b3ce0 con 0x7f5aec0d31e0 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af3c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5aec0cf950 con 0x7f5aec0ccb00 2024-09-21T07:51:33.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 0x7f5aec0b5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 0x7f5aec0b5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34418/0 (socket says 172.21.15.88:34418) 2024-09-21T07:51:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 -- 172.21.15.88:0/1274599984 learned_addr learned my addr 172.21.15.88:0/1274599984 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5aebe00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aec0d31e0 0x7f5aec1f6200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aec0ccb00 0x7f5aec0cd6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 -- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aec0ccb00 msgr2=0x7f5aec0cd6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aec0ccb00 0x7f5aec0cd6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 -- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aec0d31e0 msgr2=0x7f5aec1f6200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aec0d31e0 0x7f5aec1f6200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1200640 1 -- 172.21.15.88:0/1274599984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5aec0cba60 con 0x7f5aec0b9b40 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.856+0000 7f5af1c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aec0ccb00 0x7f5aec0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.857+0000 7f5af1200640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 0x7f5aec0b5580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c3d0 tx=0x7f5ae000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.857+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0017070 con 0x7f5aec0b9b40 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.857+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ae00053c0 con 0x7f5aec0b9b40 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.857+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0003850 con 0x7f5aec0b9b40 2024-09-21T07:51:33.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.857+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5aec0cbd40 con 0x7f5aec0b9b40 2024-09-21T07:51:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.857+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5aec0cc300 con 0x7f5aec0b9b40 2024-09-21T07:51:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.858+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5aec0cfe20 con 0x7f5aec0b9b40 2024-09-21T07:51:33.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.860+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f5ae001a070 con 0x7f5aec0b9b40 2024-09-21T07:51:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.861+0000 7f5ae8c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5acc078cb0 0x7f5acc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.861+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f5ae009a2c0 con 0x7f5aec0b9b40 2024-09-21T07:51:33.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.861+0000 7f5aebe00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5acc078cb0 0x7f5acc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:33.863 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.862+0000 7f5aebe00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5acc078cb0 0x7f5acc07b1c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8033000 tx=0x7f5ad8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:33.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:33.866+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ae0061e90 con 0x7f5aec0b9b40 2024-09-21T07:51:34.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.052+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5aec0d1d30 con 0x7f5aec0b9b40 2024-09-21T07:51:34.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.054+0000 7f5ae8c00640 1 -- 172.21.15.88:0/1274599984 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v105) ==== 74+0+16755 (secure 0 0 0) 0x7f5ae00668c0 con 0x7f5aec0b9b40 2024-09-21T07:51:34.055 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:34.055 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":105,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:51:31.089076+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:51:27.552445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T07:51:28.544258+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"105","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:51:34.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5acc078cb0 msgr2=0x7f5acc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5acc078cb0 0x7f5acc07b1c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f5ad8033000 tx=0x7f5ad8004730 comp rx=0 tx=0).stop 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 msgr2=0x7f5aec0b5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 0x7f5aec0b5580 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000c3d0 tx=0x7f5ae000c890 comp rx=0 tx=0).stop 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 shutdown_connections 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5acc078cb0 0x7f5acc07b1c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5aec0ccb00 0x7f5aec0cd6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5aec0d31e0 0x7f5aec1f6200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 --2- 172.21.15.88:0/1274599984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5aec0b9b40 0x7f5aec0b5580 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 >> 172.21.15.88:0/1274599984 conn(0x7f5aec0d4680 msgr2=0x7f5aec161770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 shutdown_connections 2024-09-21T07:51:34.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.056+0000 7f5af3c00640 1 -- 172.21.15.88:0/1274599984 wait complete. 2024-09-21T07:51:34.082 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:51:34.082 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:51:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 -- 172.21.15.88:0/4151457738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc15fae0 msgr2=0x7f99bc0b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 --2- 172.21.15.88:0/4151457738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc15fae0 0x7f99bc0b3750 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7f99bc072810 tx=0x7f99b402aa30 comp rx=0 tx=0).stop 2024-09-21T07:51:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 -- 172.21.15.88:0/4151457738 shutdown_connections 2024-09-21T07:51:34.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 --2- 172.21.15.88:0/4151457738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc15fae0 0x7f99bc0b3750 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 --2- 172.21.15.88:0/4151457738 >> v2:172.21.15.175:3300/0 conn(0x7f99bc0aebc0 0x7f99bc152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 -- 172.21.15.88:0/4151457738 >> 172.21.15.88:0/4151457738 conn(0x7f99bc06d730 msgr2=0x7f99bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 -- 172.21.15.88:0/4151457738 shutdown_connections 2024-09-21T07:51:34.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.301+0000 7f99c3800640 1 -- 172.21.15.88:0/4151457738 wait complete. 2024-09-21T07:51:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.303+0000 7f99c3800640 1 Processor -- start 2024-09-21T07:51:34.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.303+0000 7f99c3800640 1 -- start start 2024-09-21T07:51:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.304+0000 7f99c3800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 0x7f99bc0b1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.304+0000 7f99c3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99bc0b2750 0x7f99bc1580f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.304+0000 7f99c3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99bc157260 0x7f99bc158630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.304+0000 7f99c3800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc0b40d0 con 0x7f99bc0aebc0 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.304+0000 7f99c3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99bc0b3f50 con 0x7f99bc157260 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.304+0000 7f99c3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99bc0b4250 con 0x7f99bc0b2750 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 0x7f99bc0b1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 0x7f99bc0b1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34430/0 (socket says 172.21.15.88:34430) 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 -- 172.21.15.88:0/3459314158 learned_addr learned my addr 172.21.15.88:0/3459314158 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c1800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99bc157260 0x7f99bc158630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99b3e00640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99bc0b2750 0x7f99bc1580f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 -- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99bc0b2750 msgr2=0x7f99bc1580f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99bc0b2750 0x7f99bc1580f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 -- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99bc157260 msgr2=0x7f99bc158630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99bc157260 0x7f99bc158630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.305+0000 7f99c0e00640 1 -- 172.21.15.88:0/3459314158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f99bc0b21b0 con 0x7f99bc0aebc0 2024-09-21T07:51:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.306+0000 7f99c0e00640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 0x7f99bc0b1860 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00d740 tx=0x7f99ac00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.306+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac017070 con 0x7f99bc0aebc0 2024-09-21T07:51:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.306+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f99ac009d20 con 0x7f99bc0aebc0 2024-09-21T07:51:34.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.306+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99ac0106c0 con 0x7f99bc0aebc0 2024-09-21T07:51:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.306+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99bc1564a0 con 0x7f99bc0aebc0 2024-09-21T07:51:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.306+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99bc156a60 con 0x7f99bc0aebc0 2024-09-21T07:51:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.307+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f99bc0b4770 con 0x7f99bc0aebc0 2024-09-21T07:51:34.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.308+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f99ac01a070 con 0x7f99bc0aebc0 2024-09-21T07:51:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.310+0000 7f99b9600640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9998078ce0 0x7f999807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.310+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f99ac09b000 con 0x7f99bc0aebc0 2024-09-21T07:51:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.310+0000 7f99b3e00640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9998078ce0 0x7f999807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.311+0000 7f99b3e00640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9998078ce0 0x7f999807b1f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000f60 tx=0x7f99a4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:34.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.314+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f99ac062bd0 con 0x7f99bc0aebc0 2024-09-21T07:51:34.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.498+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f99bc069940 con 0x7f9998078ce0 2024-09-21T07:51:34.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.504+0000 7f99b9600640 1 -- 172.21.15.88:0/3459314158 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f99bc069940 con 0x7f9998078ce0 2024-09-21T07:51:34.508 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:34.509 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9998078ce0 msgr2=0x7f999807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9998078ce0 0x7f999807b1f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f99a4000f60 tx=0x7f99a4009040 comp rx=0 tx=0).stop 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 msgr2=0x7f99bc0b1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 0x7f99bc0b1860 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f99ac00d740 tx=0x7f99ac00dc00 comp rx=0 tx=0).stop 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 shutdown_connections 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9998078ce0 0x7f999807b1f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f99bc0b2750 0x7f99bc1580f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f99bc157260 0x7f99bc158630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 --2- 172.21.15.88:0/3459314158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f99bc0aebc0 0x7f99bc0b1860 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 >> 172.21.15.88:0/3459314158 conn(0x7f99bc06d730 msgr2=0x7f99bc15f1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 shutdown_connections 2024-09-21T07:51:34.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.508+0000 7f99c3800640 1 -- 172.21.15.88:0/3459314158 wait complete. 2024-09-21T07:51:34.533 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:51:33.705889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260796,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713924,"statfs":{"total":773094113280,"available":772827058176,"internally_reserved":0,"allocated":11546624,"data_stored":5690120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.628693"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412557+0000","last_change":"2024-09-21T07:51:30.121795+0000","last_active":"2024-09-21T07:51:31.412557+0000","last_peered":"2024-09-21T07:51:31.412557+0000","last_clean":"2024-09-21T07:51:31.412557+0000","last_became_active":"2024-09-21T07:51:30.120674+0000","last_became_peered":"2024-09-21T07:51:30.120674+0000","last_unstale":"2024-09-21T07:51:31.412557+0000","last_undegraded":"2024-09-21T07:51:31.412557+0000","last_fullsized":"2024-09-21T07:51:31.412557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:22.975118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410828+0000","last_change":"2024-09-21T07:51:30.113664+0000","last_active":"2024-09-21T07:51:31.410828+0000","last_peered":"2024-09-21T07:51:31.410828+0000","last_clean":"2024-09-21T07:51:31.410828+0000","last_became_active":"2024-09-21T07:51:30.113366+0000","last_became_peered":"2024-09-21T07:51:30.113366+0000","last_unstale":"2024-09-21T07:51:31.410828+0000","last_undegraded":"2024-09-21T07:51:31.410828+0000","last_fullsized":"2024-09-21T07:51:31.410828+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:53.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094041+0000","last_change":"2024-09-21T07:51:30.104176+0000","last_active":"2024-09-21T07:51:31.094041+0000","last_peered":"2024-09-21T07:51:31.094041+0000","last_clean":"2024-09-21T07:51:31.094041+0000","last_became_active":"2024-09-21T07:51:30.103868+0000","last_became_peered":"2024-09-21T07:51:30.103868+0000","last_unstale":"2024-09-21T07:51:31.094041+0000","last_undegraded":"2024-09-21T07:51:31.094041+0000","last_fullsized":"2024-09-21T07:51:31.094041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:42.764315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.097042+0000","last_change":"2024-09-21T07:51:30.114417+0000","last_active":"2024-09-21T07:51:31.097042+0000","last_peered":"2024-09-21T07:51:31.097042+0000","last_clean":"2024-09-21T07:51:31.097042+0000","last_became_active":"2024-09-21T07:51:30.113302+0000","last_became_peered":"2024-09-21T07:51:30.113302+0000","last_unstale":"2024-09-21T07:51:31.097042+0000","last_undegraded":"2024-09-21T07:51:31.097042+0000","last_fullsized":"2024-09-21T07:51:31.097042+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:43.247321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412012+0000","last_change":"2024-09-21T07:51:30.118323+0000","last_active":"2024-09-21T07:51:31.412012+0000","last_peered":"2024-09-21T07:51:31.412012+0000","last_clean":"2024-09-21T07:51:31.412012+0000","last_became_active":"2024-09-21T07:51:30.117786+0000","last_became_peered":"2024-09-21T07:51:30.117786+0000","last_unstale":"2024-09-21T07:51:31.412012+0000","last_undegraded":"2024-09-21T07:51:31.412012+0000","last_fullsized":"2024-09-21T07:51:31.412012+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:35.412139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095526+0000","last_change":"2024-09-21T07:51:30.112685+0000","last_active":"2024-09-21T07:51:31.095526+0000","last_peered":"2024-09-21T07:51:31.095526+0000","last_clean":"2024-09-21T07:51:31.095526+0000","last_became_active":"2024-09-21T07:51:30.110374+0000","last_became_peered":"2024-09-21T07:51:30.110374+0000","last_unstale":"2024-09-21T07:51:31.095526+0000","last_undegraded":"2024-09-21T07:51:31.095526+0000","last_fullsized":"2024-09-21T07:51:31.095526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.262560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129840+0000","last_change":"2024-09-21T07:51:30.114412+0000","last_active":"2024-09-21T07:51:31.129840+0000","last_peered":"2024-09-21T07:51:31.129840+0000","last_clean":"2024-09-21T07:51:31.129840+0000","last_became_active":"2024-09-21T07:51:30.114100+0000","last_became_peered":"2024-09-21T07:51:30.114100+0000","last_unstale":"2024-09-21T07:51:31.129840+0000","last_undegraded":"2024-09-21T07:51:31.129840+0000","last_fullsized":"2024-09-21T07:51:31.129840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:06.531960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129862+0000","last_change":"2024-09-21T07:51:30.110283+0000","last_active":"2024-09-21T07:51:31.129862+0000","last_peered":"2024-09-21T07:51:31.129862+0000","last_clean":"2024-09-21T07:51:31.129862+0000","last_became_active":"2024-09-21T07:51:30.110016+0000","last_became_peered":"2024-09-21T07:51:30.110016+0000","last_unstale":"2024-09-21T07:51:31.129862+0000","last_undegraded":"2024-09-21T07:51:31.129862+0000","last_fullsized":"2024-09-21T07:51:31.129862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:12.297924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412905+0000","last_change":"2024-09-21T07:51:30.116396+0000","last_active":"2024-09-21T07:51:31.412905+0000","last_peered":"2024-09-21T07:51:31.412905+0000","last_clean":"2024-09-21T07:51:31.412905+0000","last_became_active":"2024-09-21T07:51:30.116039+0000","last_became_peered":"2024-09-21T07:51:30.116039+0000","last_unstale":"2024-09-21T07:51:31.412905+0000","last_undegraded":"2024-09-21T07:51:31.412905+0000","last_fullsized":"2024-09-21T07:51:31.412905+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.903907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412349+0000","last_change":"2024-09-21T07:51:30.119720+0000","last_active":"2024-09-21T07:51:31.412349+0000","last_peered":"2024-09-21T07:51:31.412349+0000","last_clean":"2024-09-21T07:51:31.412349+0000","last_became_active":"2024-09-21T07:51:30.119403+0000","last_became_peered":"2024-09-21T07:51:30.119403+0000","last_unstale":"2024-09-21T07:51:31.412349+0000","last_undegraded":"2024-09-21T07:51:31.412349+0000","last_fullsized":"2024-09-21T07:51:31.412349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:39.078116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130156+0000","last_change":"2024-09-21T07:51:30.124931+0000","last_active":"2024-09-21T07:51:31.130156+0000","last_peered":"2024-09-21T07:51:31.130156+0000","last_clean":"2024-09-21T07:51:31.130156+0000","last_became_active":"2024-09-21T07:51:30.116722+0000","last_became_peered":"2024-09-21T07:51:30.116722+0000","last_unstale":"2024-09-21T07:51:31.130156+0000","last_undegraded":"2024-09-21T07:51:31.130156+0000","last_fullsized":"2024-09-21T07:51:31.130156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:53.736398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412038+0000","last_change":"2024-09-21T07:51:30.125056+0000","last_active":"2024-09-21T07:51:31.412038+0000","last_peered":"2024-09-21T07:51:31.412038+0000","last_clean":"2024-09-21T07:51:31.412038+0000","last_became_active":"2024-09-21T07:51:30.124422+0000","last_became_peered":"2024-09-21T07:51:30.124422+0000","last_unstale":"2024-09-21T07:51:31.412038+0000","last_undegraded":"2024-09-21T07:51:31.412038+0000","last_fullsized":"2024-09-21T07:51:31.412038+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:03.441092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410515+0000","last_change":"2024-09-21T07:51:30.116213+0000","last_active":"2024-09-21T07:51:31.410515+0000","last_peered":"2024-09-21T07:51:31.410515+0000","last_clean":"2024-09-21T07:51:31.410515+0000","last_became_active":"2024-09-21T07:51:30.115872+0000","last_became_peered":"2024-09-21T07:51:30.115872+0000","last_unstale":"2024-09-21T07:51:31.410515+0000","last_undegraded":"2024-09-21T07:51:31.410515+0000","last_fullsized":"2024-09-21T07:51:31.410515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:52.122436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410226+0000","last_change":"2024-09-21T07:51:30.121205+0000","last_active":"2024-09-21T07:51:31.410226+0000","last_peered":"2024-09-21T07:51:31.410226+0000","last_clean":"2024-09-21T07:51:31.410226+0000","last_became_active":"2024-09-21T07:51:30.120946+0000","last_became_peered":"2024-09-21T07:51:30.120946+0000","last_unstale":"2024-09-21T07:51:31.410226+0000","last_undegraded":"2024-09-21T07:51:31.410226+0000","last_fullsized":"2024-09-21T07:51:31.410226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:10.354369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412095+0000","last_change":"2024-09-21T07:51:30.120980+0000","last_active":"2024-09-21T07:51:31.412095+0000","last_peered":"2024-09-21T07:51:31.412095+0000","last_clean":"2024-09-21T07:51:31.412095+0000","last_became_active":"2024-09-21T07:51:30.120633+0000","last_became_peered":"2024-09-21T07:51:30.120633+0000","last_unstale":"2024-09-21T07:51:31.412095+0000","last_undegraded":"2024-09-21T07:51:31.412095+0000","last_fullsized":"2024-09-21T07:51:31.412095+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.977426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411649+0000","last_change":"2024-09-21T07:51:30.118518+0000","last_active":"2024-09-21T07:51:31.411649+0000","last_peered":"2024-09-21T07:51:31.411649+0000","last_clean":"2024-09-21T07:51:31.411649+0000","last_became_active":"2024-09-21T07:51:30.118164+0000","last_became_peered":"2024-09-21T07:51:30.118164+0000","last_unstale":"2024-09-21T07:51:31.411649+0000","last_undegraded":"2024-09-21T07:51:31.411649+0000","last_fullsized":"2024-09-21T07:51:31.411649+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:40.369272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094053+0000","last_change":"2024-09-21T07:51:30.122189+0000","last_active":"2024-09-21T07:51:31.094053+0000","last_peered":"2024-09-21T07:51:31.094053+0000","last_clean":"2024-09-21T07:51:31.094053+0000","last_became_active":"2024-09-21T07:51:30.121875+0000","last_became_peered":"2024-09-21T07:51:30.121875+0000","last_unstale":"2024-09-21T07:51:31.094053+0000","last_undegraded":"2024-09-21T07:51:31.094053+0000","last_fullsized":"2024-09-21T07:51:31.094053+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:33.893765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130324+0000","last_change":"2024-09-21T07:51:30.125832+0000","last_active":"2024-09-21T07:51:31.130324+0000","last_peered":"2024-09-21T07:51:31.130324+0000","last_clean":"2024-09-21T07:51:31.130324+0000","last_became_active":"2024-09-21T07:51:30.124689+0000","last_became_peered":"2024-09-21T07:51:30.124689+0000","last_unstale":"2024-09-21T07:51:31.130324+0000","last_undegraded":"2024-09-21T07:51:31.130324+0000","last_fullsized":"2024-09-21T07:51:31.130324+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:27.012868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094083+0000","last_change":"2024-09-21T07:51:30.113362+0000","last_active":"2024-09-21T07:51:31.094083+0000","last_peered":"2024-09-21T07:51:31.094083+0000","last_clean":"2024-09-21T07:51:31.094083+0000","last_became_active":"2024-09-21T07:51:30.113073+0000","last_became_peered":"2024-09-21T07:51:30.113073+0000","last_unstale":"2024-09-21T07:51:31.094083+0000","last_undegraded":"2024-09-21T07:51:31.094083+0000","last_fullsized":"2024-09-21T07:51:31.094083+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.885890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093874+0000","last_change":"2024-09-21T07:51:30.108842+0000","last_active":"2024-09-21T07:51:31.093874+0000","last_peered":"2024-09-21T07:51:31.093874+0000","last_clean":"2024-09-21T07:51:31.093874+0000","last_became_active":"2024-09-21T07:51:30.108515+0000","last_became_peered":"2024-09-21T07:51:30.108515+0000","last_unstale":"2024-09-21T07:51:31.093874+0000","last_undegraded":"2024-09-21T07:51:31.093874+0000","last_fullsized":"2024-09-21T07:51:31.093874+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.217766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130177+0000","last_change":"2024-09-21T07:51:30.112139+0000","last_active":"2024-09-21T07:51:31.130177+0000","last_peered":"2024-09-21T07:51:31.130177+0000","last_clean":"2024-09-21T07:51:31.130177+0000","last_became_active":"2024-09-21T07:51:30.111837+0000","last_became_peered":"2024-09-21T07:51:30.111837+0000","last_unstale":"2024-09-21T07:51:31.130177+0000","last_undegraded":"2024-09-21T07:51:31.130177+0000","last_fullsized":"2024-09-21T07:51:31.130177+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:34.035739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096617+0000","last_change":"2024-09-21T07:51:30.120017+0000","last_active":"2024-09-21T07:51:31.096617+0000","last_peered":"2024-09-21T07:51:31.096617+0000","last_clean":"2024-09-21T07:51:31.096617+0000","last_became_active":"2024-09-21T07:51:30.114536+0000","last_became_peered":"2024-09-21T07:51:30.114536+0000","last_unstale":"2024-09-21T07:51:31.096617+0000","last_undegraded":"2024-09-21T07:51:31.096617+0000","last_fullsized":"2024-09-21T07:51:31.096617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:55.601527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410300+0000","last_change":"2024-09-21T07:51:30.120808+0000","last_active":"2024-09-21T07:51:31.410300+0000","last_peered":"2024-09-21T07:51:31.410300+0000","last_clean":"2024-09-21T07:51:31.410300+0000","last_became_active":"2024-09-21T07:51:30.120305+0000","last_became_peered":"2024-09-21T07:51:30.120305+0000","last_unstale":"2024-09-21T07:51:31.410300+0000","last_undegraded":"2024-09-21T07:51:31.410300+0000","last_fullsized":"2024-09-21T07:51:31.410300+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:18.990000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095102+0000","last_change":"2024-09-21T07:51:30.121244+0000","last_active":"2024-09-21T07:51:31.095102+0000","last_peered":"2024-09-21T07:51:31.095102+0000","last_clean":"2024-09-21T07:51:31.095102+0000","last_became_active":"2024-09-21T07:51:30.116845+0000","last_became_peered":"2024-09-21T07:51:30.116845+0000","last_unstale":"2024-09-21T07:51:31.095102+0000","last_undegraded":"2024-09-21T07:51:31.095102+0000","last_fullsized":"2024-09-21T07:51:31.095102+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:08.109641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130321+0000","last_change":"2024-09-21T07:51:30.118834+0000","last_active":"2024-09-21T07:51:31.130321+0000","last_peered":"2024-09-21T07:51:31.130321+0000","last_clean":"2024-09-21T07:51:31.130321+0000","last_became_active":"2024-09-21T07:51:30.118535+0000","last_became_peered":"2024-09-21T07:51:30.118535+0000","last_unstale":"2024-09-21T07:51:31.130321+0000","last_undegraded":"2024-09-21T07:51:31.130321+0000","last_fullsized":"2024-09-21T07:51:31.130321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:50.269600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096284+0000","last_change":"2024-09-21T07:51:30.107355+0000","last_active":"2024-09-21T07:51:31.096284+0000","last_peered":"2024-09-21T07:51:31.096284+0000","last_clean":"2024-09-21T07:51:31.096284+0000","last_became_active":"2024-09-21T07:51:30.106468+0000","last_became_peered":"2024-09-21T07:51:30.106468+0000","last_unstale":"2024-09-21T07:51:31.096284+0000","last_undegraded":"2024-09-21T07:51:31.096284+0000","last_fullsized":"2024-09-21T07:51:31.096284+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:21.553538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096840+0000","last_change":"2024-09-21T07:51:30.120090+0000","last_active":"2024-09-21T07:51:31.096840+0000","last_peered":"2024-09-21T07:51:31.096840+0000","last_clean":"2024-09-21T07:51:31.096840+0000","last_became_active":"2024-09-21T07:51:30.119777+0000","last_became_peered":"2024-09-21T07:51:30.119777+0000","last_unstale":"2024-09-21T07:51:31.096840+0000","last_undegraded":"2024-09-21T07:51:31.096840+0000","last_fullsized":"2024-09-21T07:51:31.096840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.425313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096286+0000","last_change":"2024-09-21T07:51:30.114867+0000","last_active":"2024-09-21T07:51:31.096286+0000","last_peered":"2024-09-21T07:51:31.096286+0000","last_clean":"2024-09-21T07:51:31.096286+0000","last_became_active":"2024-09-21T07:51:30.110477+0000","last_became_peered":"2024-09-21T07:51:30.110477+0000","last_unstale":"2024-09-21T07:51:31.096286+0000","last_undegraded":"2024-09-21T07:51:31.096286+0000","last_fullsized":"2024-09-21T07:51:31.096286+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:53.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411339+0000","last_change":"2024-09-21T07:51:30.131859+0000","last_active":"2024-09-21T07:51:31.411339+0000","last_peered":"2024-09-21T07:51:31.411339+0000","last_clean":"2024-09-21T07:51:31.411339+0000","last_became_active":"2024-09-21T07:51:30.130522+0000","last_became_peered":"2024-09-21T07:51:30.130522+0000","last_unstale":"2024-09-21T07:51:31.411339+0000","last_undegraded":"2024-09-21T07:51:31.411339+0000","last_fullsized":"2024-09-21T07:51:31.411339+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.629937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130040+0000","last_change":"2024-09-21T07:51:30.106763+0000","last_active":"2024-09-21T07:51:31.130040+0000","last_peered":"2024-09-21T07:51:31.130040+0000","last_clean":"2024-09-21T07:51:31.130040+0000","last_became_active":"2024-09-21T07:51:30.106444+0000","last_became_peered":"2024-09-21T07:51:30.106444+0000","last_unstale":"2024-09-21T07:51:31.130040+0000","last_undegraded":"2024-09-21T07:51:31.130040+0000","last_fullsized":"2024-09-21T07:51:31.130040+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:29.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411625+0000","last_change":"2024-09-21T07:51:30.115224+0000","last_active":"2024-09-21T07:51:31.411625+0000","last_peered":"2024-09-21T07:51:31.411625+0000","last_clean":"2024-09-21T07:51:31.411625+0000","last_became_active":"2024-09-21T07:51:30.114838+0000","last_became_peered":"2024-09-21T07:51:30.114838+0000","last_unstale":"2024-09-21T07:51:31.411625+0000","last_undegraded":"2024-09-21T07:51:31.411625+0000","last_fullsized":"2024-09-21T07:51:31.411625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:16.220152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411141+0000","last_change":"2024-09-21T07:51:30.125225+0000","last_active":"2024-09-21T07:51:31.411141+0000","last_peered":"2024-09-21T07:51:31.411141+0000","last_clean":"2024-09-21T07:51:31.411141+0000","last_became_active":"2024-09-21T07:51:30.124749+0000","last_became_peered":"2024-09-21T07:51:30.124749+0000","last_unstale":"2024-09-21T07:51:31.411141+0000","last_undegraded":"2024-09-21T07:51:31.411141+0000","last_fullsized":"2024-09-21T07:51:31.411141+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:16.269889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412567+0000","last_change":"2024-09-21T07:51:30.115125+0000","last_active":"2024-09-21T07:51:31.412567+0000","last_peered":"2024-09-21T07:51:31.412567+0000","last_clean":"2024-09-21T07:51:31.412567+0000","last_became_active":"2024-09-21T07:51:30.113440+0000","last_became_peered":"2024-09-21T07:51:30.113440+0000","last_unstale":"2024-09-21T07:51:31.412567+0000","last_undegraded":"2024-09-21T07:51:31.412567+0000","last_fullsized":"2024-09-21T07:51:31.412567+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:32.743165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"104'4","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410970+0000","last_change":"2024-09-21T07:51:29.116867+0000","last_active":"2024-09-21T07:51:31.410970+0000","last_peered":"2024-09-21T07:51:31.410970+0000","last_clean":"2024-09-21T07:51:31.410970+0000","last_became_active":"2024-09-21T07:51:29.116331+0000","last_became_peered":"2024-09-21T07:51:29.116331+0000","last_unstale":"2024-09-21T07:51:31.410970+0000","last_undegraded":"2024-09-21T07:51:31.410970+0000","last_fullsized":"2024-09-21T07:51:31.410970+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.738717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410840+0000","last_change":"2024-09-21T07:51:30.121328+0000","last_active":"2024-09-21T07:51:31.410840+0000","last_peered":"2024-09-21T07:51:31.410840+0000","last_clean":"2024-09-21T07:51:31.410840+0000","last_became_active":"2024-09-21T07:51:30.121035+0000","last_became_peered":"2024-09-21T07:51:30.121035+0000","last_unstale":"2024-09-21T07:51:31.410840+0000","last_undegraded":"2024-09-21T07:51:31.410840+0000","last_fullsized":"2024-09-21T07:51:31.410840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:45.839409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411186+0000","last_change":"2024-09-21T07:51:29.104097+0000","last_active":"2024-09-21T07:51:31.411186+0000","last_peered":"2024-09-21T07:51:31.411186+0000","last_clean":"2024-09-21T07:51:31.411186+0000","last_became_active":"2024-09-21T07:51:29.103769+0000","last_became_peered":"2024-09-21T07:51:29.103769+0000","last_unstale":"2024-09-21T07:51:31.411186+0000","last_undegraded":"2024-09-21T07:51:31.411186+0000","last_fullsized":"2024-09-21T07:51:31.411186+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:44.827104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096755+0000","last_change":"2024-09-21T07:51:30.116641+0000","last_active":"2024-09-21T07:51:31.096755+0000","last_peered":"2024-09-21T07:51:31.096755+0000","last_clean":"2024-09-21T07:51:31.096755+0000","last_became_active":"2024-09-21T07:51:30.116344+0000","last_became_peered":"2024-09-21T07:51:30.116344+0000","last_unstale":"2024-09-21T07:51:31.096755+0000","last_undegraded":"2024-09-21T07:51:31.096755+0000","last_fullsized":"2024-09-21T07:51:31.096755+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:54.128145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096552+0000","last_change":"2024-09-21T07:51:29.111194+0000","last_active":"2024-09-21T07:51:31.096552+0000","last_peered":"2024-09-21T07:51:31.096552+0000","last_clean":"2024-09-21T07:51:31.096552+0000","last_became_active":"2024-09-21T07:51:29.110367+0000","last_became_peered":"2024-09-21T07:51:29.110367+0000","last_unstale":"2024-09-21T07:51:31.096552+0000","last_undegraded":"2024-09-21T07:51:31.096552+0000","last_fullsized":"2024-09-21T07:51:31.096552+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:09.066960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094180+0000","last_change":"2024-09-21T07:51:30.115756+0000","last_active":"2024-09-21T07:51:31.094180+0000","last_peered":"2024-09-21T07:51:31.094180+0000","last_clean":"2024-09-21T07:51:31.094180+0000","last_became_active":"2024-09-21T07:51:30.115317+0000","last_became_peered":"2024-09-21T07:51:30.115317+0000","last_unstale":"2024-09-21T07:51:31.094180+0000","last_undegraded":"2024-09-21T07:51:31.094180+0000","last_fullsized":"2024-09-21T07:51:31.094180+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:56.697822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096684+0000","last_change":"2024-09-21T07:51:29.111266+0000","last_active":"2024-09-21T07:51:31.096684+0000","last_peered":"2024-09-21T07:51:31.096684+0000","last_clean":"2024-09-21T07:51:31.096684+0000","last_became_active":"2024-09-21T07:51:29.110878+0000","last_became_peered":"2024-09-21T07:51:29.110878+0000","last_unstale":"2024-09-21T07:51:31.096684+0000","last_undegraded":"2024-09-21T07:51:31.096684+0000","last_fullsized":"2024-09-21T07:51:31.096684+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:31.139669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095534+0000","last_change":"2024-09-21T07:51:30.125700+0000","last_active":"2024-09-21T07:51:31.095534+0000","last_peered":"2024-09-21T07:51:31.095534+0000","last_clean":"2024-09-21T07:51:31.095534+0000","last_became_active":"2024-09-21T07:51:30.124533+0000","last_became_peered":"2024-09-21T07:51:30.124533+0000","last_unstale":"2024-09-21T07:51:31.095534+0000","last_undegraded":"2024-09-21T07:51:31.095534+0000","last_fullsized":"2024-09-21T07:51:31.095534+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:02.433903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093601+0000","last_change":"2024-09-21T07:51:29.097120+0000","last_active":"2024-09-21T07:51:31.093601+0000","last_peered":"2024-09-21T07:51:31.093601+0000","last_clean":"2024-09-21T07:51:31.093601+0000","last_became_active":"2024-09-21T07:51:29.096790+0000","last_became_peered":"2024-09-21T07:51:29.096790+0000","last_unstale":"2024-09-21T07:51:31.093601+0000","last_undegraded":"2024-09-21T07:51:31.093601+0000","last_fullsized":"2024-09-21T07:51:31.093601+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:00.021303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130112+0000","last_change":"2024-09-21T07:51:30.110308+0000","last_active":"2024-09-21T07:51:31.130112+0000","last_peered":"2024-09-21T07:51:31.130112+0000","last_clean":"2024-09-21T07:51:31.130112+0000","last_became_active":"2024-09-21T07:51:30.110068+0000","last_became_peered":"2024-09-21T07:51:30.110068+0000","last_unstale":"2024-09-21T07:51:31.130112+0000","last_undegraded":"2024-09-21T07:51:31.130112+0000","last_fullsized":"2024-09-21T07:51:31.130112+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:25.798301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411359+0000","last_change":"2024-09-21T07:51:29.107419+0000","last_active":"2024-09-21T07:51:31.411359+0000","last_peered":"2024-09-21T07:51:31.411359+0000","last_clean":"2024-09-21T07:51:31.411359+0000","last_became_active":"2024-09-21T07:51:29.107067+0000","last_became_peered":"2024-09-21T07:51:29.107067+0000","last_unstale":"2024-09-21T07:51:31.411359+0000","last_undegraded":"2024-09-21T07:51:31.411359+0000","last_fullsized":"2024-09-21T07:51:31.411359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:55.685516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411408+0000","last_change":"2024-09-21T07:51:30.115458+0000","last_active":"2024-09-21T07:51:31.411408+0000","last_peered":"2024-09-21T07:51:31.411408+0000","last_clean":"2024-09-21T07:51:31.411408+0000","last_became_active":"2024-09-21T07:51:30.115036+0000","last_became_peered":"2024-09-21T07:51:30.115036+0000","last_unstale":"2024-09-21T07:51:31.411408+0000","last_undegraded":"2024-09-21T07:51:31.411408+0000","last_fullsized":"2024-09-21T07:51:31.411408+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:49.609617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129981+0000","last_change":"2024-09-21T07:51:29.111025+0000","last_active":"2024-09-21T07:51:31.129981+0000","last_peered":"2024-09-21T07:51:31.129981+0000","last_clean":"2024-09-21T07:51:31.129981+0000","last_became_active":"2024-09-21T07:51:29.110724+0000","last_became_peered":"2024-09-21T07:51:29.110724+0000","last_unstale":"2024-09-21T07:51:31.129981+0000","last_undegraded":"2024-09-21T07:51:31.129981+0000","last_fullsized":"2024-09-21T07:51:31.129981+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:34.257512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129883+0000","last_change":"2024-09-21T07:51:30.119307+0000","last_active":"2024-09-21T07:51:31.129883+0000","last_peered":"2024-09-21T07:51:31.129883+0000","last_clean":"2024-09-21T07:51:31.129883+0000","last_became_active":"2024-09-21T07:51:30.118341+0000","last_became_peered":"2024-09-21T07:51:30.118341+0000","last_unstale":"2024-09-21T07:51:31.129883+0000","last_undegraded":"2024-09-21T07:51:31.129883+0000","last_fullsized":"2024-09-21T07:51:31.129883+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:23.513817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411553+0000","last_change":"2024-09-21T07:51:29.117436+0000","last_active":"2024-09-21T07:51:31.411553+0000","last_peered":"2024-09-21T07:51:31.411553+0000","last_clean":"2024-09-21T07:51:31.411553+0000","last_became_active":"2024-09-21T07:51:29.117093+0000","last_became_peered":"2024-09-21T07:51:29.117093+0000","last_unstale":"2024-09-21T07:51:31.411553+0000","last_undegraded":"2024-09-21T07:51:31.411553+0000","last_fullsized":"2024-09-21T07:51:31.411553+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:45.614379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411049+0000","last_change":"2024-09-21T07:51:30.119633+0000","last_active":"2024-09-21T07:51:31.411049+0000","last_peered":"2024-09-21T07:51:31.411049+0000","last_clean":"2024-09-21T07:51:31.411049+0000","last_became_active":"2024-09-21T07:51:30.118450+0000","last_became_peered":"2024-09-21T07:51:30.118450+0000","last_unstale":"2024-09-21T07:51:31.411049+0000","last_undegraded":"2024-09-21T07:51:31.411049+0000","last_fullsized":"2024-09-21T07:51:31.411049+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:41.198532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096916+0000","last_change":"2024-09-21T07:51:29.113044+0000","last_active":"2024-09-21T07:51:31.096916+0000","last_peered":"2024-09-21T07:51:31.096916+0000","last_clean":"2024-09-21T07:51:31.096916+0000","last_became_active":"2024-09-21T07:51:29.112737+0000","last_became_peered":"2024-09-21T07:51:29.112737+0000","last_unstale":"2024-09-21T07:51:31.096916+0000","last_undegraded":"2024-09-21T07:51:31.096916+0000","last_fullsized":"2024-09-21T07:51:31.096916+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:31.730860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410950+0000","last_change":"2024-09-21T07:51:30.116235+0000","last_active":"2024-09-21T07:51:31.410950+0000","last_peered":"2024-09-21T07:51:31.410950+0000","last_clean":"2024-09-21T07:51:31.410950+0000","last_became_active":"2024-09-21T07:51:30.115825+0000","last_became_peered":"2024-09-21T07:51:30.115825+0000","last_unstale":"2024-09-21T07:51:31.410950+0000","last_undegraded":"2024-09-21T07:51:31.410950+0000","last_fullsized":"2024-09-21T07:51:31.410950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:26.424322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412206+0000","last_change":"2024-09-21T07:51:29.117836+0000","last_active":"2024-09-21T07:51:31.412206+0000","last_peered":"2024-09-21T07:51:31.412206+0000","last_clean":"2024-09-21T07:51:31.412206+0000","last_became_active":"2024-09-21T07:51:29.117017+0000","last_became_peered":"2024-09-21T07:51:29.117017+0000","last_unstale":"2024-09-21T07:51:31.412206+0000","last_undegraded":"2024-09-21T07:51:31.412206+0000","last_fullsized":"2024-09-21T07:51:31.412206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:33.574734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096816+0000","last_change":"2024-09-21T07:51:30.121035+0000","last_active":"2024-09-21T07:51:31.096816+0000","last_peered":"2024-09-21T07:51:31.096816+0000","last_clean":"2024-09-21T07:51:31.096816+0000","last_became_active":"2024-09-21T07:51:30.120689+0000","last_became_peered":"2024-09-21T07:51:30.120689+0000","last_unstale":"2024-09-21T07:51:31.096816+0000","last_undegraded":"2024-09-21T07:51:31.096816+0000","last_fullsized":"2024-09-21T07:51:31.096816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:52.651103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411804+0000","last_change":"2024-09-21T07:51:29.114736+0000","last_active":"2024-09-21T07:51:31.411804+0000","last_peered":"2024-09-21T07:51:31.411804+0000","last_clean":"2024-09-21T07:51:31.411804+0000","last_became_active":"2024-09-21T07:51:29.114297+0000","last_became_peered":"2024-09-21T07:51:29.114297+0000","last_unstale":"2024-09-21T07:51:31.411804+0000","last_undegraded":"2024-09-21T07:51:31.411804+0000","last_fullsized":"2024-09-21T07:51:31.411804+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:11.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130242+0000","last_change":"2024-09-21T07:51:30.110596+0000","last_active":"2024-09-21T07:51:31.130242+0000","last_peered":"2024-09-21T07:51:31.130242+0000","last_clean":"2024-09-21T07:51:31.130242+0000","last_became_active":"2024-09-21T07:51:30.107904+0000","last_became_peered":"2024-09-21T07:51:30.107904+0000","last_unstale":"2024-09-21T07:51:31.130242+0000","last_undegraded":"2024-09-21T07:51:31.130242+0000","last_fullsized":"2024-09-21T07:51:31.130242+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:47.538159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096819+0000","last_change":"2024-09-21T07:51:29.113787+0000","last_active":"2024-09-21T07:51:31.096819+0000","last_peered":"2024-09-21T07:51:31.096819+0000","last_clean":"2024-09-21T07:51:31.096819+0000","last_became_active":"2024-09-21T07:51:29.113295+0000","last_became_peered":"2024-09-21T07:51:29.113295+0000","last_unstale":"2024-09-21T07:51:31.096819+0000","last_undegraded":"2024-09-21T07:51:31.096819+0000","last_fullsized":"2024-09-21T07:51:31.096819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:42.156091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411178+0000","last_change":"2024-09-21T07:51:30.118654+0000","last_active":"2024-09-21T07:51:31.411178+0000","last_peered":"2024-09-21T07:51:31.411178+0000","last_clean":"2024-09-21T07:51:31.411178+0000","last_became_active":"2024-09-21T07:51:30.118067+0000","last_became_peered":"2024-09-21T07:51:30.118067+0000","last_unstale":"2024-09-21T07:51:31.411178+0000","last_undegraded":"2024-09-21T07:51:31.411178+0000","last_fullsized":"2024-09-21T07:51:31.411178+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:00.790432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411923+0000","last_change":"2024-09-21T07:51:29.114979+0000","last_active":"2024-09-21T07:51:31.411923+0000","last_peered":"2024-09-21T07:51:31.411923+0000","last_clean":"2024-09-21T07:51:31.411923+0000","last_became_active":"2024-09-21T07:51:29.114376+0000","last_became_peered":"2024-09-21T07:51:29.114376+0000","last_unstale":"2024-09-21T07:51:31.411923+0000","last_undegraded":"2024-09-21T07:51:31.411923+0000","last_fullsized":"2024-09-21T07:51:31.411923+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:41.295424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096167+0000","last_change":"2024-09-21T07:51:30.123712+0000","last_active":"2024-09-21T07:51:31.096167+0000","last_peered":"2024-09-21T07:51:31.096167+0000","last_clean":"2024-09-21T07:51:31.096167+0000","last_became_active":"2024-09-21T07:51:30.123396+0000","last_became_peered":"2024-09-21T07:51:30.123396+0000","last_unstale":"2024-09-21T07:51:31.096167+0000","last_undegraded":"2024-09-21T07:51:31.096167+0000","last_fullsized":"2024-09-21T07:51:31.096167+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:36.255258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093687+0000","last_change":"2024-09-21T07:51:29.100662+0000","last_active":"2024-09-21T07:51:31.093687+0000","last_peered":"2024-09-21T07:51:31.093687+0000","last_clean":"2024-09-21T07:51:31.093687+0000","last_became_active":"2024-09-21T07:51:29.100353+0000","last_became_peered":"2024-09-21T07:51:29.100353+0000","last_unstale":"2024-09-21T07:51:31.093687+0000","last_undegraded":"2024-09-21T07:51:31.093687+0000","last_fullsized":"2024-09-21T07:51:31.093687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:28.950459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096237+0000","last_change":"2024-09-21T07:51:30.107089+0000","last_active":"2024-09-21T07:51:31.096237+0000","last_peered":"2024-09-21T07:51:31.096237+0000","last_clean":"2024-09-21T07:51:31.096237+0000","last_became_active":"2024-09-21T07:51:30.106714+0000","last_became_peered":"2024-09-21T07:51:30.106714+0000","last_unstale":"2024-09-21T07:51:31.096237+0000","last_undegraded":"2024-09-21T07:51:31.096237+0000","last_fullsized":"2024-09-21T07:51:31.096237+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:57.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411336+0000","last_change":"2024-09-21T07:51:29.105733+0000","last_active":"2024-09-21T07:51:31.411336+0000","last_peered":"2024-09-21T07:51:31.411336+0000","last_clean":"2024-09-21T07:51:31.411336+0000","last_became_active":"2024-09-21T07:51:29.103945+0000","last_became_peered":"2024-09-21T07:51:29.103945+0000","last_unstale":"2024-09-21T07:51:31.411336+0000","last_undegraded":"2024-09-21T07:51:31.411336+0000","last_fullsized":"2024-09-21T07:51:31.411336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:42.867682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412614+0000","last_change":"2024-09-21T07:51:30.116548+0000","last_active":"2024-09-21T07:51:31.412614+0000","last_peered":"2024-09-21T07:51:31.412614+0000","last_clean":"2024-09-21T07:51:31.412614+0000","last_became_active":"2024-09-21T07:51:30.116170+0000","last_became_peered":"2024-09-21T07:51:30.116170+0000","last_unstale":"2024-09-21T07:51:31.412614+0000","last_undegraded":"2024-09-21T07:51:31.412614+0000","last_fullsized":"2024-09-21T07:51:31.412614+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:52.122436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095937+0000","last_change":"2024-09-21T07:51:29.106076+0000","last_active":"2024-09-21T07:51:31.095937+0000","last_peered":"2024-09-21T07:51:31.095937+0000","last_clean":"2024-09-21T07:51:31.095937+0000","last_became_active":"2024-09-21T07:51:29.104518+0000","last_became_peered":"2024-09-21T07:51:29.104518+0000","last_unstale":"2024-09-21T07:51:31.095937+0000","last_undegraded":"2024-09-21T07:51:31.095937+0000","last_fullsized":"2024-09-21T07:51:31.095937+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:11.508457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412897+0000","last_change":"2024-09-21T07:51:30.116309+0000","last_active":"2024-09-21T07:51:31.412897+0000","last_peered":"2024-09-21T07:51:31.412897+0000","last_clean":"2024-09-21T07:51:31.412897+0000","last_became_active":"2024-09-21T07:51:30.115962+0000","last_became_peered":"2024-09-21T07:51:30.115962+0000","last_unstale":"2024-09-21T07:51:31.412897+0000","last_undegraded":"2024-09-21T07:51:31.412897+0000","last_fullsized":"2024-09-21T07:51:31.412897+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:53.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412300+0000","last_change":"2024-09-21T07:51:29.116935+0000","last_active":"2024-09-21T07:51:31.412300+0000","last_peered":"2024-09-21T07:51:31.412300+0000","last_clean":"2024-09-21T07:51:31.412300+0000","last_became_active":"2024-09-21T07:51:29.116549+0000","last_became_peered":"2024-09-21T07:51:29.116549+0000","last_unstale":"2024-09-21T07:51:31.412300+0000","last_undegraded":"2024-09-21T07:51:31.412300+0000","last_fullsized":"2024-09-21T07:51:31.412300+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:12.969327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130444+0000","last_change":"2024-09-21T07:51:30.120480+0000","last_active":"2024-09-21T07:51:31.130444+0000","last_peered":"2024-09-21T07:51:31.130444+0000","last_clean":"2024-09-21T07:51:31.130444+0000","last_became_active":"2024-09-21T07:51:30.120192+0000","last_became_peered":"2024-09-21T07:51:30.120192+0000","last_unstale":"2024-09-21T07:51:31.130444+0000","last_undegraded":"2024-09-21T07:51:31.130444+0000","last_fullsized":"2024-09-21T07:51:31.130444+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:07.055052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411059+0000","last_change":"2024-09-21T07:51:29.105588+0000","last_active":"2024-09-21T07:51:31.411059+0000","last_peered":"2024-09-21T07:51:31.411059+0000","last_clean":"2024-09-21T07:51:31.411059+0000","last_became_active":"2024-09-21T07:51:29.105303+0000","last_became_peered":"2024-09-21T07:51:29.105303+0000","last_unstale":"2024-09-21T07:51:31.411059+0000","last_undegraded":"2024-09-21T07:51:31.411059+0000","last_fullsized":"2024-09-21T07:51:31.411059+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:49.862179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411287+0000","last_change":"2024-09-21T07:51:30.119659+0000","last_active":"2024-09-21T07:51:31.411287+0000","last_peered":"2024-09-21T07:51:31.411287+0000","last_clean":"2024-09-21T07:51:31.411287+0000","last_became_active":"2024-09-21T07:51:30.119304+0000","last_became_peered":"2024-09-21T07:51:30.119304+0000","last_unstale":"2024-09-21T07:51:31.411287+0000","last_undegraded":"2024-09-21T07:51:31.411287+0000","last_fullsized":"2024-09-21T07:51:31.411287+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:22.975118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410501+0000","last_change":"2024-09-21T07:51:29.112319+0000","last_active":"2024-09-21T07:51:31.410501+0000","last_peered":"2024-09-21T07:51:31.410501+0000","last_clean":"2024-09-21T07:51:31.410501+0000","last_became_active":"2024-09-21T07:51:29.112055+0000","last_became_peered":"2024-09-21T07:51:29.112055+0000","last_unstale":"2024-09-21T07:51:31.410501+0000","last_undegraded":"2024-09-21T07:51:31.410501+0000","last_fullsized":"2024-09-21T07:51:31.410501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:53.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093555+0000","last_change":"2024-09-21T07:51:30.104623+0000","last_active":"2024-09-21T07:51:31.093555+0000","last_peered":"2024-09-21T07:51:31.093555+0000","last_clean":"2024-09-21T07:51:31.093555+0000","last_became_active":"2024-09-21T07:51:30.102939+0000","last_became_peered":"2024-09-21T07:51:30.102939+0000","last_unstale":"2024-09-21T07:51:31.093555+0000","last_undegraded":"2024-09-21T07:51:31.093555+0000","last_fullsized":"2024-09-21T07:51:31.093555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:44.051383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412068+0000","last_change":"2024-09-21T07:51:29.116684+0000","last_active":"2024-09-21T07:51:31.412068+0000","last_peered":"2024-09-21T07:51:31.412068+0000","last_clean":"2024-09-21T07:51:31.412068+0000","last_became_active":"2024-09-21T07:51:29.116351+0000","last_became_peered":"2024-09-21T07:51:29.116351+0000","last_unstale":"2024-09-21T07:51:31.412068+0000","last_undegraded":"2024-09-21T07:51:31.412068+0000","last_fullsized":"2024-09-21T07:51:31.412068+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:48.485754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411964+0000","last_change":"2024-09-21T07:51:30.122458+0000","last_active":"2024-09-21T07:51:31.411964+0000","last_peered":"2024-09-21T07:51:31.411964+0000","last_clean":"2024-09-21T07:51:31.411964+0000","last_became_active":"2024-09-21T07:51:30.121949+0000","last_became_peered":"2024-09-21T07:51:30.121949+0000","last_unstale":"2024-09-21T07:51:31.411964+0000","last_undegraded":"2024-09-21T07:51:31.411964+0000","last_fullsized":"2024-09-21T07:51:31.411964+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.481048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412011+0000","last_change":"2024-09-21T07:51:29.106803+0000","last_active":"2024-09-21T07:51:31.412011+0000","last_peered":"2024-09-21T07:51:31.412011+0000","last_clean":"2024-09-21T07:51:31.412011+0000","last_became_active":"2024-09-21T07:51:29.106426+0000","last_became_peered":"2024-09-21T07:51:29.106426+0000","last_unstale":"2024-09-21T07:51:31.412011+0000","last_undegraded":"2024-09-21T07:51:31.412011+0000","last_fullsized":"2024-09-21T07:51:31.412011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:32.881460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.409983+0000","last_change":"2024-09-21T07:51:30.122034+0000","last_active":"2024-09-21T07:51:31.409983+0000","last_peered":"2024-09-21T07:51:31.409983+0000","last_clean":"2024-09-21T07:51:31.409983+0000","last_became_active":"2024-09-21T07:51:30.121694+0000","last_became_peered":"2024-09-21T07:51:30.121694+0000","last_unstale":"2024-09-21T07:51:31.409983+0000","last_undegraded":"2024-09-21T07:51:31.409983+0000","last_fullsized":"2024-09-21T07:51:31.409983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:55.119810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096404+0000","last_change":"2024-09-21T07:51:29.105500+0000","last_active":"2024-09-21T07:51:31.096404+0000","last_peered":"2024-09-21T07:51:31.096404+0000","last_clean":"2024-09-21T07:51:31.096404+0000","last_became_active":"2024-09-21T07:51:29.105156+0000","last_became_peered":"2024-09-21T07:51:29.105156+0000","last_unstale":"2024-09-21T07:51:31.096404+0000","last_undegraded":"2024-09-21T07:51:31.096404+0000","last_fullsized":"2024-09-21T07:51:31.096404+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:22.377753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411221+0000","last_change":"2024-09-21T07:51:30.117790+0000","last_active":"2024-09-21T07:51:31.411221+0000","last_peered":"2024-09-21T07:51:31.411221+0000","last_clean":"2024-09-21T07:51:31.411221+0000","last_became_active":"2024-09-21T07:51:30.117479+0000","last_became_peered":"2024-09-21T07:51:30.117479+0000","last_unstale":"2024-09-21T07:51:31.411221+0000","last_undegraded":"2024-09-21T07:51:31.411221+0000","last_fullsized":"2024-09-21T07:51:31.411221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:23.126201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095276+0000","last_change":"2024-09-21T07:51:29.104861+0000","last_active":"2024-09-21T07:51:31.095276+0000","last_peered":"2024-09-21T07:51:31.095276+0000","last_clean":"2024-09-21T07:51:31.095276+0000","last_became_active":"2024-09-21T07:51:29.103746+0000","last_became_peered":"2024-09-21T07:51:29.103746+0000","last_unstale":"2024-09-21T07:51:31.095276+0000","last_undegraded":"2024-09-21T07:51:31.095276+0000","last_fullsized":"2024-09-21T07:51:31.095276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:54.367561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411055+0000","last_change":"2024-09-21T07:51:30.122163+0000","last_active":"2024-09-21T07:51:31.411055+0000","last_peered":"2024-09-21T07:51:31.411055+0000","last_clean":"2024-09-21T07:51:31.411055+0000","last_became_active":"2024-09-21T07:51:30.121830+0000","last_became_peered":"2024-09-21T07:51:30.121830+0000","last_unstale":"2024-09-21T07:51:31.411055+0000","last_undegraded":"2024-09-21T07:51:31.411055+0000","last_fullsized":"2024-09-21T07:51:31.411055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.977426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096995+0000","last_change":"2024-09-21T07:51:29.115728+0000","last_active":"2024-09-21T07:51:31.096995+0000","last_peered":"2024-09-21T07:51:31.096995+0000","last_clean":"2024-09-21T07:51:31.096995+0000","last_became_active":"2024-09-21T07:51:29.115280+0000","last_became_peered":"2024-09-21T07:51:29.115280+0000","last_unstale":"2024-09-21T07:51:31.096995+0000","last_undegraded":"2024-09-21T07:51:31.096995+0000","last_fullsized":"2024-09-21T07:51:31.096995+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:30.814504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096055+0000","last_change":"2024-09-21T07:51:30.121219+0000","last_active":"2024-09-21T07:51:31.096055+0000","last_peered":"2024-09-21T07:51:31.096055+0000","last_clean":"2024-09-21T07:51:31.096055+0000","last_became_active":"2024-09-21T07:51:30.120922+0000","last_became_peered":"2024-09-21T07:51:30.120922+0000","last_unstale":"2024-09-21T07:51:31.096055+0000","last_undegraded":"2024-09-21T07:51:31.096055+0000","last_fullsized":"2024-09-21T07:51:31.096055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:27.907841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"105'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.132000+0000","last_change":"2024-09-21T07:51:29.103439+0000","last_active":"2024-09-21T07:51:31.132000+0000","last_peered":"2024-09-21T07:51:31.132000+0000","last_clean":"2024-09-21T07:51:31.132000+0000","last_became_active":"2024-09-21T07:51:29.103106+0000","last_became_peered":"2024-09-21T07:51:29.103106+0000","last_unstale":"2024-09-21T07:51:31.132000+0000","last_undegraded":"2024-09-21T07:51:31.132000+0000","last_fullsized":"2024-09-21T07:51:31.132000+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:42.329445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411630+0000","last_change":"2024-09-21T07:51:30.121561+0000","last_active":"2024-09-21T07:51:31.411630+0000","last_peered":"2024-09-21T07:51:31.411630+0000","last_clean":"2024-09-21T07:51:31.411630+0000","last_became_active":"2024-09-21T07:51:30.120944+0000","last_became_peered":"2024-09-21T07:51:30.120944+0000","last_unstale":"2024-09-21T07:51:31.411630+0000","last_undegraded":"2024-09-21T07:51:31.411630+0000","last_fullsized":"2024-09-21T07:51:31.411630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:30.664143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096673+0000","last_change":"2024-09-21T07:51:29.105325+0000","last_active":"2024-09-21T07:51:31.096673+0000","last_peered":"2024-09-21T07:51:31.096673+0000","last_clean":"2024-09-21T07:51:31.096673+0000","last_became_active":"2024-09-21T07:51:29.104965+0000","last_became_peered":"2024-09-21T07:51:29.104965+0000","last_unstale":"2024-09-21T07:51:31.096673+0000","last_undegraded":"2024-09-21T07:51:31.096673+0000","last_fullsized":"2024-09-21T07:51:31.096673+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:29.651837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410933+0000","last_change":"2024-09-21T07:51:30.128743+0000","last_active":"2024-09-21T07:51:31.410933+0000","last_peered":"2024-09-21T07:51:31.410933+0000","last_clean":"2024-09-21T07:51:31.410933+0000","last_became_active":"2024-09-21T07:51:30.127622+0000","last_became_peered":"2024-09-21T07:51:30.127622+0000","last_unstale":"2024-09-21T07:51:31.410933+0000","last_undegraded":"2024-09-21T07:51:31.410933+0000","last_fullsized":"2024-09-21T07:51:31.410933+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.481048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096944+0000","last_change":"2024-09-21T07:51:29.104567+0000","last_active":"2024-09-21T07:51:31.096944+0000","last_peered":"2024-09-21T07:51:31.096944+0000","last_clean":"2024-09-21T07:51:31.096944+0000","last_became_active":"2024-09-21T07:51:29.104275+0000","last_became_peered":"2024-09-21T07:51:29.104275+0000","last_unstale":"2024-09-21T07:51:31.096944+0000","last_undegraded":"2024-09-21T07:51:31.096944+0000","last_fullsized":"2024-09-21T07:51:31.096944+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:54.341891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093584+0000","last_change":"2024-09-21T07:51:30.101101+0000","last_active":"2024-09-21T07:51:31.093584+0000","last_peered":"2024-09-21T07:51:31.093584+0000","last_clean":"2024-09-21T07:51:31.093584+0000","last_became_active":"2024-09-21T07:51:30.100761+0000","last_became_peered":"2024-09-21T07:51:30.100761+0000","last_unstale":"2024-09-21T07:51:31.093584+0000","last_undegraded":"2024-09-21T07:51:31.093584+0000","last_fullsized":"2024-09-21T07:51:31.093584+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:24.676842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130240+0000","last_change":"2024-09-21T07:51:29.108009+0000","last_active":"2024-09-21T07:51:31.130240+0000","last_peered":"2024-09-21T07:51:31.130240+0000","last_clean":"2024-09-21T07:51:31.130240+0000","last_became_active":"2024-09-21T07:51:29.107149+0000","last_became_peered":"2024-09-21T07:51:29.107149+0000","last_unstale":"2024-09-21T07:51:31.130240+0000","last_undegraded":"2024-09-21T07:51:31.130240+0000","last_fullsized":"2024-09-21T07:51:31.130240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:42.765582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410666+0000","last_change":"2024-09-21T07:51:30.123553+0000","last_active":"2024-09-21T07:51:31.410666+0000","last_peered":"2024-09-21T07:51:31.410666+0000","last_clean":"2024-09-21T07:51:31.410666+0000","last_became_active":"2024-09-21T07:51:30.123262+0000","last_became_peered":"2024-09-21T07:51:30.123262+0000","last_unstale":"2024-09-21T07:51:31.410666+0000","last_undegraded":"2024-09-21T07:51:31.410666+0000","last_fullsized":"2024-09-21T07:51:31.410666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:16.220152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410338+0000","last_change":"2024-09-21T07:51:29.109165+0000","last_active":"2024-09-21T07:51:31.410338+0000","last_peered":"2024-09-21T07:51:31.410338+0000","last_clean":"2024-09-21T07:51:31.410338+0000","last_became_active":"2024-09-21T07:51:29.108857+0000","last_became_peered":"2024-09-21T07:51:29.108857+0000","last_unstale":"2024-09-21T07:51:31.410338+0000","last_undegraded":"2024-09-21T07:51:31.410338+0000","last_fullsized":"2024-09-21T07:51:31.410338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:56.334823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410057+0000","last_change":"2024-09-21T07:51:30.120627+0000","last_active":"2024-09-21T07:51:31.410057+0000","last_peered":"2024-09-21T07:51:31.410057+0000","last_clean":"2024-09-21T07:51:31.410057+0000","last_became_active":"2024-09-21T07:51:30.120108+0000","last_became_peered":"2024-09-21T07:51:30.120108+0000","last_unstale":"2024-09-21T07:51:31.410057+0000","last_undegraded":"2024-09-21T07:51:31.410057+0000","last_fullsized":"2024-09-21T07:51:31.410057+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:44.170781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411842+0000","last_change":"2024-09-21T07:51:29.105332+0000","last_active":"2024-09-21T07:51:31.411842+0000","last_peered":"2024-09-21T07:51:31.411842+0000","last_clean":"2024-09-21T07:51:31.411842+0000","last_became_active":"2024-09-21T07:51:29.104821+0000","last_became_peered":"2024-09-21T07:51:29.104821+0000","last_unstale":"2024-09-21T07:51:31.411842+0000","last_undegraded":"2024-09-21T07:51:31.411842+0000","last_fullsized":"2024-09-21T07:51:31.411842+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:06.873585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095039+0000","last_change":"2024-09-21T07:51:30.125953+0000","last_active":"2024-09-21T07:51:31.095039+0000","last_peered":"2024-09-21T07:51:31.095039+0000","last_clean":"2024-09-21T07:51:31.095039+0000","last_became_active":"2024-09-21T07:51:30.125506+0000","last_became_peered":"2024-09-21T07:51:30.125506+0000","last_unstale":"2024-09-21T07:51:31.095039+0000","last_undegraded":"2024-09-21T07:51:31.095039+0000","last_fullsized":"2024-09-21T07:51:31.095039+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:04.374254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094998+0000","last_change":"2024-09-21T07:51:29.107130+0000","last_active":"2024-09-21T07:51:31.094998+0000","last_peered":"2024-09-21T07:51:31.094998+0000","last_clean":"2024-09-21T07:51:31.094998+0000","last_became_active":"2024-09-21T07:51:29.106778+0000","last_became_peered":"2024-09-21T07:51:29.106778+0000","last_unstale":"2024-09-21T07:51:31.094998+0000","last_undegraded":"2024-09-21T07:51:31.094998+0000","last_fullsized":"2024-09-21T07:51:31.094998+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:55.062920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095499+0000","last_change":"2024-09-21T07:51:30.109723+0000","last_active":"2024-09-21T07:51:31.095499+0000","last_peered":"2024-09-21T07:51:31.095499+0000","last_clean":"2024-09-21T07:51:31.095499+0000","last_became_active":"2024-09-21T07:51:30.109405+0000","last_became_peered":"2024-09-21T07:51:30.109405+0000","last_unstale":"2024-09-21T07:51:31.095499+0000","last_undegraded":"2024-09-21T07:51:31.095499+0000","last_fullsized":"2024-09-21T07:51:31.095499+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:37.352715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410317+0000","last_change":"2024-09-21T07:51:29.102933+0000","last_active":"2024-09-21T07:51:31.410317+0000","last_peered":"2024-09-21T07:51:31.410317+0000","last_clean":"2024-09-21T07:51:31.410317+0000","last_became_active":"2024-09-21T07:51:29.102705+0000","last_became_peered":"2024-09-21T07:51:29.102705+0000","last_unstale":"2024-09-21T07:51:31.410317+0000","last_undegraded":"2024-09-21T07:51:31.410317+0000","last_fullsized":"2024-09-21T07:51:31.410317+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:15.207847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":296,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093328+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:51:31.093328+0000","last_peered":"2024-09-21T07:51:31.093328+0000","last_clean":"2024-09-21T07:51:31.093328+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:51:31.093328+0000","last_undegraded":"2024-09-21T07:51:31.093328+0000","last_fullsized":"2024-09-21T07:51:31.093328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542215,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1302528,"data_stored":562444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":560708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":1157468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1310720,"data_stored":568710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476809,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":560708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476809,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1294336,"data_stored":561346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1875968,"data_stored":1151076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476806,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1302528,"data_stored":567660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:51:34.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 -- 172.21.15.88:0/967217705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 msgr2=0x7f49c8152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 --2- 172.21.15.88:0/967217705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c8152920 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7f49b8005fd0 tx=0x7f49b802a540 comp rx=0 tx=0).stop 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 -- 172.21.15.88:0/967217705 shutdown_connections 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 --2- 172.21.15.88:0/967217705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c80b2750 0x7f49c815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 --2- 172.21.15.88:0/967217705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c8152920 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 -- 172.21.15.88:0/967217705 >> 172.21.15.88:0/967217705 conn(0x7f49c806d730 msgr2=0x7f49c80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 -- 172.21.15.88:0/967217705 shutdown_connections 2024-09-21T07:51:34.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.790+0000 7f49cd800640 1 -- 172.21.15.88:0/967217705 wait complete. 2024-09-21T07:51:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.793+0000 7f49cd800640 1 Processor -- start 2024-09-21T07:51:34.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.793+0000 7f49cd800640 1 -- start start 2024-09-21T07:51:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49cd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c814b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49cd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c80b2750 0x7f49c80b0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49cd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815f910 0x7f49c80ca7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49cd800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49c815c4a0 con 0x7f49c80aebc0 2024-09-21T07:51:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49cd800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49c80b4a20 con 0x7f49c80b2750 2024-09-21T07:51:34.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49cd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49c815c620 con 0x7f49c815f910 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.794+0000 7f49c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c814b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c814b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34458/0 (socket says 172.21.15.88:34458) 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 -- 172.21.15.88:0/232046257 learned_addr learned my addr 172.21.15.88:0/232046257 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6000640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c80b2750 0x7f49c80b0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c7400640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815f910 0x7f49c80ca7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 -- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815f910 msgr2=0x7f49c80ca7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815f910 0x7f49c80ca7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 -- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c80b2750 msgr2=0x7f49c80b0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c80b2750 0x7f49c80b0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 -- 172.21.15.88:0/232046257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49c80b1f00 con 0x7f49c80aebc0 2024-09-21T07:51:34.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49c6a00640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c814b7b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000c00 tx=0x7f49b8003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.795+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b803e030 con 0x7f49c80aebc0 2024-09-21T07:51:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.796+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49b8003e60 con 0x7f49c80aebc0 2024-09-21T07:51:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.796+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49c80b1c20 con 0x7f49c80aebc0 2024-09-21T07:51:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.796+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49b8035040 con 0x7f49c80aebc0 2024-09-21T07:51:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.796+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49c81541a0 con 0x7f49c80aebc0 2024-09-21T07:51:34.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.797+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49c815cb40 con 0x7f49c80aebc0 2024-09-21T07:51:34.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.798+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f49b8041030 con 0x7f49c80aebc0 2024-09-21T07:51:34.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.800+0000 7f49bea00640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49a8078cb0 0x7f49a807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:34.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.800+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f49b80bb850 con 0x7f49c80aebc0 2024-09-21T07:51:34.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.800+0000 7f49c6000640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49a8078cb0 0x7f49a807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:34.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.801+0000 7f49c6000640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49a8078cb0 0x7f49a807b1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49c815beb0 tx=0x7f49b400a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:34.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.804+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f49b8083410 con 0x7f49c80aebc0 2024-09-21T07:51:34.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.990+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f49c815bc00 con 0x7f49a8078cb0 2024-09-21T07:51:34.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.994+0000 7f49bea00640 1 -- 172.21.15.88:0/232046257 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252472 (secure 0 0 0) 0x7f49c815bc00 con 0x7f49a8078cb0 2024-09-21T07:51:34.999 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:35.000 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:51:35.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49a8078cb0 msgr2=0x7f49a807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49a8078cb0 0x7f49a807b1c0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f49c815beb0 tx=0x7f49b400a250 comp rx=0 tx=0).stop 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 msgr2=0x7f49c814b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c814b7b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f49b8000c00 tx=0x7f49b8003970 comp rx=0 tx=0).stop 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 shutdown_connections 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49a8078cb0 0x7f49a807b1c0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49c815f910 0x7f49c80ca7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49c80b2750 0x7f49c80b0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 --2- 172.21.15.88:0/232046257 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49c80aebc0 0x7f49c814b7b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.998+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 >> 172.21.15.88:0/232046257 conn(0x7f49c806d730 msgr2=0x7f49c80cb8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.999+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 shutdown_connections 2024-09-21T07:51:35.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:34.999+0000 7f49cd800640 1 -- 172.21.15.88:0/232046257 wait complete. 2024-09-21T07:51:35.023 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2024-09-21T07:51:33.705889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":161,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":260796,"kb_used_data":11276,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754713924,"statfs":{"total":773094113280,"available":772827058176,"internally_reserved":0,"allocated":11546624,"data_stored":5690120,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":27,"apply_latency_ms":27,"commit_latency_ns":27000000,"apply_latency_ns":27000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.628693"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412557+0000","last_change":"2024-09-21T07:51:30.121795+0000","last_active":"2024-09-21T07:51:31.412557+0000","last_peered":"2024-09-21T07:51:31.412557+0000","last_clean":"2024-09-21T07:51:31.412557+0000","last_became_active":"2024-09-21T07:51:30.120674+0000","last_became_peered":"2024-09-21T07:51:30.120674+0000","last_unstale":"2024-09-21T07:51:31.412557+0000","last_undegraded":"2024-09-21T07:51:31.412557+0000","last_fullsized":"2024-09-21T07:51:31.412557+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:22.975118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410828+0000","last_change":"2024-09-21T07:51:30.113664+0000","last_active":"2024-09-21T07:51:31.410828+0000","last_peered":"2024-09-21T07:51:31.410828+0000","last_clean":"2024-09-21T07:51:31.410828+0000","last_became_active":"2024-09-21T07:51:30.113366+0000","last_became_peered":"2024-09-21T07:51:30.113366+0000","last_unstale":"2024-09-21T07:51:31.410828+0000","last_undegraded":"2024-09-21T07:51:31.410828+0000","last_fullsized":"2024-09-21T07:51:31.410828+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:53.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094041+0000","last_change":"2024-09-21T07:51:30.104176+0000","last_active":"2024-09-21T07:51:31.094041+0000","last_peered":"2024-09-21T07:51:31.094041+0000","last_clean":"2024-09-21T07:51:31.094041+0000","last_became_active":"2024-09-21T07:51:30.103868+0000","last_became_peered":"2024-09-21T07:51:30.103868+0000","last_unstale":"2024-09-21T07:51:31.094041+0000","last_undegraded":"2024-09-21T07:51:31.094041+0000","last_fullsized":"2024-09-21T07:51:31.094041+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:42.764315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.097042+0000","last_change":"2024-09-21T07:51:30.114417+0000","last_active":"2024-09-21T07:51:31.097042+0000","last_peered":"2024-09-21T07:51:31.097042+0000","last_clean":"2024-09-21T07:51:31.097042+0000","last_became_active":"2024-09-21T07:51:30.113302+0000","last_became_peered":"2024-09-21T07:51:30.113302+0000","last_unstale":"2024-09-21T07:51:31.097042+0000","last_undegraded":"2024-09-21T07:51:31.097042+0000","last_fullsized":"2024-09-21T07:51:31.097042+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:43.247321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412012+0000","last_change":"2024-09-21T07:51:30.118323+0000","last_active":"2024-09-21T07:51:31.412012+0000","last_peered":"2024-09-21T07:51:31.412012+0000","last_clean":"2024-09-21T07:51:31.412012+0000","last_became_active":"2024-09-21T07:51:30.117786+0000","last_became_peered":"2024-09-21T07:51:30.117786+0000","last_unstale":"2024-09-21T07:51:31.412012+0000","last_undegraded":"2024-09-21T07:51:31.412012+0000","last_fullsized":"2024-09-21T07:51:31.412012+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:35.412139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095526+0000","last_change":"2024-09-21T07:51:30.112685+0000","last_active":"2024-09-21T07:51:31.095526+0000","last_peered":"2024-09-21T07:51:31.095526+0000","last_clean":"2024-09-21T07:51:31.095526+0000","last_became_active":"2024-09-21T07:51:30.110374+0000","last_became_peered":"2024-09-21T07:51:30.110374+0000","last_unstale":"2024-09-21T07:51:31.095526+0000","last_undegraded":"2024-09-21T07:51:31.095526+0000","last_fullsized":"2024-09-21T07:51:31.095526+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:00:47.262560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129840+0000","last_change":"2024-09-21T07:51:30.114412+0000","last_active":"2024-09-21T07:51:31.129840+0000","last_peered":"2024-09-21T07:51:31.129840+0000","last_clean":"2024-09-21T07:51:31.129840+0000","last_became_active":"2024-09-21T07:51:30.114100+0000","last_became_peered":"2024-09-21T07:51:30.114100+0000","last_unstale":"2024-09-21T07:51:31.129840+0000","last_undegraded":"2024-09-21T07:51:31.129840+0000","last_fullsized":"2024-09-21T07:51:31.129840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:37:06.531960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129862+0000","last_change":"2024-09-21T07:51:30.110283+0000","last_active":"2024-09-21T07:51:31.129862+0000","last_peered":"2024-09-21T07:51:31.129862+0000","last_clean":"2024-09-21T07:51:31.129862+0000","last_became_active":"2024-09-21T07:51:30.110016+0000","last_became_peered":"2024-09-21T07:51:30.110016+0000","last_unstale":"2024-09-21T07:51:31.129862+0000","last_undegraded":"2024-09-21T07:51:31.129862+0000","last_fullsized":"2024-09-21T07:51:31.129862+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:12.297924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412905+0000","last_change":"2024-09-21T07:51:30.116396+0000","last_active":"2024-09-21T07:51:31.412905+0000","last_peered":"2024-09-21T07:51:31.412905+0000","last_clean":"2024-09-21T07:51:31.412905+0000","last_became_active":"2024-09-21T07:51:30.116039+0000","last_became_peered":"2024-09-21T07:51:30.116039+0000","last_unstale":"2024-09-21T07:51:31.412905+0000","last_undegraded":"2024-09-21T07:51:31.412905+0000","last_fullsized":"2024-09-21T07:51:31.412905+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:59:34.903907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412349+0000","last_change":"2024-09-21T07:51:30.119720+0000","last_active":"2024-09-21T07:51:31.412349+0000","last_peered":"2024-09-21T07:51:31.412349+0000","last_clean":"2024-09-21T07:51:31.412349+0000","last_became_active":"2024-09-21T07:51:30.119403+0000","last_became_peered":"2024-09-21T07:51:30.119403+0000","last_unstale":"2024-09-21T07:51:31.412349+0000","last_undegraded":"2024-09-21T07:51:31.412349+0000","last_fullsized":"2024-09-21T07:51:31.412349+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:39.078116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130156+0000","last_change":"2024-09-21T07:51:30.124931+0000","last_active":"2024-09-21T07:51:31.130156+0000","last_peered":"2024-09-21T07:51:31.130156+0000","last_clean":"2024-09-21T07:51:31.130156+0000","last_became_active":"2024-09-21T07:51:30.116722+0000","last_became_peered":"2024-09-21T07:51:30.116722+0000","last_unstale":"2024-09-21T07:51:31.130156+0000","last_undegraded":"2024-09-21T07:51:31.130156+0000","last_fullsized":"2024-09-21T07:51:31.130156+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:06:53.736398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412038+0000","last_change":"2024-09-21T07:51:30.125056+0000","last_active":"2024-09-21T07:51:31.412038+0000","last_peered":"2024-09-21T07:51:31.412038+0000","last_clean":"2024-09-21T07:51:31.412038+0000","last_became_active":"2024-09-21T07:51:30.124422+0000","last_became_peered":"2024-09-21T07:51:30.124422+0000","last_unstale":"2024-09-21T07:51:31.412038+0000","last_undegraded":"2024-09-21T07:51:31.412038+0000","last_fullsized":"2024-09-21T07:51:31.412038+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:03.441092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410515+0000","last_change":"2024-09-21T07:51:30.116213+0000","last_active":"2024-09-21T07:51:31.410515+0000","last_peered":"2024-09-21T07:51:31.410515+0000","last_clean":"2024-09-21T07:51:31.410515+0000","last_became_active":"2024-09-21T07:51:30.115872+0000","last_became_peered":"2024-09-21T07:51:30.115872+0000","last_unstale":"2024-09-21T07:51:31.410515+0000","last_undegraded":"2024-09-21T07:51:31.410515+0000","last_fullsized":"2024-09-21T07:51:31.410515+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:52.122436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410226+0000","last_change":"2024-09-21T07:51:30.121205+0000","last_active":"2024-09-21T07:51:31.410226+0000","last_peered":"2024-09-21T07:51:31.410226+0000","last_clean":"2024-09-21T07:51:31.410226+0000","last_became_active":"2024-09-21T07:51:30.120946+0000","last_became_peered":"2024-09-21T07:51:30.120946+0000","last_unstale":"2024-09-21T07:51:31.410226+0000","last_undegraded":"2024-09-21T07:51:31.410226+0000","last_fullsized":"2024-09-21T07:51:31.410226+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:10.354369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412095+0000","last_change":"2024-09-21T07:51:30.120980+0000","last_active":"2024-09-21T07:51:31.412095+0000","last_peered":"2024-09-21T07:51:31.412095+0000","last_clean":"2024-09-21T07:51:31.412095+0000","last_became_active":"2024-09-21T07:51:30.120633+0000","last_became_peered":"2024-09-21T07:51:30.120633+0000","last_unstale":"2024-09-21T07:51:31.412095+0000","last_undegraded":"2024-09-21T07:51:31.412095+0000","last_fullsized":"2024-09-21T07:51:31.412095+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.977426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411649+0000","last_change":"2024-09-21T07:51:30.118518+0000","last_active":"2024-09-21T07:51:31.411649+0000","last_peered":"2024-09-21T07:51:31.411649+0000","last_clean":"2024-09-21T07:51:31.411649+0000","last_became_active":"2024-09-21T07:51:30.118164+0000","last_became_peered":"2024-09-21T07:51:30.118164+0000","last_unstale":"2024-09-21T07:51:31.411649+0000","last_undegraded":"2024-09-21T07:51:31.411649+0000","last_fullsized":"2024-09-21T07:51:31.411649+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:29:40.369272+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094053+0000","last_change":"2024-09-21T07:51:30.122189+0000","last_active":"2024-09-21T07:51:31.094053+0000","last_peered":"2024-09-21T07:51:31.094053+0000","last_clean":"2024-09-21T07:51:31.094053+0000","last_became_active":"2024-09-21T07:51:30.121875+0000","last_became_peered":"2024-09-21T07:51:30.121875+0000","last_unstale":"2024-09-21T07:51:31.094053+0000","last_undegraded":"2024-09-21T07:51:31.094053+0000","last_fullsized":"2024-09-21T07:51:31.094053+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:33.893765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130324+0000","last_change":"2024-09-21T07:51:30.125832+0000","last_active":"2024-09-21T07:51:31.130324+0000","last_peered":"2024-09-21T07:51:31.130324+0000","last_clean":"2024-09-21T07:51:31.130324+0000","last_became_active":"2024-09-21T07:51:30.124689+0000","last_became_peered":"2024-09-21T07:51:30.124689+0000","last_unstale":"2024-09-21T07:51:31.130324+0000","last_undegraded":"2024-09-21T07:51:31.130324+0000","last_fullsized":"2024-09-21T07:51:31.130324+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:27.012868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094083+0000","last_change":"2024-09-21T07:51:30.113362+0000","last_active":"2024-09-21T07:51:31.094083+0000","last_peered":"2024-09-21T07:51:31.094083+0000","last_clean":"2024-09-21T07:51:31.094083+0000","last_became_active":"2024-09-21T07:51:30.113073+0000","last_became_peered":"2024-09-21T07:51:30.113073+0000","last_unstale":"2024-09-21T07:51:31.094083+0000","last_undegraded":"2024-09-21T07:51:31.094083+0000","last_fullsized":"2024-09-21T07:51:31.094083+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:07.885890+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093874+0000","last_change":"2024-09-21T07:51:30.108842+0000","last_active":"2024-09-21T07:51:31.093874+0000","last_peered":"2024-09-21T07:51:31.093874+0000","last_clean":"2024-09-21T07:51:31.093874+0000","last_became_active":"2024-09-21T07:51:30.108515+0000","last_became_peered":"2024-09-21T07:51:30.108515+0000","last_unstale":"2024-09-21T07:51:31.093874+0000","last_undegraded":"2024-09-21T07:51:31.093874+0000","last_fullsized":"2024-09-21T07:51:31.093874+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:47:35.217766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130177+0000","last_change":"2024-09-21T07:51:30.112139+0000","last_active":"2024-09-21T07:51:31.130177+0000","last_peered":"2024-09-21T07:51:31.130177+0000","last_clean":"2024-09-21T07:51:31.130177+0000","last_became_active":"2024-09-21T07:51:30.111837+0000","last_became_peered":"2024-09-21T07:51:30.111837+0000","last_unstale":"2024-09-21T07:51:31.130177+0000","last_undegraded":"2024-09-21T07:51:31.130177+0000","last_fullsized":"2024-09-21T07:51:31.130177+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:34.035739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096617+0000","last_change":"2024-09-21T07:51:30.120017+0000","last_active":"2024-09-21T07:51:31.096617+0000","last_peered":"2024-09-21T07:51:31.096617+0000","last_clean":"2024-09-21T07:51:31.096617+0000","last_became_active":"2024-09-21T07:51:30.114536+0000","last_became_peered":"2024-09-21T07:51:30.114536+0000","last_unstale":"2024-09-21T07:51:31.096617+0000","last_undegraded":"2024-09-21T07:51:31.096617+0000","last_fullsized":"2024-09-21T07:51:31.096617+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:23:55.601527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410300+0000","last_change":"2024-09-21T07:51:30.120808+0000","last_active":"2024-09-21T07:51:31.410300+0000","last_peered":"2024-09-21T07:51:31.410300+0000","last_clean":"2024-09-21T07:51:31.410300+0000","last_became_active":"2024-09-21T07:51:30.120305+0000","last_became_peered":"2024-09-21T07:51:30.120305+0000","last_unstale":"2024-09-21T07:51:31.410300+0000","last_undegraded":"2024-09-21T07:51:31.410300+0000","last_fullsized":"2024-09-21T07:51:31.410300+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:27:18.990000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095102+0000","last_change":"2024-09-21T07:51:30.121244+0000","last_active":"2024-09-21T07:51:31.095102+0000","last_peered":"2024-09-21T07:51:31.095102+0000","last_clean":"2024-09-21T07:51:31.095102+0000","last_became_active":"2024-09-21T07:51:30.116845+0000","last_became_peered":"2024-09-21T07:51:30.116845+0000","last_unstale":"2024-09-21T07:51:31.095102+0000","last_undegraded":"2024-09-21T07:51:31.095102+0000","last_fullsized":"2024-09-21T07:51:31.095102+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:01:08.109641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130321+0000","last_change":"2024-09-21T07:51:30.118834+0000","last_active":"2024-09-21T07:51:31.130321+0000","last_peered":"2024-09-21T07:51:31.130321+0000","last_clean":"2024-09-21T07:51:31.130321+0000","last_became_active":"2024-09-21T07:51:30.118535+0000","last_became_peered":"2024-09-21T07:51:30.118535+0000","last_unstale":"2024-09-21T07:51:31.130321+0000","last_undegraded":"2024-09-21T07:51:31.130321+0000","last_fullsized":"2024-09-21T07:51:31.130321+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:50.269600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096284+0000","last_change":"2024-09-21T07:51:30.107355+0000","last_active":"2024-09-21T07:51:31.096284+0000","last_peered":"2024-09-21T07:51:31.096284+0000","last_clean":"2024-09-21T07:51:31.096284+0000","last_became_active":"2024-09-21T07:51:30.106468+0000","last_became_peered":"2024-09-21T07:51:30.106468+0000","last_unstale":"2024-09-21T07:51:31.096284+0000","last_undegraded":"2024-09-21T07:51:31.096284+0000","last_fullsized":"2024-09-21T07:51:31.096284+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:21.553538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096840+0000","last_change":"2024-09-21T07:51:30.120090+0000","last_active":"2024-09-21T07:51:31.096840+0000","last_peered":"2024-09-21T07:51:31.096840+0000","last_clean":"2024-09-21T07:51:31.096840+0000","last_became_active":"2024-09-21T07:51:30.119777+0000","last_became_peered":"2024-09-21T07:51:30.119777+0000","last_unstale":"2024-09-21T07:51:31.096840+0000","last_undegraded":"2024-09-21T07:51:31.096840+0000","last_fullsized":"2024-09-21T07:51:31.096840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:10.425313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096286+0000","last_change":"2024-09-21T07:51:30.114867+0000","last_active":"2024-09-21T07:51:31.096286+0000","last_peered":"2024-09-21T07:51:31.096286+0000","last_clean":"2024-09-21T07:51:31.096286+0000","last_became_active":"2024-09-21T07:51:30.110477+0000","last_became_peered":"2024-09-21T07:51:30.110477+0000","last_unstale":"2024-09-21T07:51:31.096286+0000","last_undegraded":"2024-09-21T07:51:31.096286+0000","last_fullsized":"2024-09-21T07:51:31.096286+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:53.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411339+0000","last_change":"2024-09-21T07:51:30.131859+0000","last_active":"2024-09-21T07:51:31.411339+0000","last_peered":"2024-09-21T07:51:31.411339+0000","last_clean":"2024-09-21T07:51:31.411339+0000","last_became_active":"2024-09-21T07:51:30.130522+0000","last_became_peered":"2024-09-21T07:51:30.130522+0000","last_unstale":"2024-09-21T07:51:31.411339+0000","last_undegraded":"2024-09-21T07:51:31.411339+0000","last_fullsized":"2024-09-21T07:51:31.411339+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:30:46.629937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130040+0000","last_change":"2024-09-21T07:51:30.106763+0000","last_active":"2024-09-21T07:51:31.130040+0000","last_peered":"2024-09-21T07:51:31.130040+0000","last_clean":"2024-09-21T07:51:31.130040+0000","last_became_active":"2024-09-21T07:51:30.106444+0000","last_became_peered":"2024-09-21T07:51:30.106444+0000","last_unstale":"2024-09-21T07:51:31.130040+0000","last_undegraded":"2024-09-21T07:51:31.130040+0000","last_fullsized":"2024-09-21T07:51:31.130040+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:26:29.452438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411625+0000","last_change":"2024-09-21T07:51:30.115224+0000","last_active":"2024-09-21T07:51:31.411625+0000","last_peered":"2024-09-21T07:51:31.411625+0000","last_clean":"2024-09-21T07:51:31.411625+0000","last_became_active":"2024-09-21T07:51:30.114838+0000","last_became_peered":"2024-09-21T07:51:30.114838+0000","last_unstale":"2024-09-21T07:51:31.411625+0000","last_undegraded":"2024-09-21T07:51:31.411625+0000","last_fullsized":"2024-09-21T07:51:31.411625+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:16.220152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411141+0000","last_change":"2024-09-21T07:51:30.125225+0000","last_active":"2024-09-21T07:51:31.411141+0000","last_peered":"2024-09-21T07:51:31.411141+0000","last_clean":"2024-09-21T07:51:31.411141+0000","last_became_active":"2024-09-21T07:51:30.124749+0000","last_became_peered":"2024-09-21T07:51:30.124749+0000","last_unstale":"2024-09-21T07:51:31.411141+0000","last_undegraded":"2024-09-21T07:51:31.411141+0000","last_fullsized":"2024-09-21T07:51:31.411141+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:16.269889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412567+0000","last_change":"2024-09-21T07:51:30.115125+0000","last_active":"2024-09-21T07:51:31.412567+0000","last_peered":"2024-09-21T07:51:31.412567+0000","last_clean":"2024-09-21T07:51:31.412567+0000","last_became_active":"2024-09-21T07:51:30.113440+0000","last_became_peered":"2024-09-21T07:51:30.113440+0000","last_unstale":"2024-09-21T07:51:31.412567+0000","last_undegraded":"2024-09-21T07:51:31.412567+0000","last_fullsized":"2024-09-21T07:51:31.412567+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:32.743165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1f","version":"104'4","reported_seq":19,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410970+0000","last_change":"2024-09-21T07:51:29.116867+0000","last_active":"2024-09-21T07:51:31.410970+0000","last_peered":"2024-09-21T07:51:31.410970+0000","last_clean":"2024-09-21T07:51:31.410970+0000","last_became_active":"2024-09-21T07:51:29.116331+0000","last_became_peered":"2024-09-21T07:51:29.116331+0000","last_unstale":"2024-09-21T07:51:31.410970+0000","last_undegraded":"2024-09-21T07:51:31.410970+0000","last_fullsized":"2024-09-21T07:51:31.410970+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:39.738717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410840+0000","last_change":"2024-09-21T07:51:30.121328+0000","last_active":"2024-09-21T07:51:31.410840+0000","last_peered":"2024-09-21T07:51:31.410840+0000","last_clean":"2024-09-21T07:51:31.410840+0000","last_became_active":"2024-09-21T07:51:30.121035+0000","last_became_peered":"2024-09-21T07:51:30.121035+0000","last_unstale":"2024-09-21T07:51:31.410840+0000","last_undegraded":"2024-09-21T07:51:31.410840+0000","last_fullsized":"2024-09-21T07:51:31.410840+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:45.839409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1e","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411186+0000","last_change":"2024-09-21T07:51:29.104097+0000","last_active":"2024-09-21T07:51:31.411186+0000","last_peered":"2024-09-21T07:51:31.411186+0000","last_clean":"2024-09-21T07:51:31.411186+0000","last_became_active":"2024-09-21T07:51:29.103769+0000","last_became_peered":"2024-09-21T07:51:29.103769+0000","last_unstale":"2024-09-21T07:51:31.411186+0000","last_undegraded":"2024-09-21T07:51:31.411186+0000","last_fullsized":"2024-09-21T07:51:31.411186+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:44.827104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096755+0000","last_change":"2024-09-21T07:51:30.116641+0000","last_active":"2024-09-21T07:51:31.096755+0000","last_peered":"2024-09-21T07:51:31.096755+0000","last_clean":"2024-09-21T07:51:31.096755+0000","last_became_active":"2024-09-21T07:51:30.116344+0000","last_became_peered":"2024-09-21T07:51:30.116344+0000","last_unstale":"2024-09-21T07:51:31.096755+0000","last_undegraded":"2024-09-21T07:51:31.096755+0000","last_fullsized":"2024-09-21T07:51:31.096755+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:02:54.128145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.1d","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096552+0000","last_change":"2024-09-21T07:51:29.111194+0000","last_active":"2024-09-21T07:51:31.096552+0000","last_peered":"2024-09-21T07:51:31.096552+0000","last_clean":"2024-09-21T07:51:31.096552+0000","last_became_active":"2024-09-21T07:51:29.110367+0000","last_became_peered":"2024-09-21T07:51:29.110367+0000","last_unstale":"2024-09-21T07:51:31.096552+0000","last_undegraded":"2024-09-21T07:51:31.096552+0000","last_fullsized":"2024-09-21T07:51:31.096552+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:09.066960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094180+0000","last_change":"2024-09-21T07:51:30.115756+0000","last_active":"2024-09-21T07:51:31.094180+0000","last_peered":"2024-09-21T07:51:31.094180+0000","last_clean":"2024-09-21T07:51:31.094180+0000","last_became_active":"2024-09-21T07:51:30.115317+0000","last_became_peered":"2024-09-21T07:51:30.115317+0000","last_unstale":"2024-09-21T07:51:31.094180+0000","last_undegraded":"2024-09-21T07:51:31.094180+0000","last_fullsized":"2024-09-21T07:51:31.094180+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:56.697822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1c","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096684+0000","last_change":"2024-09-21T07:51:29.111266+0000","last_active":"2024-09-21T07:51:31.096684+0000","last_peered":"2024-09-21T07:51:31.096684+0000","last_clean":"2024-09-21T07:51:31.096684+0000","last_became_active":"2024-09-21T07:51:29.110878+0000","last_became_peered":"2024-09-21T07:51:29.110878+0000","last_unstale":"2024-09-21T07:51:31.096684+0000","last_undegraded":"2024-09-21T07:51:31.096684+0000","last_fullsized":"2024-09-21T07:51:31.096684+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:06:31.139669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095534+0000","last_change":"2024-09-21T07:51:30.125700+0000","last_active":"2024-09-21T07:51:31.095534+0000","last_peered":"2024-09-21T07:51:31.095534+0000","last_clean":"2024-09-21T07:51:31.095534+0000","last_became_active":"2024-09-21T07:51:30.124533+0000","last_became_peered":"2024-09-21T07:51:30.124533+0000","last_unstale":"2024-09-21T07:51:31.095534+0000","last_undegraded":"2024-09-21T07:51:31.095534+0000","last_fullsized":"2024-09-21T07:51:31.095534+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:33:02.433903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1b","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093601+0000","last_change":"2024-09-21T07:51:29.097120+0000","last_active":"2024-09-21T07:51:31.093601+0000","last_peered":"2024-09-21T07:51:31.093601+0000","last_clean":"2024-09-21T07:51:31.093601+0000","last_became_active":"2024-09-21T07:51:29.096790+0000","last_became_peered":"2024-09-21T07:51:29.096790+0000","last_unstale":"2024-09-21T07:51:31.093601+0000","last_undegraded":"2024-09-21T07:51:31.093601+0000","last_fullsized":"2024-09-21T07:51:31.093601+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:17:00.021303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130112+0000","last_change":"2024-09-21T07:51:30.110308+0000","last_active":"2024-09-21T07:51:31.130112+0000","last_peered":"2024-09-21T07:51:31.130112+0000","last_clean":"2024-09-21T07:51:31.130112+0000","last_became_active":"2024-09-21T07:51:30.110068+0000","last_became_peered":"2024-09-21T07:51:30.110068+0000","last_unstale":"2024-09-21T07:51:31.130112+0000","last_undegraded":"2024-09-21T07:51:31.130112+0000","last_fullsized":"2024-09-21T07:51:31.130112+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:25.798301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.1a","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411359+0000","last_change":"2024-09-21T07:51:29.107419+0000","last_active":"2024-09-21T07:51:31.411359+0000","last_peered":"2024-09-21T07:51:31.411359+0000","last_clean":"2024-09-21T07:51:31.411359+0000","last_became_active":"2024-09-21T07:51:29.107067+0000","last_became_peered":"2024-09-21T07:51:29.107067+0000","last_unstale":"2024-09-21T07:51:31.411359+0000","last_undegraded":"2024-09-21T07:51:31.411359+0000","last_fullsized":"2024-09-21T07:51:31.411359+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:12:55.685516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411408+0000","last_change":"2024-09-21T07:51:30.115458+0000","last_active":"2024-09-21T07:51:31.411408+0000","last_peered":"2024-09-21T07:51:31.411408+0000","last_clean":"2024-09-21T07:51:31.411408+0000","last_became_active":"2024-09-21T07:51:30.115036+0000","last_became_peered":"2024-09-21T07:51:30.115036+0000","last_unstale":"2024-09-21T07:51:31.411408+0000","last_undegraded":"2024-09-21T07:51:31.411408+0000","last_fullsized":"2024-09-21T07:51:31.411408+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:49.609617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129981+0000","last_change":"2024-09-21T07:51:29.111025+0000","last_active":"2024-09-21T07:51:31.129981+0000","last_peered":"2024-09-21T07:51:31.129981+0000","last_clean":"2024-09-21T07:51:31.129981+0000","last_became_active":"2024-09-21T07:51:29.110724+0000","last_became_peered":"2024-09-21T07:51:29.110724+0000","last_unstale":"2024-09-21T07:51:31.129981+0000","last_undegraded":"2024-09-21T07:51:31.129981+0000","last_fullsized":"2024-09-21T07:51:31.129981+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:17:34.257512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.129883+0000","last_change":"2024-09-21T07:51:30.119307+0000","last_active":"2024-09-21T07:51:31.129883+0000","last_peered":"2024-09-21T07:51:31.129883+0000","last_clean":"2024-09-21T07:51:31.129883+0000","last_became_active":"2024-09-21T07:51:30.118341+0000","last_became_peered":"2024-09-21T07:51:30.118341+0000","last_unstale":"2024-09-21T07:51:31.129883+0000","last_undegraded":"2024-09-21T07:51:31.129883+0000","last_fullsized":"2024-09-21T07:51:31.129883+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:28:23.513817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411553+0000","last_change":"2024-09-21T07:51:29.117436+0000","last_active":"2024-09-21T07:51:31.411553+0000","last_peered":"2024-09-21T07:51:31.411553+0000","last_clean":"2024-09-21T07:51:31.411553+0000","last_became_active":"2024-09-21T07:51:29.117093+0000","last_became_peered":"2024-09-21T07:51:29.117093+0000","last_unstale":"2024-09-21T07:51:31.411553+0000","last_undegraded":"2024-09-21T07:51:31.411553+0000","last_fullsized":"2024-09-21T07:51:31.411553+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:09:45.614379+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411049+0000","last_change":"2024-09-21T07:51:30.119633+0000","last_active":"2024-09-21T07:51:31.411049+0000","last_peered":"2024-09-21T07:51:31.411049+0000","last_clean":"2024-09-21T07:51:31.411049+0000","last_became_active":"2024-09-21T07:51:30.118450+0000","last_became_peered":"2024-09-21T07:51:30.118450+0000","last_unstale":"2024-09-21T07:51:31.411049+0000","last_undegraded":"2024-09-21T07:51:31.411049+0000","last_fullsized":"2024-09-21T07:51:31.411049+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:52:41.198532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096916+0000","last_change":"2024-09-21T07:51:29.113044+0000","last_active":"2024-09-21T07:51:31.096916+0000","last_peered":"2024-09-21T07:51:31.096916+0000","last_clean":"2024-09-21T07:51:31.096916+0000","last_became_active":"2024-09-21T07:51:29.112737+0000","last_became_peered":"2024-09-21T07:51:29.112737+0000","last_unstale":"2024-09-21T07:51:31.096916+0000","last_undegraded":"2024-09-21T07:51:31.096916+0000","last_fullsized":"2024-09-21T07:51:31.096916+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:31.730860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410950+0000","last_change":"2024-09-21T07:51:30.116235+0000","last_active":"2024-09-21T07:51:31.410950+0000","last_peered":"2024-09-21T07:51:31.410950+0000","last_clean":"2024-09-21T07:51:31.410950+0000","last_became_active":"2024-09-21T07:51:30.115825+0000","last_became_peered":"2024-09-21T07:51:30.115825+0000","last_unstale":"2024-09-21T07:51:31.410950+0000","last_undegraded":"2024-09-21T07:51:31.410950+0000","last_fullsized":"2024-09-21T07:51:31.410950+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:26.424322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412206+0000","last_change":"2024-09-21T07:51:29.117836+0000","last_active":"2024-09-21T07:51:31.412206+0000","last_peered":"2024-09-21T07:51:31.412206+0000","last_clean":"2024-09-21T07:51:31.412206+0000","last_became_active":"2024-09-21T07:51:29.117017+0000","last_became_peered":"2024-09-21T07:51:29.117017+0000","last_unstale":"2024-09-21T07:51:31.412206+0000","last_undegraded":"2024-09-21T07:51:31.412206+0000","last_fullsized":"2024-09-21T07:51:31.412206+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:31:33.574734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096816+0000","last_change":"2024-09-21T07:51:30.121035+0000","last_active":"2024-09-21T07:51:31.096816+0000","last_peered":"2024-09-21T07:51:31.096816+0000","last_clean":"2024-09-21T07:51:31.096816+0000","last_became_active":"2024-09-21T07:51:30.120689+0000","last_became_peered":"2024-09-21T07:51:30.120689+0000","last_unstale":"2024-09-21T07:51:31.096816+0000","last_undegraded":"2024-09-21T07:51:31.096816+0000","last_fullsized":"2024-09-21T07:51:31.096816+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:35:52.651103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.15","version":"104'3","reported_seq":18,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411804+0000","last_change":"2024-09-21T07:51:29.114736+0000","last_active":"2024-09-21T07:51:31.411804+0000","last_peered":"2024-09-21T07:51:31.411804+0000","last_clean":"2024-09-21T07:51:31.411804+0000","last_became_active":"2024-09-21T07:51:29.114297+0000","last_became_peered":"2024-09-21T07:51:29.114297+0000","last_unstale":"2024-09-21T07:51:31.411804+0000","last_undegraded":"2024-09-21T07:51:31.411804+0000","last_fullsized":"2024-09-21T07:51:31.411804+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:11.137559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130242+0000","last_change":"2024-09-21T07:51:30.110596+0000","last_active":"2024-09-21T07:51:31.130242+0000","last_peered":"2024-09-21T07:51:31.130242+0000","last_clean":"2024-09-21T07:51:31.130242+0000","last_became_active":"2024-09-21T07:51:30.107904+0000","last_became_peered":"2024-09-21T07:51:30.107904+0000","last_unstale":"2024-09-21T07:51:31.130242+0000","last_undegraded":"2024-09-21T07:51:31.130242+0000","last_fullsized":"2024-09-21T07:51:31.130242+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:25:47.538159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.14","version":"104'6","reported_seq":21,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096819+0000","last_change":"2024-09-21T07:51:29.113787+0000","last_active":"2024-09-21T07:51:31.096819+0000","last_peered":"2024-09-21T07:51:31.096819+0000","last_clean":"2024-09-21T07:51:31.096819+0000","last_became_active":"2024-09-21T07:51:29.113295+0000","last_became_peered":"2024-09-21T07:51:29.113295+0000","last_unstale":"2024-09-21T07:51:31.096819+0000","last_undegraded":"2024-09-21T07:51:31.096819+0000","last_fullsized":"2024-09-21T07:51:31.096819+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:42.156091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411178+0000","last_change":"2024-09-21T07:51:30.118654+0000","last_active":"2024-09-21T07:51:31.411178+0000","last_peered":"2024-09-21T07:51:31.411178+0000","last_clean":"2024-09-21T07:51:31.411178+0000","last_became_active":"2024-09-21T07:51:30.118067+0000","last_became_peered":"2024-09-21T07:51:30.118067+0000","last_unstale":"2024-09-21T07:51:31.411178+0000","last_undegraded":"2024-09-21T07:51:31.411178+0000","last_fullsized":"2024-09-21T07:51:31.411178+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:19:00.790432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.13","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411923+0000","last_change":"2024-09-21T07:51:29.114979+0000","last_active":"2024-09-21T07:51:31.411923+0000","last_peered":"2024-09-21T07:51:31.411923+0000","last_clean":"2024-09-21T07:51:31.411923+0000","last_became_active":"2024-09-21T07:51:29.114376+0000","last_became_peered":"2024-09-21T07:51:29.114376+0000","last_unstale":"2024-09-21T07:51:31.411923+0000","last_undegraded":"2024-09-21T07:51:31.411923+0000","last_fullsized":"2024-09-21T07:51:31.411923+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:56:41.295424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096167+0000","last_change":"2024-09-21T07:51:30.123712+0000","last_active":"2024-09-21T07:51:31.096167+0000","last_peered":"2024-09-21T07:51:31.096167+0000","last_clean":"2024-09-21T07:51:31.096167+0000","last_became_active":"2024-09-21T07:51:30.123396+0000","last_became_peered":"2024-09-21T07:51:30.123396+0000","last_unstale":"2024-09-21T07:51:31.096167+0000","last_undegraded":"2024-09-21T07:51:31.096167+0000","last_fullsized":"2024-09-21T07:51:31.096167+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:36.255258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093687+0000","last_change":"2024-09-21T07:51:29.100662+0000","last_active":"2024-09-21T07:51:31.093687+0000","last_peered":"2024-09-21T07:51:31.093687+0000","last_clean":"2024-09-21T07:51:31.093687+0000","last_became_active":"2024-09-21T07:51:29.100353+0000","last_became_peered":"2024-09-21T07:51:29.100353+0000","last_unstale":"2024-09-21T07:51:31.093687+0000","last_undegraded":"2024-09-21T07:51:31.093687+0000","last_fullsized":"2024-09-21T07:51:31.093687+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:28.950459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096237+0000","last_change":"2024-09-21T07:51:30.107089+0000","last_active":"2024-09-21T07:51:31.096237+0000","last_peered":"2024-09-21T07:51:31.096237+0000","last_clean":"2024-09-21T07:51:31.096237+0000","last_became_active":"2024-09-21T07:51:30.106714+0000","last_became_peered":"2024-09-21T07:51:30.106714+0000","last_unstale":"2024-09-21T07:51:31.096237+0000","last_undegraded":"2024-09-21T07:51:31.096237+0000","last_fullsized":"2024-09-21T07:51:31.096237+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:57.190339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.10","version":"104'8","reported_seq":23,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411336+0000","last_change":"2024-09-21T07:51:29.105733+0000","last_active":"2024-09-21T07:51:31.411336+0000","last_peered":"2024-09-21T07:51:31.411336+0000","last_clean":"2024-09-21T07:51:31.411336+0000","last_became_active":"2024-09-21T07:51:29.103945+0000","last_became_peered":"2024-09-21T07:51:29.103945+0000","last_unstale":"2024-09-21T07:51:31.411336+0000","last_undegraded":"2024-09-21T07:51:31.411336+0000","last_fullsized":"2024-09-21T07:51:31.411336+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:42.867682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412614+0000","last_change":"2024-09-21T07:51:30.116548+0000","last_active":"2024-09-21T07:51:31.412614+0000","last_peered":"2024-09-21T07:51:31.412614+0000","last_clean":"2024-09-21T07:51:31.412614+0000","last_became_active":"2024-09-21T07:51:30.116170+0000","last_became_peered":"2024-09-21T07:51:30.116170+0000","last_unstale":"2024-09-21T07:51:31.412614+0000","last_undegraded":"2024-09-21T07:51:31.412614+0000","last_fullsized":"2024-09-21T07:51:31.412614+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:52.122436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.f","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095937+0000","last_change":"2024-09-21T07:51:29.106076+0000","last_active":"2024-09-21T07:51:31.095937+0000","last_peered":"2024-09-21T07:51:31.095937+0000","last_clean":"2024-09-21T07:51:31.095937+0000","last_became_active":"2024-09-21T07:51:29.104518+0000","last_became_peered":"2024-09-21T07:51:29.104518+0000","last_unstale":"2024-09-21T07:51:31.095937+0000","last_undegraded":"2024-09-21T07:51:31.095937+0000","last_fullsized":"2024-09-21T07:51:31.095937+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:11.508457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412897+0000","last_change":"2024-09-21T07:51:30.116309+0000","last_active":"2024-09-21T07:51:31.412897+0000","last_peered":"2024-09-21T07:51:31.412897+0000","last_clean":"2024-09-21T07:51:31.412897+0000","last_became_active":"2024-09-21T07:51:30.115962+0000","last_became_peered":"2024-09-21T07:51:30.115962+0000","last_unstale":"2024-09-21T07:51:31.412897+0000","last_undegraded":"2024-09-21T07:51:31.412897+0000","last_fullsized":"2024-09-21T07:51:31.412897+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:53.680214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.e","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412300+0000","last_change":"2024-09-21T07:51:29.116935+0000","last_active":"2024-09-21T07:51:31.412300+0000","last_peered":"2024-09-21T07:51:31.412300+0000","last_clean":"2024-09-21T07:51:31.412300+0000","last_became_active":"2024-09-21T07:51:29.116549+0000","last_became_peered":"2024-09-21T07:51:29.116549+0000","last_unstale":"2024-09-21T07:51:31.412300+0000","last_undegraded":"2024-09-21T07:51:31.412300+0000","last_fullsized":"2024-09-21T07:51:31.412300+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:42:12.969327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130444+0000","last_change":"2024-09-21T07:51:30.120480+0000","last_active":"2024-09-21T07:51:31.130444+0000","last_peered":"2024-09-21T07:51:31.130444+0000","last_clean":"2024-09-21T07:51:31.130444+0000","last_became_active":"2024-09-21T07:51:30.120192+0000","last_became_peered":"2024-09-21T07:51:30.120192+0000","last_unstale":"2024-09-21T07:51:31.130444+0000","last_undegraded":"2024-09-21T07:51:31.130444+0000","last_fullsized":"2024-09-21T07:51:31.130444+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:18:07.055052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.d","version":"104'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411059+0000","last_change":"2024-09-21T07:51:29.105588+0000","last_active":"2024-09-21T07:51:31.411059+0000","last_peered":"2024-09-21T07:51:31.411059+0000","last_clean":"2024-09-21T07:51:31.411059+0000","last_became_active":"2024-09-21T07:51:29.105303+0000","last_became_peered":"2024-09-21T07:51:29.105303+0000","last_unstale":"2024-09-21T07:51:31.411059+0000","last_undegraded":"2024-09-21T07:51:31.411059+0000","last_fullsized":"2024-09-21T07:51:31.411059+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:01:49.862179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411287+0000","last_change":"2024-09-21T07:51:30.119659+0000","last_active":"2024-09-21T07:51:31.411287+0000","last_peered":"2024-09-21T07:51:31.411287+0000","last_clean":"2024-09-21T07:51:31.411287+0000","last_became_active":"2024-09-21T07:51:30.119304+0000","last_became_peered":"2024-09-21T07:51:30.119304+0000","last_unstale":"2024-09-21T07:51:31.411287+0000","last_undegraded":"2024-09-21T07:51:31.411287+0000","last_fullsized":"2024-09-21T07:51:31.411287+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:22.975118+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410501+0000","last_change":"2024-09-21T07:51:29.112319+0000","last_active":"2024-09-21T07:51:31.410501+0000","last_peered":"2024-09-21T07:51:31.410501+0000","last_clean":"2024-09-21T07:51:31.410501+0000","last_became_active":"2024-09-21T07:51:29.112055+0000","last_became_peered":"2024-09-21T07:51:29.112055+0000","last_unstale":"2024-09-21T07:51:31.410501+0000","last_undegraded":"2024-09-21T07:51:31.410501+0000","last_fullsized":"2024-09-21T07:51:31.410501+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:53.973059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093555+0000","last_change":"2024-09-21T07:51:30.104623+0000","last_active":"2024-09-21T07:51:31.093555+0000","last_peered":"2024-09-21T07:51:31.093555+0000","last_clean":"2024-09-21T07:51:31.093555+0000","last_became_active":"2024-09-21T07:51:30.102939+0000","last_became_peered":"2024-09-21T07:51:30.102939+0000","last_unstale":"2024-09-21T07:51:31.093555+0000","last_undegraded":"2024-09-21T07:51:31.093555+0000","last_fullsized":"2024-09-21T07:51:31.093555+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:07:44.051383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.b","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412068+0000","last_change":"2024-09-21T07:51:29.116684+0000","last_active":"2024-09-21T07:51:31.412068+0000","last_peered":"2024-09-21T07:51:31.412068+0000","last_clean":"2024-09-21T07:51:31.412068+0000","last_became_active":"2024-09-21T07:51:29.116351+0000","last_became_peered":"2024-09-21T07:51:29.116351+0000","last_unstale":"2024-09-21T07:51:31.412068+0000","last_undegraded":"2024-09-21T07:51:31.412068+0000","last_fullsized":"2024-09-21T07:51:31.412068+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:48.485754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411964+0000","last_change":"2024-09-21T07:51:30.122458+0000","last_active":"2024-09-21T07:51:31.411964+0000","last_peered":"2024-09-21T07:51:31.411964+0000","last_clean":"2024-09-21T07:51:31.411964+0000","last_became_active":"2024-09-21T07:51:30.121949+0000","last_became_peered":"2024-09-21T07:51:30.121949+0000","last_unstale":"2024-09-21T07:51:31.411964+0000","last_undegraded":"2024-09-21T07:51:31.411964+0000","last_fullsized":"2024-09-21T07:51:31.411964+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.481048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.412011+0000","last_change":"2024-09-21T07:51:29.106803+0000","last_active":"2024-09-21T07:51:31.412011+0000","last_peered":"2024-09-21T07:51:31.412011+0000","last_clean":"2024-09-21T07:51:31.412011+0000","last_became_active":"2024-09-21T07:51:29.106426+0000","last_became_peered":"2024-09-21T07:51:29.106426+0000","last_unstale":"2024-09-21T07:51:31.412011+0000","last_undegraded":"2024-09-21T07:51:31.412011+0000","last_fullsized":"2024-09-21T07:51:31.412011+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:34:32.881460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.409983+0000","last_change":"2024-09-21T07:51:30.122034+0000","last_active":"2024-09-21T07:51:31.409983+0000","last_peered":"2024-09-21T07:51:31.409983+0000","last_clean":"2024-09-21T07:51:31.409983+0000","last_became_active":"2024-09-21T07:51:30.121694+0000","last_became_peered":"2024-09-21T07:51:30.121694+0000","last_unstale":"2024-09-21T07:51:31.409983+0000","last_undegraded":"2024-09-21T07:51:31.409983+0000","last_fullsized":"2024-09-21T07:51:31.409983+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:55.119810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096404+0000","last_change":"2024-09-21T07:51:29.105500+0000","last_active":"2024-09-21T07:51:31.096404+0000","last_peered":"2024-09-21T07:51:31.096404+0000","last_clean":"2024-09-21T07:51:31.096404+0000","last_became_active":"2024-09-21T07:51:29.105156+0000","last_became_peered":"2024-09-21T07:51:29.105156+0000","last_unstale":"2024-09-21T07:51:31.096404+0000","last_undegraded":"2024-09-21T07:51:31.096404+0000","last_fullsized":"2024-09-21T07:51:31.096404+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:22.377753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411221+0000","last_change":"2024-09-21T07:51:30.117790+0000","last_active":"2024-09-21T07:51:31.411221+0000","last_peered":"2024-09-21T07:51:31.411221+0000","last_clean":"2024-09-21T07:51:31.411221+0000","last_became_active":"2024-09-21T07:51:30.117479+0000","last_became_peered":"2024-09-21T07:51:30.117479+0000","last_unstale":"2024-09-21T07:51:31.411221+0000","last_undegraded":"2024-09-21T07:51:31.411221+0000","last_fullsized":"2024-09-21T07:51:31.411221+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:59:23.126201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095276+0000","last_change":"2024-09-21T07:51:29.104861+0000","last_active":"2024-09-21T07:51:31.095276+0000","last_peered":"2024-09-21T07:51:31.095276+0000","last_clean":"2024-09-21T07:51:31.095276+0000","last_became_active":"2024-09-21T07:51:29.103746+0000","last_became_peered":"2024-09-21T07:51:29.103746+0000","last_unstale":"2024-09-21T07:51:31.095276+0000","last_undegraded":"2024-09-21T07:51:31.095276+0000","last_fullsized":"2024-09-21T07:51:31.095276+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:54.367561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411055+0000","last_change":"2024-09-21T07:51:30.122163+0000","last_active":"2024-09-21T07:51:31.411055+0000","last_peered":"2024-09-21T07:51:31.411055+0000","last_clean":"2024-09-21T07:51:31.411055+0000","last_became_active":"2024-09-21T07:51:30.121830+0000","last_became_peered":"2024-09-21T07:51:30.121830+0000","last_unstale":"2024-09-21T07:51:31.411055+0000","last_undegraded":"2024-09-21T07:51:31.411055+0000","last_fullsized":"2024-09-21T07:51:31.411055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:21.977426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.7","version":"104'2","reported_seq":17,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096995+0000","last_change":"2024-09-21T07:51:29.115728+0000","last_active":"2024-09-21T07:51:31.096995+0000","last_peered":"2024-09-21T07:51:31.096995+0000","last_clean":"2024-09-21T07:51:31.096995+0000","last_became_active":"2024-09-21T07:51:29.115280+0000","last_became_peered":"2024-09-21T07:51:29.115280+0000","last_unstale":"2024-09-21T07:51:31.096995+0000","last_undegraded":"2024-09-21T07:51:31.096995+0000","last_fullsized":"2024-09-21T07:51:31.096995+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:30.814504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096055+0000","last_change":"2024-09-21T07:51:30.121219+0000","last_active":"2024-09-21T07:51:31.096055+0000","last_peered":"2024-09-21T07:51:31.096055+0000","last_clean":"2024-09-21T07:51:31.096055+0000","last_became_active":"2024-09-21T07:51:30.120922+0000","last_became_peered":"2024-09-21T07:51:30.120922+0000","last_unstale":"2024-09-21T07:51:31.096055+0000","last_undegraded":"2024-09-21T07:51:31.096055+0000","last_fullsized":"2024-09-21T07:51:31.096055+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:13:27.907841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.6","version":"105'1","reported_seq":16,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.132000+0000","last_change":"2024-09-21T07:51:29.103439+0000","last_active":"2024-09-21T07:51:31.132000+0000","last_peered":"2024-09-21T07:51:31.132000+0000","last_clean":"2024-09-21T07:51:31.132000+0000","last_became_active":"2024-09-21T07:51:29.103106+0000","last_became_peered":"2024-09-21T07:51:29.103106+0000","last_unstale":"2024-09-21T07:51:31.132000+0000","last_undegraded":"2024-09-21T07:51:31.132000+0000","last_fullsized":"2024-09-21T07:51:31.132000+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:26:42.329445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411630+0000","last_change":"2024-09-21T07:51:30.121561+0000","last_active":"2024-09-21T07:51:31.411630+0000","last_peered":"2024-09-21T07:51:31.411630+0000","last_clean":"2024-09-21T07:51:31.411630+0000","last_became_active":"2024-09-21T07:51:30.120944+0000","last_became_peered":"2024-09-21T07:51:30.120944+0000","last_unstale":"2024-09-21T07:51:31.411630+0000","last_undegraded":"2024-09-21T07:51:31.411630+0000","last_fullsized":"2024-09-21T07:51:31.411630+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:30.664143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096673+0000","last_change":"2024-09-21T07:51:29.105325+0000","last_active":"2024-09-21T07:51:31.096673+0000","last_peered":"2024-09-21T07:51:31.096673+0000","last_clean":"2024-09-21T07:51:31.096673+0000","last_became_active":"2024-09-21T07:51:29.104965+0000","last_became_peered":"2024-09-21T07:51:29.104965+0000","last_unstale":"2024-09-21T07:51:31.096673+0000","last_undegraded":"2024-09-21T07:51:31.096673+0000","last_fullsized":"2024-09-21T07:51:31.096673+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:42:29.651837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410933+0000","last_change":"2024-09-21T07:51:30.128743+0000","last_active":"2024-09-21T07:51:31.410933+0000","last_peered":"2024-09-21T07:51:31.410933+0000","last_clean":"2024-09-21T07:51:31.410933+0000","last_became_active":"2024-09-21T07:51:30.127622+0000","last_became_peered":"2024-09-21T07:51:30.127622+0000","last_unstale":"2024-09-21T07:51:31.410933+0000","last_undegraded":"2024-09-21T07:51:31.410933+0000","last_fullsized":"2024-09-21T07:51:31.410933+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:25:35.481048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.096944+0000","last_change":"2024-09-21T07:51:29.104567+0000","last_active":"2024-09-21T07:51:31.096944+0000","last_peered":"2024-09-21T07:51:31.096944+0000","last_clean":"2024-09-21T07:51:31.096944+0000","last_became_active":"2024-09-21T07:51:29.104275+0000","last_became_peered":"2024-09-21T07:51:29.104275+0000","last_unstale":"2024-09-21T07:51:31.096944+0000","last_undegraded":"2024-09-21T07:51:31.096944+0000","last_fullsized":"2024-09-21T07:51:31.096944+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:50:54.341891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093584+0000","last_change":"2024-09-21T07:51:30.101101+0000","last_active":"2024-09-21T07:51:31.093584+0000","last_peered":"2024-09-21T07:51:31.093584+0000","last_clean":"2024-09-21T07:51:31.093584+0000","last_became_active":"2024-09-21T07:51:30.100761+0000","last_became_peered":"2024-09-21T07:51:30.100761+0000","last_unstale":"2024-09-21T07:51:31.093584+0000","last_undegraded":"2024-09-21T07:51:31.093584+0000","last_fullsized":"2024-09-21T07:51:31.093584+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:50:24.676842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.130240+0000","last_change":"2024-09-21T07:51:29.108009+0000","last_active":"2024-09-21T07:51:31.130240+0000","last_peered":"2024-09-21T07:51:31.130240+0000","last_clean":"2024-09-21T07:51:31.130240+0000","last_became_active":"2024-09-21T07:51:29.107149+0000","last_became_peered":"2024-09-21T07:51:29.107149+0000","last_unstale":"2024-09-21T07:51:31.130240+0000","last_undegraded":"2024-09-21T07:51:31.130240+0000","last_fullsized":"2024-09-21T07:51:31.130240+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:23:42.765582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410666+0000","last_change":"2024-09-21T07:51:30.123553+0000","last_active":"2024-09-21T07:51:31.410666+0000","last_peered":"2024-09-21T07:51:31.410666+0000","last_clean":"2024-09-21T07:51:31.410666+0000","last_became_active":"2024-09-21T07:51:30.123262+0000","last_became_peered":"2024-09-21T07:51:30.123262+0000","last_unstale":"2024-09-21T07:51:31.410666+0000","last_undegraded":"2024-09-21T07:51:31.410666+0000","last_fullsized":"2024-09-21T07:51:31.410666+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:16.220152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410338+0000","last_change":"2024-09-21T07:51:29.109165+0000","last_active":"2024-09-21T07:51:31.410338+0000","last_peered":"2024-09-21T07:51:31.410338+0000","last_clean":"2024-09-21T07:51:31.410338+0000","last_became_active":"2024-09-21T07:51:29.108857+0000","last_became_peered":"2024-09-21T07:51:29.108857+0000","last_unstale":"2024-09-21T07:51:31.410338+0000","last_undegraded":"2024-09-21T07:51:31.410338+0000","last_fullsized":"2024-09-21T07:51:31.410338+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:33:56.334823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410057+0000","last_change":"2024-09-21T07:51:30.120627+0000","last_active":"2024-09-21T07:51:31.410057+0000","last_peered":"2024-09-21T07:51:31.410057+0000","last_clean":"2024-09-21T07:51:31.410057+0000","last_became_active":"2024-09-21T07:51:30.120108+0000","last_became_peered":"2024-09-21T07:51:30.120108+0000","last_unstale":"2024-09-21T07:51:31.410057+0000","last_undegraded":"2024-09-21T07:51:31.410057+0000","last_fullsized":"2024-09-21T07:51:31.410057+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:44.170781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.411842+0000","last_change":"2024-09-21T07:51:29.105332+0000","last_active":"2024-09-21T07:51:31.411842+0000","last_peered":"2024-09-21T07:51:31.411842+0000","last_clean":"2024-09-21T07:51:31.411842+0000","last_became_active":"2024-09-21T07:51:29.104821+0000","last_became_peered":"2024-09-21T07:51:29.104821+0000","last_unstale":"2024-09-21T07:51:31.411842+0000","last_undegraded":"2024-09-21T07:51:31.411842+0000","last_fullsized":"2024-09-21T07:51:31.411842+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:32:06.873585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095039+0000","last_change":"2024-09-21T07:51:30.125953+0000","last_active":"2024-09-21T07:51:31.095039+0000","last_peered":"2024-09-21T07:51:31.095039+0000","last_clean":"2024-09-21T07:51:31.095039+0000","last_became_active":"2024-09-21T07:51:30.125506+0000","last_became_peered":"2024-09-21T07:51:30.125506+0000","last_unstale":"2024-09-21T07:51:31.095039+0000","last_undegraded":"2024-09-21T07:51:31.095039+0000","last_fullsized":"2024-09-21T07:51:31.095039+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:30:04.374254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.094998+0000","last_change":"2024-09-21T07:51:29.107130+0000","last_active":"2024-09-21T07:51:31.094998+0000","last_peered":"2024-09-21T07:51:31.094998+0000","last_clean":"2024-09-21T07:51:31.094998+0000","last_became_active":"2024-09-21T07:51:29.106778+0000","last_became_peered":"2024-09-21T07:51:29.106778+0000","last_unstale":"2024-09-21T07:51:31.094998+0000","last_undegraded":"2024-09-21T07:51:31.094998+0000","last_fullsized":"2024-09-21T07:51:31.094998+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:20:55.062920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":13,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.095499+0000","last_change":"2024-09-21T07:51:30.109723+0000","last_active":"2024-09-21T07:51:31.095499+0000","last_peered":"2024-09-21T07:51:31.095499+0000","last_clean":"2024-09-21T07:51:31.095499+0000","last_became_active":"2024-09-21T07:51:30.109405+0000","last_became_peered":"2024-09-21T07:51:30.109405+0000","last_unstale":"2024-09-21T07:51:31.095499+0000","last_undegraded":"2024-09-21T07:51:31.095499+0000","last_fullsized":"2024-09-21T07:51:31.095499+0000","mapping_epoch":103,"log_start":"0'0","ondisk_log_start":"0'0","created":103,"last_epoch_clean":104,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:29.076880+0000","last_clean_scrub_stamp":"2024-09-21T07:51:29.076880+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:37.352715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":15,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.410317+0000","last_change":"2024-09-21T07:51:29.102933+0000","last_active":"2024-09-21T07:51:31.410317+0000","last_peered":"2024-09-21T07:51:31.410317+0000","last_clean":"2024-09-21T07:51:31.410317+0000","last_became_active":"2024-09-21T07:51:29.102705+0000","last_became_peered":"2024-09-21T07:51:29.102705+0000","last_unstale":"2024-09-21T07:51:31.410317+0000","last_undegraded":"2024-09-21T07:51:31.410317+0000","last_fullsized":"2024-09-21T07:51:31.410317+0000","mapping_epoch":102,"log_start":"0'0","ondisk_log_start":"0'0","created":102,"last_epoch_clean":103,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:51:28.064574+0000","last_clean_scrub_stamp":"2024-09-21T07:51:28.064574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:25:15.207847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":296,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-09-21T07:51:31.093328+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:51:31.093328+0000","last_peered":"2024-09-21T07:51:31.093328+0000","last_clean":"2024-09-21T07:51:31.093328+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:51:31.093328+0000","last_undegraded":"2024-09-21T07:51:31.093328+0000","last_fullsized":"2024-09-21T07:51:31.093328+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542215,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32464,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339376,"statfs":{"total":96636764160,"available":96603521024,"internally_reserved":0,"allocated":1302528,"data_stored":562444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509510,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":560708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509511,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33044,"kb_used_data":1852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338796,"statfs":{"total":96636764160,"available":96602927104,"internally_reserved":0,"allocated":1896448,"data_stored":1157468,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509511,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32472,"kb_used_data":1280,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339368,"statfs":{"total":96636764160,"available":96603512832,"internally_reserved":0,"allocated":1310720,"data_stored":568710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476809,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32440,"kb_used_data":1252,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339400,"statfs":{"total":96636764160,"available":96603545600,"internally_reserved":0,"allocated":1282048,"data_stored":560708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476809,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32448,"kb_used_data":1264,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339392,"statfs":{"total":96636764160,"available":96603537408,"internally_reserved":0,"allocated":1294336,"data_stored":561346,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476806,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33020,"kb_used_data":1832,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338820,"statfs":{"total":96636764160,"available":96602951680,"internally_reserved":0,"allocated":1875968,"data_stored":1151076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476806,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32468,"kb_used_data":1272,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339372,"statfs":{"total":96636764160,"available":96603516928,"internally_reserved":0,"allocated":1302528,"data_stored":567660,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:51:35.026 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:51:35.027 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:51:35.060 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-09-21T07:51:35.059+0000 7f7724e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:51:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 -- 172.21.15.88:0/2422923195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00cd240 msgr2=0x7f6cd00b3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 --2- 172.21.15.88:0/2422923195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00cd240 0x7f6cd00b3fc0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc005fd0 tx=0x7f6ccc02aa90 comp rx=0 tx=0).stop 2024-09-21T07:51:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 -- 172.21.15.88:0/2422923195 shutdown_connections 2024-09-21T07:51:35.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 --2- 172.21.15.88:0/2422923195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00cd240 0x7f6cd00b3fc0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 --2- 172.21.15.88:0/2422923195 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cd00af4e0 0x7f6cd015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 -- 172.21.15.88:0/2422923195 >> 172.21.15.88:0/2422923195 conn(0x7f6cd00b1dd0 msgr2=0x7f6cd00b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 -- 172.21.15.88:0/2422923195 shutdown_connections 2024-09-21T07:51:35.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.286+0000 7f6cd7200640 1 -- 172.21.15.88:0/2422923195 wait complete. 2024-09-21T07:51:35.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.288+0000 7f6cd7200640 1 Processor -- start 2024-09-21T07:51:35.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.288+0000 7f6cd7200640 1 -- start start 2024-09-21T07:51:35.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 0x7f6cd015be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd0160370 0x7f6cd01f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cd009b710 0x7f6cd0155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd00b4500 con 0x7f6cd00af4e0 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6cd0160990 con 0x7f6cd0160370 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6cd00b4680 con 0x7f6cd009b710 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 0x7f6cd015be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 0x7f6cd015be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:34472/0 (socket says 172.21.15.88:34472) 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd5e00640 1 -- 172.21.15.88:0/1789540677 learned_addr learned my addr 172.21.15.88:0/1789540677 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd5400640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd0160370 0x7f6cd01f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd6800640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cd009b710 0x7f6cd0155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd5e00640 1 -- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cd009b710 msgr2=0x7f6cd0155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.290+0000 7f6cd5e00640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cd009b710 0x7f6cd0155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd5e00640 1 -- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd0160370 msgr2=0x7f6cd01f57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd5e00640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd0160370 0x7f6cd01f57b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd5e00640 1 -- 172.21.15.88:0/1789540677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cd0072ed0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd5400640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd0160370 0x7f6cd01f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:35.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd5e00640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 0x7f6cd015be20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400d740 tx=0x7f6cc400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4017070 con 0x7f6cd00af4e0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6cc4009d20 con 0x7f6cd00af4e0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cd01f5fe0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc40106c0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.291+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cd01f65a0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.292+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6cd00b4b80 con 0x7f6cd00af4e0 2024-09-21T07:51:35.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.294+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f6cc401a070 con 0x7f6cd00af4e0 2024-09-21T07:51:35.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.294+0000 7f6cca000640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6cb4078cb0 0x7f6cb407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.295+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f6cc409b1a0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.295+0000 7f6cd5400640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6cb4078cb0 0x7f6cb407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.295+0000 7f6cd5400640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6cb4078cb0 0x7f6cb407b1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6cd009c920 tx=0x7f6cc0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:35.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.299+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6cc40633b0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.557+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f6cd0051040 con 0x7f6cd00af4e0 2024-09-21T07:51:35.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.560+0000 7f6cca000640 1 -- 172.21.15.88:0/1789540677 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) ==== 190+0+147 (secure 0 0 0) 0x7f6cc4062cb0 con 0x7f6cd00af4e0 2024-09-21T07:51:35.561 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:51:35.561 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:51:35.561 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:51:35.561 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:51:35.561 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:51:35.562 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.562+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6cb4078cb0 msgr2=0x7f6cb407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.562+0000 7f6cd7200640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6cb4078cb0 0x7f6cb407b1c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6cd009c920 tx=0x7f6cc0009040 comp rx=0 tx=0).stop 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.562+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 msgr2=0x7f6cd015be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.562+0000 7f6cd7200640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 0x7f6cd015be20 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400d740 tx=0x7f6cc400dc00 comp rx=0 tx=0).stop 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 shutdown_connections 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6cb4078cb0 0x7f6cb407b1c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6cd009b710 0x7f6cd0155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6cd0160370 0x7f6cd01f57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 --2- 172.21.15.88:0/1789540677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6cd00af4e0 0x7f6cd015be20 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 >> 172.21.15.88:0/1789540677 conn(0x7f6cd00b1dd0 msgr2=0x7f6cd006ef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:35.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 shutdown_connections 2024-09-21T07:51:35.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.563+0000 7f6cd7200640 1 -- 172.21.15.88:0/1789540677 wait complete. 2024-09-21T07:51:35.588 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:51:35.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 -- 172.21.15.88:0/4081656074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 msgr2=0x7f10640b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 --2- 172.21.15.88:0/4081656074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f10640b3960 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f105c005fd0 tx=0x7f105c02aa10 comp rx=0 tx=0).stop 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 -- 172.21.15.88:0/4081656074 shutdown_connections 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 --2- 172.21.15.88:0/4081656074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f10640b3960 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 --2- 172.21.15.88:0/4081656074 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f106415d760 0x7f10641535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 -- 172.21.15.88:0/4081656074 >> 172.21.15.88:0/4081656074 conn(0x7f106406d730 msgr2=0x7f10640b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 -- 172.21.15.88:0/4081656074 shutdown_connections 2024-09-21T07:51:35.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.808+0000 7f1068e00640 1 -- 172.21.15.88:0/4081656074 wait complete. 2024-09-21T07:51:35.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.810+0000 7f1068e00640 1 Processor -- start 2024-09-21T07:51:35.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.811+0000 7f1068e00640 1 -- start start 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1068e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640ad0e0 0x7f10641f63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1068e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f1064150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1068e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415d760 0x7f1064099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1068e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10640b4da0 con 0x7f10640b3550 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1068e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10640b4c20 con 0x7f106415d760 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1068e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10640b3ea0 con 0x7f10640ad0e0 2024-09-21T07:51:35.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1061400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f1064150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1061400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f1064150b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44464/0 (socket says 172.21.15.88:44464) 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1061400640 1 -- 172.21.15.88:0/1850407417 learned_addr learned my addr 172.21.15.88:0/1850407417 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1061e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640ad0e0 0x7f10641f63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.812+0000 7f1062800640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415d760 0x7f1064099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1061400640 1 -- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640ad0e0 msgr2=0x7f10641f63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1061400640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640ad0e0 0x7f10641f63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1061400640 1 -- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415d760 msgr2=0x7f1064099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1061400640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415d760 0x7f1064099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1061400640 1 -- 172.21.15.88:0/1850407417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1064151080 con 0x7f10640b3550 2024-09-21T07:51:35.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1061400640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f1064150b40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f105000c390 tx=0x7f105000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050017070 con 0x7f10640b3550 2024-09-21T07:51:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10500035f0 con 0x7f10640b3550 2024-09-21T07:51:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f106414faa0 con 0x7f10640b3550 2024-09-21T07:51:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.813+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1050003a20 con 0x7f10640b3550 2024-09-21T07:51:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.814+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1064150060 con 0x7f10640b3550 2024-09-21T07:51:35.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.814+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10640b4330 con 0x7f10640b3550 2024-09-21T07:51:35.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.816+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f105001a070 con 0x7f10640b3550 2024-09-21T07:51:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.817+0000 7f105a000640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1044078cb0 0x7f104407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.817+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 5709+0+0 (secure 0 0 0) 0x7f105009a1d0 con 0x7f10640b3550 2024-09-21T07:51:35.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.817+0000 7f1061e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1044078cb0 0x7f104407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:35.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.818+0000 7f1061e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1044078cb0 0x7f104407b1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1054005fd0 tx=0x7f105400a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:35.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:35.821+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1050061d90 con 0x7f10640b3550 2024-09-21T07:51:36.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.043+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f106415f730 con 0x7f10640b3550 2024-09-21T07:51:36.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.043+0000 7f105a000640 1 -- 172.21.15.88:0/1850407417 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 70 v70) ==== 94+0+5225 (secure 0 0 0) 0x7f10500667c0 con 0x7f10640b3550 2024-09-21T07:51:36.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:51:36.045 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":70,"btime":"2024-09-21T07:51:33:116260+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:51:29.581113+0000","modified":"2024-09-21T07:51:32.603721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6975":{"gid":6975,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3744288878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3744288878},{"type":"v1","addr":"172.21.15.88:6835","nonce":3744288878}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6975,"qdb_cluster":[6975]},"id":8}]} 2024-09-21T07:51:36.045 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 70 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1044078cb0 msgr2=0x7f104407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1044078cb0 0x7f104407b1c0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f1054005fd0 tx=0x7f105400a250 comp rx=0 tx=0).stop 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 msgr2=0x7f1064150b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f1064150b40 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f105000c390 tx=0x7f105000c850 comp rx=0 tx=0).stop 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 shutdown_connections 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1044078cb0 0x7f104407b1c0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f106415d760 0x7f1064099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10640b3550 0x7f1064150b40 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 --2- 172.21.15.88:0/1850407417 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10640ad0e0 0x7f10641f63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 >> 172.21.15.88:0/1850407417 conn(0x7f106406d730 msgr2=0x7f10641602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 shutdown_connections 2024-09-21T07:51:36.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.046+0000 7f1068e00640 1 -- 172.21.15.88:0/1850407417 wait complete. 2024-09-21T07:51:36.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 70, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:51:29.581113+0000', 'modified': '2024-09-21T07:51:32.603721+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6975}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6975': {'gid': 6975, 'name': 'a', 'rank': 0, 'incarnation': 66, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/3744288878', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 3744288878}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 3744288878}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 6975, 'qdb_cluster': [6975]} 2024-09-21T07:51:36.471 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:51:36.471 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:51:36.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 -- 172.21.15.88:0/2214952971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b100cf490 msgr2=0x7f0b1015cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 --2- 172.21.15.88:0/2214952971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b100cf490 0x7f0b1015cc30 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0b10074df0 tx=0x7f0b0002a530 comp rx=0 tx=0).stop 2024-09-21T07:51:36.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 -- 172.21.15.88:0/2214952971 shutdown_connections 2024-09-21T07:51:36.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 --2- 172.21.15.88:0/2214952971 >> v2:172.21.15.175:3300/0 conn(0x7f0b100d2990 0x7f0b100d0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 --2- 172.21.15.88:0/2214952971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b100cf490 0x7f0b1015cc30 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 -- 172.21.15.88:0/2214952971 >> 172.21.15.88:0/2214952971 conn(0x7f0b1006d1c0 msgr2=0x7f0b1006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 -- 172.21.15.88:0/2214952971 shutdown_connections 2024-09-21T07:51:36.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.695+0000 7f0b14a00640 1 -- 172.21.15.88:0/2214952971 wait complete. 2024-09-21T07:51:36.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.697+0000 7f0b14a00640 1 Processor -- start 2024-09-21T07:51:36.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.697+0000 7f0b14a00640 1 -- start start 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b14a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100cf490 0x7f0b1014f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b14a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100cf8a0 0x7f0b100cb280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b14a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 0x7f0b100cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b14a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100d0780 con 0x7f0b10157910 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b14a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b1015d380 con 0x7f0b100cf490 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b14a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100d0900 con 0x7f0b100cf8a0 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 0x7f0b100cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 0x7f0b100cb7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44474/0 (socket says 172.21.15.88:44474) 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b0fe00640 1 -- 172.21.15.88:0/21611926 learned_addr learned my addr 172.21.15.88:0/21611926 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b0f400640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100cf490 0x7f0b1014f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.699+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100cf8a0 0x7f0b100cb280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b0fe00640 1 -- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100cf8a0 msgr2=0x7f0b100cb280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100cf8a0 0x7f0b100cb280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b0fe00640 1 -- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100cf490 msgr2=0x7f0b1014f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100cf490 0x7f0b1014f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b0fe00640 1 -- 172.21.15.88:0/21611926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b101f9ca0 con 0x7f0b10157910 2024-09-21T07:51:36.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 0x7f0b100cb7c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800e960 tx=0x7f0b0800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b07400640 1 -- 172.21.15.88:0/21611926 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08004d20 con 0x7f0b10157910 2024-09-21T07:51:36.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b07400640 1 -- 172.21.15.88:0/21611926 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b0800cb20 con 0x7f0b10157910 2024-09-21T07:51:36.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b101f8f20 con 0x7f0b10157910 2024-09-21T07:51:36.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b07400640 1 -- 172.21.15.88:0/21611926 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08005020 con 0x7f0b10157910 2024-09-21T07:51:36.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.700+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b101f94a0 con 0x7f0b10157910 2024-09-21T07:51:36.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.704+0000 7f0b07400640 1 -- 172.21.15.88:0/21611926 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f0b080051c0 con 0x7f0b10157910 2024-09-21T07:51:36.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.705+0000 7f0b07400640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0af0078cb0 0x7f0af007b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.705+0000 7f0b07400640 1 -- 172.21.15.88:0/21611926 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f0b0809b8a0 con 0x7f0b10157910 2024-09-21T07:51:36.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.705+0000 7f0b0f400640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0af0078cb0 0x7f0af007b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.705+0000 7f0b1d039740 1 Processor -- start 2024-09-21T07:51:36.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.706+0000 7f0b1d039740 1 -- start start 2024-09-21T07:51:36.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.706+0000 7f0b0f400640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0af0078cb0 0x7f0af007b1c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b00034000 tx=0x7f0b00004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f3fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f400a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f10150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b1d039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563b83e99430 con 0x563b83b463b0 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b83e895d0 con 0x563b83b4e590 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563b83e78a00 con 0x563b83b6f270 2024-09-21T07:51:36.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b0ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44480/0 (socket says 172.21.15.88:44480) 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b0ea00640 1 -- 172.21.15.88:0/3264408420 learned_addr learned my addr 172.21.15.88:0/3264408420 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f10150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.707+0000 7f0b0f400640 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f3fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0ea00640 1 -- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 msgr2=0x563b83f3fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f3fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0ea00640 1 -- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 msgr2=0x563b83f10150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f10150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0ea00640 1 -- 172.21.15.88:0/3264408420 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b83e41520 con 0x563b83b463b0 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0f400640 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f3fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f400a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0017f0 tx=0x7f0afc007db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de8e8ff5c8e3f3e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0afb400640 1 -- 172.21.15.88:0/3264408420 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc0052e0 con 0x563b83b463b0 2024-09-21T07:51:36.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0afb400640 1 -- 172.21.15.88:0/3264408420 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0afc018070 con 0x563b83b463b0 2024-09-21T07:51:36.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.708+0000 7f0afb400640 1 -- 172.21.15.88:0/3264408420 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc00f9d0 con 0x563b83b463b0 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 -- 172.21.15.88:0/3264408420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 msgr2=0x563b83f400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f400a0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0afc0017f0 tx=0x7f0afc007db0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 -- 172.21.15.88:0/3264408420 shutdown_connections 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f10150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f400a0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3264408420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f3fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 -- 172.21.15.88:0/3264408420 >> 172.21.15.88:0/3264408420 conn(0x563b83b633c0 msgr2=0x563b83f15b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 -- 172.21.15.88:0/3264408420 shutdown_connections 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.709+0000 7f0b1d039740 1 -- 172.21.15.88:0/3264408420 wait complete. 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.710+0000 7f0b1d039740 1 -- start start 2024-09-21T07:51:36.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.711+0000 7f0b1d039740 1 Processor -- start 2024-09-21T07:51:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f5b200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f35e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f36350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563b83e1f330 con 0x563b83b463b0 2024-09-21T07:51:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b83e7c810 con 0x563b83b4e590 2024-09-21T07:51:36.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563b83be9620 con 0x563b83b6f270 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b1d039740 10 client.0 _ll_register_callbacks cb 0x563b83e24420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f5b200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f5b200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44492/0 (socket says 172.21.15.88:44492) 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.712+0000 7f0b0f400640 1 -- 172.21.15.88:0/2459090382 learned_addr learned my addr 172.21.15.88:0/2459090382 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f36350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f35e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0f400640 1 -- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 msgr2=0x563b83f36350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0f400640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f36350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0f400640 1 -- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 msgr2=0x563b83f35e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0f400640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f35e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0f400640 1 -- 172.21.15.88:0/2459090382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b83bda850 con 0x563b83b463b0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b0f400640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f5b200 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009920 tx=0x7f0b00005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2a146f40a16483 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0aefe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:51:36.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0aefe00640 1 -- 172.21.15.88:0/2459090382 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0003cb70 con 0x563b83b463b0 2024-09-21T07:51:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0aefe00640 1 -- 172.21.15.88:0/2459090382 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f0b0003cd10 con 0x563b83b463b0 2024-09-21T07:51:36.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.713+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0afc018070 con 0x563b83b463b0 2024-09-21T07:51:36.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0aefe00640 1 -- 172.21.15.88:0/2459090382 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0b00058050 con 0x563b83b463b0 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 10 client.7095 fetch_fsmap learned FSMap version 70 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x563b83c516b0 con 0x563b83b463b0 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0aefe00640 1 -- 172.21.15.88:0/2459090382 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0afc018070 con 0x563b83b463b0 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0aefe00640 1 -- 172.21.15.88:0/2459090382 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f0b00005020 con 0x563b83b463b0 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 10 client.7095 fetch_fsmap finished waiting for FSMap version 70 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 10 client.7095 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 20 client.7095 populate_metadata read hostname 'smithi088' 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83b2de40 0x563b83f3bac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 4 client.7095 mds_command: new command op to 6975 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 --> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b83e8b520 con 0x563b83b2de40 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.714+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83b2de40 0x563b83f3bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.715+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83b2de40 0x563b83f3bac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eeeb8981977c6f7 server_cookie=a45c111e1f620274 in_seq=0 out_seq=0 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.715+0000 7f0aefe00640 10 client.7095 ms_handle_connect on v2:172.21.15.88:6832/3744288878 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0aefe00640 1 -- 172.21.15.88:0/2459090382 <== mds.0 v2:172.21.15.88:6832/3744288878 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563b83e8b520 con 0x563b83b2de40 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0aefe00640 10 client.7095 handle_command_reply: tid=0 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 client.7095 shutdown 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 msgr2=0x563b83f5b200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f5b200 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009920 tx=0x7f0b00005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 shutdown_connections 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83b2de40 0x563b83f3bac0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f36350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f35e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 --2- 172.21.15.88:0/2459090382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83f5b200 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0aefe00640 0 client.7095 ms_handle_reset on v2:172.21.15.88:6832/3744288878 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 >> 172.21.15.88:0/2459090382 conn(0x563b83b633c0 msgr2=0x563b83f5def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 shutdown_connections 2024-09-21T07:51:36.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 1 -- 172.21.15.88:0/2459090382 wait complete. 2024-09-21T07:51:36.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.716+0000 7f0b1d039740 20 client.7095 trim_cache size 0 max 16384 2024-09-21T07:51:36.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.735+0000 7f0b1d039740 1 Processor -- start 2024-09-21T07:51:36.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.735+0000 7f0b1d039740 1 -- start start 2024-09-21T07:51:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.736+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b463b0 0x563b83f35fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.736+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b4e590 0x563b83f11620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 0x563b83f538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b1d039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b10201230 con 0x563b83b6f270 2024-09-21T07:51:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b100b6040 con 0x563b83b463b0 2024-09-21T07:51:36.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b100b7190 con 0x563b83b4e590 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 0x563b83f538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 0x563b83f538a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44496/0 (socket says 172.21.15.88:44496) 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 -- 172.21.15.88:0/3907008970 learned_addr learned my addr 172.21.15.88:0/3907008970 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0f400640 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b463b0 0x563b83f35fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b4e590 0x563b83f11620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 -- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b4e590 msgr2=0x563b83f11620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b4e590 0x563b83f11620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 -- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b463b0 msgr2=0x563b83f35fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b463b0 0x563b83f35fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 -- 172.21.15.88:0/3907008970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b83f366b0 con 0x563b83b6f270 2024-09-21T07:51:36.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.737+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 0x563b83f538a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0b08063430 tx=0x7f0b08062d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfeb93aa86ac027c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0afaa00640 1 -- 172.21.15.88:0/3907008970 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0801f040 con 0x563b83b6f270 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0afaa00640 1 -- 172.21.15.88:0/3907008970 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b0806cd80 con 0x563b83b6f270 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0afaa00640 1 -- 172.21.15.88:0/3907008970 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b08085dc0 con 0x563b83b6f270 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 -- 172.21.15.88:0/3907008970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 msgr2=0x563b83f538a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 0x563b83f538a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0b08063430 tx=0x7f0b08062d30 comp rx=0 tx=0).stop 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 -- 172.21.15.88:0/3907008970 shutdown_connections 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b4e590 0x563b83f11620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b6f270 0x563b83f538a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 --2- 172.21.15.88:0/3907008970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b463b0 0x563b83f35fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 -- 172.21.15.88:0/3907008970 >> 172.21.15.88:0/3907008970 conn(0x563b83b633c0 msgr2=0x563b83f0b5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 -- 172.21.15.88:0/3907008970 shutdown_connections 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.738+0000 7f0b1d039740 1 -- 172.21.15.88:0/3907008970 wait complete. 2024-09-21T07:51:36.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.739+0000 7f0b1d039740 1 -- start start 2024-09-21T07:51:36.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.740+0000 7f0b1d039740 1 Processor -- start 2024-09-21T07:51:36.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83fd0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f7d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f7da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563b83f554f0 con 0x563b83b463b0 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x563b83f54f30 con 0x563b83b4e590 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x563b83f39220 con 0x563b83b6f270 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b1d039740 10 client.0 _ll_register_callbacks cb 0x7f0b00003a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83fd0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b0f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83fd0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44510/0 (socket says 172.21.15.88:44510) 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.741+0000 7f0b0f400640 1 -- 172.21.15.88:0/181514487 learned_addr learned my addr 172.21.15.88:0/181514487 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f7d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0fe00640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f7da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0f400640 1 -- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 msgr2=0x563b83f7da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0f400640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f7da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0f400640 1 -- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 msgr2=0x563b83f7d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0f400640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f7d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0f400640 1 -- 172.21.15.88:0/181514487 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b83fd11b0 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b0f400640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83fd0c70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009920 tx=0x7f0b0003c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79d949a769b71d1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0aefe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0aefe00640 1 -- 172.21.15.88:0/181514487 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0003cd60 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0aefe00640 1 -- 172.21.15.88:0/181514487 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7f0b00033720 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.742+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b0806cd80 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0aefe00640 1 -- 172.21.15.88:0/181514487 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=70) ==== 24+0+0 (secure 0 0 0) 0x7f0b00033920 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 10 client.7101 fetch_fsmap learned FSMap version 70 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=70,monmap=2+}) -- 0x563b83f59d10 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0aefe00640 1 -- 172.21.15.88:0/181514487 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0b0806cd80 con 0x563b83b463b0 2024-09-21T07:51:36.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0aefe00640 1 -- 172.21.15.88:0/181514487 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 70) ==== 3502+0+0 (secure 0 0 0) 0x7f0b00033ac0 con 0x563b83b463b0 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 10 client.7101 fetch_fsmap finished waiting for FSMap version 70 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 10 client.7101 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 20 client.7101 populate_metadata read hostname 'smithi088' 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83fde240 0x563b83f7c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 4 client.7101 mds_command: new command op to 6975 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 --> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] -- command(tid 0: {"prefix": "status"}) -- 0x563b83e86fd0 con 0x563b83fde240 2024-09-21T07:51:36.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.743+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83fde240 0x563b83f7c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b0ea00640 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83fde240 0x563b83f7c4a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=491fb3325bddf5b4 server_cookie=a90c443a2478215b in_seq=0 out_seq=0 2024-09-21T07:51:36.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0aefe00640 10 client.7101 ms_handle_connect on v2:172.21.15.88:6832/3744288878 2024-09-21T07:51:36.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0aefe00640 1 -- 172.21.15.88:0/181514487 <== mds.0 v2:172.21.15.88:6832/3744288878 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x563b83e86fd0 con 0x563b83fde240 2024-09-21T07:51:36.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0aefe00640 10 client.7101 handle_command_reply: tid=0 2024-09-21T07:51:36.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 client.7101 shutdown 2024-09-21T07:51:36.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 msgr2=0x563b83fd0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83fd0c70 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009920 tx=0x7f0b0003c870 comp rx=0 tx=0).stop 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 shutdown_connections 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:6832/3744288878,v1:172.21.15.88:6835/3744288878] conn(0x563b83fde240 0x563b83f7c4a0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x563b83b6f270 0x563b83f7da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x563b83b4e590 0x563b83f7d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 --2- 172.21.15.88:0/181514487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b83b463b0 0x563b83fd0c70 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0aefe00640 0 client.7101 ms_handle_reset on v2:172.21.15.88:6832/3744288878 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.744+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 >> 172.21.15.88:0/181514487 conn(0x563b83b633c0 msgr2=0x563b83f3f0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 shutdown_connections 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b1d039740 1 -- 172.21.15.88:0/181514487 wait complete. 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b1d039740 20 client.7101 trim_cache size 0 max 16384 2024-09-21T07:51:36.749 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 6975, 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6402951269999999, 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 70, 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 106, 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 105, 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.711734720999999 2024-09-21T07:51:36.750 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:36.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0af0078cb0 msgr2=0x7f0af007b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0af0078cb0 0x7f0af007b1c0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f0b00034000 tx=0x7f0b00004730 comp rx=0 tx=0).stop 2024-09-21T07:51:36.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 msgr2=0x7f0b100cb7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 0x7f0b100cb7c0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0b0800e960 tx=0x7f0b0800ee20 comp rx=0 tx=0).stop 2024-09-21T07:51:36.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 shutdown_connections 2024-09-21T07:51:36.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0af0078cb0 0x7f0af007b1c0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b100cf8a0 0x7f0b100cb280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b10157910 0x7f0b100cb7c0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 --2- 172.21.15.88:0/21611926 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0b100cf490 0x7f0b1014f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.745+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 >> 172.21.15.88:0/21611926 conn(0x7f0b1006d1c0 msgr2=0x7f0b1015f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.746+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 shutdown_connections 2024-09-21T07:51:36.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.746+0000 7f0b14a00640 1 -- 172.21.15.88:0/21611926 wait complete. 2024-09-21T07:51:36.771 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 -- 172.21.15.88:0/390543159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1480ad0e0 msgr2=0x7fe1480ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 --2- 172.21.15.88:0/390543159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1480ad0e0 0x7fe1480ad4f0 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe148072d90 tx=0x7fe13402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 -- 172.21.15.88:0/390543159 shutdown_connections 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 --2- 172.21.15.88:0/390543159 >> v2:172.21.15.175:3300/0 conn(0x7fe148160310 0x7fe148160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 --2- 172.21.15.88:0/390543159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1480ad0e0 0x7fe1480ad4f0 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 --2- 172.21.15.88:0/390543159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe14815d760 0x7fe1481535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 -- 172.21.15.88:0/390543159 >> 172.21.15.88:0/390543159 conn(0x7fe14806d730 msgr2=0x7fe1480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 -- 172.21.15.88:0/390543159 shutdown_connections 2024-09-21T07:51:36.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.990+0000 7fe14ca00640 1 -- 172.21.15.88:0/390543159 wait complete. 2024-09-21T07:51:36.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.992+0000 7fe14ca00640 1 Processor -- start 2024-09-21T07:51:36.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.993+0000 7fe14ca00640 1 -- start start 2024-09-21T07:51:36.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe14ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1480ad0e0 0x7fe148159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe14ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 0x7fe1481f5cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe14ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe148160310 0x7fe1481575c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe14ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe1480b4da0 con 0x7fe14815d760 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe14ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1480b4c20 con 0x7fe148160310 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe14ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe148160c60 con 0x7fe1480ad0e0 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe145400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 0x7fe1481f5cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe145400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 0x7fe1481f5cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44520/0 (socket says 172.21.15.88:44520) 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe145400640 1 -- 172.21.15.88:0/3670553105 learned_addr learned my addr 172.21.15.88:0/3670553105 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.994+0000 7fe146800640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe148160310 0x7fe1481575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145e00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1480ad0e0 0x7fe148159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145400640 1 -- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1480ad0e0 msgr2=0x7fe148159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145400640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1480ad0e0 0x7fe148159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145400640 1 -- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe148160310 msgr2=0x7fe1481575c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:36.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145400640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe148160310 0x7fe1481575c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145400640 1 -- 172.21.15.88:0/3670553105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe148151c40 con 0x7fe14815d760 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145e00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1480ad0e0 0x7fe148159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe145400640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 0x7fe1481f5cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe134034040 tx=0x7fe1340038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe13403e030 con 0x7fe14815d760 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe134003ca0 con 0x7fe14815d760 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe134035040 con 0x7fe14815d760 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.995+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1481f6890 con 0x7fe14815d760 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.996+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1481593e0 con 0x7fe14815d760 2024-09-21T07:51:36.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.997+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1481f6600 con 0x7fe14815d760 2024-09-21T07:51:36.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.998+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fe134041030 con 0x7fe14815d760 2024-09-21T07:51:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.999+0000 7fe13e000640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe128078cb0 0x7fe12807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.999+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 5767+0+0 (secure 0 0 0) 0x7fe1340bb720 con 0x7fe14815d760 2024-09-21T07:51:37.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:36.999+0000 7fe145e00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe128078cb0 0x7fe12807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:37.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.000+0000 7fe145e00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe128078cb0 0x7fe12807b1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe138009b80 tx=0x7fe138005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:37.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.004+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe134083210 con 0x7fe14815d760 2024-09-21T07:51:37.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.227+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe1481535c0 con 0x7fe14815d760 2024-09-21T07:51:37.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.228+0000 7fe13e000640 1 -- 172.21.15.88:0/3670553105 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7fe134087c40 con 0x7fe14815d760 2024-09-21T07:51:37.229 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe128078cb0 msgr2=0x7fe12807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe128078cb0 0x7fe12807b1c0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fe138009b80 tx=0x7fe138005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 msgr2=0x7fe1481f5cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 0x7fe1481f5cb0 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7fe134034040 tx=0x7fe1340038a0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 shutdown_connections 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe128078cb0 0x7fe12807b1c0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe148160310 0x7fe1481575c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe14815d760 0x7fe1481f5cb0 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 --2- 172.21.15.88:0/3670553105 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1480ad0e0 0x7fe148159ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.230+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 >> 172.21.15.88:0/3670553105 conn(0x7fe14806d730 msgr2=0x7fe14815a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:37.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.231+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 shutdown_connections 2024-09-21T07:51:37.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37.231+0000 7fe14ca00640 1 -- 172.21.15.88:0/3670553105 wait complete. 2024-09-21T07:51:37.253 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:51:37.253 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:51:37.254 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:37.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:37.254 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:51:37.289 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:51:37.289 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:51:37.289 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:51:37.290 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42397sec preferred_lft 42397sec 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:51:37.291 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:37.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:37.292 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:51:37.386 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:51:37.388 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:37.388 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:51:37.424 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:51:37.424 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:51:37.424 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:51:37.424 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:51:37.424 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:51:37.424 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:51:37.424 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:51:37.424 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:51:37.425 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:51:37.425 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:51:37.425 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:51:37.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:51:37.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:37.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:37.622 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:51:37.656 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:51:37.657 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:51:37.657 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:37.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:37.658 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:51:37.755 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:37.755 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:37.789 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:51:37.790 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:37.790 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:37.864 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:51:37.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:37.865 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:38.059 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:38.060 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:51:38.060 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:51:38.080 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:38.080 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:51:38.080 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:51:38.080 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:38.080 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:51:38.115 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:51:38.116 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:51:38.117 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:51:38.117 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:51:38.117 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:51:38.117 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:51:38.120 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:51:38.120 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:38.120 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:51:38.175 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:51:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:51:38.263 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:38.271 INFO:teuthology.orchestra.run.smithi088.stdout:client6194 2024-09-21T07:51:38.271 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:38.271 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:38.306 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:51:38.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 -- 172.21.15.88:0/2246645408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 msgr2=0x7feefc0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 --2- 172.21.15.88:0/2246645408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 0x7feefc0af170 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feeec005fd0 tx=0x7feeec02a820 comp rx=0 tx=0).stop 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 -- 172.21.15.88:0/2246645408 shutdown_connections 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 --2- 172.21.15.88:0/2246645408 >> v2:172.21.15.175:3300/0 conn(0x7feefc0b2f60 0x7feefc15a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 --2- 172.21.15.88:0/2246645408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 0x7feefc0af170 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 -- 172.21.15.88:0/2246645408 >> 172.21.15.88:0/2246645408 conn(0x7feefc0b1e40 msgr2=0x7feefc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 -- 172.21.15.88:0/2246645408 shutdown_connections 2024-09-21T07:51:38.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.567+0000 7fef01a00640 1 -- 172.21.15.88:0/2246645408 wait complete. 2024-09-21T07:51:38.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.569+0000 7fef01a00640 1 Processor -- start 2024-09-21T07:51:38.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.570+0000 7fef01a00640 1 -- start start 2024-09-21T07:51:38.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7fef01a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 0x7feefc072e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:38.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7fef01a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 0x7feefc09dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:38.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7fef01a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feefc15d670 0x7feefc154890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:38.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7fef01a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feefc06f410 con 0x7feefc0b2f60 2024-09-21T07:51:38.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7fef01a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feefc06f290 con 0x7feefc15d670 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7fef01a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feefc06f590 con 0x7feefc0ad9c0 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7feefb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 0x7feefc09dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7feefb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 0x7feefc09dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44534/0 (socket says 172.21.15.88:44534) 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.571+0000 7feefb400640 1 -- 172.21.15.88:0/2272409945 learned_addr learned my addr 172.21.15.88:0/2272409945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefbe00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 0x7feefc072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7fef01000640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feefc15d670 0x7feefc154890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:38.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefb400640 1 -- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 msgr2=0x7feefc072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:38.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefb400640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 0x7feefc072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefb400640 1 -- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feefc15d670 msgr2=0x7feefc154890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:38.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefb400640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feefc15d670 0x7feefc154890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefb400640 1 -- 172.21.15.88:0/2272409945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feefc074510 con 0x7feefc0b2f60 2024-09-21T07:51:38.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7fef01000640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feefc15d670 0x7feefc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feefb400640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 0x7feefc09dd40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feee800e9f0 tx=0x7feee800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee800cd80 con 0x7feefc0b2f60 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feee8004020 con 0x7feefc0b2f60 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feefc0747f0 con 0x7feefc0b2f60 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.572+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feee8004450 con 0x7feefc0b2f60 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.573+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feefc074db0 con 0x7feefc0b2f60 2024-09-21T07:51:38.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.574+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feefc06a610 con 0x7feefc0b2f60 2024-09-21T07:51:38.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.575+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7feee80035f0 con 0x7feefc0b2f60 2024-09-21T07:51:38.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.576+0000 7feef3e00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feedc078cb0 0x7feedc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:38.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.576+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7feee809b9d0 con 0x7feefc0b2f60 2024-09-21T07:51:38.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.576+0000 7feefbe00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feedc078cb0 0x7feedc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:38.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.577+0000 7feefbe00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feedc078cb0 0x7feedc07b1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feeec034000 tx=0x7feeec004ab0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:38.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.583+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feefc06a610 con 0x7feefc0b2f60 2024-09-21T07:51:38.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.806+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7feefc06f9b0 con 0x7feefc0b2f60 2024-09-21T07:51:38.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.807+0000 7feef3e00640 1 -- 172.21.15.88:0/2272409945 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v70) ==== 53+0+73 (secure 0 0 0) 0x7feee8063b10 con 0x7feefc0b2f60 2024-09-21T07:51:38.808 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feedc078cb0 msgr2=0x7feedc07b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feedc078cb0 0x7feedc07b1c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7feeec034000 tx=0x7feeec004ab0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 msgr2=0x7feefc09dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 0x7feefc09dd40 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7feee800e9f0 tx=0x7feee800eeb0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 shutdown_connections 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feedc078cb0 0x7feedc07b1c0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feefc15d670 0x7feefc154890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feefc0b2f60 0x7feefc09dd40 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 --2- 172.21.15.88:0/2272409945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feefc0ad9c0 0x7feefc072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.809+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 >> 172.21.15.88:0/2272409945 conn(0x7feefc0b1e40 msgr2=0x7feefc070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.810+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 shutdown_connections 2024-09-21T07:51:38.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:38.810+0000 7fef01a00640 1 -- 172.21.15.88:0/2272409945 wait complete. 2024-09-21T07:51:38.837 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:51:38.837 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:51:38.837 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:51:38.837 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:51:38.837 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:38.838 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:51:38.838 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:51:38.838 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:51:38.838 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:51:38.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:38.838 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:51:38.878 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:38.879 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42346sec preferred_lft 42346sec 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.880 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:51:38.881 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:51:38.881 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:51:38.881 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:51:38.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:38.881 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:51:38.976 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:51:38.979 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:38.979 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:51:39.016 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:51:39.016 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:51:39.016 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:51:39.016 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:51:39.016 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:51:39.017 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:51:39.017 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:51:39.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:51:39.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:51:39.017 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:51:39.017 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:51:39.133 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:39 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:51:39.204 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:39.212 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:39.212 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:51:39.246 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:51:39.246 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:51:39.247 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:51:39.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:39.247 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:51:39.343 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:39.344 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:39.378 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:51:39.379 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:51:39.379 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:39.453 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:51:39.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:39.453 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:39.648 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:39.648 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:51:39.648 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:51:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:51:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:51:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:39.668 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:51:39.695 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:51:39.697 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:51:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:51:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:51:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:51:39.697 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:51:39.700 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:51:39.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:39.700 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:51:39.755 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:51:39.833 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:39 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:51:39.836 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:51:39 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:51:39.844 INFO:teuthology.orchestra.run.smithi175.stdout:client6203 2024-09-21T07:51:39.844 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:51:39.845 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:39.880 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.112+0000 7f8229200640 1 -- 172.21.15.88:0/4244934400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f822406d4b0 msgr2=0x7f82240b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.112+0000 7f8229200640 1 --2- 172.21.15.88:0/4244934400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f822406d4b0 0x7f82240b4620 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f8210005fd0 tx=0x7f821002a670 comp rx=0 tx=0).stop 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.113+0000 7f8229200640 1 -- 172.21.15.88:0/4244934400 shutdown_connections 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.113+0000 7f8229200640 1 --2- 172.21.15.88:0/4244934400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f822406d4b0 0x7f82240b4620 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.113+0000 7f8229200640 1 -- 172.21.15.88:0/4244934400 >> 172.21.15.88:0/4244934400 conn(0x7f82240b2540 msgr2=0x7f82240d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.113+0000 7f8229200640 1 -- 172.21.15.88:0/4244934400 shutdown_connections 2024-09-21T07:51:40.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.113+0000 7f8229200640 1 -- 172.21.15.88:0/4244934400 wait complete. 2024-09-21T07:51:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.115+0000 7f8229200640 1 Processor -- start 2024-09-21T07:51:40.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.115+0000 7f8229200640 1 -- start start 2024-09-21T07:51:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.116+0000 7f8229200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 0x7f82240c3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.116+0000 7f8229200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82240cc830 0x7f822406f640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8229200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82241530e0 0x7f822406fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8229200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f82240b3780 con 0x7f82240af100 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8229200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f82240b3600 con 0x7f82240cc830 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8229200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f82240b3900 con 0x7f82241530e0 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 0x7f82240c3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 0x7f82240c3010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44564/0 (socket says 172.21.15.88:44564) 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 -- 172.21.15.88:0/1448907705 learned_addr learned my addr 172.21.15.88:0/1448907705 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8222800640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82241530e0 0x7f822406fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221400640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82240cc830 0x7f822406f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 -- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82241530e0 msgr2=0x7f822406fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82241530e0 0x7f822406fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 -- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82240cc830 msgr2=0x7f822406f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82240cc830 0x7f822406f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.117+0000 7f8221e00640 1 -- 172.21.15.88:0/1448907705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82240c3550 con 0x7f82240af100 2024-09-21T07:51:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.118+0000 7f8221e00640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 0x7f82240c3010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f821400c430 tx=0x7f821400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.118+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214017070 con 0x7f82240af100 2024-09-21T07:51:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.118+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f82140053c0 con 0x7f82240af100 2024-09-21T07:51:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.118+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f822409b520 con 0x7f82240af100 2024-09-21T07:51:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.118+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8214003850 con 0x7f82240af100 2024-09-21T07:51:40.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.118+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f822409bae0 con 0x7f82240af100 2024-09-21T07:51:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.119+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82240b3dd0 con 0x7f82240af100 2024-09-21T07:51:40.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.120+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f821401a070 con 0x7f82240af100 2024-09-21T07:51:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.121+0000 7f821a000640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8204078ce0 0x7f820407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.121+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f821409a330 con 0x7f82240af100 2024-09-21T07:51:40.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.121+0000 7f8221400640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8204078ce0 0x7f820407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.122+0000 7f8221400640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8204078ce0 0x7f820407b1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8210033000 tx=0x7f8210004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:40.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.126+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8214061eb0 con 0x7f82240af100 2024-09-21T07:51:40.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.350+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f8224074eb0 con 0x7f82240af100 2024-09-21T07:51:40.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.591+0000 7f821a000640 1 -- 172.21.15.88:0/1448907705 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v71) ==== 136+0+0 (secure 0 0 0) 0x7f82140668e0 con 0x7f82240af100 2024-09-21T07:51:40.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8204078ce0 msgr2=0x7f820407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8204078ce0 0x7f820407b1f0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8210033000 tx=0x7f8210004730 comp rx=0 tx=0).stop 2024-09-21T07:51:40.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 msgr2=0x7f82240c3010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 0x7f82240c3010 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f821400c430 tx=0x7f821400c8f0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 shutdown_connections 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8204078ce0 0x7f820407b1f0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f82241530e0 0x7f822406fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f82240cc830 0x7f822406f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 --2- 172.21.15.88:0/1448907705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f82240af100 0x7f82240c3010 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 >> 172.21.15.88:0/1448907705 conn(0x7f82240b2540 msgr2=0x7f8224099820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 shutdown_connections 2024-09-21T07:51:40.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.595+0000 7f8229200640 1 -- 172.21.15.88:0/1448907705 wait complete. 2024-09-21T07:51:40.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 -- 172.21.15.88:0/504368178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec4074490 msgr2=0x7f7ec40b3870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/504368178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec4074490 0x7f7ec40b3870 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc005fd0 tx=0x7f7ebc02aaa0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 -- 172.21.15.88:0/504368178 shutdown_connections 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/504368178 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec4074490 0x7f7ec40b3870 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 -- 172.21.15.88:0/504368178 >> 172.21.15.88:0/504368178 conn(0x7f7ec40d1470 msgr2=0x7f7ec4073670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 -- 172.21.15.88:0/504368178 shutdown_connections 2024-09-21T07:51:40.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.840+0000 7f7ec3e00640 1 -- 172.21.15.88:0/504368178 wait complete. 2024-09-21T07:51:40.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.843+0000 7f7ec3e00640 1 Processor -- start 2024-09-21T07:51:40.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.843+0000 7f7ec3e00640 1 -- start start 2024-09-21T07:51:40.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 0x7f7ec414f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec415c510 0x7f7ec41579c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec41f62a0 0x7f7ec4157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec40b3db0 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7ec40ce310 con 0x7f7ec41f62a0 2024-09-21T07:51:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7ec40b3f30 con 0x7f7ec415c510 2024-09-21T07:51:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 0x7f7ec414f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 0x7f7ec414f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44582/0 (socket says 172.21.15.88:44582) 2024-09-21T07:51:40.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.844+0000 7f7ec2a00640 1 -- 172.21.15.88:0/2333366937 learned_addr learned my addr 172.21.15.88:0/2333366937 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec2a00640 1 -- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec415c510 msgr2=0x7f7ec41579c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec2a00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec415c510 0x7f7ec41579c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec2a00640 1 -- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec41f62a0 msgr2=0x7f7ec4157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec3400640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec41f62a0 0x7f7ec4157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec2a00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec41f62a0 0x7f7ec4157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec2a00640 1 -- 172.21.15.88:0/2333366937 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ec415ac50 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec3400640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec41f62a0 0x7f7ec4157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec2a00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 0x7f7ec414f150 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400c400 tx=0x7f7eb400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4017070 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eb40053c0 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ec415f430 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.845+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4003850 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.846+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ec415f9f0 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.847+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ec40b3870 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.848+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f7eb401a070 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.849+0000 7f7ebaa00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ea4078ce0 0x7f7ea407b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:40.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.849+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f7eb409a2a0 con 0x7f7ec40cdcf0 2024-09-21T07:51:40.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.849+0000 7f7ec2000640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ea4078ce0 0x7f7ea407b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:40.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.850+0000 7f7ec2000640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ea4078ce0 0x7f7ea407b1f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0000f60 tx=0x7f7eb00079b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:40.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:40.854+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7eb4062470 con 0x7f7ec40cdcf0 2024-09-21T07:51:41.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.061+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7f7ec4158840 con 0x7f7ea4078ce0 2024-09-21T07:51:41.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.063+0000 7f7ebaa00640 1 -- 172.21.15.88:0/2333366937 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7f7ec4158840 con 0x7f7ea4078ce0 2024-09-21T07:51:41.064 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:51:41.064 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:41.064 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:51:41.064 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:41.064 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:51:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ea4078ce0 msgr2=0x7f7ea407b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ea4078ce0 0x7f7ea407b1f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7eb0000f60 tx=0x7f7eb00079b0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 msgr2=0x7f7ec414f150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 0x7f7ec414f150 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400c400 tx=0x7f7eb400c8c0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 shutdown_connections 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ea4078ce0 0x7f7ea407b1f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7ec415c510 0x7f7ec41579c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ec41f62a0 0x7f7ec4157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 --2- 172.21.15.88:0/2333366937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ec40cdcf0 0x7f7ec414f150 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 >> 172.21.15.88:0/2333366937 conn(0x7f7ec40d1470 msgr2=0x7f7ec41515e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 shutdown_connections 2024-09-21T07:51:41.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.065+0000 7f7ec3e00640 1 -- 172.21.15.88:0/2333366937 wait complete. 2024-09-21T07:51:41.088 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:51:41.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 -- 172.21.15.88:0/3920306772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 msgr2=0x7f1d080ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 --2- 172.21.15.88:0/3920306772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d080ccac0 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x7f1cf4009830 tx=0x7f1cf402a510 comp rx=0 tx=0).stop 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 -- 172.21.15.88:0/3920306772 shutdown_connections 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 --2- 172.21.15.88:0/3920306772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d080ccac0 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 --2- 172.21.15.88:0/3920306772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 0x7f1d08156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 -- 172.21.15.88:0/3920306772 >> 172.21.15.88:0/3920306772 conn(0x7f1d0806d620 msgr2=0x7f1d080b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 -- 172.21.15.88:0/3920306772 shutdown_connections 2024-09-21T07:51:41.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.309+0000 7f1d07e00640 1 -- 172.21.15.88:0/3920306772 wait complete. 2024-09-21T07:51:41.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.311+0000 7f1d07e00640 1 Processor -- start 2024-09-21T07:51:41.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.311+0000 7f1d07e00640 1 -- start start 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 0x7f1d0814dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d080cd000 0x7f1d0809c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d0809c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1d080af960 con 0x7f1d0815d6e0 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1d080af7e0 con 0x7f1d080cd000 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1d080afae0 con 0x7f1d080b3f30 2024-09-21T07:51:41.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d0809c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d0809c710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44598/0 (socket says 172.21.15.88:44598) 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d07400640 1 -- 172.21.15.88:0/2681717716 learned_addr learned my addr 172.21.15.88:0/2681717716 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.313+0000 7f1d06000640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d080cd000 0x7f1d0809c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d06a00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 0x7f1d0814dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07400640 1 -- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 msgr2=0x7f1d0814dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07400640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 0x7f1d0814dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07400640 1 -- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d080cd000 msgr2=0x7f1d0809c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07400640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d080cd000 0x7f1d0809c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07400640 1 -- 172.21.15.88:0/2681717716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d0815c9d0 con 0x7f1d0815d6e0 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d06000640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d080cd000 0x7f1d0809c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d06a00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 0x7f1d0814dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07400640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d0809c710 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000ca80 tx=0x7f1d0000cf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00016020 con 0x7f1d0815d6e0 2024-09-21T07:51:41.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1d0000ad20 con 0x7f1d0815d6e0 2024-09-21T07:51:41.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d0809d280 con 0x7f1d0815d6e0 2024-09-21T07:51:41.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d00012680 con 0x7f1d0815d6e0 2024-09-21T07:51:41.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.314+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d0809d840 con 0x7f1d0815d6e0 2024-09-21T07:51:41.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.315+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d0815c740 con 0x7f1d0815d6e0 2024-09-21T07:51:41.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.317+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f1d00019020 con 0x7f1d0815d6e0 2024-09-21T07:51:41.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.317+0000 7f1cfea00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ce8078cb0 0x7f1ce807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.318+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f1d0009b370 con 0x7f1d0815d6e0 2024-09-21T07:51:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.318+0000 7f1d06a00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ce8078cb0 0x7f1ce807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:41.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.318+0000 7f1d06a00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ce8078cb0 0x7f1ce807b1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009aa0 tx=0x7f1cf8005db0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:41.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.322+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1d00020030 con 0x7f1d0815d6e0 2024-09-21T07:51:41.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.507+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f1d08156ca0 con 0x7f1d0815d6e0 2024-09-21T07:51:41.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.508+0000 7f1cfea00640 1 -- 172.21.15.88:0/2681717716 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v42)=0 v42) ==== 122+0+0 (secure 0 0 0) 0x7f1d000634b0 con 0x7f1d0815d6e0 2024-09-21T07:51:41.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ce8078cb0 msgr2=0x7f1ce807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ce8078cb0 0x7f1ce807b1c0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f1cf8009aa0 tx=0x7f1cf8005db0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 msgr2=0x7f1d0809c710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:41.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d0809c710 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7f1d0000ca80 tx=0x7f1d0000cf40 comp rx=0 tx=0).stop 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 shutdown_connections 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ce8078cb0 0x7f1ce807b1c0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1d0815d6e0 0x7f1d0809c710 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1d080cd000 0x7f1d0809c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 --2- 172.21.15.88:0/2681717716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1d080b3f30 0x7f1d0814dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 >> 172.21.15.88:0/2681717716 conn(0x7f1d0806d620 msgr2=0x7f1d080b0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 shutdown_connections 2024-09-21T07:51:41.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:41.510+0000 7f1d07e00640 1 -- 172.21.15.88:0/2681717716 wait complete. 2024-09-21T07:51:41.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:41.533 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e) 2024-09-21T07:51:41.599 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:41.599 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/fake) 2024-09-21T07:51:41.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:41.684 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:51:41.720 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:51:41.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:41.721 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e) 2024-09-21T07:51:41.821 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:41.822 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol88300/.meta 2024-09-21T07:51:41.881 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol88300 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3383964400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28004800 msgr2=0x7fdf28004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3383964400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28004800 0x7fdf28004c10 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7fdf30006f40 tx=0x7fdf28005830 comp rx=0 tx=0).stop 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3383964400 shutdown_connections 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3383964400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28004800 0x7fdf28004c10 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3383964400 >> 172.21.15.88:0/3383964400 conn(0x7fdf4406d1c0 msgr2=0x7fdf4406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3383964400 shutdown_connections 2024-09-21T07:51:42.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.147+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3383964400 wait complete. 2024-09-21T07:51:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.150+0000 7fdf4ae00640 1 Processor -- start 2024-09-21T07:51:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.150+0000 7fdf4ae00640 1 -- start start 2024-09-21T07:51:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf4ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf28004800 0x7fdf441f87c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf4ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf44138230 0x7fdf441f8d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf4ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 0x7fdf4413ae60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf4ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44073540 con 0x7fdf4413aa50 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf4ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf44070180 con 0x7fdf44138230 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf4ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf440736c0 con 0x7fdf28004800 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf48e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 0x7fdf4413ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf48e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 0x7fdf4413ae60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44622/0 (socket says 172.21.15.88:44622) 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.151+0000 7fdf48e00640 1 -- 172.21.15.88:0/3212467864 learned_addr learned my addr 172.21.15.88:0/3212467864 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf43e00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf28004800 0x7fdf441f87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf43400640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf44138230 0x7fdf441f8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf48e00640 1 -- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf28004800 msgr2=0x7fdf441f87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf48e00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf28004800 0x7fdf441f87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf48e00640 1 -- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf44138230 msgr2=0x7fdf441f8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf48e00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf44138230 0x7fdf441f8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf48e00640 1 -- 172.21.15.88:0/3212467864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4413b3a0 con 0x7fdf4413aa50 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf48e00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 0x7fdf4413ae60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdf440b0070 tx=0x7fdf3c010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c003970 con 0x7fdf4413aa50 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf3c015070 con 0x7fdf4413aa50 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c004030 con 0x7fdf4413aa50 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.152+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf441389c0 con 0x7fdf4413aa50 2024-09-21T07:51:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.153+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf44138f80 con 0x7fdf4413aa50 2024-09-21T07:51:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.154+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf44073c20 con 0x7fdf4413aa50 2024-09-21T07:51:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.155+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fdf3c0041d0 con 0x7fdf4413aa50 2024-09-21T07:51:42.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.156+0000 7fdf3be00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf24078cb0 0x7fdf2407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.156+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fdf3c09ba60 con 0x7fdf4413aa50 2024-09-21T07:51:42.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.156+0000 7fdf43e00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf24078cb0 0x7fdf2407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.157+0000 7fdf43e00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf24078cb0 0x7fdf2407b1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf34006fd0 tx=0x7fdf34006b10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.160+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf4413b3a0 con 0x7fdf4413aa50 2024-09-21T07:51:42.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.371+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7fdf441f9380 con 0x7fdf24078cb0 2024-09-21T07:51:42.378 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:42.377+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:42.378 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:42.377+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:42.378 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:42.377+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:42.378 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:42.377+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:42.378 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:42.377+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:42.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.422+0000 7fdf3be00640 1 -- 172.21.15.88:0/3212467864 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7fdf441f9380 con 0x7fdf24078cb0 2024-09-21T07:51:42.423 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e 2024-09-21T07:51:42.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.424+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf24078cb0 msgr2=0x7fdf2407b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf24078cb0 0x7fdf2407b1c0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fdf34006fd0 tx=0x7fdf34006b10 comp rx=0 tx=0).stop 2024-09-21T07:51:42.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 msgr2=0x7fdf4413ae60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 0x7fdf4413ae60 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7fdf440b0070 tx=0x7fdf3c010f40 comp rx=0 tx=0).stop 2024-09-21T07:51:42.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 shutdown_connections 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf24078cb0 0x7fdf2407b1c0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4413aa50 0x7fdf4413ae60 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf44138230 0x7fdf441f8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 --2- 172.21.15.88:0/3212467864 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf28004800 0x7fdf441f87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 >> 172.21.15.88:0/3212467864 conn(0x7fdf4406d1c0 msgr2=0x7fdf4413dfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 shutdown_connections 2024-09-21T07:51:42.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.425+0000 7fdf4ae00640 1 -- 172.21.15.88:0/3212467864 wait complete. 2024-09-21T07:51:42.451 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 -- 172.21.15.88:0/3947081140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 msgr2=0x7fcdfc0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 --2- 172.21.15.88:0/3947081140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc0af0d0 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7fcdf4005fd0 tx=0x7fcdf402aa90 comp rx=0 tx=0).stop 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 -- 172.21.15.88:0/3947081140 shutdown_connections 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 --2- 172.21.15.88:0/3947081140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc0af0d0 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 --2- 172.21.15.88:0/3947081140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdfc156ca0 0x7fcdfc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 -- 172.21.15.88:0/3947081140 >> 172.21.15.88:0/3947081140 conn(0x7fcdfc06d620 msgr2=0x7fcdfc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 -- 172.21.15.88:0/3947081140 shutdown_connections 2024-09-21T07:51:42.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.673+0000 7fce01a00640 1 -- 172.21.15.88:0/3947081140 wait complete. 2024-09-21T07:51:42.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.676+0000 7fce01a00640 1 Processor -- start 2024-09-21T07:51:42.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.676+0000 7fce01a00640 1 -- start start 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fce01a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdfc0b3f30 0x7fcdfc145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fce01a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc099550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fce01a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdfc156ca0 0x7fcdfc1e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fce01a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc0af9a0 con 0x7fcdfc0cd000 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fce01a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcdfc0af820 con 0x7fcdfc156ca0 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fce01a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcdfc0afb20 con 0x7fcdfc0b3f30 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.677+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc099550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc099550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44674/0 (socket says 172.21.15.88:44674) 2024-09-21T07:51:42.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 -- 172.21.15.88:0/4168619117 learned_addr learned my addr 172.21.15.88:0/4168619117 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdfbe00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdfc0b3f30 0x7fcdfc145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fce01000640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdfc156ca0 0x7fcdfc1e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 -- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdfc0b3f30 msgr2=0x7fcdfc145830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdfc0b3f30 0x7fcdfc145830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 -- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdfc156ca0 msgr2=0x7fcdfc1e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdfc156ca0 0x7fcdfc1e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 -- 172.21.15.88:0/4168619117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcdfc155c10 con 0x7fcdfc0cd000 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fce01000640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdfc156ca0 0x7fcdfc1e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.678+0000 7fcdf3400640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc099550 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d6d0 tx=0x7fcde400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.679+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4017070 con 0x7fcdfc0cd000 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.679+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcde4009b10 con 0x7fcdfc0cd000 2024-09-21T07:51:42.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.679+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcde4010460 con 0x7fcdfc0cd000 2024-09-21T07:51:42.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.679+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcdfc0739f0 con 0x7fcdfc0cd000 2024-09-21T07:51:42.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.679+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcdfc073c90 con 0x7fcdfc0cd000 2024-09-21T07:51:42.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.680+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcdfc1556a0 con 0x7fcdfc0cd000 2024-09-21T07:51:42.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.681+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7fcde401a070 con 0x7fcdfc0cd000 2024-09-21T07:51:42.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.682+0000 7fcdf8c00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcdd8078cb0 0x7fcdd807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:42.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.682+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fcde409b100 con 0x7fcdfc0cd000 2024-09-21T07:51:42.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.682+0000 7fcdfbe00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcdd8078cb0 0x7fcdd807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:42.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.683+0000 7fcdfbe00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcdd8078cb0 0x7fcdd807b1c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdec001720 tx=0x7fcdec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:42.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.687+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcde4062c80 con 0x7fcdfc0cd000 2024-09-21T07:51:42.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.897+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7fcdfc0af0d0 con 0x7fcdd8078cb0 2024-09-21T07:51:42.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.901+0000 7fcdf8c00640 1 -- 172.21.15.88:0/4168619117 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7fcdfc0af0d0 con 0x7fcdd8078cb0 2024-09-21T07:51:42.902 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:51:42.902 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:42.902 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "subvol88300" 2024-09-21T07:51:42.902 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:42.902 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcdd8078cb0 msgr2=0x7fcdd807b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcdd8078cb0 0x7fcdd807b1c0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fcdec001720 tx=0x7fcdec009040 comp rx=0 tx=0).stop 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 msgr2=0x7fcdfc099550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc099550 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7fcde400d6d0 tx=0x7fcde400db90 comp rx=0 tx=0).stop 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 shutdown_connections 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcdd8078cb0 0x7fcdd807b1c0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcdfc156ca0 0x7fcdfc1e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcdfc0cd000 0x7fcdfc099550 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 --2- 172.21.15.88:0/4168619117 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcdfc0b3f30 0x7fcdfc145830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.903+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 >> 172.21.15.88:0/4168619117 conn(0x7fcdfc06d620 msgr2=0x7fcdfc0b0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:42.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.904+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 shutdown_connections 2024-09-21T07:51:42.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:42.904+0000 7fce01a00640 1 -- 172.21.15.88:0/4168619117 wait complete. 2024-09-21T07:51:42.927 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol88300 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 -- 172.21.15.88:0/2685343101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370815d6e0 msgr2=0x7f37080ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 --2- 172.21.15.88:0/2685343101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370815d6e0 0x7f37080ccac0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f36f4009830 tx=0x7f36f402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 -- 172.21.15.88:0/2685343101 shutdown_connections 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 --2- 172.21.15.88:0/2685343101 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f370815d6e0 0x7f37080ccac0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 --2- 172.21.15.88:0/2685343101 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f37080b3f30 0x7f3708156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 -- 172.21.15.88:0/2685343101 >> 172.21.15.88:0/2685343101 conn(0x7f370806d620 msgr2=0x7f37080b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 -- 172.21.15.88:0/2685343101 shutdown_connections 2024-09-21T07:51:43.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.151+0000 7f370ee00640 1 -- 172.21.15.88:0/2685343101 wait complete. 2024-09-21T07:51:43.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.154+0000 7f370ee00640 1 Processor -- start 2024-09-21T07:51:43.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.154+0000 7f370ee00640 1 -- start start 2024-09-21T07:51:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.155+0000 7f370ee00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37080b3f30 0x7f37081f5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.155+0000 7f370ee00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 0x7f370809d180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.155+0000 7f370ee00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f370815d6e0 0x7f370809d6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.155+0000 7f370ee00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37080af960 con 0x7f37080cd000 2024-09-21T07:51:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.155+0000 7f370ee00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f37080af7e0 con 0x7f37080b3f30 2024-09-21T07:51:43.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.155+0000 7f370ee00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f37080afae0 con 0x7f370815d6e0 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 0x7f370809d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 0x7f370809d180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44680/0 (socket says 172.21.15.88:44680) 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 -- 172.21.15.88:0/981056736 learned_addr learned my addr 172.21.15.88:0/981056736 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370da00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37080b3f30 0x7f37081f5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370e400640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f370815d6e0 0x7f370809d6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 -- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f370815d6e0 msgr2=0x7f370809d6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f370815d6e0 0x7f370809d6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 -- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37080b3f30 msgr2=0x7f37081f5100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37080b3f30 0x7f37081f5100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 -- 172.21.15.88:0/981056736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3708154ed0 con 0x7f37080cd000 2024-09-21T07:51:43.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.156+0000 7f370d000640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 0x7f370809d180 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f36f402aa90 tx=0x7f36f4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:43.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.157+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4005c90 con 0x7f37080cd000 2024-09-21T07:51:43.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.157+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36f4005020 con 0x7f37080cd000 2024-09-21T07:51:43.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.157+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37080b0450 con 0x7f37080cd000 2024-09-21T07:51:43.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.157+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36f4005450 con 0x7f37080cd000 2024-09-21T07:51:43.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.157+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f370815df80 con 0x7f37080cd000 2024-09-21T07:51:43.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.158+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37081553d0 con 0x7f37080cd000 2024-09-21T07:51:43.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.159+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 101543+0+0 (secure 0 0 0) 0x7f36f40317c0 con 0x7f37080cd000 2024-09-21T07:51:43.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.160+0000 7f3705600640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36e8078ce0 0x7f36e807b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.160+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f36f40bbfb0 con 0x7f37080cd000 2024-09-21T07:51:43.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.160+0000 7f370da00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36e8078ce0 0x7f36e807b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.161+0000 7f370da00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36e8078ce0 0x7f36e807b1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009b30 tx=0x7f36fc005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:43.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.164+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36f403f030 con 0x7f37080cd000 2024-09-21T07:51:43.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.377+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7f3708154c80 con 0x7f36e8078ce0 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.402+0000 7f3705600640 1 -- 172.21.15.88:0/981056736 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+705 (secure 0 0 0) 0x7f3708154c80 con 0x7f36e8078ce0 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "atime": "2024-09-21 07:51:41", 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_pcent": "undefined", 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_quota": "infinite", 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_used": 0, 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "created_at": "2024-09-21 07:51:41", 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "ctime": "2024-09-21 07:51:42", 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "data_pool": "cephfs_data", 2024-09-21T07:51:43.403 INFO:teuthology.orchestra.run.smithi088.stdout: "earmark": "", 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "features": [ 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-clone", 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-autoprotect" 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "flavor": 1, 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "gid": 0, 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": 16877, 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "mon_addrs": [ 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.88:6789", 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.175:6789", 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.175:6790" 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "mtime": "2024-09-21 07:51:41", 2024-09-21T07:51:43.404 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e", 2024-09-21T07:51:43.405 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_namespace": "", 2024-09-21T07:51:43.405 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "complete", 2024-09-21T07:51:43.405 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "subvolume", 2024-09-21T07:51:43.405 INFO:teuthology.orchestra.run.smithi088.stdout: "uid": 0 2024-09-21T07:51:43.405 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36e8078ce0 msgr2=0x7f36e807b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36e8078ce0 0x7f36e807b1f0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f36fc009b30 tx=0x7f36fc005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 msgr2=0x7f370809d180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 0x7f370809d180 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f36f402aa90 tx=0x7f36f4003c10 comp rx=0 tx=0).stop 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 shutdown_connections 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36e8078ce0 0x7f36e807b1f0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f370815d6e0 0x7f370809d6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37080cd000 0x7f370809d180 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 --2- 172.21.15.88:0/981056736 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f37080b3f30 0x7f37081f5100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 >> 172.21.15.88:0/981056736 conn(0x7f370806d620 msgr2=0x7f37080afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 shutdown_connections 2024-09-21T07:51:43.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.404+0000 7f370ee00640 1 -- 172.21.15.88:0/981056736 wait complete. 2024-09-21T07:51:43.431 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize cephfs subvol88300 10485760 2024-09-21T07:51:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 -- 172.21.15.88:0/2670655174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0af100 msgr2=0x7f2a6c0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 --2- 172.21.15.88:0/2670655174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0af100 0x7f2a6c0af510 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f2a64005fd0 tx=0x7f2a6402aa90 comp rx=0 tx=0).stop 2024-09-21T07:51:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 -- 172.21.15.88:0/2670655174 shutdown_connections 2024-09-21T07:51:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 --2- 172.21.15.88:0/2670655174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0af100 0x7f2a6c0af510 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 --2- 172.21.15.88:0/2670655174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a6c0cc830 0x7f2a6c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 -- 172.21.15.88:0/2670655174 >> 172.21.15.88:0/2670655174 conn(0x7f2a6c0b2540 msgr2=0x7f2a6c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:43.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 -- 172.21.15.88:0/2670655174 shutdown_connections 2024-09-21T07:51:43.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.658+0000 7f2a73600640 1 -- 172.21.15.88:0/2670655174 wait complete. 2024-09-21T07:51:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.660+0000 7f2a73600640 1 Processor -- start 2024-09-21T07:51:43.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.660+0000 7f2a73600640 1 -- start start 2024-09-21T07:51:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.661+0000 7f2a73600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a6c06cda0 0x7f2a6c093050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a73600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a6c0af100 0x7f2a6c093590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a73600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 0x7f2a6c154b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a73600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c0b36a0 con 0x7f2a6c0cc830 2024-09-21T07:51:43.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a73600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a6c0b3520 con 0x7f2a6c06cda0 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a73600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a6c0b3820 con 0x7f2a6c0af100 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 0x7f2a6c154b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 0x7f2a6c154b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44700/0 (socket says 172.21.15.88:44700) 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 -- 172.21.15.88:0/1907014223 learned_addr learned my addr 172.21.15.88:0/1907014223 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a63e00640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a6c0af100 0x7f2a6c093590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a70c00640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a6c06cda0 0x7f2a6c093050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 -- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a6c0af100 msgr2=0x7f2a6c093590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a6c0af100 0x7f2a6c093590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 -- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a6c06cda0 msgr2=0x7f2a6c093050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a6c06cda0 0x7f2a6c093050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a71600640 1 -- 172.21.15.88:0/1907014223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a6c156d90 con 0x7f2a6c0cc830 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.662+0000 7f2a70c00640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a6c06cda0 0x7f2a6c093050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:43.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.663+0000 7f2a71600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 0x7f2a6c154b70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009d90 tx=0x7f2a64003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:43.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.663+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64044070 con 0x7f2a6c0cc830 2024-09-21T07:51:43.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.663+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a64004040 con 0x7f2a6c0cc830 2024-09-21T07:51:43.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.663+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a6c157070 con 0x7f2a6c0cc830 2024-09-21T07:51:43.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.663+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a64004430 con 0x7f2a6c0cc830 2024-09-21T07:51:43.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.663+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a6c155bf0 con 0x7f2a6c0cc830 2024-09-21T07:51:43.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.664+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a6c155e50 con 0x7f2a6c0cc830 2024-09-21T07:51:43.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.665+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f2a64047070 con 0x7f2a6c0cc830 2024-09-21T07:51:43.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.666+0000 7f2a69600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a48078d40 0x7f2a4807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:43.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.666+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f2a640bbf50 con 0x7f2a6c0cc830 2024-09-21T07:51:43.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.667+0000 7f2a70c00640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a48078d40 0x7f2a4807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:43.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.667+0000 7f2a70c00640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a48078d40 0x7f2a4807b250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009b80 tx=0x7f2a5c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:43.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.671+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a64032030 con 0x7f2a6c0cc830 2024-09-21T07:51:43.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.892+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "cephfs", "sub_name": "subvol88300", "new_size": "10485760", "target": ["mon-mgr", ""]}) -- 0x7f2a6c156b40 con 0x7f2a48078d40 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.918+0000 7f2a69600640 1 -- 172.21.15.88:0/1907014223 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+128 (secure 0 0 0) 0x7f2a6c156b40 con 0x7f2a48078d40 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_used": 0 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_quota": 10485760 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:43.919 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:51:43.920 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_pcent": "0.00" 2024-09-21T07:51:43.920 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:43.920 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.920+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a48078d40 msgr2=0x7f2a4807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.920+0000 7f2a73600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a48078d40 0x7f2a4807b250 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f2a5c009b80 tx=0x7f2a5c005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 msgr2=0x7f2a6c154b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 0x7f2a6c154b70 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f2a64009d90 tx=0x7f2a64003ce0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 shutdown_connections 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a48078d40 0x7f2a4807b250 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a6c0af100 0x7f2a6c093590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a6c0cc830 0x7f2a6c154b70 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 --2- 172.21.15.88:0/1907014223 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a6c06cda0 0x7f2a6c093050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:43.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 >> 172.21.15.88:0/1907014223 conn(0x7f2a6c0b2540 msgr2=0x7f2a6c0ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:43.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 shutdown_connections 2024-09-21T07:51:43.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:43.921+0000 7f2a73600640 1 -- 172.21.15.88:0/1907014223 wait complete. 2024-09-21T07:51:43.948 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol88300 2024-09-21T07:51:44.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 -- 172.21.15.88:0/2069947651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145550 msgr2=0x7f909c06fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 --2- 172.21.15.88:0/2069947651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145550 0x7f909c06fb40 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f909c1464e0 tx=0x7f908c02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 -- 172.21.15.88:0/2069947651 shutdown_connections 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 --2- 172.21.15.88:0/2069947651 >> v2:172.21.15.175:3300/0 conn(0x7f909c073c50 0x7f909c0cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 --2- 172.21.15.88:0/2069947651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145550 0x7f909c06fb40 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 -- 172.21.15.88:0/2069947651 >> 172.21.15.88:0/2069947651 conn(0x7f909c06d1c0 msgr2=0x7f909c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 -- 172.21.15.88:0/2069947651 shutdown_connections 2024-09-21T07:51:44.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.170+0000 7f90a1800640 1 -- 172.21.15.88:0/2069947651 wait complete. 2024-09-21T07:51:44.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.172+0000 7f90a1800640 1 Processor -- start 2024-09-21T07:51:44.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.173+0000 7f90a1800640 1 -- start start 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f90a1800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f909c145550 0x7f909c152f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f90a1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 0x7f909c13d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f90a1800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f909c15bd30 0x7f909c13d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f90a1800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f909c0ce130 con 0x7f909c145a40 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f90a1800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f909c0cdfb0 con 0x7f909c145550 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f90a1800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f909c0ce2b0 con 0x7f909c15bd30 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f9092000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 0x7f909c13d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f9092000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 0x7f909c13d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44704/0 (socket says 172.21.15.88:44704) 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.174+0000 7f9092000640 1 -- 172.21.15.88:0/2586974705 learned_addr learned my addr 172.21.15.88:0/2586974705 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f909b400640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f909c15bd30 0x7f909c13d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f909aa00640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f909c145550 0x7f909c152f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9092000640 1 -- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f909c15bd30 msgr2=0x7f909c13d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9092000640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f909c15bd30 0x7f909c13d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9092000640 1 -- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f909c145550 msgr2=0x7f909c152f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9092000640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f909c145550 0x7f909c152f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9092000640 1 -- 172.21.15.88:0/2586974705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f909c13c6a0 con 0x7f909c145a40 2024-09-21T07:51:44.177 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9092000640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 0x7f909c13d1c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f908400c3a0 tx=0x7f908400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084017070 con 0x7f909c145a40 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f90840053c0 con 0x7f909c145a40 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9084003850 con 0x7f909c145a40 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f909c15e1b0 con 0x7f909c145a40 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.175+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f909c15e770 con 0x7f909c145a40 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.176+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f909c13c410 con 0x7f909c145a40 2024-09-21T07:51:44.178 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.178+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f908401a070 con 0x7f909c145a40 2024-09-21T07:51:44.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.179+0000 7f9093400640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9078078d40 0x7f907807b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.179+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f908409a660 con 0x7f909c145a40 2024-09-21T07:51:44.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.179+0000 7f909aa00640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9078078d40 0x7f907807b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.180+0000 7f909aa00640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9078078d40 0x7f907807b250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f908c033000 tx=0x7f908c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:44.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.183+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f90840625b0 con 0x7f909c145a40 2024-09-21T07:51:44.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.397+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7f909c0ce7d0 con 0x7f9078078d40 2024-09-21T07:51:44.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.420+0000 7f9093400640 1 -- 172.21.15.88:0/2586974705 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+698 (secure 0 0 0) 0x7f909c0ce7d0 con 0x7f9078078d40 2024-09-21T07:51:44.421 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:44.421 INFO:teuthology.orchestra.run.smithi088.stdout: "atime": "2024-09-21 07:51:41", 2024-09-21T07:51:44.421 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_pcent": "0.00", 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_quota": 10485760, 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_used": 0, 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "created_at": "2024-09-21 07:51:41", 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "ctime": "2024-09-21 07:51:43", 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "data_pool": "cephfs_data", 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "earmark": "", 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "features": [ 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-clone", 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-autoprotect" 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "flavor": 1, 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "gid": 0, 2024-09-21T07:51:44.422 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": 16877, 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "mon_addrs": [ 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.88:6789", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.175:6789", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.175:6790" 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "mtime": "2024-09-21 07:51:41", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_namespace": "", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "complete", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "subvolume", 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout: "uid": 0 2024-09-21T07:51:44.423 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9078078d40 msgr2=0x7f907807b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9078078d40 0x7f907807b250 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f908c033000 tx=0x7f908c004730 comp rx=0 tx=0).stop 2024-09-21T07:51:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 msgr2=0x7f909c13d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 0x7f909c13d1c0 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f908400c3a0 tx=0x7f908400c860 comp rx=0 tx=0).stop 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 shutdown_connections 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9078078d40 0x7f907807b250 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f909c15bd30 0x7f909c13d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f909c145a40 0x7f909c13d1c0 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 --2- 172.21.15.88:0/2586974705 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f909c145550 0x7f909c152f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.422+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 >> 172.21.15.88:0/2586974705 conn(0x7f909c06d1c0 msgr2=0x7f909c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.423+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 shutdown_connections 2024-09-21T07:51:44.425 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.423+0000 7f90a1800640 1 -- 172.21.15.88:0/2586974705 wait complete. 2024-09-21T07:51:44.450 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs subvol88300 --mode 777 --uid 1000 --gid 1000 2024-09-21T07:51:44.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.678+0000 7f984b000640 1 -- 172.21.15.88:0/3047939546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145550 msgr2=0x7f984406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.678+0000 7f984b000640 1 --2- 172.21.15.88:0/3047939546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145550 0x7f984406fb40 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f9834005fd0 tx=0x7f983402a710 comp rx=0 tx=0).stop 2024-09-21T07:51:44.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.678+0000 7f984b000640 1 -- 172.21.15.88:0/3047939546 shutdown_connections 2024-09-21T07:51:44.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.678+0000 7f984b000640 1 --2- 172.21.15.88:0/3047939546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145550 0x7f984406fb40 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.678+0000 7f984b000640 1 -- 172.21.15.88:0/3047939546 >> 172.21.15.88:0/3047939546 conn(0x7f984406d1c0 msgr2=0x7f984406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:44.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.678+0000 7f984b000640 1 -- 172.21.15.88:0/3047939546 shutdown_connections 2024-09-21T07:51:44.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.679+0000 7f984b000640 1 -- 172.21.15.88:0/3047939546 wait complete. 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.681+0000 7f984b000640 1 Processor -- start 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.681+0000 7f984b000640 1 -- start start 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f984b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9844145550 0x7f984415abd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f984b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 0x7f984413bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f984b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440aa1d0 0x7f98440aa5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f984b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98440ce360 con 0x7f9844145a40 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f984b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98440ce1e0 con 0x7f98440aa1d0 2024-09-21T07:51:44.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f984b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98440ce4e0 con 0x7f9844145550 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f9843400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 0x7f984413bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f9843400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 0x7f984413bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44708/0 (socket says 172.21.15.88:44708) 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.682+0000 7f9843400640 1 -- 172.21.15.88:0/3428732842 learned_addr learned my addr 172.21.15.88:0/3428732842 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9849000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440aa1d0 0x7f98440aa5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843e00640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9844145550 0x7f984415abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843400640 1 -- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9844145550 msgr2=0x7f984415abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843400640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9844145550 0x7f984415abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843400640 1 -- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440aa1d0 msgr2=0x7f98440aa5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843400640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440aa1d0 0x7f98440aa5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843400640 1 -- 172.21.15.88:0/3428732842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98440aab20 con 0x7f9844145a40 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843e00640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9844145550 0x7f984415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f9843400640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 0x7f984413bf90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f983000c400 tx=0x7f983000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.683+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830017070 con 0x7f9844145a40 2024-09-21T07:51:44.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.684+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98300053c0 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.684+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f984415b540 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.684+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9830003830 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.684+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f984415bb00 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.685+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f984406fb40 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.686+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f983001a070 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.687+0000 7f983be00640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9824078d70 0x7f982407b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.687+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f983009a3d0 con 0x7f9844145a40 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.687+0000 7f9843e00640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9824078d70 0x7f982407b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.688+0000 7f9843e00640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9824078d70 0x7f982407b280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9834033000 tx=0x7f9834004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:44.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.692+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f98440aab20 con 0x7f9844145a40 2024-09-21T07:51:44.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.903+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "subvol88300", "mode": "777", "uid": 1000, "gid": 1000, "target": ["mon-mgr", ""]}) -- 0x7f984415b2d0 con 0x7f9824078d70 2024-09-21T07:51:44.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.933+0000 7f983be00640 1 -- 172.21.15.88:0/3428732842 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f984415b2d0 con 0x7f9824078d70 2024-09-21T07:51:44.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9824078d70 msgr2=0x7f982407b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9824078d70 0x7f982407b280 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f9834033000 tx=0x7f9834004730 comp rx=0 tx=0).stop 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 msgr2=0x7f984413bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 0x7f984413bf90 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f983000c400 tx=0x7f983000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 shutdown_connections 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9824078d70 0x7f982407b280 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98440aa1d0 0x7f98440aa5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9844145a40 0x7f984413bf90 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 --2- 172.21.15.88:0/3428732842 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9844145550 0x7f984415abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 >> 172.21.15.88:0/3428732842 conn(0x7f984406d1c0 msgr2=0x7f9844142eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 shutdown_connections 2024-09-21T07:51:44.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:44.935+0000 7f984b000640 1 -- 172.21.15.88:0/3428732842 wait complete. 2024-09-21T07:51:44.959 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol88300 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 -- 172.21.15.88:0/3390668042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 msgr2=0x7feeb00af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 --2- 172.21.15.88:0/3390668042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb00af0d0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7feea8005fd0 tx=0x7feea802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 -- 172.21.15.88:0/3390668042 shutdown_connections 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 --2- 172.21.15.88:0/3390668042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb00af0d0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 -- 172.21.15.88:0/3390668042 >> 172.21.15.88:0/3390668042 conn(0x7feeb006d620 msgr2=0x7feeb00b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 -- 172.21.15.88:0/3390668042 shutdown_connections 2024-09-21T07:51:45.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.184+0000 7feeb6400640 1 -- 172.21.15.88:0/3390668042 wait complete. 2024-09-21T07:51:45.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.187+0000 7feeb6400640 1 Processor -- start 2024-09-21T07:51:45.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.187+0000 7feeb6400640 1 -- start start 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb01f5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb00b3f30 0x7feeb01f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb0156ca0 0x7feeb01f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00af790 con 0x7feeb0070160 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feeb00af610 con 0x7feeb0156ca0 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feeb00af910 con 0x7feeb00b3f30 2024-09-21T07:51:45.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb01f5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb01f5660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44712/0 (socket says 172.21.15.88:44712) 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.188+0000 7feeb5000640 1 -- 172.21.15.88:0/1714079623 learned_addr learned my addr 172.21.15.88:0/1714079623 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5a00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb0156ca0 0x7feeb01f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeafe00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb00b3f30 0x7feeb01f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5000640 1 -- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb00b3f30 msgr2=0x7feeb01f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5000640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb00b3f30 0x7feeb01f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5000640 1 -- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb0156ca0 msgr2=0x7feeb01f97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5000640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb0156ca0 0x7feeb01f97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5000640 1 -- 172.21.15.88:0/1714079623 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feeb00aee30 con 0x7feeb0070160 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5a00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb0156ca0 0x7feeb01f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeafe00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb00b3f30 0x7feeb01f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:45.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeb5000640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb01f5660 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7feea000c370 tx=0x7feea000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea00052d0 con 0x7feeb0070160 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.189+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feea0007020 con 0x7feeb0070160 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.190+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feea0007470 con 0x7feeb0070160 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.190+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feeb0146560 con 0x7feeb0070160 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.190+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feeb01f9e80 con 0x7feeb0070160 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.191+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feeb00af0d0 con 0x7feeb0070160 2024-09-21T07:51:45.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.192+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7feea00067d0 con 0x7feeb0070160 2024-09-21T07:51:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.193+0000 7feeacc00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee90078d40 0x7fee9007b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:45.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.193+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7feea009a780 con 0x7feeb0070160 2024-09-21T07:51:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.194+0000 7feeafe00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee90078d40 0x7fee9007b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:45.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.194+0000 7feeafe00640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee90078d40 0x7fee9007b250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feeb00cf920 tx=0x7fee9c008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:45.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.198+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feea0062750 con 0x7feeb0070160 2024-09-21T07:51:45.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.410+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7feeb00ad510 con 0x7fee90078d40 2024-09-21T07:51:45.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.432+0000 7feeacc00640 1 -- 172.21.15.88:0/1714079623 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7feeb00ad510 con 0x7fee90078d40 2024-09-21T07:51:45.433 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee90078d40 msgr2=0x7fee9007b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee90078d40 0x7fee9007b250 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7feeb00cf920 tx=0x7fee9c008040 comp rx=0 tx=0).stop 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 msgr2=0x7feeb01f5660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb01f5660 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7feea000c370 tx=0x7feea000c830 comp rx=0 tx=0).stop 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 shutdown_connections 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee90078d40 0x7fee9007b250 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feeb00b3f30 0x7feeb01f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feeb0156ca0 0x7feeb01f97f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 --2- 172.21.15.88:0/1714079623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feeb0070160 0x7feeb01f5660 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:45.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 >> 172.21.15.88:0/1714079623 conn(0x7feeb006d620 msgr2=0x7feeb015f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:45.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 shutdown_connections 2024-09-21T07:51:45.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:45.435+0000 7feeb6400640 1 -- 172.21.15.88:0/1714079623 wait complete. 2024-09-21T07:51:45.458 DEBUG:tasks.cephfs.test_volumes:filling subvolume subvol88300 with 8 files each 1MB size under directory volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e 2024-09-21T07:51:45.458 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:45.458 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:45.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:45.493 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.0 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.019347 s, 54.2 MB/s 2024-09-21T07:51:45.590 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:45.590 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:45.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:45.626 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.1 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:45.718 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:45.719 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0157843 s, 66.4 MB/s 2024-09-21T07:51:45.720 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:45.720 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:45.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:45.755 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.2 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:45.850 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.018198 s, 57.6 MB/s 2024-09-21T07:51:45.851 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:45.851 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:45.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:45.886 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.3 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:45.986 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:45.986 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:45.986 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232264 s, 45.1 MB/s 2024-09-21T07:51:45.987 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:45.987 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:46.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:46.023 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.4 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:46.122 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:46.122 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:46.122 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0232554 s, 45.1 MB/s 2024-09-21T07:51:46.123 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:46.123 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:46.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:46.158 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.5 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:46.256 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0214532 s, 48.9 MB/s 2024-09-21T07:51:46.257 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:46.257 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:46.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:46.293 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.6 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:46.392 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:46.392 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:46.393 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0242088 s, 43.3 MB/s 2024-09-21T07:51:46.394 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:51:46.394 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:51:46.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:51:46.429 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/subvolume_file.7 bs=1M conv=fdatasync count=1 seek=0) 2024-09-21T07:51:46.523 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-09-21T07:51:46.523 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-09-21T07:51:46.523 INFO:teuthology.orchestra.run.smithi088.stderr:1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.0185021 s, 56.7 MB/s 2024-09-21T07:51:46.524 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create cephfs subvol88300 subvol_snap57170 2024-09-21T07:51:46.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 -- 172.21.15.88:0/1184927071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0157110 msgr2=0x7fd4f00ae2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 --2- 172.21.15.88:0/1184927071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0157110 0x7fd4f00ae2d0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009830 tx=0x7fd4e002a480 comp rx=0 tx=0).stop 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 -- 172.21.15.88:0/1184927071 shutdown_connections 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 --2- 172.21.15.88:0/1184927071 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015d7d0 0x7fd4f006cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 --2- 172.21.15.88:0/1184927071 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f0157110 0x7fd4f00ae2d0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 -- 172.21.15.88:0/1184927071 >> 172.21.15.88:0/1184927071 conn(0x7fd4f009aa50 msgr2=0x7fd4f00b2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 -- 172.21.15.88:0/1184927071 shutdown_connections 2024-09-21T07:51:46.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.752+0000 7fd4f5800640 1 -- 172.21.15.88:0/1184927071 wait complete. 2024-09-21T07:51:46.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.755+0000 7fd4f5800640 1 Processor -- start 2024-09-21T07:51:46.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.755+0000 7fd4f5800640 1 -- start start 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 0x7fd4f01f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f015d7d0 0x7fd4f0152dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4f5800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015dbe0 0x7fd4f00ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4f5800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f015b640 con 0x7fd4f0157110 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4f5800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f015b4c0 con 0x7fd4f015d7d0 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4f5800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd4f006dcd0 con 0x7fd4f015dbe0 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4eea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 0x7fd4f01f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4eea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 0x7fd4f01f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39568/0 (socket says 172.21.15.88:39568) 2024-09-21T07:51:46.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.756+0000 7fd4eea00640 1 -- 172.21.15.88:0/3660426241 learned_addr learned my addr 172.21.15.88:0/3660426241 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4e6000640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f015d7d0 0x7fd4f0152dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4eea00640 1 -- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015dbe0 msgr2=0x7fd4f00ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4ef400640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015dbe0 0x7fd4f00ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4eea00640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015dbe0 0x7fd4f00ccc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4eea00640 1 -- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f015d7d0 msgr2=0x7fd4f0152dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4eea00640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f015d7d0 0x7fd4f0152dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4eea00640 1 -- 172.21.15.88:0/3660426241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f00ca710 con 0x7fd4f0157110 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4ef400640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015dbe0 0x7fd4f00ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4eea00640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 0x7fd4f01f6bb0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd4e002aa70 tx=0x7fd4e0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.757+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005c90 con 0x7fd4f0157110 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.758+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd4e0031e50 con 0x7fd4f0157110 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.758+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f00ca9f0 con 0x7fd4f0157110 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.758+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0005260 con 0x7fd4f0157110 2024-09-21T07:51:46.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.758+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01568c0 con 0x7fd4f0157110 2024-09-21T07:51:46.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.759+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f006d5a0 con 0x7fd4f0157110 2024-09-21T07:51:46.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.760+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7fd4e0005400 con 0x7fd4f0157110 2024-09-21T07:51:46.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.761+0000 7fd4e7400640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd4cc078d70 0x7fd4cc07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:46.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.761+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fd4e00bc200 con 0x7fd4f0157110 2024-09-21T07:51:46.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.761+0000 7fd4e6000640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd4cc078d70 0x7fd4cc07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:46.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.762+0000 7fd4e6000640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd4cc078d70 0x7fd4cc07b280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0156c60 tx=0x7fd4d8005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:46.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.765+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd4e003f030 con 0x7fd4f0157110 2024-09-21T07:51:46.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:46.980+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "cephfs", "sub_name": "subvol88300", "snap_name": "subvol_snap57170", "target": ["mon-mgr", ""]}) -- 0x7fd4f006ff80 con 0x7fd4cc078d70 2024-09-21T07:51:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.059+0000 7fd4e7400640 1 -- 172.21.15.88:0/3660426241 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd4f006ff80 con 0x7fd4cc078d70 2024-09-21T07:51:50.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd4cc078d70 msgr2=0x7fd4cc07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd4cc078d70 0x7fd4cc07b280 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0156c60 tx=0x7fd4d8005f90 comp rx=0 tx=0).stop 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 msgr2=0x7fd4f01f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 0x7fd4f01f6bb0 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7fd4e002aa70 tx=0x7fd4e0033c20 comp rx=0 tx=0).stop 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 shutdown_connections 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd4cc078d70 0x7fd4cc07b280 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4f015dbe0 0x7fd4f00ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4f015d7d0 0x7fd4f0152dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 --2- 172.21.15.88:0/3660426241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4f0157110 0x7fd4f01f6bb0 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 >> 172.21.15.88:0/3660426241 conn(0x7fd4f009aa50 msgr2=0x7fd4f015f250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 shutdown_connections 2024-09-21T07:51:50.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.062+0000 7fd4f5800640 1 -- 172.21.15.88:0/3660426241 wait complete. 2024-09-21T07:51:50.091 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol88300 subvol_snap57170 subvol_clone33240 2024-09-21T07:51:50.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 -- 172.21.15.88:0/3503728681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620015bca0 msgr2=0x7f62000cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 --2- 172.21.15.88:0/3503728681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620015bca0 0x7f62000cf9a0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f61f0009830 tx=0x7f61f002a480 comp rx=0 tx=0).stop 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 -- 172.21.15.88:0/3503728681 shutdown_connections 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 --2- 172.21.15.88:0/3503728681 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62000cf1c0 0x7f6200070630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 --2- 172.21.15.88:0/3503728681 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f620015bca0 0x7f62000cf9a0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 -- 172.21.15.88:0/3503728681 >> 172.21.15.88:0/3503728681 conn(0x7f62000b1dd0 msgr2=0x7f62000b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 -- 172.21.15.88:0/3503728681 shutdown_connections 2024-09-21T07:51:50.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.320+0000 7f6205800640 1 -- 172.21.15.88:0/3503728681 wait complete. 2024-09-21T07:51:50.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.322+0000 7f6205800640 1 Processor -- start 2024-09-21T07:51:50.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.322+0000 7f6205800640 1 -- start start 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6205800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6200070220 0x7f6200150260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6205800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62000cf1c0 0x7f62001f2fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6205800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 0x7f62001545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6205800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62000cd2f0 con 0x7f620015bca0 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6205800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f620015d560 con 0x7f62000cf1c0 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6205800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62000cd470 con 0x7f6200070220 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6204e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 0x7f62001545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6204e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 0x7f62001545b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39582/0 (socket says 172.21.15.88:39582) 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f6204e00640 1 -- 172.21.15.88:0/3630280612 learned_addr learned my addr 172.21.15.88:0/3630280612 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f61f7400640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62000cf1c0 0x7f62001f2fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.324+0000 7f61ffe00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6200070220 0x7f6200150260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6204e00640 1 -- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6200070220 msgr2=0x7f6200150260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6204e00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6200070220 0x7f6200150260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6204e00640 1 -- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62000cf1c0 msgr2=0x7f62001f2fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6204e00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62000cf1c0 0x7f62001f2fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6204e00640 1 -- 172.21.15.88:0/3630280612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62001f3cd0 con 0x7f620015bca0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6204e00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 0x7f62001545b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f61f800eab0 tx=0x7f61f800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f800cea0 con 0x7f620015bca0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f61f8004b20 con 0x7f620015bca0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f61f8005020 con 0x7f620015bca0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62001f3fb0 con 0x7f620015bca0 2024-09-21T07:51:50.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.325+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62001f4570 con 0x7f620015bca0 2024-09-21T07:51:50.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.326+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62001f3a40 con 0x7f620015bca0 2024-09-21T07:51:50.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.328+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7f61f80051c0 con 0x7f620015bca0 2024-09-21T07:51:50.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.329+0000 7f61fcc00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61dc078d40 0x7f61dc07b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.329+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f61f809be90 con 0x7f620015bca0 2024-09-21T07:51:50.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.329+0000 7f61ffe00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61dc078d40 0x7f61dc07b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.330+0000 7f61ffe00640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61dc078d40 0x7f61dc07b250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61f0034000 tx=0x7f61f0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:50.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.333+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f61f8063820 con 0x7f620015bca0 2024-09-21T07:51:50.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.551+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol88300", "snap_name": "subvol_snap57170", "target_sub_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f6200070630 con 0x7f61dc078d40 2024-09-21T07:51:50.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.624+0000 7f61fcc00640 1 -- 172.21.15.88:0/3630280612 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6200070630 con 0x7f61dc078d40 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.626+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61dc078d40 msgr2=0x7f61dc07b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.626+0000 7f6205800640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61dc078d40 0x7f61dc07b250 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f61f0034000 tx=0x7f61f0004730 comp rx=0 tx=0).stop 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 msgr2=0x7f62001545b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 0x7f62001545b0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f61f800eab0 tx=0x7f61f800ef70 comp rx=0 tx=0).stop 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 shutdown_connections 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f61dc078d40 0x7f61dc07b250 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f620015bca0 0x7f62001545b0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62000cf1c0 0x7f62001f2fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 --2- 172.21.15.88:0/3630280612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6200070220 0x7f6200150260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 >> 172.21.15.88:0/3630280612 conn(0x7f62000b1dd0 msgr2=0x7f620015e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:50.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 shutdown_connections 2024-09-21T07:51:50.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.627+0000 7f6205800640 1 -- 172.21.15.88:0/3630280612 wait complete. 2024-09-21T07:51:50.630 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.629+0000 7facaf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.630 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.629+0000 7facaf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.631 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.629+0000 7facaf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.631 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.629+0000 7facaf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.631 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.629+0000 7facaf400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.649 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.648+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.650 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.648+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.650 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.648+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.650 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.648+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.650 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:50.648+0000 7facb0800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:50.658 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:50.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 -- 172.21.15.88:0/791507906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc0073c50 msgr2=0x7effc01674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 --2- 172.21.15.88:0/791507906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc0073c50 0x7effc01674a0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7effb8009830 tx=0x7effb802a650 comp rx=0 tx=0).stop 2024-09-21T07:51:50.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 -- 172.21.15.88:0/791507906 shutdown_connections 2024-09-21T07:51:50.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 --2- 172.21.15.88:0/791507906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc0073c50 0x7effc01674a0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 -- 172.21.15.88:0/791507906 >> 172.21.15.88:0/791507906 conn(0x7effc006d1c0 msgr2=0x7effc006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:50.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 -- 172.21.15.88:0/791507906 shutdown_connections 2024-09-21T07:51:50.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.899+0000 7effc7400640 1 -- 172.21.15.88:0/791507906 wait complete. 2024-09-21T07:51:50.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.901+0000 7effc7400640 1 Processor -- start 2024-09-21T07:51:50.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.901+0000 7effc7400640 1 -- start start 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.902+0000 7effc7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc0145550 0x7effc01f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.902+0000 7effc7400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc0145a40 0x7effc015c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.902+0000 7effc7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 0x7effc015d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc7400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7effc0167d70 con 0x7effc015fbe0 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc7400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7effc0167bf0 con 0x7effc0145550 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc7400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7effc0167ef0 con 0x7effc0145a40 2024-09-21T07:51:50.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 0x7effc015d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 0x7effc015d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39630/0 (socket says 172.21.15.88:39630) 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 -- 172.21.15.88:0/1307970674 learned_addr learned my addr 172.21.15.88:0/1307970674 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc4a00640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc0145550 0x7effc01f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effbfe00640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc0145a40 0x7effc015c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 -- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc0145a40 msgr2=0x7effc015c4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc0145a40 0x7effc015c4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 -- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc0145550 msgr2=0x7effc01f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:50.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc0145550 0x7effc01f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.903+0000 7effc5400640 1 -- 172.21.15.88:0/1307970674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7effc015cd60 con 0x7effc015fbe0 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.904+0000 7effc5400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 0x7effc015d540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7effb8009950 tx=0x7effb8033ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.904+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8031d60 con 0x7effc015fbe0 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.904+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7effb8005020 con 0x7effc015fbe0 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.904+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7effb8005450 con 0x7effc015fbe0 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.904+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7effc01f3d40 con 0x7effc015fbe0 2024-09-21T07:51:50.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.904+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7effc01f4300 con 0x7effc015fbe0 2024-09-21T07:51:50.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.905+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7effc006fb40 con 0x7effc015fbe0 2024-09-21T07:51:50.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.906+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 18) ==== 101594+0+0 (secure 0 0 0) 0x7effb80047a0 con 0x7effc015fbe0 2024-09-21T07:51:50.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.908+0000 7effbcc00640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff9c078d70 0x7eff9c07b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:50.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.908+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7effb80bc260 con 0x7effc015fbe0 2024-09-21T07:51:50.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.908+0000 7effc4a00640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff9c078d70 0x7eff9c07b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:50.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.909+0000 7effc4a00640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff9c078d70 0x7eff9c07b280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7effb0001e20 tx=0x7effb0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:50.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:50.912+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7effb803f030 con 0x7effc015fbe0 2024-09-21T07:51:51.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.132+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7effc01609a0 con 0x7eff9c078d70 2024-09-21T07:51:51.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.157+0000 7effbcc00640 1 -- 172.21.15.88:0/1307970674 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7effc01609a0 con 0x7eff9c078d70 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "in-progress", 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "subvol_snap57170" 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "progress_report": { 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "percentage cloned": "0.003%", 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "amount cloned": "213/8.0M", 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: "files cloned": "3/9" 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:51.159 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:51.160 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:51.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff9c078d70 msgr2=0x7eff9c07b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:51.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff9c078d70 0x7eff9c07b280 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7effb0001e20 tx=0x7effb0009040 comp rx=0 tx=0).stop 2024-09-21T07:51:51.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 msgr2=0x7effc015d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:51.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 0x7effc015d540 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7effb8009950 tx=0x7effb8033ce0 comp rx=0 tx=0).stop 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 shutdown_connections 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff9c078d70 0x7eff9c07b280 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7effc0145a40 0x7effc015c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7effc015fbe0 0x7effc015d540 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 --2- 172.21.15.88:0/1307970674 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7effc0145550 0x7effc01f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 >> 172.21.15.88:0/1307970674 conn(0x7effc006d1c0 msgr2=0x7effc0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 shutdown_connections 2024-09-21T07:51:51.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:51.160+0000 7effc7400640 1 -- 172.21.15.88:0/1307970674 wait complete. 2024-09-21T07:51:51.186 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T07:51:51.632 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:51.631+0000 7fac91400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:51.633 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:51.631+0000 7fac91400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:51.633 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:51.631+0000 7fac91400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:51.633 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:51.631+0000 7fac91400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:51.633 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:51:51.631+0000 7fac91400640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:51:52.187 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e19e00640 1 -- 172.21.15.88:0/952352214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0df8003c10 msgr2=0x7f0df8004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e19e00640 1 --2- 172.21.15.88:0/952352214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0df8003c10 0x7f0df8004060 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f0e0400c4a0 tx=0x7f0df8005830 comp rx=0 tx=0).stop 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e13400640 1 -- 172.21.15.88:0/952352214 reap_dead start 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e19e00640 1 -- 172.21.15.88:0/952352214 shutdown_connections 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e19e00640 1 -- 172.21.15.88:0/952352214 >> 172.21.15.88:0/952352214 conn(0x7f0e1406d1c0 msgr2=0x7f0e1406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e19e00640 1 -- 172.21.15.88:0/952352214 shutdown_connections 2024-09-21T07:51:52.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.410+0000 7f0e19e00640 1 -- 172.21.15.88:0/952352214 wait complete. 2024-09-21T07:51:52.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.413+0000 7f0e19e00640 1 Processor -- start 2024-09-21T07:51:52.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.413+0000 7f0e19e00640 1 -- start start 2024-09-21T07:51:52.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df8003c10 0x7f0e1413e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 0x7f0e140b4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e19e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1413c410 0x7f0e140b4d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e19e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140cdea0 con 0x7f0e1415be80 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e19e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e140cdd20 con 0x7f0df8003c10 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e19e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e140ce020 con 0x7f0e1413c410 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 0x7f0e140b4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 0x7f0e140b4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39662/0 (socket says 172.21.15.88:39662) 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.414+0000 7f0e0be00640 1 -- 172.21.15.88:0/1507910050 learned_addr learned my addr 172.21.15.88:0/1507910050 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e13400640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1413c410 0x7f0e140b4d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e12a00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df8003c10 0x7f0e1413e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0be00640 1 -- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1413c410 msgr2=0x7f0e140b4d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0be00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1413c410 0x7f0e140b4d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0be00640 1 -- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df8003c10 msgr2=0x7f0e1413e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0be00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df8003c10 0x7f0e1413e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0be00640 1 -- 172.21.15.88:0/1507910050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e140b3790 con 0x7f0e1415be80 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e13400640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1413c410 0x7f0e140b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e12a00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df8003c10 0x7f0e1413e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0be00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 0x7f0e140b4830 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc011980 tx=0x7f0dfc011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc01f070 con 0x7f0e1415be80 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dfc00fde0 con 0x7f0e1415be80 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.415+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc0126b0 con 0x7f0e1415be80 2024-09-21T07:51:52.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.416+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e140b3a70 con 0x7f0e1415be80 2024-09-21T07:51:52.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.416+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e140b4030 con 0x7f0e1415be80 2024-09-21T07:51:52.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.417+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e140ce4f0 con 0x7f0e1415be80 2024-09-21T07:51:52.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.418+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 19) ==== 101696+0+0 (secure 0 0 0) 0x7f0dfc022070 con 0x7f0e1415be80 2024-09-21T07:51:52.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.420+0000 7f0e0b400640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0df0078e60 0x7f0df007b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:52.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.420+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f0dfc09bc20 con 0x7f0e1415be80 2024-09-21T07:51:52.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.420+0000 7f0e12a00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0df0078e60 0x7f0df007b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:52.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.420+0000 7f0e12a00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0df0078e60 0x7f0df007b370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e04033000 tx=0x7f0e0402d070 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:52.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.424+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0dfc0635c0 con 0x7f0e1415be80 2024-09-21T07:51:52.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.645+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f0e140b52b0 con 0x7f0df0078e60 2024-09-21T07:51:52.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.663+0000 7f0e0b400640 1 -- 172.21.15.88:0/1507910050 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f0e04001920 con 0x7f0df0078e60 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "in-progress", 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "subvol_snap57170" 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "progress_report": { 2024-09-21T07:51:52.666 INFO:teuthology.orchestra.run.smithi088.stdout: "percentage cloned": "0.003%", 2024-09-21T07:51:52.667 INFO:teuthology.orchestra.run.smithi088.stdout: "amount cloned": "213/8.0M", 2024-09-21T07:51:52.667 INFO:teuthology.orchestra.run.smithi088.stdout: "files cloned": "3/9" 2024-09-21T07:51:52.667 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:52.667 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:52.667 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0df0078e60 msgr2=0x7f0df007b370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0df0078e60 0x7f0df007b370 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f0e04033000 tx=0x7f0e0402d070 comp rx=0 tx=0).stop 2024-09-21T07:51:52.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 msgr2=0x7f0e140b4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 0x7f0e140b4830 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc011980 tx=0x7f0dfc011e40 comp rx=0 tx=0).stop 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 shutdown_connections 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0df0078e60 0x7f0df007b370 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e1413c410 0x7f0e140b4d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e1415be80 0x7f0e140b4830 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 --2- 172.21.15.88:0/1507910050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0df8003c10 0x7f0e1413e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 >> 172.21.15.88:0/1507910050 conn(0x7f0e1406d1c0 msgr2=0x7f0e140cea20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 shutdown_connections 2024-09-21T07:51:52.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:52.666+0000 7f0e19e00640 1 -- 172.21.15.88:0/1507910050 wait complete. 2024-09-21T07:51:52.694 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T07:51:53.695 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 -- 172.21.15.88:0/2230490278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c0b3f30 msgr2=0x7f6b8c156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2230490278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c156ca0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f6b8c0b3160 tx=0x7f6b7c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 -- 172.21.15.88:0/2230490278 shutdown_connections 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2230490278 >> v2:172.21.15.175:3300/0 conn(0x7f6b8c15d6e0 0x7f6b8c0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2230490278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c0b3f30 0x7f6b8c156ca0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 -- 172.21.15.88:0/2230490278 >> 172.21.15.88:0/2230490278 conn(0x7f6b8c06d620 msgr2=0x7f6b8c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 -- 172.21.15.88:0/2230490278 shutdown_connections 2024-09-21T07:51:53.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.923+0000 7f6b8b200640 1 -- 172.21.15.88:0/2230490278 wait complete. 2024-09-21T07:51:53.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.925+0000 7f6b8b200640 1 Processor -- start 2024-09-21T07:51:53.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.926+0000 7f6b8b200640 1 -- start start 2024-09-21T07:51:53.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b8c0b3f30 0x7f6b8c1f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b8c0cd000 0x7f6b8c145990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 0x7f6b8c15c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c0af960 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b8c0af7e0 con 0x7f6b8c0cd000 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b8c0afae0 con 0x7f6b8c0b3f30 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 0x7f6b8c15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 0x7f6b8c15c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39672/0 (socket says 172.21.15.88:39672) 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b8a800640 1 -- 172.21.15.88:0/2804488785 learned_addr learned my addr 172.21.15.88:0/2804488785 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.927+0000 7f6b89400640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b8c0cd000 0x7f6b8c145990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b89e00640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b8c0b3f30 0x7f6b8c1f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8a800640 1 -- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b8c0b3f30 msgr2=0x7f6b8c1f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:53.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8a800640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b8c0b3f30 0x7f6b8c1f8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8a800640 1 -- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b8c0cd000 msgr2=0x7f6b8c145990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8a800640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b8c0cd000 0x7f6b8c145990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8a800640 1 -- 172.21.15.88:0/2804488785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b8c1f94a0 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8a800640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 0x7f6b8c15c110 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c730 tx=0x7f6b8400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b84017070 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b84003c60 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b840052d0 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.928+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b8c1f9780 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.929+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b8c1f9d40 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.930+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b8c0b10c0 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.931+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f6b8401a070 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.932+0000 7f6b82000640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b6c078ef0 0x7f6b6c07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:53.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.933+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f6b8409ab50 con 0x7f6b8c15d6e0 2024-09-21T07:51:53.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.933+0000 7f6b89e00640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b6c078ef0 0x7f6b6c07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:53.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.933+0000 7f6b89e00640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b6c078ef0 0x7f6b6c07b400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c033000 tx=0x7f6b7c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:53.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:53.937+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b8c0b10c0 con 0x7f6b8c15d6e0 2024-09-21T07:51:54.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.166+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f6b8c156ca0 con 0x7f6b6c078ef0 2024-09-21T07:51:54.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.182+0000 7f6b82000640 1 -- 172.21.15.88:0/2804488785 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+301 (secure 0 0 0) 0x7f6b8c156ca0 con 0x7f6b6c078ef0 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "in-progress", 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:51:54.185 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "subvol_snap57170" 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: "progress_report": { 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: "percentage cloned": "0.003%", 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: "amount cloned": "213/8.0M", 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: "files cloned": "3/9" 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:54.186 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:54.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b6c078ef0 msgr2=0x7f6b6c07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b6c078ef0 0x7f6b6c07b400 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c033000 tx=0x7f6b7c004730 comp rx=0 tx=0).stop 2024-09-21T07:51:54.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 msgr2=0x7f6b8c15c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 0x7f6b8c15c110 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f6b8400c730 tx=0x7f6b8400cbf0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 shutdown_connections 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b6c078ef0 0x7f6b6c07b400 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b8c15d6e0 0x7f6b8c15c110 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b8c0cd000 0x7f6b8c145990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 --2- 172.21.15.88:0/2804488785 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b8c0b3f30 0x7f6b8c1f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 >> 172.21.15.88:0/2804488785 conn(0x7f6b8c06d620 msgr2=0x7f6b8c0b12e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 shutdown_connections 2024-09-21T07:51:54.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:54.185+0000 7f6b8b200640 1 -- 172.21.15.88:0/2804488785 wait complete. 2024-09-21T07:51:54.211 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T07:51:55.212 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 -- 172.21.15.88:0/445259315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90440cd610 msgr2=0x7f90440cda20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 --2- 172.21.15.88:0/445259315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90440cd610 0x7f90440cda20 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7f9044072bd0 tx=0x7f903c02aa30 comp rx=0 tx=0).stop 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 -- 172.21.15.88:0/445259315 shutdown_connections 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 --2- 172.21.15.88:0/445259315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f90440cd610 0x7f90440cda20 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 --2- 172.21.15.88:0/445259315 >> v2:172.21.15.175:3300/0 conn(0x7f9044145450 0x7f904406f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 -- 172.21.15.88:0/445259315 >> 172.21.15.88:0/445259315 conn(0x7f904406d1c0 msgr2=0x7f904406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:55.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.433+0000 7f904b400640 1 -- 172.21.15.88:0/445259315 shutdown_connections 2024-09-21T07:51:55.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.434+0000 7f904b400640 1 -- 172.21.15.88:0/445259315 wait complete. 2024-09-21T07:51:55.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.436+0000 7f904b400640 1 Processor -- start 2024-09-21T07:51:55.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.436+0000 7f904b400640 1 -- start start 2024-09-21T07:51:55.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.437+0000 7f904b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440735e0 0x7f904413df90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.437+0000 7f904b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90440cd610 0x7f90441f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.437+0000 7f904b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 0x7f90441f92d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.437+0000 7f904b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f90440ce2f0 con 0x7f9044145450 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.437+0000 7f904b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90440ce170 con 0x7f90440cd610 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.437+0000 7f904b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f90440ce470 con 0x7f90440735e0 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 0x7f90441f92d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 0x7f90441f92d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:39686/0 (socket says 172.21.15.88:39686) 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 -- 172.21.15.88:0/2991204596 learned_addr learned my addr 172.21.15.88:0/2991204596 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f903be00640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90440cd610 0x7f90441f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:55.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9048a00640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440735e0 0x7f904413df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 -- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440735e0 msgr2=0x7f904413df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440735e0 0x7f904413df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 -- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90440cd610 msgr2=0x7f90441f8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90440cd610 0x7f90441f8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f9049400640 1 -- 172.21.15.88:0/2991204596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90440b3aa0 con 0x7f9044145450 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.438+0000 7f903be00640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90440cd610 0x7f90441f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.439+0000 7f9049400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 0x7f90441f92d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f903c02b020 tx=0x7f903c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.439+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c033050 con 0x7f9044145450 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.439+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f903c004450 con 0x7f9044145450 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.439+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f903c004840 con 0x7f9044145450 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.439+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90440aee50 con 0x7f9044145450 2024-09-21T07:51:55.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.439+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90440af410 con 0x7f9044145450 2024-09-21T07:51:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.440+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90440cda20 con 0x7f9044145450 2024-09-21T07:51:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.441+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f903c036070 con 0x7f9044145450 2024-09-21T07:51:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.442+0000 7f9041600640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9020078ef0 0x7f902007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:55.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.442+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f903c0bcad0 con 0x7f9044145450 2024-09-21T07:51:55.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.442+0000 7f9048a00640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9020078ef0 0x7f902007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:55.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.443+0000 7f9048a00640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9020078ef0 0x7f902007b400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9034005fd0 tx=0x7f9034005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:55.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.447+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f903c037e90 con 0x7f9044145450 2024-09-21T07:51:55.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.669+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f904413ee80 con 0x7f9020078ef0 2024-09-21T07:51:55.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.687+0000 7f9041600640 1 -- 172.21.15.88:0/2991204596 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7f904413ee80 con 0x7f9020078ef0 2024-09-21T07:51:55.689 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:55.689 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "in-progress", 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "subvol_snap57170" 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "progress_report": { 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "percentage cloned": "75.003%", 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "amount cloned": "6.0M/8.0M", 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: "files cloned": "10/9" 2024-09-21T07:51:55.690 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:55.691 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:55.691 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:55.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9020078ef0 msgr2=0x7f902007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:55.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9020078ef0 0x7f902007b400 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f9034005fd0 tx=0x7f9034005850 comp rx=0 tx=0).stop 2024-09-21T07:51:55.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 msgr2=0x7f90441f92d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 0x7f90441f92d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f903c02b020 tx=0x7f903c003f10 comp rx=0 tx=0).stop 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 shutdown_connections 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9020078ef0 0x7f902007b400 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9044145450 0x7f90441f92d0 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f90440cd610 0x7f90441f8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 --2- 172.21.15.88:0/2991204596 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90440735e0 0x7f904413df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 >> 172.21.15.88:0/2991204596 conn(0x7f904406d1c0 msgr2=0x7f9044141560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 shutdown_connections 2024-09-21T07:51:55.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:55.690+0000 7f904b400640 1 -- 172.21.15.88:0/2991204596 wait complete. 2024-09-21T07:51:55.718 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T07:51:56.719 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:56.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.951+0000 7fd89cc00640 1 -- 172.21.15.88:0/3596785844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 msgr2=0x7fd898156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.951+0000 7fd89cc00640 1 --2- 172.21.15.88:0/3596785844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd898156ca0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fd888009830 tx=0x7fd88802a5b0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.951+0000 7fd89cc00640 1 -- 172.21.15.88:0/3596785844 shutdown_connections 2024-09-21T07:51:56.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.951+0000 7fd89cc00640 1 --2- 172.21.15.88:0/3596785844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd898156ca0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.952+0000 7fd89cc00640 1 -- 172.21.15.88:0/3596785844 >> 172.21.15.88:0/3596785844 conn(0x7fd89806d620 msgr2=0x7fd8980b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:56.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.952+0000 7fd89cc00640 1 -- 172.21.15.88:0/3596785844 shutdown_connections 2024-09-21T07:51:56.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.952+0000 7fd89cc00640 1 -- 172.21.15.88:0/3596785844 wait complete. 2024-09-21T07:51:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.954+0000 7fd89cc00640 1 Processor -- start 2024-09-21T07:51:56.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.954+0000 7fd89cc00640 1 -- start start 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.955+0000 7fd89cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd8981f5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.955+0000 7fd89cc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8980cd1d0 0x7fd8981f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.955+0000 7fd89cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd89815d6e0 0x7fd89815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.955+0000 7fd89cc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd8980af7e0 con 0x7fd8980b3f30 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.955+0000 7fd89cc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd898070370 con 0x7fd8980cd1d0 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.955+0000 7fd89cc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd8980af960 con 0x7fd89815d6e0 2024-09-21T07:51:56.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd8981f5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd8981f5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57902/0 (socket says 172.21.15.88:57902) 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 -- 172.21.15.88:0/2551370962 learned_addr learned my addr 172.21.15.88:0/2551370962 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd896a00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8980cd1d0 0x7fd8981f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 -- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd89815d6e0 msgr2=0x7fd89815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd89815d6e0 0x7fd89815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 -- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8980cd1d0 msgr2=0x7fd8981f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8980cd1d0 0x7fd8981f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 -- 172.21.15.88:0/2551370962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8981f5760 con 0x7fd8980b3f30 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd896a00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8980cd1d0 0x7fd8981f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:51:56.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.956+0000 7fd897400640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd8981f5220 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd888009950 tx=0x7fd888033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.957+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888031d60 con 0x7fd8980b3f30 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.957+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd8880330d0 con 0x7fd8980b3f30 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.957+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8981f9df0 con 0x7fd8980b3f30 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.957+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd888033520 con 0x7fd8980b3f30 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.957+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd89809a840 con 0x7fd8980b3f30 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.958+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8981567e0 con 0x7fd8980b3f30 2024-09-21T07:51:56.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.959+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fd8880035f0 con 0x7fd8980b3f30 2024-09-21T07:51:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.960+0000 7fd88f400640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd874078ef0 0x7fd87407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.960+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fd8880bca70 con 0x7fd8980b3f30 2024-09-21T07:51:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.960+0000 7fd896a00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd874078ef0 0x7fd87407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:56.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.961+0000 7fd896a00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd874078ef0 0x7fd87407b400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd898073010 tx=0x7fd880009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:56.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:56.964+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd88804a020 con 0x7fd8980b3f30 2024-09-21T07:51:57.034 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:57.038 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:51:57.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.192+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7fd89815e4b0 con 0x7fd874078ef0 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.211+0000 7fd88f400640 1 -- 172.21.15.88:0/2551370962 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7fd89815e4b0 con 0x7fd874078ef0 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "in-progress", 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:51:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "subvol_snap57170" 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: "progress_report": { 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: "percentage cloned": "75.003%", 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: "amount cloned": "6.0M/8.0M", 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: "files cloned": "10/9" 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:57.214 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:57.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd874078ef0 msgr2=0x7fd87407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd874078ef0 0x7fd87407b400 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd898073010 tx=0x7fd880009040 comp rx=0 tx=0).stop 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 msgr2=0x7fd8981f5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd8981f5220 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fd888009950 tx=0x7fd888033d60 comp rx=0 tx=0).stop 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 shutdown_connections 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd874078ef0 0x7fd87407b400 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd89815d6e0 0x7fd89815b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd8980cd1d0 0x7fd8981f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 --2- 172.21.15.88:0/2551370962 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd8980b3f30 0x7fd8981f5220 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 >> 172.21.15.88:0/2551370962 conn(0x7fd89806d620 msgr2=0x7fd898070580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 shutdown_connections 2024-09-21T07:51:57.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:57.213+0000 7fd89cc00640 1 -- 172.21.15.88:0/2551370962 wait complete. 2024-09-21T07:51:57.239 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T07:51:58.241 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 -- 172.21.15.88:0/2632322942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68145a40 msgr2=0x7f6f68073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 --2- 172.21.15.88:0/2632322942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68145a40 0x7f6f68073540 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f6f54005fd0 tx=0x7f6f5402a670 comp rx=0 tx=0).stop 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 -- 172.21.15.88:0/2632322942 shutdown_connections 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 --2- 172.21.15.88:0/2632322942 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f68073a80 0x7f6f680cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 --2- 172.21.15.88:0/2632322942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68145a40 0x7f6f68073540 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 -- 172.21.15.88:0/2632322942 >> 172.21.15.88:0/2632322942 conn(0x7f6f6806d1c0 msgr2=0x7f6f6806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 -- 172.21.15.88:0/2632322942 shutdown_connections 2024-09-21T07:51:58.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.464+0000 7f6f6d000640 1 -- 172.21.15.88:0/2632322942 wait complete. 2024-09-21T07:51:58.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.467+0000 7f6f6d000640 1 Processor -- start 2024-09-21T07:51:58.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.467+0000 7f6f6d000640 1 -- start start 2024-09-21T07:51:58.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f6d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 0x7f6f680ae710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f6d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f68145550 0x7f6f6812ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f6d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f68145a40 0x7f6f6812f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f6d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680ce160 con 0x7f6f68073a80 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f6d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f680cdfe0 con 0x7f6f68145a40 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f6d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f680ce2e0 con 0x7f6f68145550 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f65e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 0x7f6f680ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f65e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 0x7f6f680ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57916/0 (socket says 172.21.15.88:57916) 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.468+0000 7f6f65e00640 1 -- 172.21.15.88:0/333363643 learned_addr learned my addr 172.21.15.88:0/333363643 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65400640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f68145550 0x7f6f6812ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f66800640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f68145a40 0x7f6f6812f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65e00640 1 -- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f68145550 msgr2=0x7f6f6812ed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65e00640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f68145550 0x7f6f6812ed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65e00640 1 -- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f68145a40 msgr2=0x7f6f6812f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65e00640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f68145a40 0x7f6f6812f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65e00640 1 -- 172.21.15.88:0/333363643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f680a9ea0 con 0x7f6f68073a80 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f66800640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f68145a40 0x7f6f6812f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f65e00640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 0x7f6f680ae710 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800c3d0 tx=0x7f6f5800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58017070 con 0x7f6f68073a80 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6f580053c0 con 0x7f6f68073a80 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f58003850 con 0x7f6f68073a80 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.469+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f680aa180 con 0x7f6f68073a80 2024-09-21T07:51:58.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.470+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f680aa740 con 0x7f6f68073a80 2024-09-21T07:51:58.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.470+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f680ce790 con 0x7f6f68073a80 2024-09-21T07:51:58.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.472+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f6f5801a070 con 0x7f6f68073a80 2024-09-21T07:51:58.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.473+0000 7f6f5e000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6f48078f20 0x7f6f4807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:58.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.473+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f6f5809a6a0 con 0x7f6f68073a80 2024-09-21T07:51:58.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.473+0000 7f6f65400640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6f48078f20 0x7f6f4807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:58.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.474+0000 7f6f65400640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6f48078f20 0x7f6f4807b430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6f54033000 tx=0x7f6f54004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:58.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.477+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6f68073e90 con 0x7f6f68073a80 2024-09-21T07:51:58.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.700+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f6f6806e140 con 0x7f6f48078f20 2024-09-21T07:51:58.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.720+0000 7f6f5e000640 1 -- 172.21.15.88:0/333363643 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+304 (secure 0 0 0) 0x7f6f6806e140 con 0x7f6f48078f20 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "in-progress", 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "subvol_snap57170" 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:51:58.722 INFO:teuthology.orchestra.run.smithi088.stdout: "progress_report": { 2024-09-21T07:51:58.723 INFO:teuthology.orchestra.run.smithi088.stdout: "percentage cloned": "75.003%", 2024-09-21T07:51:58.723 INFO:teuthology.orchestra.run.smithi088.stdout: "amount cloned": "6.0M/8.0M", 2024-09-21T07:51:58.723 INFO:teuthology.orchestra.run.smithi088.stdout: "files cloned": "10/9" 2024-09-21T07:51:58.723 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:58.723 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:51:58.723 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6f48078f20 msgr2=0x7f6f4807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6f48078f20 0x7f6f4807b430 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f6f54033000 tx=0x7f6f54004730 comp rx=0 tx=0).stop 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 msgr2=0x7f6f680ae710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 0x7f6f680ae710 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800c3d0 tx=0x7f6f5800c890 comp rx=0 tx=0).stop 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 shutdown_connections 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6f48078f20 0x7f6f4807b430 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f68145550 0x7f6f6812ed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f68145a40 0x7f6f6812f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 --2- 172.21.15.88:0/333363643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f68073a80 0x7f6f680ae710 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.722+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 >> 172.21.15.88:0/333363643 conn(0x7f6f6806d1c0 msgr2=0x7f6f6806f9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.723+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 shutdown_connections 2024-09-21T07:51:58.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:58.723+0000 7f6f6d000640 1 -- 172.21.15.88:0/333363643 wait complete. 2024-09-21T07:51:58.750 DEBUG:tasks.cephfs.test_volumes:current clone state = in-progress 2024-09-21T07:51:59.751 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone33240 2024-09-21T07:51:59.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 -- 172.21.15.88:0/2311360623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac415f9e0 msgr2=0x7f5ac4147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/2311360623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac415f9e0 0x7f5ac4147b90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4005fd0 tx=0x7f5ab402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 -- 172.21.15.88:0/2311360623 shutdown_connections 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/2311360623 >> v2:172.21.15.175:3300/0 conn(0x7f5ac4073c50 0x7f5ac40af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/2311360623 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac415f9e0 0x7f5ac4147b90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 -- 172.21.15.88:0/2311360623 >> 172.21.15.88:0/2311360623 conn(0x7f5ac406d1c0 msgr2=0x7f5ac406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 -- 172.21.15.88:0/2311360623 shutdown_connections 2024-09-21T07:51:59.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.985+0000 7f5ac9a00640 1 -- 172.21.15.88:0/2311360623 wait complete. 2024-09-21T07:51:59.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.987+0000 7f5ac9a00640 1 Processor -- start 2024-09-21T07:51:59.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.988+0000 7f5ac9a00640 1 -- start start 2024-09-21T07:51:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac40aee00 0x7f5ac40b16d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 0x7f5ac41f5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac41f63f0 0x7f5ac413e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac406fb60 con 0x7f5ac415f9e0 2024-09-21T07:51:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ac406f9e0 con 0x7f5ac41f63f0 2024-09-21T07:51:59.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ac406fce0 con 0x7f5ac40aee00 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 0x7f5ac41f5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 0x7f5ac41f5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57918/0 (socket says 172.21.15.88:57918) 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.989+0000 7f5ac2000640 1 -- 172.21.15.88:0/4164334114 learned_addr learned my addr 172.21.15.88:0/4164334114 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac40aee00 0x7f5ac40b16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac3400640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac41f63f0 0x7f5ac413e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2000640 1 -- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac40aee00 msgr2=0x7f5ac40b16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2000640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac40aee00 0x7f5ac40b16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2000640 1 -- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac41f63f0 msgr2=0x7f5ac413e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2000640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac41f63f0 0x7f5ac413e5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2000640 1 -- 172.21.15.88:0/4164334114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ac40b04b0 con 0x7f5ac415f9e0 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac3400640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac41f63f0 0x7f5ac413e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:51:59.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5ac2000640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 0x7f5ac41f5eb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c370 tx=0x7f5ab000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:51:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0017070 con 0x7f5ac415f9e0 2024-09-21T07:51:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.990+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ab00053c0 con 0x7f5ac415f9e0 2024-09-21T07:51:59.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.991+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ac40b0790 con 0x7f5ac415f9e0 2024-09-21T07:51:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.991+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ab0003850 con 0x7f5ac415f9e0 2024-09-21T07:51:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.991+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ac40b0d50 con 0x7f5ac415f9e0 2024-09-21T07:51:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.992+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ac40b0220 con 0x7f5ac415f9e0 2024-09-21T07:51:59.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.993+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f5ab001a070 con 0x7f5ac415f9e0 2024-09-21T07:51:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.994+0000 7f5abaa00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5aa4078ef0 0x7f5aa407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:51:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.994+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f5ab009a690 con 0x7f5ac415f9e0 2024-09-21T07:51:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.994+0000 7f5ac2a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5aa4078ef0 0x7f5aa407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:51:59.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.995+0000 7f5ac2a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5aa4078ef0 0x7f5aa407b400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4033000 tx=0x7f5ab4004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:00.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:51:59.998+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ab0061f70 con 0x7f5ac415f9e0 2024-09-21T07:52:00.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:00.247+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f5ac40af210 con 0x7f5aa4078ef0 2024-09-21T07:52:05.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.045+0000 7f5abaa00640 1 -- 172.21.15.88:0/4164334114 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f5ac40af210 con 0x7f5aa4078ef0 2024-09-21T07:52:05.048 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:05.048 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:52:05.049 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "complete" 2024-09-21T07:52:05.049 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:05.049 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.048+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5aa4078ef0 msgr2=0x7f5aa407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.048+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5aa4078ef0 0x7f5aa407b400 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f5ab4033000 tx=0x7f5ab4004730 comp rx=0 tx=0).stop 2024-09-21T07:52:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 msgr2=0x7f5ac41f5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 0x7f5ac41f5eb0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7f5ab000c370 tx=0x7f5ab000c830 comp rx=0 tx=0).stop 2024-09-21T07:52:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 shutdown_connections 2024-09-21T07:52:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5aa4078ef0 0x7f5aa407b400 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ac41f63f0 0x7f5ac413e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ac415f9e0 0x7f5ac41f5eb0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 --2- 172.21.15.88:0/4164334114 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ac40aee00 0x7f5ac40b16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 >> 172.21.15.88:0/4164334114 conn(0x7f5ac406d1c0 msgr2=0x7f5ac41607f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 shutdown_connections 2024-09-21T07:52:05.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.049+0000 7f5ac9a00640 1 -- 172.21.15.88:0/4164334114 wait complete. 2024-09-21T07:52:05.080 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-21T07:52:05.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:05.080 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone33240/.meta) 2024-09-21T07:52:05.140 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:52:05.141 INFO:teuthology.orchestra.run.smithi088.stdout:version = 2 2024-09-21T07:52:05.141 INFO:teuthology.orchestra.run.smithi088.stdout:type = clone 2024-09-21T07:52:05.141 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079 2024-09-21T07:52:05.141 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:52:05.141 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:05.144 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol88300 2024-09-21T07:52:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f50200640 1 -- 172.21.15.88:0/1960910319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 msgr2=0x7f7f30005220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f50200640 1 --2- 172.21.15.88:0/1960910319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f30005220 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f7f4000c560 tx=0x7f7f30005830 comp rx=0 tx=0).stop 2024-09-21T07:52:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f4e200640 1 -- 172.21.15.88:0/1960910319 reap_dead start 2024-09-21T07:52:05.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f50200640 1 -- 172.21.15.88:0/1960910319 shutdown_connections 2024-09-21T07:52:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f50200640 1 -- 172.21.15.88:0/1960910319 >> 172.21.15.88:0/1960910319 conn(0x7f7f4806d1c0 msgr2=0x7f7f4806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f50200640 1 -- 172.21.15.88:0/1960910319 shutdown_connections 2024-09-21T07:52:05.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.367+0000 7f7f50200640 1 -- 172.21.15.88:0/1960910319 wait complete. 2024-09-21T07:52:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.369+0000 7f7f50200640 1 Processor -- start 2024-09-21T07:52:05.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.370+0000 7f7f50200640 1 -- start start 2024-09-21T07:52:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f50200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f48139dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f50200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f480af0b0 0x7f7f48138780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f50200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f48138cc0 0x7f7f481f8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f50200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480cdea0 con 0x7f7f30004d90 2024-09-21T07:52:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f50200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480cdd20 con 0x7f7f480af0b0 2024-09-21T07:52:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f50200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7f480ce020 con 0x7f7f48138cc0 2024-09-21T07:52:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f4d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f48139dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f4d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f48139dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57932/0 (socket says 172.21.15.88:57932) 2024-09-21T07:52:05.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.371+0000 7f7f4d800640 1 -- 172.21.15.88:0/154408034 learned_addr learned my addr 172.21.15.88:0/154408034 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4e200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f48138cc0 0x7f7f481f8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4ce00640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f480af0b0 0x7f7f48138780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4d800640 1 -- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f48138cc0 msgr2=0x7f7f481f8a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4d800640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f48138cc0 0x7f7f481f8a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4d800640 1 -- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f480af0b0 msgr2=0x7f7f48138780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4d800640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f480af0b0 0x7f7f48138780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4d800640 1 -- 172.21.15.88:0/154408034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f481390d0 con 0x7f7f30004d90 2024-09-21T07:52:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4e200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f48138cc0 0x7f7f481f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4ce00640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f480af0b0 0x7f7f48138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f4d800640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f48139dd0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c011ae0 tx=0x7f7f3c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c01e070 con 0x7f7f30004d90 2024-09-21T07:52:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f3c00fe70 con 0x7f7f30004d90 2024-09-21T07:52:05.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f480ae010 con 0x7f7f30004d90 2024-09-21T07:52:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f3c004290 con 0x7f7f30004d90 2024-09-21T07:52:05.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.372+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f480ae5d0 con 0x7f7f30004d90 2024-09-21T07:52:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.374+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f480ce580 con 0x7f7f30004d90 2024-09-21T07:52:05.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.375+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f7f3c021070 con 0x7f7f30004d90 2024-09-21T07:52:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.376+0000 7f7f45600640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f28078ef0 0x7f7f2807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.376+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f7f3c09bef0 con 0x7f7f30004d90 2024-09-21T07:52:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.376+0000 7f7f4ce00640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f28078ef0 0x7f7f2807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.377+0000 7f7f4ce00640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f28078ef0 0x7f7f2807b400 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7f34000c00 tx=0x7f7f340069e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:05.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.380+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f3c0637d0 con 0x7f7f30004d90 2024-09-21T07:52:05.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.594+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7f7f48144cf0 con 0x7f7f28078ef0 2024-09-21T07:52:05.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.608+0000 7f7f45600640 1 -- 172.21.15.88:0/154408034 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f7f48144cf0 con 0x7f7f28078ef0 2024-09-21T07:52:05.610 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e 2024-09-21T07:52:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f28078ef0 msgr2=0x7f7f2807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f28078ef0 0x7f7f2807b400 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f7f34000c00 tx=0x7f7f340069e0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 msgr2=0x7f7f48139dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f48139dd0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f7f3c011ae0 tx=0x7f7f3c009e90 comp rx=0 tx=0).stop 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 shutdown_connections 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f28078ef0 0x7f7f2807b400 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f48138cc0 0x7f7f481f8a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f480af0b0 0x7f7f48138780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 --2- 172.21.15.88:0/154408034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f30004d90 0x7f7f48139dd0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 >> 172.21.15.88:0/154408034 conn(0x7f7f4806d1c0 msgr2=0x7f7f481428b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 shutdown_connections 2024-09-21T07:52:05.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.611+0000 7f7f50200640 1 -- 172.21.15.88:0/154408034 wait complete. 2024-09-21T07:52:05.638 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol_clone33240 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 -- 172.21.15.88:0/51004371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145a40 msgr2=0x7fdc94073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 --2- 172.21.15.88:0/51004371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145a40 0x7fdc94073540 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7fdc80005fd0 tx=0x7fdc8002a720 comp rx=0 tx=0).stop 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 -- 172.21.15.88:0/51004371 shutdown_connections 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 --2- 172.21.15.88:0/51004371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145a40 0x7fdc94073540 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 -- 172.21.15.88:0/51004371 >> 172.21.15.88:0/51004371 conn(0x7fdc9406d1c0 msgr2=0x7fdc9406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 -- 172.21.15.88:0/51004371 shutdown_connections 2024-09-21T07:52:05.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.863+0000 7fdc99400640 1 -- 172.21.15.88:0/51004371 wait complete. 2024-09-21T07:52:05.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.866+0000 7fdc99400640 1 Processor -- start 2024-09-21T07:52:05.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.866+0000 7fdc99400640 1 -- start start 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc99400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94073a80 0x7fdc9415cda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc99400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 0x7fdc9415d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc99400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94145a40 0x7fdc9415d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc99400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdc940ce350 con 0x7fdc94145550 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc99400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdc940ce1d0 con 0x7fdc94145a40 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc99400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdc940ce4d0 con 0x7fdc94073a80 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 0x7fdc9415d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc91e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 0x7fdc9415d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45600/0 (socket says 172.21.15.88:45600) 2024-09-21T07:52:05.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.867+0000 7fdc91e00640 1 -- 172.21.15.88:0/1442248660 learned_addr learned my addr 172.21.15.88:0/1442248660 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:05.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc91e00640 1 -- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94073a80 msgr2=0x7fdc9415cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc93200640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94145a40 0x7fdc9415d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc92800640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94073a80 0x7fdc9415cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc91e00640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94073a80 0x7fdc9415cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc91e00640 1 -- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94145a40 msgr2=0x7fdc9415d820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc91e00640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94145a40 0x7fdc9415d820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc91e00640 1 -- 172.21.15.88:0/1442248660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc9413ca80 con 0x7fdc94145550 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc93200640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94145a40 0x7fdc9415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc92800640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94073a80 0x7fdc9415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc91e00640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 0x7fdc9415d2e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fdc80034040 tx=0x7fdc80003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.868+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc8003e030 con 0x7fdc94145550 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.869+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdc800047a0 con 0x7fdc94145550 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.869+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc940b1980 con 0x7fdc94145550 2024-09-21T07:52:05.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.869+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc80035040 con 0x7fdc94145550 2024-09-21T07:52:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.869+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc940b0ff0 con 0x7fdc94145550 2024-09-21T07:52:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.870+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdc940b16f0 con 0x7fdc94145550 2024-09-21T07:52:05.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.871+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fdc80041030 con 0x7fdc94145550 2024-09-21T07:52:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.872+0000 7fdc8aa00640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdc74078ef0 0x7fdc7407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.872+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fdc800bbc10 con 0x7fdc94145550 2024-09-21T07:52:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.872+0000 7fdc92800640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdc74078ef0 0x7fdc7407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:05.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.873+0000 7fdc92800640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdc74078ef0 0x7fdc7407b400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc84007fd0 tx=0x7fdc84009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:05.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:05.876+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdc80083b40 con 0x7fdc94145550 2024-09-21T07:52:06.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.089+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7fdc94073540 con 0x7fdc74078ef0 2024-09-21T07:52:06.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.095+0000 7fdc8aa00640 1 -- 172.21.15.88:0/1442248660 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7fdc94073540 con 0x7fdc74078ef0 2024-09-21T07:52:06.097 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079 2024-09-21T07:52:06.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdc74078ef0 msgr2=0x7fdc7407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdc74078ef0 0x7fdc7407b400 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fdc84007fd0 tx=0x7fdc84009040 comp rx=0 tx=0).stop 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 msgr2=0x7fdc9415d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 0x7fdc9415d2e0 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7fdc80034040 tx=0x7fdc80003970 comp rx=0 tx=0).stop 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 shutdown_connections 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdc74078ef0 0x7fdc7407b400 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdc94145a40 0x7fdc9415d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc94145550 0x7fdc9415d2e0 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 --2- 172.21.15.88:0/1442248660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdc94073a80 0x7fdc9415cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 >> 172.21.15.88:0/1442248660 conn(0x7fdc9406d1c0 msgr2=0x7fdc9413f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 shutdown_connections 2024-09-21T07:52:06.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.097+0000 7fdc99400640 1 -- 172.21.15.88:0/1442248660 wait complete. 2024-09-21T07:52:06.122 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:06.122 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:06.161 INFO:teuthology.orchestra.run.smithi088.stdout:9 2024-09-21T07:52:06.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:06.161 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.rentries volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079) 2024-09-21T07:52:06.238 INFO:teuthology.orchestra.run.smithi088.stdout:9 2024-09-21T07:52:06.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info cephfs subvol_clone33240 2024-09-21T07:52:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f231b600640 1 -- 172.21.15.88:0/2002982958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 msgr2=0x7f22f8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f231b600640 1 --2- 172.21.15.88:0/2002982958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f22f8003e70 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f2304009b50 tx=0x7f22f8005830 comp rx=0 tx=0).stop 2024-09-21T07:52:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f2319600640 1 -- 172.21.15.88:0/2002982958 reap_dead start 2024-09-21T07:52:06.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f231b600640 1 -- 172.21.15.88:0/2002982958 shutdown_connections 2024-09-21T07:52:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f231b600640 1 -- 172.21.15.88:0/2002982958 >> 172.21.15.88:0/2002982958 conn(0x7f231406d1c0 msgr2=0x7f231406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f231b600640 1 -- 172.21.15.88:0/2002982958 shutdown_connections 2024-09-21T07:52:06.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.499+0000 7f231b600640 1 -- 172.21.15.88:0/2002982958 wait complete. 2024-09-21T07:52:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.501+0000 7f231b600640 1 Processor -- start 2024-09-21T07:52:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.501+0000 7f231b600640 1 -- start start 2024-09-21T07:52:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f231b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f231415f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f231b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f231413e0e0 0x7f23140aef70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f231b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f231414cc50 0x7f231414d060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f231b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2314073540 con 0x7f22f8003a00 2024-09-21T07:52:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f231b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f23140ce940 con 0x7f231413e0e0 2024-09-21T07:52:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f231b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f23140736c0 con 0x7f231414cc50 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f231415f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f231415f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45618/0 (socket says 172.21.15.88:45618) 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 -- 172.21.15.88:0/3859283180 learned_addr learned my addr 172.21.15.88:0/3859283180 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2319600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f231414cc50 0x7f231414d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f230be00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f231413e0e0 0x7f23140aef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 -- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f231414cc50 msgr2=0x7f231414d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f231414cc50 0x7f231414d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 -- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f231413e0e0 msgr2=0x7f23140aef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f231413e0e0 0x7f23140aef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f2318c00640 1 -- 172.21.15.88:0/3859283180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f231414d5a0 con 0x7f22f8003a00 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.503+0000 7f230be00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f231413e0e0 0x7f23140aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.504+0000 7f2318c00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f231415f580 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2304009a70 tx=0x7f2304033c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.504+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304043070 con 0x7f22f8003a00 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.504+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2304004030 con 0x7f22f8003a00 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.504+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2304038040 con 0x7f22f8003a00 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.504+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f231413e730 con 0x7f22f8003a00 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.504+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f231415ed80 con 0x7f22f8003a00 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.505+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f231406fb40 con 0x7f22f8003a00 2024-09-21T07:52:06.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.507+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f23040035f0 con 0x7f22f8003a00 2024-09-21T07:52:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.507+0000 7f2311600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22f0078f20 0x7f22f007b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.508+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f23040bbd60 con 0x7f22f8003a00 2024-09-21T07:52:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.508+0000 7f230be00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22f0078f20 0x7f22f007b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.508+0000 7f230be00640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22f0078f20 0x7f22f007b430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23140af9b0 tx=0x7f22fc005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:06.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.512+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2304083680 con 0x7f22f8003a00 2024-09-21T07:52:06.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.725+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "cephfs", "sub_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7f2314141810 con 0x7f22f0078f20 2024-09-21T07:52:06.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.732+0000 7f2311600640 1 -- 172.21.15.88:0/3859283180 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+743 (secure 0 0 0) 0x7f2314141810 con 0x7f22f0078f20 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "atime": "2024-09-21 07:51:41", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_pcent": "80.00", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_quota": 10485760, 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "bytes_used": 8388608, 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "created_at": "2024-09-21 07:51:50", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "ctime": "2024-09-21 07:52:00", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "data_pool": "cephfs_data", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "earmark": "", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "features": [ 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-clone", 2024-09-21T07:52:06.735 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-autoprotect", 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot-retention" 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "flavor": 2, 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "gid": 1000, 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "mode": 16895, 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "mon_addrs": [ 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.88:6789", 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.175:6789", 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "172.21.15.175:6790" 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: ], 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "mtime": "2024-09-21 07:51:46", 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "path": "/volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079", 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "pool_namespace": "", 2024-09-21T07:52:06.736 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "complete", 2024-09-21T07:52:06.737 INFO:teuthology.orchestra.run.smithi088.stdout: "type": "clone", 2024-09-21T07:52:06.737 INFO:teuthology.orchestra.run.smithi088.stdout: "uid": 1000 2024-09-21T07:52:06.737 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:06.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22f0078f20 msgr2=0x7f22f007b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22f0078f20 0x7f22f007b430 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f23140af9b0 tx=0x7f22fc005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 msgr2=0x7f231415f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f231415f580 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f2304009a70 tx=0x7f2304033c30 comp rx=0 tx=0).stop 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 shutdown_connections 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22f0078f20 0x7f22f007b430 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f231414cc50 0x7f231414d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f231413e0e0 0x7f23140aef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 --2- 172.21.15.88:0/3859283180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22f8003a00 0x7f231415f580 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 >> 172.21.15.88:0/3859283180 conn(0x7f231406d1c0 msgr2=0x7f2314068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 shutdown_connections 2024-09-21T07:52:06.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:06.735+0000 7f231b600640 1 -- 172.21.15.88:0/3859283180 wait complete. 2024-09-21T07:52:06.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:06.762 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.quota.max_bytes volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:06.797 INFO:teuthology.orchestra.run.smithi088.stderr:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170: ceph.quota.max_bytes: No such attribute 2024-09-21T07:52:06.798 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:52:06.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:06.798 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:06.875 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:52:06.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:06.875 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool_namespace volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:06.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:06.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec find volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:07.030 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170 2024-09-21T07:52:07.030 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.2 2024-09-21T07:52:07.030 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.5 2024-09-21T07:52:07.031 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.4 2024-09-21T07:52:07.031 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.7 2024-09-21T07:52:07.031 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.6 2024-09-21T07:52:07.031 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.0 2024-09-21T07:52:07.031 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.3 2024-09-21T07:52:07.031 INFO:teuthology.orchestra.run.smithi088.stdout:volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.1 2024-09-21T07:52:07.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.032 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:07.107 INFO:teuthology.orchestra.run.smithi088.stdout:41ff 2024-09-21T07:52:07.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.108 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/) 2024-09-21T07:52:07.182 INFO:teuthology.orchestra.run.smithi088.stdout:41ff 2024-09-21T07:52:07.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.184 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:07.259 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:07.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.260 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/) 2024-09-21T07:52:07.335 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:07.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.336 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:07.410 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:07.411 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.411 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/) 2024-09-21T07:52:07.487 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:07.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.488 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170) 2024-09-21T07:52:07.562 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:07.563 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.563 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/) 2024-09-21T07:52:07.638 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:07.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.639 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.2) 2024-09-21T07:52:07.716 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:07.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.717 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.2) 2024-09-21T07:52:07.792 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:07.793 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.793 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.2) 2024-09-21T07:52:07.869 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:07.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.870 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.2) 2024-09-21T07:52:07.945 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:07.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:07.946 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.2) 2024-09-21T07:52:08.022 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:08.023 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.023 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.2) 2024-09-21T07:52:08.097 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:08.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.098 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.2) 2024-09-21T07:52:08.178 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:08.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.180 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.2) 2024-09-21T07:52:08.254 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:08.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.256 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.5) 2024-09-21T07:52:08.332 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:08.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.333 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.5) 2024-09-21T07:52:08.408 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:08.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.409 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.5) 2024-09-21T07:52:08.483 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:08.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.484 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.5) 2024-09-21T07:52:08.559 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:08.560 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.560 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.5) 2024-09-21T07:52:08.634 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:08.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.635 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.5) 2024-09-21T07:52:08.710 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:08.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.711 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.5) 2024-09-21T07:52:08.789 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:08.791 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.791 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.5) 2024-09-21T07:52:08.866 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:08.867 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.867 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.4) 2024-09-21T07:52:08.943 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:08.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:08.944 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.4) 2024-09-21T07:52:09.020 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:09.021 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.021 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.4) 2024-09-21T07:52:09.097 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:09.098 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.098 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.4) 2024-09-21T07:52:09.173 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:09.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.174 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.4) 2024-09-21T07:52:09.249 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:09.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.251 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.4) 2024-09-21T07:52:09.325 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:09.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.326 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.4) 2024-09-21T07:52:09.405 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:09.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.406 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.4) 2024-09-21T07:52:09.480 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:09.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.481 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.7) 2024-09-21T07:52:09.558 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:09.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.559 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.7) 2024-09-21T07:52:09.634 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:09.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.635 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.7) 2024-09-21T07:52:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:09.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.710 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.7) 2024-09-21T07:52:09.784 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:09.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.785 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.7) 2024-09-21T07:52:09.861 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:09.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.862 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.7) 2024-09-21T07:52:09.936 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:09.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:09.937 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.7) 2024-09-21T07:52:10.016 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:10.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.017 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.7) 2024-09-21T07:52:10.091 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:10.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.092 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.6) 2024-09-21T07:52:10.170 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:10.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.171 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.6) 2024-09-21T07:52:10.246 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:10.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.295 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.6) 2024-09-21T07:52:10.330 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:10.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.331 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.6) 2024-09-21T07:52:10.412 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:10.413 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.413 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.6) 2024-09-21T07:52:10.493 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:10.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.494 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.6) 2024-09-21T07:52:10.570 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:10.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.571 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.6) 2024-09-21T07:52:10.652 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:10.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.653 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.6) 2024-09-21T07:52:10.729 INFO:teuthology.orchestra.run.smithi088.stdout:1726905106 2024-09-21T07:52:10.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.731 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.0) 2024-09-21T07:52:10.808 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:10.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.809 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.0) 2024-09-21T07:52:10.886 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:10.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.887 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.0) 2024-09-21T07:52:10.962 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:10.963 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:10.963 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.0) 2024-09-21T07:52:11.037 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:11.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.039 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.0) 2024-09-21T07:52:11.113 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:11.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.114 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.0) 2024-09-21T07:52:11.189 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:11.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.190 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.0) 2024-09-21T07:52:11.265 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:11.266 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.266 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.0) 2024-09-21T07:52:11.341 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:11.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.342 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.3) 2024-09-21T07:52:11.418 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:11.419 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.419 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.3) 2024-09-21T07:52:11.495 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:11.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.496 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.3) 2024-09-21T07:52:11.571 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:11.572 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.572 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.3) 2024-09-21T07:52:11.647 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:11.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.648 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.3) 2024-09-21T07:52:11.723 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:11.724 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.724 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.3) 2024-09-21T07:52:11.799 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:11.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.800 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.3) 2024-09-21T07:52:11.874 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:11.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.875 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.3) 2024-09-21T07:52:11.950 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:11.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:11.951 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.1) 2024-09-21T07:52:12.027 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:12.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.028 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%f volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.1) 2024-09-21T07:52:12.103 INFO:teuthology.orchestra.run.smithi088.stdout:81a4 2024-09-21T07:52:12.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.104 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.1) 2024-09-21T07:52:12.178 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:12.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.179 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%u volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.1) 2024-09-21T07:52:12.254 INFO:teuthology.orchestra.run.smithi088.stdout:1000 2024-09-21T07:52:12.255 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.1) 2024-09-21T07:52:12.330 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:12.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.331 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%g volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.1) 2024-09-21T07:52:12.408 INFO:teuthology.orchestra.run.smithi088.stdout:1333 2024-09-21T07:52:12.409 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.409 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e/.snap/subvol_snap57170/subvolume_file.1) 2024-09-21T07:52:12.483 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:12.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:12.484 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stat -c%Y volumes/_nogroup/subvol_clone33240/fcf393b3-1c9e-42a7-bda9-27e62b4a4079/subvolume_file.1) 2024-09-21T07:52:12.559 INFO:teuthology.orchestra.run.smithi088.stdout:1726905105 2024-09-21T07:52:12.560 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone cephfs subvol88300 fake subvol_clone85521 2024-09-21T07:52:12.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 -- 172.21.15.88:0/4018003490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 msgr2=0x7fa5940d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:12.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 --2- 172.21.15.88:0/4018003490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5940d1090 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa59406ee50 tx=0x7fa58c02aa10 comp rx=0 tx=0).stop 2024-09-21T07:52:12.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 -- 172.21.15.88:0/4018003490 shutdown_connections 2024-09-21T07:52:12.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 --2- 172.21.15.88:0/4018003490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5940d1090 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:12.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 --2- 172.21.15.88:0/4018003490 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa59406e890 0x7fa5940cff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:12.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 --2- 172.21.15.88:0/4018003490 >> v2:172.21.15.175:3300/0 conn(0x7fa5940b95e0 0x7fa5940b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:12.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 -- 172.21.15.88:0/4018003490 >> 172.21.15.88:0/4018003490 conn(0x7fa59409d860 msgr2=0x7fa59408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:12.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 -- 172.21.15.88:0/4018003490 shutdown_connections 2024-09-21T07:52:12.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.825+0000 7fa593e00640 1 -- 172.21.15.88:0/4018003490 wait complete. 2024-09-21T07:52:12.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.827+0000 7fa593e00640 1 Processor -- start 2024-09-21T07:52:12.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.827+0000 7fa593e00640 1 -- start start 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.828+0000 7fa593e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa59406e890 0x7fa59409e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.828+0000 7fa593e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5940b95e0 0x7fa59415fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.828+0000 7fa593e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5941f58c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.828+0000 7fa593e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5940d2e30 con 0x7fa5940d0c80 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.828+0000 7fa593e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5940d2cb0 con 0x7fa5940b95e0 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.828+0000 7fa593e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa5940d2fb0 con 0x7fa59406e890 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5941f58c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5941f58c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45642/0 (socket says 172.21.15.88:45642) 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 -- 172.21.15.88:0/806169663 learned_addr learned my addr 172.21.15.88:0/806169663 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:12.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa592a00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa59406e890 0x7fa59409e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa592000640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5940b95e0 0x7fa59415fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 -- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa59406e890 msgr2=0x7fa59409e670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa59406e890 0x7fa59409e670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 -- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5940b95e0 msgr2=0x7fa59415fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5940b95e0 0x7fa59415fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 -- 172.21.15.88:0/806169663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa59409ee30 con 0x7fa5940d0c80 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa592000640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5940b95e0 0x7fa59415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.829+0000 7fa593400640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5941f58c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.830+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c033050 con 0x7fa5940d0c80 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.830+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa58c004550 con 0x7fa5940d0c80 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.830+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5940a1280 con 0x7fa5940d0c80 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.830+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa58c038450 con 0x7fa5940d0c80 2024-09-21T07:52:12.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.830+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5940a1a80 con 0x7fa5940d0c80 2024-09-21T07:52:12.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.831+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa59406f210 con 0x7fa5940d0c80 2024-09-21T07:52:12.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.832+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fa58c036070 con 0x7fa5940d0c80 2024-09-21T07:52:12.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.833+0000 7fa58aa00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa574078ef0 0x7fa57407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:12.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.833+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fa58c0bc8d0 con 0x7fa5940d0c80 2024-09-21T07:52:12.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.833+0000 7fa592a00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa574078ef0 0x7fa57407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:12.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.834+0000 7fa592a00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa574078ef0 0x7fa57407b400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa58400ab20 tx=0x7fa58400a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:12.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:12.838+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa59406f210 con 0x7fa5940d0c80 2024-09-21T07:52:13.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.055+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "cephfs", "sub_name": "subvol88300", "snap_name": "fake", "target_sub_name": "subvol_clone85521", "target": ["mon-mgr", ""]}) -- 0x7fa5941f6750 con 0x7fa574078ef0 2024-09-21T07:52:13.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.134+0000 7fa58aa00640 1 -- 172.21.15.88:0/806169663 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa5941f6750 con 0x7fa574078ef0 2024-09-21T07:52:13.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa574078ef0 msgr2=0x7fa57407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa574078ef0 0x7fa57407b400 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa58400ab20 tx=0x7fa58400a250 comp rx=0 tx=0).stop 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 msgr2=0x7fa5941f58c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5941f58c0 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7fa58c005fd0 tx=0x7fa58c003f10 comp rx=0 tx=0).stop 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 shutdown_connections 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa574078ef0 0x7fa57407b400 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa5940d0c80 0x7fa5941f58c0 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5940b95e0 0x7fa59415fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 --2- 172.21.15.88:0/806169663 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa59406e890 0x7fa59409e670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 >> 172.21.15.88:0/806169663 conn(0x7fa59409d860 msgr2=0x7fa5940b7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 shutdown_connections 2024-09-21T07:52:13.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.137+0000 7fa593e00640 1 -- 172.21.15.88:0/806169663 wait complete. 2024-09-21T07:52:13.165 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone85521 2024-09-21T07:52:13.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f53a2000640 1 -- 172.21.15.88:0/2901730990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5380004d40 msgr2=0x7f5380005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f53a2000640 1 --2- 172.21.15.88:0/2901730990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5380004d40 0x7f5380005150 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f5394009b20 tx=0x7f5380005830 comp rx=0 tx=0).stop 2024-09-21T07:52:13.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f539be00640 1 -- 172.21.15.88:0/2901730990 reap_dead start 2024-09-21T07:52:13.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f53a2000640 1 -- 172.21.15.88:0/2901730990 shutdown_connections 2024-09-21T07:52:13.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f53a2000640 1 -- 172.21.15.88:0/2901730990 >> 172.21.15.88:0/2901730990 conn(0x7f539c06d1c0 msgr2=0x7f539c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:13.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f53a2000640 1 -- 172.21.15.88:0/2901730990 shutdown_connections 2024-09-21T07:52:13.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.388+0000 7f53a2000640 1 -- 172.21.15.88:0/2901730990 wait complete. 2024-09-21T07:52:13.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.390+0000 7f53a2000640 1 Processor -- start 2024-09-21T07:52:13.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.390+0000 7f53a2000640 1 -- start start 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f53a2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5380004d40 0x7f539c0b4e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f53a2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 0x7f539c1f8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f53a2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f539c1f8ad0 0x7f539c0b3580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f53a2000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0cdcd0 con 0x7f539c13db10 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f53a2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f539c0cdb50 con 0x7f5380004d40 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f53a2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f539c0cde50 con 0x7f539c1f8ad0 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 0x7f539c1f8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 0x7f539c1f8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:45658/0 (socket says 172.21.15.88:45658) 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 -- 172.21.15.88:0/2289667517 learned_addr learned my addr 172.21.15.88:0/2289667517 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f539b400640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5380004d40 0x7f539c0b4e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f539be00640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f539c1f8ad0 0x7f539c0b3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 -- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f539c1f8ad0 msgr2=0x7f539c0b3580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f539c1f8ad0 0x7f539c0b3580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 -- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5380004d40 msgr2=0x7f539c0b4e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:13.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5380004d40 0x7f539c0b4e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:13.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.392+0000 7f5392a00640 1 -- 172.21.15.88:0/2289667517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f539c15f890 con 0x7f539c13db10 2024-09-21T07:52:13.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.393+0000 7f5392a00640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 0x7f539c1f8590 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5384006630 tx=0x7f53840039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.393+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384015070 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.393+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53840119c0 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.393+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5384011e10 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.393+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f539c15fb70 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.393+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f539c15f410 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.394+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f539c15fdb0 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.395+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f5384018070 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.396+0000 7f5393e00640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5378078f20 0x7f537807b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.396+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f538409c210 con 0x7f539c13db10 2024-09-21T07:52:13.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.397+0000 7f539b400640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5378078f20 0x7f537807b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:13.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.397+0000 7f539b400640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5378078f20 0x7f537807b430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f538c005fd0 tx=0x7f538c009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:13.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.401+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f539c15fdb0 con 0x7f539c13db10 2024-09-21T07:52:13.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:13.621+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone85521", "target": ["mon-mgr", ""]}) -- 0x7f539c0cd610 con 0x7f5378078f20 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.059+0000 7f5393e00640 1 -- 172.21.15.88:0/2289667517 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+153 (secure 0 0 0) 0x7f539c0cd610 con 0x7f5378078f20 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "pending", 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "cephfs", 2024-09-21T07:52:15.061 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "subvol88300", 2024-09-21T07:52:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "fake" 2024-09-21T07:52:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:15.062 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:15.062 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5378078f20 msgr2=0x7f537807b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5378078f20 0x7f537807b430 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f538c005fd0 tx=0x7f538c009500 comp rx=0 tx=0).stop 2024-09-21T07:52:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 msgr2=0x7f539c1f8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:15.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 0x7f539c1f8590 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f5384006630 tx=0x7f53840039c0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 shutdown_connections 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5378078f20 0x7f537807b430 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f539c1f8ad0 0x7f539c0b3580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f539c13db10 0x7f539c1f8590 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 --2- 172.21.15.88:0/2289667517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5380004d40 0x7f539c0b4e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 >> 172.21.15.88:0/2289667517 conn(0x7f539c06d1c0 msgr2=0x7f539c0b2a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 shutdown_connections 2024-09-21T07:52:15.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:15.062+0000 7f53a2000640 1 -- 172.21.15.88:0/2289667517 wait complete. 2024-09-21T07:52:15.098 DEBUG:tasks.cephfs.test_volumes:current clone state = pending 2024-09-21T07:52:16.099 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol_clone85521 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 -- 172.21.15.88:0/3130825443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 msgr2=0x7f0968073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 --2- 172.21.15.88:0/3130825443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f0968073540 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f0954005fd0 tx=0x7f095402a670 comp rx=0 tx=0).stop 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 -- 172.21.15.88:0/3130825443 shutdown_connections 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 --2- 172.21.15.88:0/3130825443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f0968073540 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 --2- 172.21.15.88:0/3130825443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968145550 0x7f096806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 -- 172.21.15.88:0/3130825443 >> 172.21.15.88:0/3130825443 conn(0x7f096806d1c0 msgr2=0x7f096806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 -- 172.21.15.88:0/3130825443 shutdown_connections 2024-09-21T07:52:16.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.331+0000 7f0970400640 1 -- 172.21.15.88:0/3130825443 wait complete. 2024-09-21T07:52:16.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.334+0000 7f0970400640 1 Processor -- start 2024-09-21T07:52:16.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.334+0000 7f0970400640 1 -- start start 2024-09-21T07:52:16.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f0970400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968073a80 0x7f096815e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:16.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f0970400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0968145550 0x7f096815ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:16.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f0970400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f096812fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f0970400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0968160a30 con 0x7f0968145a40 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f0970400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09680ce940 con 0x7f0968145550 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f0970400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0968160bb0 con 0x7f0968073a80 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f096e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f096812fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f096e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f096812fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51930/0 (socket says 172.21.15.88:51930) 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.335+0000 7f096e400640 1 -- 172.21.15.88:0/559081921 learned_addr learned my addr 172.21.15.88:0/559081921 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096da00640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968073a80 0x7f096815e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096d000640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0968145550 0x7f096815ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096e400640 1 -- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968073a80 msgr2=0x7f096815e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096e400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968073a80 0x7f096815e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096e400640 1 -- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0968145550 msgr2=0x7f096815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096e400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0968145550 0x7f096815ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:16.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096e400640 1 -- 172.21.15.88:0/559081921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09680add20 con 0x7f0968145a40 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096da00640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968073a80 0x7f096815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f096e400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f096812fc60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0960002230 tx=0x7f096000cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960003b70 con 0x7f0968145a40 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0960003d10 con 0x7f0968145a40 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0960005330 con 0x7f0968145a40 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.336+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09680a9980 con 0x7f0968145a40 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.337+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0968130850 con 0x7f0968145a40 2024-09-21T07:52:16.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.337+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09681610d0 con 0x7f0968145a40 2024-09-21T07:52:16.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.339+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f09600056e0 con 0x7f0968145a40 2024-09-21T07:52:16.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.340+0000 7f0965600640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0948078ef0 0x7f094807b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:16.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.340+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f096009b3a0 con 0x7f0968145a40 2024-09-21T07:52:16.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.340+0000 7f096da00640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0948078ef0 0x7f094807b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:16.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.341+0000 7f096da00640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0948078ef0 0x7f094807b400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c00f360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:16.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.344+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0968073e90 con 0x7f0968145a40 2024-09-21T07:52:16.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:16.565+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol_clone85521", "target": ["mon-mgr", ""]}) -- 0x7f0968130af0 con 0x7f0948078ef0 2024-09-21T07:52:20.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.047+0000 7f0965600640 1 -- 172.21.15.88:0/559081921 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+46 (secure 0 0 0) 0x7f0968130af0 con 0x7f0948078ef0 2024-09-21T07:52:20.050 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:52:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "complete" 2024-09-21T07:52:20.050 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:20.050 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0948078ef0 msgr2=0x7f094807b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0948078ef0 0x7f094807b400 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f095c005fd0 tx=0x7f095c00f360 comp rx=0 tx=0).stop 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 msgr2=0x7f096812fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f096812fc60 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f0960002230 tx=0x7f096000cb90 comp rx=0 tx=0).stop 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 shutdown_connections 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0948078ef0 0x7f094807b400 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0968145a40 0x7f096812fc60 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0968145550 0x7f096815ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 --2- 172.21.15.88:0/559081921 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0968073a80 0x7f096815e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 >> 172.21.15.88:0/559081921 conn(0x7f096806d1c0 msgr2=0x7f0968140130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 shutdown_connections 2024-09-21T07:52:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.050+0000 7f0970400640 1 -- 172.21.15.88:0/559081921 wait complete. 2024-09-21T07:52:20.079 DEBUG:tasks.cephfs.test_volumes:current clone state = complete 2024-09-21T07:52:20.080 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:20.080 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol_clone85521/.meta) 2024-09-21T07:52:20.138 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:52:20.138 INFO:teuthology.orchestra.run.smithi088.stdout:version = 2 2024-09-21T07:52:20.138 INFO:teuthology.orchestra.run.smithi088.stdout:type = clone 2024-09-21T07:52:20.138 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol_clone85521/1c3566c0-0c90-4c00-8aa7-e106ff77f7a5 2024-09-21T07:52:20.138 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:52:20.138 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:20.141 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info cephfs subvol88300 subvol_snap57170 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5c400640 1 -- 172.21.15.88:0/1693485542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c004800 msgr2=0x7f0a3c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5c400640 1 --2- 172.21.15.88:0/1693485542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a3c004800 0x7f0a3c004c10 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f0a40009b50 tx=0x7f0a3c005830 comp rx=0 tx=0).stop 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5a400640 1 -- 172.21.15.88:0/1693485542 reap_dead start 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5c400640 1 -- 172.21.15.88:0/1693485542 shutdown_connections 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5c400640 1 -- 172.21.15.88:0/1693485542 >> 172.21.15.88:0/1693485542 conn(0x7f0a540b2540 msgr2=0x7f0a540d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5c400640 1 -- 172.21.15.88:0/1693485542 shutdown_connections 2024-09-21T07:52:20.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.365+0000 7f0a5c400640 1 -- 172.21.15.88:0/1693485542 wait complete. 2024-09-21T07:52:20.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.368+0000 7f0a5c400640 1 Processor -- start 2024-09-21T07:52:20.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.368+0000 7f0a5c400640 1 -- start start 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a3c004800 0x7f0a5409a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a54156570 0x7f0a5409dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 0x7f0a541f9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b3120 con 0x7f0a541532f0 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0a540b48f0 con 0x7f0a3c004800 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0a540b32a0 con 0x7f0a54156570 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 0x7f0a541f9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 0x7f0a541f9f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51958/0 (socket says 172.21.15.88:51958) 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.369+0000 7f0a5a400640 1 -- 172.21.15.88:0/2280431233 learned_addr learned my addr 172.21.15.88:0/2280431233 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a59000640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a54156570 0x7f0a5409dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a59a00640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a3c004800 0x7f0a5409a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5a400640 1 -- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a54156570 msgr2=0x7f0a5409dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5a400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a54156570 0x7f0a5409dca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5a400640 1 -- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a3c004800 msgr2=0x7f0a5409a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5a400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a3c004800 0x7f0a5409a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5a400640 1 -- 172.21.15.88:0/2280431233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5409a810 con 0x7f0a541532f0 2024-09-21T07:52:20.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5a400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 0x7f0a541f9f80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0a540704e0 tx=0x7f0a4c00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:20.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c00c9a0 con 0x7f0a541532f0 2024-09-21T07:52:20.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a4c018070 con 0x7f0a541532f0 2024-09-21T07:52:20.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5409e1e0 con 0x7f0a541532f0 2024-09-21T07:52:20.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.370+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4c011460 con 0x7f0a541532f0 2024-09-21T07:52:20.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.371+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a54156b10 con 0x7f0a541532f0 2024-09-21T07:52:20.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.372+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a5406d1b0 con 0x7f0a541532f0 2024-09-21T07:52:20.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.373+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f0a4c011600 con 0x7f0a541532f0 2024-09-21T07:52:20.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.374+0000 7f0a51600640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a34078ef0 0x7f0a3407b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.374+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f0a4c09b8c0 con 0x7f0a541532f0 2024-09-21T07:52:20.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.374+0000 7f0a59a00640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a34078ef0 0x7f0a3407b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.375+0000 7f0a59a00640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a34078ef0 0x7f0a3407b400 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005eb0 tx=0x7f0a48005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:20.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.379+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a4c063760 con 0x7f0a541532f0 2024-09-21T07:52:20.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.598+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "cephfs", "sub_name": "subvol88300", "snap_name": "subvol_snap57170", "target": ["mon-mgr", ""]}) -- 0x7f0a54070e10 con 0x7f0a34078ef0 2024-09-21T07:52:20.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.613+0000 7f0a51600640 1 -- 172.21.15.88:0/2280431233 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+115 (secure 0 0 0) 0x7f0a54070e10 con 0x7f0a34078ef0 2024-09-21T07:52:20.616 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:20.616 INFO:teuthology.orchestra.run.smithi088.stdout: "created_at": "2024-09-21 07:51:50.052680", 2024-09-21T07:52:20.616 INFO:teuthology.orchestra.run.smithi088.stdout: "data_pool": "cephfs_data", 2024-09-21T07:52:20.616 INFO:teuthology.orchestra.run.smithi088.stdout: "has_pending_clones": "no" 2024-09-21T07:52:20.616 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:20.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a34078ef0 msgr2=0x7f0a3407b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a34078ef0 0x7f0a3407b400 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f0a48005eb0 tx=0x7f0a48005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:20.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 msgr2=0x7f0a541f9f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 0x7f0a541f9f80 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f0a540704e0 tx=0x7f0a4c00ed10 comp rx=0 tx=0).stop 2024-09-21T07:52:20.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 shutdown_connections 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a34078ef0 0x7f0a3407b400 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0a54156570 0x7f0a5409dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a541532f0 0x7f0a541f9f80 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 --2- 172.21.15.88:0/2280431233 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0a3c004800 0x7f0a5409a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 >> 172.21.15.88:0/2280431233 conn(0x7f0a540b2540 msgr2=0x7f0a54069230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 shutdown_connections 2024-09-21T07:52:20.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.616+0000 7f0a5c400640 1 -- 172.21.15.88:0/2280431233 wait complete. 2024-09-21T07:52:20.642 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls cephfs subvol88300 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 -- 172.21.15.88:0/4172852204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2980705b0 msgr2=0x7fd2980af3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 --2- 172.21.15.88:0/4172852204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2980705b0 0x7fd2980af3c0 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7fd29806ddf0 tx=0x7fd28002a560 comp rx=0 tx=0).stop 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 -- 172.21.15.88:0/4172852204 shutdown_connections 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 --2- 172.21.15.88:0/4172852204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2980705b0 0x7fd2980af3c0 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 --2- 172.21.15.88:0/4172852204 >> v2:172.21.15.175:3300/0 conn(0x7fd2980b3ec0 0x7fd2980b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 -- 172.21.15.88:0/4172852204 >> 172.21.15.88:0/4172852204 conn(0x7fd2980b1dd0 msgr2=0x7fd2980b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 -- 172.21.15.88:0/4172852204 shutdown_connections 2024-09-21T07:52:20.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.867+0000 7fd29ca00640 1 -- 172.21.15.88:0/4172852204 wait complete. 2024-09-21T07:52:20.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.870+0000 7fd29ca00640 1 Processor -- start 2024-09-21T07:52:20.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.870+0000 7fd29ca00640 1 -- start start 2024-09-21T07:52:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd29ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2980705b0 0x7fd298156330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd29ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2980b3ec0 0x7fd298142b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd29ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 0x7fd2981431e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd29ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd298153a80 con 0x7fd2981532e0 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd29ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd298153900 con 0x7fd2980b3ec0 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd29ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd298153c00 con 0x7fd2980705b0 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd297200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 0x7fd2981431e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd297200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 0x7fd2981431e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51980/0 (socket says 172.21.15.88:51980) 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.871+0000 7fd297200640 1 -- 172.21.15.88:0/114360448 learned_addr learned my addr 172.21.15.88:0/114360448 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd28de00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2980b3ec0 0x7fd298142b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd296800640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2980705b0 0x7fd298156330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd297200640 1 -- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2980705b0 msgr2=0x7fd298156330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd297200640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2980705b0 0x7fd298156330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd297200640 1 -- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2980b3ec0 msgr2=0x7fd298142b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd297200640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2980b3ec0 0x7fd298142b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd297200640 1 -- 172.21.15.88:0/114360448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd298155290 con 0x7fd2981532e0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd297200640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 0x7fd2981431e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c730 tx=0x7fd29000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290003b70 con 0x7fd2981532e0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd290003d10 con 0x7fd2981532e0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd298155570 con 0x7fd2981532e0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.872+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290005330 con 0x7fd2981532e0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.873+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd298155b30 con 0x7fd2981532e0 2024-09-21T07:52:20.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.873+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd298155000 con 0x7fd2981532e0 2024-09-21T07:52:20.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.875+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fd2900054d0 con 0x7fd2981532e0 2024-09-21T07:52:20.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.876+0000 7fd28f400640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd274078f20 0x7fd27407b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:20.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.876+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fd29009b2b0 con 0x7fd2981532e0 2024-09-21T07:52:20.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.876+0000 7fd296800640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd274078f20 0x7fd27407b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:20.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.877+0000 7fd296800640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd274078f20 0x7fd27407b430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd28800bb00 tx=0x7fd288005e30 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:20.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:20.880+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2980b4ec0 con 0x7fd2981532e0 2024-09-21T07:52:21.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.097+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7fd2980af3c0 con 0x7fd274078f20 2024-09-21T07:52:21.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.110+0000 7fd28f400640 1 -- 172.21.15.88:0/114360448 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+87 (secure 0 0 0) 0x7fd2980af3c0 con 0x7fd274078f20 2024-09-21T07:52:21.112 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:52:21.112 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:21.113 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "fake" 2024-09-21T07:52:21.113 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:21.113 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:21.113 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "subvol_snap57170" 2024-09-21T07:52:21.113 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:21.113 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:52:21.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd274078f20 msgr2=0x7fd27407b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd274078f20 0x7fd27407b430 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd28800bb00 tx=0x7fd288005e30 comp rx=0 tx=0).stop 2024-09-21T07:52:21.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 msgr2=0x7fd2981431e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 0x7fd2981431e0 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7fd29000c730 tx=0x7fd29000cb30 comp rx=0 tx=0).stop 2024-09-21T07:52:21.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 shutdown_connections 2024-09-21T07:52:21.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd274078f20 0x7fd27407b430 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2981532e0 0x7fd2981431e0 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2980b3ec0 0x7fd298142b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 --2- 172.21.15.88:0/114360448 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2980705b0 0x7fd298156330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 >> 172.21.15.88:0/114360448 conn(0x7fd2980b1dd0 msgr2=0x7fd29806ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:21.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 shutdown_connections 2024-09-21T07:52:21.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.113+0000 7fd29ca00640 1 -- 172.21.15.88:0/114360448 wait complete. 2024-09-21T07:52:21.140 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol88300 subvol_snap57170 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 -- 172.21.15.88:0/1360502003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd608004800 msgr2=0x7fd608004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 --2- 172.21.15.88:0/1360502003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd608004800 0x7fd608004c10 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7fd60c00c2e0 tx=0x7fd608005830 comp rx=0 tx=0).stop 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 -- 172.21.15.88:0/1360502003 shutdown_connections 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 --2- 172.21.15.88:0/1360502003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd608004800 0x7fd608004c10 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 -- 172.21.15.88:0/1360502003 >> 172.21.15.88:0/1360502003 conn(0x7fd6200b2540 msgr2=0x7fd6200d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 -- 172.21.15.88:0/1360502003 shutdown_connections 2024-09-21T07:52:21.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.364+0000 7fd627600640 1 -- 172.21.15.88:0/1360502003 wait complete. 2024-09-21T07:52:21.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.366+0000 7fd627600640 1 Processor -- start 2024-09-21T07:52:21.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.367+0000 7fd627600640 1 -- start start 2024-09-21T07:52:21.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd627600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608004800 0x7fd6201f6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd627600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620157be0 0x7fd6201f7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd627600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 0x7fd6201f7580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd627600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200af280 con 0x7fd620157ff0 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd627600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6200af100 con 0x7fd608004800 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd627600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6200af400 con 0x7fd620157be0 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd625600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 0x7fd6201f7580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd625600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 0x7fd6201f7580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51996/0 (socket says 172.21.15.88:51996) 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.368+0000 7fd625600640 1 -- 172.21.15.88:0/1648730236 learned_addr learned my addr 172.21.15.88:0/1648730236 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd624c00640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608004800 0x7fd6201f6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd61fe00640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620157be0 0x7fd6201f7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd625600640 1 -- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620157be0 msgr2=0x7fd6201f7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd625600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620157be0 0x7fd6201f7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd625600640 1 -- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608004800 msgr2=0x7fd6201f6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd625600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608004800 0x7fd6201f6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd625600640 1 -- 172.21.15.88:0/1648730236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd62009b130 con 0x7fd620157ff0 2024-09-21T07:52:21.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd624c00640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608004800 0x7fd6201f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd625600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 0x7fd6201f7580 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd6200cb230 tx=0x7fd618010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618003970 con 0x7fd620157ff0 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd618015070 con 0x7fd620157ff0 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd62009b410 con 0x7fd620157ff0 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.369+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd618004030 con 0x7fd620157ff0 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.370+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd62009b9d0 con 0x7fd620157ff0 2024-09-21T07:52:21.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.371+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd62006cf10 con 0x7fd620157ff0 2024-09-21T07:52:21.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.372+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fd6180041d0 con 0x7fd620157ff0 2024-09-21T07:52:21.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.373+0000 7fd61cc00640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd600078ef0 0x7fd60007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.373+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fd61809c100 con 0x7fd620157ff0 2024-09-21T07:52:21.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.373+0000 7fd624c00640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd600078ef0 0x7fd60007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.374+0000 7fd624c00640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd600078ef0 0x7fd60007b400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6200521d0 tx=0x7fd614006ba0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:21.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.377+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd618063f30 con 0x7fd620157ff0 2024-09-21T07:52:21.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.593+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol88300", "snap_name": "subvol_snap57170", "target": ["mon-mgr", ""]}) -- 0x7fd6200d1030 con 0x7fd600078ef0 2024-09-21T07:52:21.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.624+0000 7fd61cc00640 1 -- 172.21.15.88:0/1648730236 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fd6200d1030 con 0x7fd600078ef0 2024-09-21T07:52:21.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd600078ef0 msgr2=0x7fd60007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd600078ef0 0x7fd60007b400 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd6200521d0 tx=0x7fd614006ba0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 msgr2=0x7fd6201f7580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 0x7fd6201f7580 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7fd6200cb230 tx=0x7fd618010f40 comp rx=0 tx=0).stop 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 shutdown_connections 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd600078ef0 0x7fd60007b400 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd620157be0 0x7fd6201f7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd620157ff0 0x7fd6201f7580 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 --2- 172.21.15.88:0/1648730236 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd608004800 0x7fd6201f6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 >> 172.21.15.88:0/1648730236 conn(0x7fd6200b2540 msgr2=0x7fd6200c9be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 shutdown_connections 2024-09-21T07:52:21.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.626+0000 7fd627600640 1 -- 172.21.15.88:0/1648730236 wait complete. 2024-09-21T07:52:21.650 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol88300 fake 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 -- 172.21.15.88:0/190910925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26001532e0 msgr2=0x7f26001536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 --2- 172.21.15.88:0/190910925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26001532e0 0x7f26001536f0 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7f260006a110 tx=0x7f25f802a980 comp rx=0 tx=0).stop 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 -- 172.21.15.88:0/190910925 shutdown_connections 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 --2- 172.21.15.88:0/190910925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26001532e0 0x7f26001536f0 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 --2- 172.21.15.88:0/190910925 >> v2:172.21.15.175:3300/0 conn(0x7f26000705b0 0x7f26000af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 --2- 172.21.15.88:0/190910925 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 0x7f26000b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 -- 172.21.15.88:0/190910925 >> 172.21.15.88:0/190910925 conn(0x7f26000b1dd0 msgr2=0x7f26000b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 -- 172.21.15.88:0/190910925 shutdown_connections 2024-09-21T07:52:21.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.880+0000 7f2605a00640 1 -- 172.21.15.88:0/190910925 wait complete. 2024-09-21T07:52:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.882+0000 7f2605a00640 1 Processor -- start 2024-09-21T07:52:21.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.883+0000 7f2605a00640 1 -- start start 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f2605a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 0x7f260015b040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f2605a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 0x7f260009c570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f2605a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26001532e0 0x7f2600147f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f2605a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2600160860 con 0x7f26000705b0 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f2605a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f26001606e0 con 0x7f26001532e0 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f2605a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f26001609e0 con 0x7f26000b3ec0 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f25ffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 0x7f260015b040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f25ffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 0x7f260015b040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52018/0 (socket says 172.21.15.88:52018) 2024-09-21T07:52:21.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.884+0000 7f25ffe00640 1 -- 172.21.15.88:0/991160116 learned_addr learned my addr 172.21.15.88:0/991160116 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ff400640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 0x7f260009c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ffe00640 1 -- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 msgr2=0x7f260009c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ffe00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 0x7f260009c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ffe00640 1 -- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26001532e0 msgr2=0x7f2600147f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ffe00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26001532e0 0x7f2600147f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ffe00640 1 -- 172.21.15.88:0/991160116 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f260009cab0 con 0x7f26000705b0 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ff400640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 0x7f260009c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:21.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25ffe00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 0x7f260015b040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f25f000e7c0 tx=0x7f25f000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f000ccc0 con 0x7f26000705b0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f25f000ce80 con 0x7f26000705b0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25f0005280 con 0x7f26000705b0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.885+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2600158e80 con 0x7f26000705b0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.886+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2600159120 con 0x7f26000705b0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.887+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26000afe00 con 0x7f26000705b0 2024-09-21T07:52:21.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.888+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f25f0005420 con 0x7f26000705b0 2024-09-21T07:52:21.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.889+0000 7f25f7e00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25e0078ef0 0x7f25e007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:21.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.889+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f25f009bf30 con 0x7f26000705b0 2024-09-21T07:52:21.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.889+0000 7f25ff400640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25e0078ef0 0x7f25e007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:21.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.890+0000 7f25ff400640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25e0078ef0 0x7f25e007b400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f260015a7d0 tx=0x7f25ec005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:21.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:21.893+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f25f0063cd0 con 0x7f26000705b0 2024-09-21T07:52:22.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.111+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol88300", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f260006de20 con 0x7f25e0078ef0 2024-09-21T07:52:22.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.142+0000 7f25f7e00640 1 -- 172.21.15.88:0/991160116 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f260006de20 con 0x7f25e0078ef0 2024-09-21T07:52:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25e0078ef0 msgr2=0x7f25e007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25e0078ef0 0x7f25e007b400 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f260015a7d0 tx=0x7f25ec005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 msgr2=0x7f260015b040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 0x7f260015b040 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7f25f000e7c0 tx=0x7f25f000ec80 comp rx=0 tx=0).stop 2024-09-21T07:52:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 shutdown_connections 2024-09-21T07:52:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25e0078ef0 0x7f25e007b400 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f26000b3ec0 0x7f260009c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f26001532e0 0x7f2600147f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 --2- 172.21.15.88:0/991160116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26000705b0 0x7f260015b040 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 >> 172.21.15.88:0/991160116 conn(0x7f26000b1dd0 msgr2=0x7f260006f6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 shutdown_connections 2024-09-21T07:52:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.145+0000 7f2605a00640 1 -- 172.21.15.88:0/991160116 wait complete. 2024-09-21T07:52:22.171 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:22.171 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol88300/.meta) 2024-09-21T07:52:22.227 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:52:22.227 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:52:22.227 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:52:22.227 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol88300/cfd0f96c-e768-4837-a880-10dc1a068a4e 2024-09-21T07:52:22.227 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:52:22.227 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:22.230 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol88300 2024-09-21T07:52:22.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.451+0000 7f3997000640 1 -- 172.21.15.88:0/3013613571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990145a40 msgr2=0x7f3990073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.451+0000 7f3997000640 1 --2- 172.21.15.88:0/3013613571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990145a40 0x7f3990073540 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7f3990072bd0 tx=0x7f397c02a680 comp rx=0 tx=0).stop 2024-09-21T07:52:22.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.452+0000 7f3997000640 1 -- 172.21.15.88:0/3013613571 shutdown_connections 2024-09-21T07:52:22.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.452+0000 7f3997000640 1 --2- 172.21.15.88:0/3013613571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990145a40 0x7f3990073540 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.452+0000 7f3997000640 1 --2- 172.21.15.88:0/3013613571 >> v2:172.21.15.175:3300/0 conn(0x7f3990145550 0x7f399006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.452+0000 7f3997000640 1 -- 172.21.15.88:0/3013613571 >> 172.21.15.88:0/3013613571 conn(0x7f399006d1c0 msgr2=0x7f399006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.452+0000 7f3997000640 1 -- 172.21.15.88:0/3013613571 shutdown_connections 2024-09-21T07:52:22.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.452+0000 7f3997000640 1 -- 172.21.15.88:0/3013613571 wait complete. 2024-09-21T07:52:22.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.454+0000 7f3997000640 1 Processor -- start 2024-09-21T07:52:22.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.454+0000 7f3997000640 1 -- start start 2024-09-21T07:52:22.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.455+0000 7f3997000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 0x7f39901f3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f3997000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3990145550 0x7f3990152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f3997000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3990145a40 0x7f3990153200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f3997000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3990160a30 con 0x7f3990073a80 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f3997000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f39900ce870 con 0x7f3990145a40 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f3997000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3990160bb0 con 0x7f3990145550 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 0x7f39901f3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 0x7f39901f3230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52040/0 (socket says 172.21.15.88:52040) 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 -- 172.21.15.88:0/3764856052 learned_addr learned my addr 172.21.15.88:0/3764856052 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398f400640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3990145550 0x7f3990152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f3995000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3990145a40 0x7f3990153200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 -- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3990145550 msgr2=0x7f3990152cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3990145550 0x7f3990152cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 -- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3990145a40 msgr2=0x7f3990153200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3990145a40 0x7f3990153200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.456+0000 7f398fe00640 1 -- 172.21.15.88:0/3764856052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3990136690 con 0x7f3990073a80 2024-09-21T07:52:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.457+0000 7f398fe00640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 0x7f39901f3230 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f398000dac0 tx=0x7f398000df80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.457+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3980017070 con 0x7f3990073a80 2024-09-21T07:52:22.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.457+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3980003970 con 0x7f3990073a80 2024-09-21T07:52:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.457+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3990136970 con 0x7f3990073a80 2024-09-21T07:52:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.457+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f398000be60 con 0x7f3990073a80 2024-09-21T07:52:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.457+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3990136f30 con 0x7f3990073a80 2024-09-21T07:52:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.458+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39901610d0 con 0x7f3990073a80 2024-09-21T07:52:22.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.459+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7f3980004020 con 0x7f3990073a80 2024-09-21T07:52:22.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.460+0000 7f3987e00640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3970078ef0 0x7f397007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.460+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f398009b8f0 con 0x7f3990073a80 2024-09-21T07:52:22.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.461+0000 7f398f400640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3970078ef0 0x7f397007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.463 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.461+0000 7f398f400640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3970078ef0 0x7f397007b400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f397c033000 tx=0x7f397c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:22.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.465+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3980020030 con 0x7f3990073a80 2024-09-21T07:52:22.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.676+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol88300", "target": ["mon-mgr", ""]}) -- 0x7f3990140d20 con 0x7f3970078ef0 2024-09-21T07:52:22.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.720+0000 7f3987e00640 1 -- 172.21.15.88:0/3764856052 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f3990140d20 con 0x7f3970078ef0 2024-09-21T07:52:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3970078ef0 msgr2=0x7f397007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3970078ef0 0x7f397007b400 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f397c033000 tx=0x7f397c004730 comp rx=0 tx=0).stop 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 msgr2=0x7f39901f3230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 0x7f39901f3230 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7f398000dac0 tx=0x7f398000df80 comp rx=0 tx=0).stop 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 shutdown_connections 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3970078ef0 0x7f397007b400 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3990145550 0x7f3990152cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3990145a40 0x7f3990153200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 --2- 172.21.15.88:0/3764856052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3990073a80 0x7f39901f3230 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 >> 172.21.15.88:0/3764856052 conn(0x7f399006d1c0 msgr2=0x7f3990143090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 shutdown_connections 2024-09-21T07:52:22.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.723+0000 7f3997000640 1 -- 172.21.15.88:0/3764856052 wait complete. 2024-09-21T07:52:22.727 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.725+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.727 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.725+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.727 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.725+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.727 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.725+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.727 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.725+0000 7facaae00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.745 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.743+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.745 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.743+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.745 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.743+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.745 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.743+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.745 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:52:22.743+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:52:22.755 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone33240 2024-09-21T07:52:22.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 -- 172.21.15.88:0/2711907182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e0005250 msgr2=0x7fe9e000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 --2- 172.21.15.88:0/2711907182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e0005250 0x7fe9e000b9c0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4001d90 tx=0x7fe9e0005830 comp rx=0 tx=0).stop 2024-09-21T07:52:22.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 -- 172.21.15.88:0/2711907182 shutdown_connections 2024-09-21T07:52:22.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 --2- 172.21.15.88:0/2711907182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9e0005250 0x7fe9e000b9c0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 -- 172.21.15.88:0/2711907182 >> 172.21.15.88:0/2711907182 conn(0x7fe9fc06d620 msgr2=0x7fe9fc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:22.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 -- 172.21.15.88:0/2711907182 shutdown_connections 2024-09-21T07:52:22.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.985+0000 7fe9fb200640 1 -- 172.21.15.88:0/2711907182 wait complete. 2024-09-21T07:52:22.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.987+0000 7fe9fb200640 1 Processor -- start 2024-09-21T07:52:22.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.987+0000 7fe9fb200640 1 -- start start 2024-09-21T07:52:22.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9e0005250 0x7fe9fc1f6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc13cd80 0x7fe9fc13d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 0x7fe9fc1f4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc157030 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe9fc156eb0 con 0x7fe9fc13cd80 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe9fc0ccac0 con 0x7fe9e0005250 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fa800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 0x7fe9fc1f4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fa800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 0x7fe9fc1f4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52070/0 (socket says 172.21.15.88:52070) 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9fa800640 1 -- 172.21.15.88:0/1088610940 learned_addr learned my addr 172.21.15.88:0/1088610940 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.989+0000 7fe9f9e00640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9e0005250 0x7fe9fc1f6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9f9400640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc13cd80 0x7fe9fc13d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fa800640 1 -- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9e0005250 msgr2=0x7fe9fc1f6270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fa800640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9e0005250 0x7fe9fc1f6270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fa800640 1 -- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc13cd80 msgr2=0x7fe9fc13d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fa800640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc13cd80 0x7fe9fc13d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fa800640 1 -- 172.21.15.88:0/1088610940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9fc157f30 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fa800640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 0x7fe9fc1f4760 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4001370 tx=0x7fe9f4006d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4003d80 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9f4005b80 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9fc158210 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.990+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4005fd0 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.991+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9fc1587d0 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.992+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe9fc0af0d0 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.993+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fe9f4005030 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.994+0000 7fe9f2000640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9dc078ef0 0x7fe9dc07b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.994+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fe9f40bb420 con 0x7fe9fc13d6d0 2024-09-21T07:52:22.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.994+0000 7fe9f9e00640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9dc078ef0 0x7fe9dc07b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:22.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.995+0000 7fe9f9e00640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9dc078ef0 0x7fe9dc07b400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009ad0 tx=0x7fe9ec005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:23.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:22.999+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9f4043060 con 0x7fe9fc13d6d0 2024-09-21T07:52:23.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.212+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone33240", "target": ["mon-mgr", ""]}) -- 0x7fe9fc157c80 con 0x7fe9dc078ef0 2024-09-21T07:52:23.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.230+0000 7fe9f2000640 1 -- 172.21.15.88:0/1088610940 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe9fc157c80 con 0x7fe9dc078ef0 2024-09-21T07:52:23.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9dc078ef0 msgr2=0x7fe9dc07b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9dc078ef0 0x7fe9dc07b400 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fe9ec009ad0 tx=0x7fe9ec005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 msgr2=0x7fe9fc1f4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 0x7fe9fc1f4760 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fe9f4001370 tx=0x7fe9f4006d40 comp rx=0 tx=0).stop 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 shutdown_connections 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9dc078ef0 0x7fe9dc07b400 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9fc13d6d0 0x7fe9fc1f4760 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9fc13cd80 0x7fe9fc13d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 --2- 172.21.15.88:0/1088610940 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9e0005250 0x7fe9fc1f6270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 >> 172.21.15.88:0/1088610940 conn(0x7fe9fc06d620 msgr2=0x7fe9fc0cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 shutdown_connections 2024-09-21T07:52:23.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.232+0000 7fe9fb200640 1 -- 172.21.15.88:0/1088610940 wait complete. 2024-09-21T07:52:23.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol_clone85521 2024-09-21T07:52:23.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 -- 172.21.15.88:0/2254913859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145550 msgr2=0x7fcb0406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 --2- 172.21.15.88:0/2254913859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145550 0x7fcb0406fb40 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4005fd0 tx=0x7fcaf402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 -- 172.21.15.88:0/2254913859 shutdown_connections 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 --2- 172.21.15.88:0/2254913859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb04073c50 0x7fcb040cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 --2- 172.21.15.88:0/2254913859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145550 0x7fcb0406fb40 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 -- 172.21.15.88:0/2254913859 >> 172.21.15.88:0/2254913859 conn(0x7fcb0406d1c0 msgr2=0x7fcb0406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 -- 172.21.15.88:0/2254913859 shutdown_connections 2024-09-21T07:52:23.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.487+0000 7fcb09200640 1 -- 172.21.15.88:0/2254913859 wait complete. 2024-09-21T07:52:23.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.489+0000 7fcb09200640 1 Processor -- start 2024-09-21T07:52:23.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.490+0000 7fcb09200640 1 -- start start 2024-09-21T07:52:23.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcb09200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb04145550 0x7fcb0413db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcb09200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 0x7fcb0415e560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcb09200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040b4b00 0x7fcb040b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcb09200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040ce150 con 0x7fcb04145a40 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcb09200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcb040cdfd0 con 0x7fcb04145550 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcb09200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcb040ce2d0 con 0x7fcb040b4b00 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcaf9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 0x7fcb0415e560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcaf9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 0x7fcb0415e560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52076/0 (socket says 172.21.15.88:52076) 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.491+0000 7fcaf9400640 1 -- 172.21.15.88:0/1470864729 learned_addr learned my addr 172.21.15.88:0/1470864729 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcb02800640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040b4b00 0x7fcb040b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcb01e00640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb04145550 0x7fcb0413db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcaf9400640 1 -- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040b4b00 msgr2=0x7fcb040b4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcaf9400640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040b4b00 0x7fcb040b4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcaf9400640 1 -- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb04145550 msgr2=0x7fcb0413db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcaf9400640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb04145550 0x7fcb0413db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcaf9400640 1 -- 172.21.15.88:0/1470864729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb040b30a0 con 0x7fcb04145a40 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcaf9400640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 0x7fcb0415e560 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b3c0 tx=0x7fcaec00b880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011830 con 0x7fcb04145a40 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcaec016070 con 0x7fcb04145a40 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.492+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb040b5450 con 0x7fcb04145a40 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.493+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaec011da0 con 0x7fcb04145a40 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.493+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb0415eeb0 con 0x7fcb04145a40 2024-09-21T07:52:23.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.494+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb040ce830 con 0x7fcb04145a40 2024-09-21T07:52:23.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.495+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 20) ==== 101747+0+0 (secure 0 0 0) 0x7fcaec01b460 con 0x7fcb04145a40 2024-09-21T07:52:23.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.496+0000 7fcafaa00640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcae0078ef0 0x7fcae007b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:23.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.496+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fcaec09ad70 con 0x7fcb04145a40 2024-09-21T07:52:23.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.496+0000 7fcb01e00640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcae0078ef0 0x7fcae007b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:23.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.497+0000 7fcb01e00640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcae0078ef0 0x7fcae007b400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4033000 tx=0x7fcaf4004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:23.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.501+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcaec020020 con 0x7fcb04145a40 2024-09-21T07:52:23.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.711+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol_clone85521", "target": ["mon-mgr", ""]}) -- 0x7fcb0413e0b0 con 0x7fcae0078ef0 2024-09-21T07:52:23.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.729+0000 7fcafaa00640 1 -- 172.21.15.88:0/1470864729 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fcaf4004020 con 0x7fcae0078ef0 2024-09-21T07:52:23.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcae0078ef0 msgr2=0x7fcae007b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcae0078ef0 0x7fcae007b400 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fcaf4033000 tx=0x7fcaf4004730 comp rx=0 tx=0).stop 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 msgr2=0x7fcb0415e560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 0x7fcb0415e560 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7fcaec00b3c0 tx=0x7fcaec00b880 comp rx=0 tx=0).stop 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 shutdown_connections 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcae0078ef0 0x7fcae007b400 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcb040b4b00 0x7fcb040b4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcb04145a40 0x7fcb0415e560 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 --2- 172.21.15.88:0/1470864729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcb04145550 0x7fcb0413db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 >> 172.21.15.88:0/1470864729 conn(0x7fcb0406d1c0 msgr2=0x7fcb04142ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 shutdown_connections 2024-09-21T07:52:23.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23.733+0000 7fcb09200640 1 -- 172.21.15.88:0/1470864729 wait complete. 2024-09-21T07:52:23.762 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:52:23.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:52:23.865 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:52:23.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:23 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:23.874 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:52:23.874 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:52:23.964 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:52:24.072 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:52:24.072 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:52:24.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:24.072 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:24.108 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:24.108 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:24.184 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:52:24.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1530150113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bbc003a50 msgr2=0x7f8bbc003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1530150113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bbc003a50 0x7f8bbc003ec0 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f8bc800baf0 tx=0x7f8bbc005830 comp rx=0 tx=0).stop 2024-09-21T07:52:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1530150113 shutdown_connections 2024-09-21T07:52:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1530150113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bbc003a50 0x7f8bbc003ec0 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1530150113 >> 172.21.15.88:0/1530150113 conn(0x7f8bd406d3d0 msgr2=0x7f8bd40b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1530150113 shutdown_connections 2024-09-21T07:52:24.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.446+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1530150113 wait complete. 2024-09-21T07:52:24.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.448+0000 7f8bdbe00640 1 Processor -- start 2024-09-21T07:52:24.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.448+0000 7f8bdbe00640 1 -- start start 2024-09-21T07:52:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.449+0000 7f8bdbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bbc003a50 0x7f8bd41f8670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bdbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bd4074f30 0x7f8bd41574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bdbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 0x7f8bd41f8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bdbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd40709c0 con 0x7f8bd4157a30 2024-09-21T07:52:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bdbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8bd4157320 con 0x7f8bbc003a50 2024-09-21T07:52:24.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bdbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8bd4070b40 con 0x7f8bd4074f30 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 0x7f8bd41f8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 0x7f8bd41f8d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52096/0 (socket says 172.21.15.88:52096) 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 -- 172.21.15.88:0/1435569095 learned_addr learned my addr 172.21.15.88:0/1435569095 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9400640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bbc003a50 0x7f8bd41f8670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd8a00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bd4074f30 0x7f8bd41574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 -- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bd4074f30 msgr2=0x7f8bd41574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bd4074f30 0x7f8bd41574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 -- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bbc003a50 msgr2=0x7f8bd41f8670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bbc003a50 0x7f8bd41f8670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.450+0000 7f8bd9e00640 1 -- 172.21.15.88:0/1435569095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bd4157e40 con 0x7f8bd4157a30 2024-09-21T07:52:24.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.451+0000 7f8bd9e00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 0x7f8bd41f8d10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40ce210 tx=0x7f8bcc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:24.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.451+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc017070 con 0x7f8bd4157a30 2024-09-21T07:52:24.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.451+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8bcc0108c0 con 0x7f8bd4157a30 2024-09-21T07:52:24.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.451+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bcc010cd0 con 0x7f8bd4157a30 2024-09-21T07:52:24.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.451+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bd40c3530 con 0x7f8bd4157a30 2024-09-21T07:52:24.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.451+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bd4074380 con 0x7f8bd4157a30 2024-09-21T07:52:24.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.452+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bd40c2f50 con 0x7f8bd4157a30 2024-09-21T07:52:24.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.453+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f8bcc01a070 con 0x7f8bd4157a30 2024-09-21T07:52:24.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.454+0000 7f8bd1600640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8bb4079010 0x7f8bb407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:24.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.454+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7f8bcc09b430 con 0x7f8bd4157a30 2024-09-21T07:52:24.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.454+0000 7f8bd9400640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8bb4079010 0x7f8bb407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:24.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.455+0000 7f8bd9400640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8bb4079010 0x7f8bb407b520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8036000 tx=0x7f8bc803d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:24.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.459+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8bd4157e40 con 0x7f8bd4157a30 2024-09-21T07:52:24.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.655+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f8bd40745e0 con 0x7f8bd4157a30 2024-09-21T07:52:24.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.656+0000 7f8bd1600640 1 -- 172.21.15.88:0/1435569095 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7f8bcc0630f0 con 0x7f8bd4157a30 2024-09-21T07:52:24.657 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8bb4079010 msgr2=0x7f8bb407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8bb4079010 0x7f8bb407b520 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8bc8036000 tx=0x7f8bc803d040 comp rx=0 tx=0).stop 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 msgr2=0x7f8bd41f8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 0x7f8bd41f8d10 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f8bd40ce210 tx=0x7f8bcc0019b0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 shutdown_connections 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8bb4079010 0x7f8bb407b520 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8bd4074f30 0x7f8bd41574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8bd4157a30 0x7f8bd41f8d10 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 --2- 172.21.15.88:0/1435569095 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8bbc003a50 0x7f8bd41f8670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:24.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 >> 172.21.15.88:0/1435569095 conn(0x7f8bd406d3d0 msgr2=0x7f8bd415f3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:24.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.658+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 shutdown_connections 2024-09-21T07:52:24.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:24.659+0000 7f8bdbe00640 1 -- 172.21.15.88:0/1435569095 wait complete. 2024-09-21T07:52:24.680 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:52:24.680 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:24.681 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:24.765 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:52:24.765 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:24.765 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:52:24.816 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:52:24.817 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:52:24.817 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:24.855 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:52:24.855 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:24.932 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 -- 172.21.15.88:0/2946062862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 msgr2=0x7fdccc0d2280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/2946062862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 0x7fdccc0d2280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcb4009830 tx=0x7fdcb402a510 comp rx=0 tx=0).stop 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 -- 172.21.15.88:0/2946062862 shutdown_connections 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/2946062862 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 0x7fdccc0d2280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/2946062862 >> v2:172.21.15.175:3300/0 conn(0x7fdccc0cf490 0x7fdccc15cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 -- 172.21.15.88:0/2946062862 >> 172.21.15.88:0/2946062862 conn(0x7fdccc06d1c0 msgr2=0x7fdccc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 -- 172.21.15.88:0/2946062862 shutdown_connections 2024-09-21T07:52:25.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.155+0000 7fdcd0c00640 1 -- 172.21.15.88:0/2946062862 wait complete. 2024-09-21T07:52:25.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.158+0000 7fdcd0c00640 1 Processor -- start 2024-09-21T07:52:25.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.158+0000 7fdcd0c00640 1 -- start start 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.159+0000 7fdcd0c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdccc0cf490 0x7fdccc1e2d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.159+0000 7fdcd0c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 0x7fdccc1e3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.159+0000 7fdcd0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 0x7fdccc14b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.159+0000 7fdcd0c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0d27c0 con 0x7fdccc15d170 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.159+0000 7fdcd0c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdccc0d0b60 con 0x7fdccc0cf490 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.159+0000 7fdcd0c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdccc0d2940 con 0x7fdccc0cf8a0 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 0x7fdccc14b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 0x7fdccc14b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52104/0 (socket says 172.21.15.88:52104) 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 -- 172.21.15.88:0/4206161454 learned_addr learned my addr 172.21.15.88:0/4206161454 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccb400640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdccc0cf490 0x7fdccc1e2d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdcc2a00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 0x7fdccc1e3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 -- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 msgr2=0x7fdccc1e3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 0x7fdccc1e3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 -- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdccc0cf490 msgr2=0x7fdccc1e2d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdccc0cf490 0x7fdccc1e2d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 -- 172.21.15.88:0/4206161454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdccc1e65a0 con 0x7fdccc15d170 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.160+0000 7fdccbe00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 0x7fdccc14b280 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c3c0 tx=0x7fdcc4001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.161+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4017070 con 0x7fdccc15d170 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.161+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcc4006d90 con 0x7fdccc15d170 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.161+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcc4007290 con 0x7fdccc15d170 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.161+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdccc1e6880 con 0x7fdccc15d170 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.161+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdccc06e190 con 0x7fdccc15d170 2024-09-21T07:52:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.162+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdccc15cc30 con 0x7fdccc15d170 2024-09-21T07:52:25.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.163+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fdcc401a070 con 0x7fdccc15d170 2024-09-21T07:52:25.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.164+0000 7fdcc3e00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca8079010 0x7fdca807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.164+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fdcc409a760 con 0x7fdccc15d170 2024-09-21T07:52:25.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.165+0000 7fdccb400640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca8079010 0x7fdca807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.165+0000 7fdccb400640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca8079010 0x7fdca807b520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0099a0 tx=0x7fdcbc009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:25.173 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.172+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdccc0cea30 con 0x7fdccc15d170 2024-09-21T07:52:25.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.369+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdccc0d2010 con 0x7fdccc15d170 2024-09-21T07:52:25.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.369+0000 7fdcc3e00640 1 -- 172.21.15.88:0/4206161454 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v107) ==== 80+0+0 (secure 0 0 0) 0x7fdcc4061f00 con 0x7fdccc15d170 2024-09-21T07:52:25.371 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:52:25.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca8079010 msgr2=0x7fdca807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca8079010 0x7fdca807b520 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0099a0 tx=0x7fdcbc009500 comp rx=0 tx=0).stop 2024-09-21T07:52:25.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 msgr2=0x7fdccc14b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 0x7fdccc14b280 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7fdcc400c3c0 tx=0x7fdcc4001e70 comp rx=0 tx=0).stop 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 shutdown_connections 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca8079010 0x7fdca807b520 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdccc0cf8a0 0x7fdccc1e3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdccc15d170 0x7fdccc14b280 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 --2- 172.21.15.88:0/4206161454 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdccc0cf490 0x7fdccc1e2d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 >> 172.21.15.88:0/4206161454 conn(0x7fdccc06d1c0 msgr2=0x7fdccc0b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 shutdown_connections 2024-09-21T07:52:25.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.372+0000 7fdcd0c00640 1 -- 172.21.15.88:0/4206161454 wait complete. 2024-09-21T07:52:25.399 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:52:25.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:25.399 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:52:25.464 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:52:25.464 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:25.464 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:52:25.501 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:52:25.501 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:25.501 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:25.536 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:52:25.536 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:52:25.536 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:52:25.537 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:25.575 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:52:25.576 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:25.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a7200640 1 -- 172.21.15.88:0/503232937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa184004800 msgr2=0x7fa184004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a7200640 1 --2- 172.21.15.88:0/503232937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa184004800 0x7fa184004c10 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fa18c009b20 tx=0x7fa184005830 comp rx=0 tx=0).stop 2024-09-21T07:52:25.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a5200640 1 -- 172.21.15.88:0/503232937 reap_dead start 2024-09-21T07:52:25.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a7200640 1 -- 172.21.15.88:0/503232937 shutdown_connections 2024-09-21T07:52:25.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a7200640 1 -- 172.21.15.88:0/503232937 >> 172.21.15.88:0/503232937 conn(0x7fa1a006d730 msgr2=0x7fa1a00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:25.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a7200640 1 -- 172.21.15.88:0/503232937 shutdown_connections 2024-09-21T07:52:25.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.796+0000 7fa1a7200640 1 -- 172.21.15.88:0/503232937 wait complete. 2024-09-21T07:52:25.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.799+0000 7fa1a7200640 1 Processor -- start 2024-09-21T07:52:25.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.799+0000 7fa1a7200640 1 -- start start 2024-09-21T07:52:25.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa1a7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa184004800 0x7fa1a01517c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa1a7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 0x7fa1a013f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa1a7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1a013fab0 0x7fa1a01f8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa1a7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00b4da0 con 0x7fa1a01f51a0 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa1a7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a00b4c20 con 0x7fa1a013fab0 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa1a7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa1a00b3960 con 0x7fa184004800 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa19f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 0x7fa1a013f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa19f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 0x7fa1a013f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41146/0 (socket says 172.21.15.88:41146) 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.800+0000 7fa19f400640 1 -- 172.21.15.88:0/3603839786 learned_addr learned my addr 172.21.15.88:0/3603839786 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19fe00640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa184004800 0x7fa1a01517c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa1a5200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1a013fab0 0x7fa1a01f8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19f400640 1 -- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa184004800 msgr2=0x7fa1a01517c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19f400640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa184004800 0x7fa1a01517c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19f400640 1 -- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1a013fab0 msgr2=0x7fa1a01f8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19f400640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1a013fab0 0x7fa1a01f8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:25.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19f400640 1 -- 172.21.15.88:0/3603839786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a013fec0 con 0x7fa1a01f51a0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa1a5200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1a013fab0 0x7fa1a01f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19fe00640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa184004800 0x7fa1a01517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa19f400640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 0x7fa1a013f570 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa18c036040 tx=0x7fa18c033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c038050 con 0x7fa1a01f51a0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa18c039040 con 0x7fa1a01f51a0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a0156bc0 con 0x7fa1a01f51a0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.801+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa18c033c50 con 0x7fa1a01f51a0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.802+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a0150fc0 con 0x7fa1a01f51a0 2024-09-21T07:52:25.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.803+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1a00ad4f0 con 0x7fa1a01f51a0 2024-09-21T07:52:25.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.804+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fa18c046020 con 0x7fa1a01f51a0 2024-09-21T07:52:25.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.805+0000 7fa197e00640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa180079010 0x7fa18007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:25.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.805+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 5767+0+0 (secure 0 0 0) 0x7fa18c0bc170 con 0x7fa1a01f51a0 2024-09-21T07:52:25.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.805+0000 7fa19fe00640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa180079010 0x7fa18007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:25.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.806+0000 7fa19fe00640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa180079010 0x7fa18007b520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1900013f0 tx=0x7fa190001240 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:25.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:25.810+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa18c04d020 con 0x7fa1a01f51a0 2024-09-21T07:52:26.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.043+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1a0152eb0 con 0x7fa1a01f51a0 2024-09-21T07:52:26.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.043+0000 7fa197e00640 1 -- 172.21.15.88:0/3603839786 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fa18c083e30 con 0x7fa1a01f51a0 2024-09-21T07:52:26.045 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:26.045 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":71,"btime":"2024-09-21T07:51:40:585430+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:51:29.581113+0000","modified":"2024-09-21T07:51:32.603721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6975":{"gid":6975,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3744288878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3744288878},{"type":"v1","addr":"172.21.15.88:6835","nonce":3744288878}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6975,"qdb_cluster":[6975]},"id":8}]} 2024-09-21T07:52:26.045 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 71 2024-09-21T07:52:26.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa180079010 msgr2=0x7fa18007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa180079010 0x7fa18007b520 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fa1900013f0 tx=0x7fa190001240 comp rx=0 tx=0).stop 2024-09-21T07:52:26.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 msgr2=0x7fa1a013f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 0x7fa1a013f570 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fa18c036040 tx=0x7fa18c033670 comp rx=0 tx=0).stop 2024-09-21T07:52:26.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 shutdown_connections 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa180079010 0x7fa18007b520 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa1a013fab0 0x7fa1a01f8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1a01f51a0 0x7fa1a013f570 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 --2- 172.21.15.88:0/3603839786 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa184004800 0x7fa1a01517c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 >> 172.21.15.88:0/3603839786 conn(0x7fa1a006d730 msgr2=0x7fa1a00b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 shutdown_connections 2024-09-21T07:52:26.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.046+0000 7fa1a7200640 1 -- 172.21.15.88:0/3603839786 wait complete. 2024-09-21T07:52:26.068 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 -- 172.21.15.88:0/183301835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf80ad0e0 msgr2=0x7fbaf80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 --2- 172.21.15.88:0/183301835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf80ad0e0 0x7fbaf80ad4f0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fbaf8075250 tx=0x7fbae402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 -- 172.21.15.88:0/183301835 shutdown_connections 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 --2- 172.21.15.88:0/183301835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf80ad0e0 0x7fbaf80ad4f0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 --2- 172.21.15.88:0/183301835 >> v2:172.21.15.175:3300/0 conn(0x7fbaf815d760 0x7fbaf81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 -- 172.21.15.88:0/183301835 >> 172.21.15.88:0/183301835 conn(0x7fbaf806d730 msgr2=0x7fbaf80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 -- 172.21.15.88:0/183301835 shutdown_connections 2024-09-21T07:52:26.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.298+0000 7fbaff800640 1 -- 172.21.15.88:0/183301835 wait complete. 2024-09-21T07:52:26.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.301+0000 7fbaff800640 1 Processor -- start 2024-09-21T07:52:26.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.301+0000 7fbaff800640 1 -- start start 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbaff800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaf80ad0e0 0x7fbaf814a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbaff800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf815d760 0x7fbaf8157e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbaff800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 0x7fbaf8158350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbaff800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf80b4c20 con 0x7fbaf8160310 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbaff800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbaf8153c70 con 0x7fbaf815d760 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbaff800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbaf80b4da0 con 0x7fbaf80ad0e0 2024-09-21T07:52:26.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbafd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 0x7fbaf8158350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbafd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 0x7fbaf8158350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41162/0 (socket says 172.21.15.88:41162) 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.302+0000 7fbafd800640 1 -- 172.21.15.88:0/3115195912 learned_addr learned my addr 172.21.15.88:0/3115195912 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbaf7e00640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf815d760 0x7fbaf8157e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafce00640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaf80ad0e0 0x7fbaf814a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafd800640 1 -- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaf80ad0e0 msgr2=0x7fbaf814a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafd800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaf80ad0e0 0x7fbaf814a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafd800640 1 -- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf815d760 msgr2=0x7fbaf8157e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafd800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf815d760 0x7fbaf8157e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafd800640 1 -- 172.21.15.88:0/3115195912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbaf8154050 con 0x7fbaf8160310 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafce00640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaf80ad0e0 0x7fbaf814a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:26.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbafd800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 0x7fbaf8158350 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000caa0 tx=0x7fbaf000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:26.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0016020 con 0x7fbaf8160310 2024-09-21T07:52:26.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbaf000acf0 con 0x7fbaf8160310 2024-09-21T07:52:26.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbaf8154330 con 0x7fbaf8160310 2024-09-21T07:52:26.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.303+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbaf0012680 con 0x7fbaf8160310 2024-09-21T07:52:26.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.304+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbaf8074cc0 con 0x7fbaf8160310 2024-09-21T07:52:26.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.305+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbaf8074ef0 con 0x7fbaf8160310 2024-09-21T07:52:26.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.306+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fbaf0019020 con 0x7fbaf8160310 2024-09-21T07:52:26.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.307+0000 7fbaf4c00640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbad8079010 0x7fbad807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.307+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7fbaf009bd40 con 0x7fbaf8160310 2024-09-21T07:52:26.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.307+0000 7fbafce00640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbad8079010 0x7fbad807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.307+0000 7fbafce00640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbad8079010 0x7fbad807b520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009b80 tx=0x7fbae8005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:26.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.311+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbaf00a2080 con 0x7fbaf8160310 2024-09-21T07:52:26.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.534+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbaf8160ff0 con 0x7fbaf8160310 2024-09-21T07:52:26.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.535+0000 7fbaf4c00640 1 -- 172.21.15.88:0/3115195912 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7fbaf0020020 con 0x7fbaf8160310 2024-09-21T07:52:26.536 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:26.536 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":71,"btime":"2024-09-21T07:51:40:585430+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:51:29.581113+0000","modified":"2024-09-21T07:51:32.603721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6975":{"gid":6975,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3744288878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3744288878},{"type":"v1","addr":"172.21.15.88:6835","nonce":3744288878}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6975,"qdb_cluster":[6975]},"id":8}]} 2024-09-21T07:52:26.536 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 71 2024-09-21T07:52:26.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbad8079010 msgr2=0x7fbad807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbad8079010 0x7fbad807b520 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbae8009b80 tx=0x7fbae8005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 msgr2=0x7fbaf8158350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 0x7fbaf8158350 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fbaf000caa0 tx=0x7fbaf000cf60 comp rx=0 tx=0).stop 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 shutdown_connections 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbad8079010 0x7fbad807b520 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbaf8160310 0x7fbaf8158350 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbaf815d760 0x7fbaf8157e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 --2- 172.21.15.88:0/3115195912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbaf80ad0e0 0x7fbaf814a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 >> 172.21.15.88:0/3115195912 conn(0x7fbaf806d730 msgr2=0x7fbaf806f340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 shutdown_connections 2024-09-21T07:52:26.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.537+0000 7fbaff800640 1 -- 172.21.15.88:0/3115195912 wait complete. 2024-09-21T07:52:26.563 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:52:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 -- 172.21.15.88:0/4183986994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc0735c0 msgr2=0x7f71fc0739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 --2- 172.21.15.88:0/4183986994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc0735c0 0x7f71fc0739d0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f71e800a940 tx=0x7f71e802b8f0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 -- 172.21.15.88:0/4183986994 shutdown_connections 2024-09-21T07:52:26.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 --2- 172.21.15.88:0/4183986994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc0735c0 0x7f71fc0739d0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 -- 172.21.15.88:0/4183986994 >> 172.21.15.88:0/4183986994 conn(0x7f71fc0b1e40 msgr2=0x7f71fc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 -- 172.21.15.88:0/4183986994 shutdown_connections 2024-09-21T07:52:26.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.784+0000 7f7202a00640 1 -- 172.21.15.88:0/4183986994 wait complete. 2024-09-21T07:52:26.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.786+0000 7f7202a00640 1 Processor -- start 2024-09-21T07:52:26.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.786+0000 7f7202a00640 1 -- start start 2024-09-21T07:52:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc0735c0 0x7f71fc1f9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71fc0b3520 0x7f71fc14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 0x7f71fc09d220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc06f360 con 0x7f71fc1518d0 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71fc071160 con 0x7f71fc0735c0 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71fc0b4040 con 0x7f71fc0b3520 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 0x7f71fc09d220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 0x7f71fc09d220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41184/0 (socket says 172.21.15.88:41184) 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7202000640 1 -- 172.21.15.88:0/2213534427 learned_addr learned my addr 172.21.15.88:0/2213534427 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.788+0000 7f7201600640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc0735c0 0x7f71fc1f9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7200c00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71fc0b3520 0x7f71fc14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202000640 1 -- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71fc0b3520 msgr2=0x7f71fc14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202000640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71fc0b3520 0x7f71fc14fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202000640 1 -- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc0735c0 msgr2=0x7f71fc1f9950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202000640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc0735c0 0x7f71fc1f9950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202000640 1 -- 172.21.15.88:0/2213534427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71fc146090 con 0x7f71fc1518d0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202000640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 0x7f71fc09d220 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f71f400cab0 tx=0x7f71f400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4016020 con 0x7f71fc1518d0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71f400ad20 con 0x7f71fc1518d0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f4012680 con 0x7f71fc1518d0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71fc145510 con 0x7f71fc1518d0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.789+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71fc145ad0 con 0x7f71fc1518d0 2024-09-21T07:52:26.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.790+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71fc0b45d0 con 0x7f71fc1518d0 2024-09-21T07:52:26.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.792+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f71f4019020 con 0x7f71fc1518d0 2024-09-21T07:52:26.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.793+0000 7f71f9600640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71dc079010 0x7f71dc07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:26.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.793+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f71f409bab0 con 0x7f71fc1518d0 2024-09-21T07:52:26.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.793+0000 7f7201600640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71dc079010 0x7f71dc07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:26.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.794+0000 7f7201600640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71dc079010 0x7f71dc07b520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f71f00015e0 tx=0x7f71f0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:26.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.797+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71f4020030 con 0x7f71fc1518d0 2024-09-21T07:52:26.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.986+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f71fc1f59c0 con 0x7f71fc1518d0 2024-09-21T07:52:26.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.987+0000 7f71f9600640 1 -- 172.21.15.88:0/2213534427 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16950 (secure 0 0 0) 0x7f71fc073f10 con 0x7f71fc1518d0 2024-09-21T07:52:26.989 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:26.989 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":108,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:52:25.849403+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:51:27.552445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T07:51:28.544258+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.1","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71dc079010 msgr2=0x7f71dc07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71dc079010 0x7f71dc07b520 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f71f00015e0 tx=0x7f71f0009040 comp rx=0 tx=0).stop 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 msgr2=0x7f71fc09d220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 0x7f71fc09d220 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7f71f400cab0 tx=0x7f71f400cf70 comp rx=0 tx=0).stop 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 shutdown_connections 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71dc079010 0x7f71dc07b520 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71fc0b3520 0x7f71fc14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71fc1518d0 0x7f71fc09d220 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 --2- 172.21.15.88:0/2213534427 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71fc0735c0 0x7f71fc1f9950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.989+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 >> 172.21.15.88:0/2213534427 conn(0x7f71fc0b1e40 msgr2=0x7f71fc152910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.990+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 shutdown_connections 2024-09-21T07:52:26.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:26.990+0000 7f7202a00640 1 -- 172.21.15.88:0/2213534427 wait complete. 2024-09-21T07:52:27.012 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:52:27.013 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:27.131 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:27.135 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.235+0000 7f15c3e00640 1 -- 172.21.15.88:0/1015382288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0ad3c0 msgr2=0x7f15bc0ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.235+0000 7f15c3e00640 1 --2- 172.21.15.88:0/1015382288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0ad3c0 0x7f15bc0ad7d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f15a8009830 tx=0x7f15a802a480 comp rx=0 tx=0).stop 2024-09-21T07:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.235+0000 7f15c3e00640 1 -- 172.21.15.88:0/1015382288 shutdown_connections 2024-09-21T07:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.235+0000 7f15c3e00640 1 --2- 172.21.15.88:0/1015382288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc0add10 0x7f15bc1646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.235+0000 7f15c3e00640 1 --2- 172.21.15.88:0/1015382288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0ad3c0 0x7f15bc0ad7d0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.236+0000 7f15c3e00640 1 -- 172.21.15.88:0/1015382288 >> 172.21.15.88:0/1015382288 conn(0x7f15bc06d730 msgr2=0x7f15bc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:27.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.236+0000 7f15c3e00640 1 -- 172.21.15.88:0/1015382288 shutdown_connections 2024-09-21T07:52:27.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.236+0000 7f15c3e00640 1 -- 172.21.15.88:0/1015382288 wait complete. 2024-09-21T07:52:27.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.238+0000 7f15c3e00640 1 Processor -- start 2024-09-21T07:52:27.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.238+0000 7f15c3e00640 1 -- start start 2024-09-21T07:52:27.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.239+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 0x7f15bc15de10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.239+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc0ad3c0 0x7f15bc0999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.239+0000 7f15c3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0add10 0x7f15bc0b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.239+0000 7f15c3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc0712b0 con 0x7f15bc0704b0 2024-09-21T07:52:27.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f15bc071130 con 0x7f15bc0add10 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f15bc164c20 con 0x7f15bc0ad3c0 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 0x7f15bc15de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 0x7f15bc15de10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41188/0 (socket says 172.21.15.88:41188) 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 -- 172.21.15.88:0/2785061914 learned_addr learned my addr 172.21.15.88:0/2785061914 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c0a00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc0ad3c0 0x7f15bc0999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0add10 0x7f15bc0b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 -- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc0ad3c0 msgr2=0x7f15bc0999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc0ad3c0 0x7f15bc0999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 -- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0add10 msgr2=0x7f15bc0b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0add10 0x7f15bc0b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.240+0000 7f15c1400640 1 -- 172.21.15.88:0/2785061914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15bc099ee0 con 0x7f15bc0704b0 2024-09-21T07:52:27.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.241+0000 7f15c1400640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 0x7f15bc15de10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c430 tx=0x7f15b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.241+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0011790 con 0x7f15bc0704b0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.241+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f15b0017070 con 0x7f15bc0704b0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.241+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15bc152800 con 0x7f15bc0704b0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.241+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15b0011ce0 con 0x7f15bc0704b0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.241+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15bc152fb0 con 0x7f15bc0704b0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.242+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15bc0ad7d0 con 0x7f15bc0704b0 2024-09-21T07:52:27.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.243+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f15b001c460 con 0x7f15bc0704b0 2024-09-21T07:52:27.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.244+0000 7f15b9600640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f159c079010 0x7f159c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.245+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f15b009b430 con 0x7f15bc0704b0 2024-09-21T07:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.245+0000 7f15c0a00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f159c079010 0x7f159c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.245+0000 7f15c0a00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f159c079010 0x7f159c07b520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15a8034000 tx=0x7f15a8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:27.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.249+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f15bc099ee0 con 0x7f15bc0704b0 2024-09-21T07:52:27.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.472+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15bc15e420 con 0x7f15bc0704b0 2024-09-21T07:52:27.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.473+0000 7f15b9600640 1 -- 172.21.15.88:0/2785061914 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7f15b0063130 con 0x7f15bc0704b0 2024-09-21T07:52:27.474 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:27.474 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":71,"btime":"2024-09-21T07:51:40:585430+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:51:29.581113+0000","modified":"2024-09-21T07:51:32.603721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6975":{"gid":6975,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3744288878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3744288878},{"type":"v1","addr":"172.21.15.88:6835","nonce":3744288878}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6975,"qdb_cluster":[6975]},"id":8}]} 2024-09-21T07:52:27.475 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 71 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f159c079010 msgr2=0x7f159c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f159c079010 0x7f159c07b520 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f15a8034000 tx=0x7f15a8004730 comp rx=0 tx=0).stop 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 msgr2=0x7f15bc15de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 0x7f15bc15de10 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7f15b000c430 tx=0x7f15b000c8f0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 shutdown_connections 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f159c079010 0x7f159c07b520 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f15bc0ad3c0 0x7f15bc0999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15bc0add10 0x7f15bc0b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 --2- 172.21.15.88:0/2785061914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15bc0704b0 0x7f15bc15de10 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.475+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 >> 172.21.15.88:0/2785061914 conn(0x7f15bc06d730 msgr2=0x7f15bc0ae260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:27.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.476+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 shutdown_connections 2024-09-21T07:52:27.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.476+0000 7f15c3e00640 1 -- 172.21.15.88:0/2785061914 wait complete. 2024-09-21T07:52:27.502 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:27.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 -- 172.21.15.88:0/2257700248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d80af630 msgr2=0x7f11d8073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 --2- 172.21.15.88:0/2257700248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d80af630 0x7f11d8073500 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f11c8009830 tx=0x7f11c802a5b0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 -- 172.21.15.88:0/2257700248 shutdown_connections 2024-09-21T07:52:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 --2- 172.21.15.88:0/2257700248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d80af630 0x7f11d8073500 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 -- 172.21.15.88:0/2257700248 >> 172.21.15.88:0/2257700248 conn(0x7f11d806d730 msgr2=0x7f11d80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 -- 172.21.15.88:0/2257700248 shutdown_connections 2024-09-21T07:52:27.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.760+0000 7f11de600640 1 -- 172.21.15.88:0/2257700248 wait complete. 2024-09-21T07:52:27.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.762+0000 7f11de600640 1 Processor -- start 2024-09-21T07:52:27.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.762+0000 7f11de600640 1 -- start start 2024-09-21T07:52:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11de600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 0x7f11d809beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11de600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8073a40 0x7f11d809d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11de600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d80af630 0x7f11d81f6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11de600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80b3d90 con 0x7f11d806f7b0 2024-09-21T07:52:27.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11de600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11d80b3c10 con 0x7f11d80af630 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11de600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f11d80b3f10 con 0x7f11d8073a40 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 0x7f11d809beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11d7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 0x7f11d809beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41218/0 (socket says 172.21.15.88:41218) 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11d7400640 1 -- 172.21.15.88:0/2764724434 learned_addr learned my addr 172.21.15.88:0/2764724434 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11cea00640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8073a40 0x7f11d809d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.764+0000 7f11d7e00640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d80af630 0x7f11d81f6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11d7400640 1 -- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8073a40 msgr2=0x7f11d809d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11d7400640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8073a40 0x7f11d809d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11d7400640 1 -- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d80af630 msgr2=0x7f11d81f6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11d7400640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d80af630 0x7f11d81f6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11d7400640 1 -- 172.21.15.88:0/2764724434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11d81f6b90 con 0x7f11d806f7b0 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11d7400640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 0x7f11d809beb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c8033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:27.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8031d60 con 0x7f11d806f7b0 2024-09-21T07:52:27.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11c80330d0 con 0x7f11d806f7b0 2024-09-21T07:52:27.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11d81f9670 con 0x7f11d806f7b0 2024-09-21T07:52:27.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11c8033520 con 0x7f11d806f7b0 2024-09-21T07:52:27.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.765+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11d81f9c30 con 0x7f11d806f7b0 2024-09-21T07:52:27.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.767+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11d80b43c0 con 0x7f11d806f7b0 2024-09-21T07:52:27.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.767+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f11c80035f0 con 0x7f11d806f7b0 2024-09-21T07:52:27.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.768+0000 7f11cfe00640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11b4079010 0x7f11b407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:27.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.769+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f11c80bccb0 con 0x7f11d806f7b0 2024-09-21T07:52:27.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.769+0000 7f11cea00640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11b4079010 0x7f11b407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:27.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.769+0000 7f11cea00640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11b4079010 0x7f11b407b520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f11d8156150 tx=0x7f11c0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:27.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.773+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11c803f030 con 0x7f11d806f7b0 2024-09-21T07:52:27.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.996+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f11d80ae340 con 0x7f11d806f7b0 2024-09-21T07:52:27.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.996+0000 7f11cfe00640 1 -- 172.21.15.88:0/2764724434 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) ==== 94+0+5225 (secure 0 0 0) 0x7f11c80842f0 con 0x7f11d806f7b0 2024-09-21T07:52:27.997 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:27.998 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":71,"btime":"2024-09-21T07:51:40:585430+0000","default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":70,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:51:29.581113+0000","modified":"2024-09-21T07:51:32.603721+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":6975},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6975":{"gid":6975,"name":"a","rank":0,"incarnation":66,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3744288878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3744288878},{"type":"v1","addr":"172.21.15.88:6835","nonce":3744288878}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":6975,"qdb_cluster":[6975]},"id":8}]} 2024-09-21T07:52:27.998 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 71 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11b4079010 msgr2=0x7f11b407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11b4079010 0x7f11b407b520 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f11d8156150 tx=0x7f11c0009040 comp rx=0 tx=0).stop 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 msgr2=0x7f11d809beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 0x7f11d809beb0 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f11c8000c00 tx=0x7f11c8033d60 comp rx=0 tx=0).stop 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 shutdown_connections 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11b4079010 0x7f11b407b520 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11d8073a40 0x7f11d809d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11d80af630 0x7f11d81f6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 --2- 172.21.15.88:0/2764724434 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11d806f7b0 0x7f11d809beb0 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 >> 172.21.15.88:0/2764724434 conn(0x7f11d806d730 msgr2=0x7f11d80aeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:28.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 shutdown_connections 2024-09-21T07:52:28.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:27.999+0000 7f11de600640 1 -- 172.21.15.88:0/2764724434 wait complete. 2024-09-21T07:52:28.025 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:52:28.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 -- 172.21.15.88:0/3657022101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0b3550 msgr2=0x7f055c0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 --2- 172.21.15.88:0/3657022101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0b3550 0x7f055c0b3960 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f055c072d60 tx=0x7f055402a9a0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 -- 172.21.15.88:0/3657022101 shutdown_connections 2024-09-21T07:52:28.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 --2- 172.21.15.88:0/3657022101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0b3550 0x7f055c0b3960 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 --2- 172.21.15.88:0/3657022101 >> v2:172.21.15.175:3300/0 conn(0x7f055c0ad0e0 0x7f055c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 -- 172.21.15.88:0/3657022101 >> 172.21.15.88:0/3657022101 conn(0x7f055c06d730 msgr2=0x7f055c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:28.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 -- 172.21.15.88:0/3657022101 shutdown_connections 2024-09-21T07:52:28.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.245+0000 7f0563400640 1 -- 172.21.15.88:0/3657022101 wait complete. 2024-09-21T07:52:28.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.247+0000 7f0563400640 1 Processor -- start 2024-09-21T07:52:28.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.247+0000 7f0563400640 1 -- start start 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0563400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 0x7f055c1440f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0563400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0b3550 0x7f055c09d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0563400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c15d760 0x7f055c156480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0563400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b4da0 con 0x7f055c0ad0e0 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0563400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b4c20 con 0x7f055c15d760 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0563400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f055c0b3ea0 con 0x7f055c0b3550 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0560a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 0x7f055c1440f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0560a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 0x7f055c1440f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41236/0 (socket says 172.21.15.88:41236) 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0560a00640 1 -- 172.21.15.88:0/3936763768 learned_addr learned my addr 172.21.15.88:0/3936763768 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0561400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c15d760 0x7f055c156480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f055be00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0b3550 0x7f055c09d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0560a00640 1 -- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0b3550 msgr2=0x7f055c09d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.249+0000 7f0560a00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0b3550 0x7f055c09d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0560a00640 1 -- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c15d760 msgr2=0x7f055c156480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0560a00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c15d760 0x7f055c156480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0560a00640 1 -- 172.21.15.88:0/3936763768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c09dde0 con 0x7f055c0ad0e0 2024-09-21T07:52:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0560a00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 0x7f055c1440f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f055000b3e0 tx=0x7f055000b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:28.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550015020 con 0x7f055c0ad0e0 2024-09-21T07:52:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0550011940 con 0x7f055c0ad0e0 2024-09-21T07:52:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0550011d90 con 0x7f055c0ad0e0 2024-09-21T07:52:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c1556c0 con 0x7f055c0ad0e0 2024-09-21T07:52:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.250+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c155c80 con 0x7f055c0ad0e0 2024-09-21T07:52:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.251+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f055c0b43e0 con 0x7f055c0ad0e0 2024-09-21T07:52:28.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.252+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f0550018020 con 0x7f055c0ad0e0 2024-09-21T07:52:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.253+0000 7f0558c00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f053c079040 0x7f053c07b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.253+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f055009b5b0 con 0x7f055c0ad0e0 2024-09-21T07:52:28.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.254+0000 7f055be00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f053c079040 0x7f053c07b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.254+0000 7f055be00640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f053c079040 0x7f053c07b550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0548006fd0 tx=0x7f0548008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:28.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.258+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f055001f020 con 0x7f055c0ad0e0 2024-09-21T07:52:28.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.444+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f055c15f4f0 con 0x7f055c0ad0e0 2024-09-21T07:52:28.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.445+0000 7f0558c00640 1 -- 172.21.15.88:0/3936763768 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+16950 (secure 0 0 0) 0x7f0550063240 con 0x7f055c0ad0e0 2024-09-21T07:52:28.447 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:28.447 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":108,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:52:25.849403+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:51:27.552445+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"104","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2024-09-21T07:51:28.544258+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"108","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":108,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"16.4","mappings":[{"from":4,"to":3}]},{"pgid":"17.1","mappings":[{"from":6,"to":1}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:52:28.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f053c079040 msgr2=0x7f053c07b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f053c079040 0x7f053c07b550 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f0548006fd0 tx=0x7f0548008040 comp rx=0 tx=0).stop 2024-09-21T07:52:28.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 msgr2=0x7f055c1440f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 0x7f055c1440f0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f055000b3e0 tx=0x7f055000b8a0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 shutdown_connections 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f053c079040 0x7f053c07b550 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0b3550 0x7f055c09d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c15d760 0x7f055c156480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 --2- 172.21.15.88:0/3936763768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c0ad0e0 0x7f055c1440f0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 >> 172.21.15.88:0/3936763768 conn(0x7f055c06d730 msgr2=0x7f055c15ebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 shutdown_connections 2024-09-21T07:52:28.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.448+0000 7f0563400640 1 -- 172.21.15.88:0/3936763768 wait complete. 2024-09-21T07:52:28.471 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:52:28.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.695+0000 7f744f400640 1 -- 172.21.15.88:0/2979697511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74480705b0 msgr2=0x7f74480709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.695+0000 7f744f400640 1 --2- 172.21.15.88:0/2979697511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74480705b0 0x7f74480709c0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f7434005fd0 tx=0x7f743402a4f0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 -- 172.21.15.88:0/2979697511 shutdown_connections 2024-09-21T07:52:28.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 --2- 172.21.15.88:0/2979697511 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f744815ce30 0x7f744815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 --2- 172.21.15.88:0/2979697511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74480705b0 0x7f74480709c0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 --2- 172.21.15.88:0/2979697511 >> v2:172.21.15.175:3300/0 conn(0x7f74480cfc80 0x7f7448156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 -- 172.21.15.88:0/2979697511 >> 172.21.15.88:0/2979697511 conn(0x7f744806d3d0 msgr2=0x7f74480b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:28.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 -- 172.21.15.88:0/2979697511 shutdown_connections 2024-09-21T07:52:28.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.696+0000 7f744f400640 1 -- 172.21.15.88:0/2979697511 wait complete. 2024-09-21T07:52:28.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.698+0000 7f744f400640 1 Processor -- start 2024-09-21T07:52:28.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.698+0000 7f744f400640 1 -- start start 2024-09-21T07:52:28.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.699+0000 7f744f400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480705b0 0x7f7448073f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.699+0000 7f744f400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74480cfc80 0x7f74480c3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.699+0000 7f744f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744815ce30 0x7f74481f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.699+0000 7f744f400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74480cce40 con 0x7f744815ce30 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.699+0000 7f744f400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74480cccc0 con 0x7f74480cfc80 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.699+0000 7f744f400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74480ccfc0 con 0x7f74480705b0 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744815ce30 0x7f74481f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480705b0 0x7f7448073f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f7447e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74480cfc80 0x7f74480c3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480705b0 0x7f7448073f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:49634/0 (socket says 172.21.15.88:49634) 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744ca00640 1 -- 172.21.15.88:0/3384600872 learned_addr learned my addr 172.21.15.88:0/3384600872 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 -- 172.21.15.88:0/3384600872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480705b0 msgr2=0x7f7448073f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480705b0 0x7f7448073f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 -- 172.21.15.88:0/3384600872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74480cfc80 msgr2=0x7f74480c3fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74480cfc80 0x7f74480c3fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 -- 172.21.15.88:0/3384600872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74481f5af0 con 0x7f744815ce30 2024-09-21T07:52:28.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.700+0000 7f744d400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744815ce30 0x7f74481f55b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f744000ea10 tx=0x7f744000eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:28.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.701+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f744000cde0 con 0x7f744815ce30 2024-09-21T07:52:28.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.701+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7440004c10 con 0x7f744815ce30 2024-09-21T07:52:28.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.701+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7440005020 con 0x7f744815ce30 2024-09-21T07:52:28.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.701+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74480747c0 con 0x7f744815ce30 2024-09-21T07:52:28.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.701+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7448074d80 con 0x7f744815ce30 2024-09-21T07:52:28.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.702+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7448076850 con 0x7f744815ce30 2024-09-21T07:52:28.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.703+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f74400051c0 con 0x7f744815ce30 2024-09-21T07:52:28.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.705+0000 7f7444c00640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7428079040 0x7f742807b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:28.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.705+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 5823+0+0 (secure 0 0 0) 0x7f744009c310 con 0x7f744815ce30 2024-09-21T07:52:28.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.705+0000 7f744ca00640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7428079040 0x7f742807b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:28.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.706+0000 7f744ca00640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7428079040 0x7f742807b550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f743c005f60 tx=0x7f743c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:28.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.709+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7448076850 con 0x7f744815ce30 2024-09-21T07:52:28.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.942+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f74480cd550 con 0x7f744815ce30 2024-09-21T07:52:28.943 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:52:28.943+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:52:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.944+0000 7f7444c00640 1 -- 172.21.15.88:0/3384600872 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) ==== 161+0+0 (secure 0 0 0) 0x7f7440063880 con 0x7f744815ce30 2024-09-21T07:52:28.945 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:52:28.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7428079040 msgr2=0x7f742807b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7428079040 0x7f742807b550 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f743c005f60 tx=0x7f743c005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:28.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744815ce30 msgr2=0x7f74481f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:28.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744815ce30 0x7f74481f55b0 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f744000ea10 tx=0x7f744000eed0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 shutdown_connections 2024-09-21T07:52:28.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7428079040 0x7f742807b550 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.947+0000 7f744f400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f744815ce30 0x7f74481f55b0 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.948+0000 7f744f400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f74480cfc80 0x7f74480c3fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.948+0000 7f744f400640 1 --2- 172.21.15.88:0/3384600872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f74480705b0 0x7f7448073f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:28.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.948+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 >> 172.21.15.88:0/3384600872 conn(0x7f744806d3d0 msgr2=0x7f744806f0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:28.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.948+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 shutdown_connections 2024-09-21T07:52:28.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:28.948+0000 7f744f400640 1 -- 172.21.15.88:0/3384600872 wait complete. 2024-09-21T07:52:28.971 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:52:29.088 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:52:29.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 -- 172.21.15.88:0/65020810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b40d4aa0 msgr2=0x7f45b40cf830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 --2- 172.21.15.88:0/65020810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b40d4aa0 0x7f45b40cf830 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f45ac005fd0 tx=0x7f45ac02a8a0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 -- 172.21.15.88:0/65020810 shutdown_connections 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 --2- 172.21.15.88:0/65020810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b40d4aa0 0x7f45b40cf830 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 --2- 172.21.15.88:0/65020810 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45b4162100 0x7f45b40d4560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 -- 172.21.15.88:0/65020810 >> 172.21.15.88:0/65020810 conn(0x7f45b408dd50 msgr2=0x7f45b409d980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 -- 172.21.15.88:0/65020810 shutdown_connections 2024-09-21T07:52:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.198+0000 7f45ba000640 1 -- 172.21.15.88:0/65020810 wait complete. 2024-09-21T07:52:29.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.200+0000 7f45ba000640 1 Processor -- start 2024-09-21T07:52:29.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.201+0000 7f45ba000640 1 -- start start 2024-09-21T07:52:29.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45ba000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45b40d24f0 0x7f45b40a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45ba000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45b40d4aa0 0x7f45b414f360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45ba000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 0x7f45b40b27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45ba000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45b40cfef0 con 0x7f45b4162100 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45ba000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f45b40cfd70 con 0x7f45b40d24f0 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45ba000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f45b40d0070 con 0x7f45b40d4aa0 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45b9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 0x7f45b40b27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45b9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 0x7f45b40b27d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41284/0 (socket says 172.21.15.88:41284) 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45b9600640 1 -- 172.21.15.88:0/716887416 learned_addr learned my addr 172.21.15.88:0/716887416 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45abe00640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45b40d4aa0 0x7f45b414f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.202+0000 7f45b8c00640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45b40d24f0 0x7f45b40a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b9600640 1 -- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45b40d4aa0 msgr2=0x7f45b414f360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b9600640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45b40d4aa0 0x7f45b414f360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b9600640 1 -- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45b40d24f0 msgr2=0x7f45b40a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b9600640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45b40d24f0 0x7f45b40a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b9600640 1 -- 172.21.15.88:0/716887416 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b4152ad0 con 0x7f45b4162100 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b9600640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 0x7f45b40b27d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009cd0 tx=0x7f45ac032d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac041070 con 0x7f45b4162100 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45ac004450 con 0x7f45b4162100 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45ac031430 con 0x7f45b4162100 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.203+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b414fe80 con 0x7f45b4162100 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.204+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b4150440 con 0x7f45b4162100 2024-09-21T07:52:29.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.205+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b40d0590 con 0x7f45b4162100 2024-09-21T07:52:29.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.206+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f45ac044070 con 0x7f45b4162100 2024-09-21T07:52:29.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.207+0000 7f45b1600640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4590079010 0x7f459007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:29.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.207+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5853+0+0 (secure 0 0 0) 0x7f45ac02e020 con 0x7f45b4162100 2024-09-21T07:52:29.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.207+0000 7f45b8c00640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4590079010 0x7f459007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:29.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.207+0000 7f45b8c00640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4590079010 0x7f459007b520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45a40016f0 tx=0x7f45a4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:29.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.211+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45ac04b030 con 0x7f45b4162100 2024-09-21T07:52:29.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.435+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f45b40cf9e0 con 0x7f45b4162100 2024-09-21T07:52:29.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.951+0000 7f45b1600640 1 -- 172.21.15.88:0/716887416 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) ==== 104+0+0 (secure 0 0 0) 0x7f45ac083c10 con 0x7f45b4162100 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4590079010 msgr2=0x7f459007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4590079010 0x7f459007b520 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f45a40016f0 tx=0x7f45a4009040 comp rx=0 tx=0).stop 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 msgr2=0x7f45b40b27d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 0x7f45b40b27d0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f45ac009cd0 tx=0x7f45ac032d30 comp rx=0 tx=0).stop 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 shutdown_connections 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4590079010 0x7f459007b520 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f45b40d4aa0 0x7f45b414f360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45b4162100 0x7f45b40b27d0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 --2- 172.21.15.88:0/716887416 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f45b40d24f0 0x7f45b40a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 >> 172.21.15.88:0/716887416 conn(0x7f45b408dd50 msgr2=0x7f45b40b3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:29.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.954+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 shutdown_connections 2024-09-21T07:52:29.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:29.955+0000 7f45ba000640 1 -- 172.21.15.88:0/716887416 wait complete. 2024-09-21T07:52:29.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 -- 172.21.15.88:0/662362167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640af100 msgr2=0x7f2f640af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 --2- 172.21.15.88:0/662362167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640af100 0x7f2f640af510 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f2f640d01c0 tx=0x7f2f5c02a530 comp rx=0 tx=0).stop 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 -- 172.21.15.88:0/662362167 shutdown_connections 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 --2- 172.21.15.88:0/662362167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640af100 0x7f2f640af510 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 --2- 172.21.15.88:0/662362167 >> v2:172.21.15.175:3300/0 conn(0x7f2f640cc830 0x7f2f640b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 -- 172.21.15.88:0/662362167 >> 172.21.15.88:0/662362167 conn(0x7f2f640b2540 msgr2=0x7f2f640d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 -- 172.21.15.88:0/662362167 shutdown_connections 2024-09-21T07:52:30.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.207+0000 7f2f69400640 1 -- 172.21.15.88:0/662362167 wait complete. 2024-09-21T07:52:30.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.209+0000 7f2f69400640 1 Processor -- start 2024-09-21T07:52:30.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.209+0000 7f2f69400640 1 -- start start 2024-09-21T07:52:30.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f69400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f6406cda0 0x7f2f641f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f69400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f640af100 0x7f2f641f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f69400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 0x7f2f6415ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f69400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2f640b3620 con 0x7f2f640cc830 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f69400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2f640b34a0 con 0x7f2f6406cda0 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f69400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2f640b37a0 con 0x7f2f640af100 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 0x7f2f6415ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 0x7f2f6415ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41308/0 (socket says 172.21.15.88:41308) 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 -- 172.21.15.88:0/2350688783 learned_addr learned my addr 172.21.15.88:0/2350688783 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f62800640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f6406cda0 0x7f2f641f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f61e00640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f640af100 0x7f2f641f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 -- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f640af100 msgr2=0x7f2f641f5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f640af100 0x7f2f641f5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 -- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f6406cda0 msgr2=0x7f2f641f57a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f6406cda0 0x7f2f641f57a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.211+0000 7f2f63200640 1 -- 172.21.15.88:0/2350688783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f6415de80 con 0x7f2f640cc830 2024-09-21T07:52:30.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.212+0000 7f2f63200640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 0x7f2f6415ef20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c02ab20 tx=0x7f2f5c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.212+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c005c90 con 0x7f2f640cc830 2024-09-21T07:52:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.212+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2f5c031e50 con 0x7f2f640cc830 2024-09-21T07:52:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.212+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f6415e160 con 0x7f2f640cc830 2024-09-21T07:52:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.212+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f5c005280 con 0x7f2f640cc830 2024-09-21T07:52:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.212+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f6415e720 con 0x7f2f640cc830 2024-09-21T07:52:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.213+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f6406a700 con 0x7f2f640cc830 2024-09-21T07:52:30.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.214+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f2f5c005420 con 0x7f2f640cc830 2024-09-21T07:52:30.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.215+0000 7f2f5aa00640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f44079040 0x7f2f4407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:30.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.215+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 5853+0+0 (secure 0 0 0) 0x7f2f5c0bc630 con 0x7f2f640cc830 2024-09-21T07:52:30.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.216+0000 7f2f62800640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f44079040 0x7f2f4407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:30.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.216+0000 7f2f62800640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f44079040 0x7f2f4407b550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005fd0 tx=0x7f2f54009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:30.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.220+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2f6406a700 con 0x7f2f640cc830 2024-09-21T07:52:30.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.406+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2f640d1030 con 0x7f2f640cc830 2024-09-21T07:52:30.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.953+0000 7f2f5aa00640 1 -- 172.21.15.88:0/2350688783 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v111) ==== 181+0+0 (secure 0 0 0) 0x7f2f5c03f030 con 0x7f2f640cc830 2024-09-21T07:52:30.955 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:52:30.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f44079040 msgr2=0x7f2f4407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f44079040 0x7f2f4407b550 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f2f54005fd0 tx=0x7f2f54009500 comp rx=0 tx=0).stop 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 msgr2=0x7f2f6415ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 0x7f2f6415ef20 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f2f5c02ab20 tx=0x7f2f5c033c20 comp rx=0 tx=0).stop 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 shutdown_connections 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2f44079040 0x7f2f4407b550 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2f640af100 0x7f2f641f5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2f640cc830 0x7f2f6415ef20 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 --2- 172.21.15.88:0/2350688783 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2f6406cda0 0x7f2f641f57a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 >> 172.21.15.88:0/2350688783 conn(0x7f2f640b2540 msgr2=0x7f2f640b15b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 shutdown_connections 2024-09-21T07:52:30.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:30.956+0000 7f2f69400640 1 -- 172.21.15.88:0/2350688783 wait complete. 2024-09-21T07:52:30.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:52:31.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 -- 172.21.15.88:0/563818478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91fc0053a0 msgr2=0x7f91fc00ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 --2- 172.21.15.88:0/563818478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91fc0053a0 0x7f91fc00ba50 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f9210005fd0 tx=0x7f91fc005830 comp rx=0 tx=0).stop 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 -- 172.21.15.88:0/563818478 shutdown_connections 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 --2- 172.21.15.88:0/563818478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91fc0053a0 0x7f91fc00ba50 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 -- 172.21.15.88:0/563818478 >> 172.21.15.88:0/563818478 conn(0x7f92180b1dd0 msgr2=0x7f92180b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 -- 172.21.15.88:0/563818478 shutdown_connections 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.207+0000 7f921ca00640 1 -- 172.21.15.88:0/563818478 wait complete. 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.209+0000 7f921ca00640 1 Processor -- start 2024-09-21T07:52:31.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.209+0000 7f921ca00640 1 -- start start 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f921ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91fc0053a0 0x7f9218073680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f921ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f921809b240 0x7f921814fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f921ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 0x7f9218073ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f921ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f92180af750 con 0x7f92181f90d0 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f921ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92180af5d0 con 0x7f91fc0053a0 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f921ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92181532e0 con 0x7f921809b240 2024-09-21T07:52:31.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 0x7f9218073ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:31.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 0x7f9218073ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41328/0 (socket says 172.21.15.88:41328) 2024-09-21T07:52:31.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 -- 172.21.15.88:0/3096026168 learned_addr learned my addr 172.21.15.88:0/3096026168 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 -- 172.21.15.88:0/3096026168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f921809b240 msgr2=0x7f921814fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f921809b240 0x7f921814fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 -- 172.21.15.88:0/3096026168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91fc0053a0 msgr2=0x7f9218073680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91fc0053a0 0x7f9218073680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.211+0000 7f9217200640 1 -- 172.21.15.88:0/3096026168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9218150140 con 0x7f92181f90d0 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.212+0000 7f9217200640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 0x7f9218073ce0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9210005fd0 tx=0x7f9210033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.212+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210038040 con 0x7f92181f90d0 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.212+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f921002b670 con 0x7f92181f90d0 2024-09-21T07:52:31.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.212+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f921809bfe0 con 0x7f92181f90d0 2024-09-21T07:52:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.212+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9210004d40 con 0x7f92181f90d0 2024-09-21T07:52:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.212+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f921809bd00 con 0x7f92181f90d0 2024-09-21T07:52:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.213+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9218153840 con 0x7f92181f90d0 2024-09-21T07:52:31.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.214+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f9210043020 con 0x7f92181f90d0 2024-09-21T07:52:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.215+0000 7f920f400640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f91f4079010 0x7f91f407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.215+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 5386+0+0 (secure 0 0 0) 0x7f92100bb540 con 0x7f92181f90d0 2024-09-21T07:52:31.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.216+0000 7f9216800640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f91f4079010 0x7f91f407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:31.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.216+0000 7f9216800640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f91f4079010 0x7f91f407b520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9208000f60 tx=0x7f9208001110 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:31.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.220+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9218150140 con 0x7f92181f90d0 2024-09-21T07:52:31.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.405+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f92180b19d0 con 0x7f92181f90d0 2024-09-21T07:52:31.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.960+0000 7f920f400640 1 -- 172.21.15.88:0/3096026168 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v112) ==== 169+0+0 (secure 0 0 0) 0x7f9210083470 con 0x7f92181f90d0 2024-09-21T07:52:31.961 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.963+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f91f4079010 msgr2=0x7f91f407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.963+0000 7f921ca00640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f91f4079010 0x7f91f407b520 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f9208000f60 tx=0x7f9208001110 comp rx=0 tx=0).stop 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.963+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 msgr2=0x7f9218073ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.963+0000 7f921ca00640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 0x7f9218073ce0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f9210005fd0 tx=0x7f9210033670 comp rx=0 tx=0).stop 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.964+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 shutdown_connections 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.964+0000 7f921ca00640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f91f4079010 0x7f91f407b520 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.964+0000 7f921ca00640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f921809b240 0x7f921814fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.964+0000 7f921ca00640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92181f90d0 0x7f9218073ce0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.964+0000 7f921ca00640 1 --2- 172.21.15.88:0/3096026168 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f91fc0053a0 0x7f9218073680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.964+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 >> 172.21.15.88:0/3096026168 conn(0x7f92180b1dd0 msgr2=0x7f9218154690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.965+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 shutdown_connections 2024-09-21T07:52:31.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:31.965+0000 7f921ca00640 1 -- 172.21.15.88:0/3096026168 wait complete. 2024-09-21T07:52:31.995 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:52:32.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 -- 172.21.15.88:0/1249394153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27815a0f0 msgr2=0x7ff27815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 --2- 172.21.15.88:0/1249394153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27815a0f0 0x7ff27815d670 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7ff270005fd0 tx=0x7ff27002a820 comp rx=0 tx=0).stop 2024-09-21T07:52:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 -- 172.21.15.88:0/1249394153 shutdown_connections 2024-09-21T07:52:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 --2- 172.21.15.88:0/1249394153 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27815a0f0 0x7ff27815d670 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 --2- 172.21.15.88:0/1249394153 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff2780ad9c0 0x7ff2780af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 -- 172.21.15.88:0/1249394153 >> 172.21.15.88:0/1249394153 conn(0x7ff2780b1e40 msgr2=0x7ff2780b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 -- 172.21.15.88:0/1249394153 shutdown_connections 2024-09-21T07:52:32.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.217+0000 7ff27d200640 1 -- 172.21.15.88:0/1249394153 wait complete. 2024-09-21T07:52:32.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.219+0000 7ff27d200640 1 Processor -- start 2024-09-21T07:52:32.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.219+0000 7ff27d200640 1 -- start start 2024-09-21T07:52:32.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff27d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2780ad9c0 0x7ff278159070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff27d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 0x7ff2781f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff27d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27815a0f0 0x7ff2781f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff27d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff27806f410 con 0x7ff2780b2f60 2024-09-21T07:52:32.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff27d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff27806f290 con 0x7ff2780ad9c0 2024-09-21T07:52:32.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff27d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff27806f590 con 0x7ff27815a0f0 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 0x7ff2781f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 0x7ff2781f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41338/0 (socket says 172.21.15.88:41338) 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 -- 172.21.15.88:0/2559963307 learned_addr learned my addr 172.21.15.88:0/2559963307 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff277e00640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27815a0f0 0x7ff2781f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 -- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27815a0f0 msgr2=0x7ff2781f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27815a0f0 0x7ff2781f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 -- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2780ad9c0 msgr2=0x7ff278159070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2780ad9c0 0x7ff278159070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.221+0000 7ff26ea00640 1 -- 172.21.15.88:0/2559963307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2780cd9c0 con 0x7ff2780b2f60 2024-09-21T07:52:32.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.222+0000 7ff26ea00640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 0x7ff2781f8e10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff26000c400 tx=0x7ff26000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:32.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.222+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260011790 con 0x7ff2780b2f60 2024-09-21T07:52:32.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.222+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2780cdca0 con 0x7ff2780b2f60 2024-09-21T07:52:32.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.222+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff260017070 con 0x7ff2780b2f60 2024-09-21T07:52:32.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.222+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff260011cc0 con 0x7ff2780b2f60 2024-09-21T07:52:32.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.222+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2780ce260 con 0x7ff2780b2f60 2024-09-21T07:52:32.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.223+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2781597c0 con 0x7ff2780b2f60 2024-09-21T07:52:32.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.224+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7ff26001c460 con 0x7ff2780b2f60 2024-09-21T07:52:32.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.225+0000 7ff26fe00640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff254079010 0x7ff25407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.225+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7ff26009afb0 con 0x7ff2780b2f60 2024-09-21T07:52:32.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.226+0000 7ff277400640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff254079010 0x7ff25407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.226+0000 7ff277400640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff254079010 0x7ff25407b520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff268005d00 tx=0x7ff268005c50 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:32.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.230+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff260063120 con 0x7ff2780b2f60 2024-09-21T07:52:32.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.425+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7ff2780cea60 con 0x7ff2780b2f60 2024-09-21T07:52:32.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.429+0000 7ff26fe00640 1 -- 172.21.15.88:0/2559963307 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v43) ==== 71+0+0 (secure 0 0 0) 0x7ff2780cea60 con 0x7ff2780b2f60 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff254079010 msgr2=0x7ff25407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff254079010 0x7ff25407b520 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff268005d00 tx=0x7ff268005c50 comp rx=0 tx=0).stop 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 msgr2=0x7ff2781f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 0x7ff2781f8e10 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7ff26000c400 tx=0x7ff26000c8c0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 shutdown_connections 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff254079010 0x7ff25407b520 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27815a0f0 0x7ff2781f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2780b2f60 0x7ff2781f8e10 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 --2- 172.21.15.88:0/2559963307 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2780ad9c0 0x7ff278159070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 >> 172.21.15.88:0/2559963307 conn(0x7ff2780b1e40 msgr2=0x7ff278074220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 shutdown_connections 2024-09-21T07:52:32.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.432+0000 7ff27d200640 1 -- 172.21.15.88:0/2559963307 wait complete. 2024-09-21T07:52:32.456 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity' 2024-09-21T07:52:32.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 -- 172.21.15.88:0/2113429333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e4015fae0 msgr2=0x7f6e40166550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 --2- 172.21.15.88:0/2113429333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e4015fae0 0x7f6e40166550 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f6e38005fd0 tx=0x7f6e3802a820 comp rx=0 tx=0).stop 2024-09-21T07:52:32.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 -- 172.21.15.88:0/2113429333 shutdown_connections 2024-09-21T07:52:32.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 --2- 172.21.15.88:0/2113429333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e4015fae0 0x7f6e40166550 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 -- 172.21.15.88:0/2113429333 >> 172.21.15.88:0/2113429333 conn(0x7f6e4006d730 msgr2=0x7f6e400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:32.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 -- 172.21.15.88:0/2113429333 shutdown_connections 2024-09-21T07:52:32.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.675+0000 7f6e48200640 1 -- 172.21.15.88:0/2113429333 wait complete. 2024-09-21T07:52:32.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.677+0000 7f6e48200640 1 Processor -- start 2024-09-21T07:52:32.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.677+0000 7f6e48200640 1 -- start start 2024-09-21T07:52:32.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.678+0000 7f6e48200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e400aebc0 0x7f6e4009c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.678+0000 7f6e48200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e40152920 0x7f6e40150c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.678+0000 7f6e48200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 0x7f6e4009adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.678+0000 7f6e48200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6e40166ca0 con 0x7f6e401511a0 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.678+0000 7f6e48200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6e400b4a70 con 0x7f6e400aebc0 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.678+0000 7f6e48200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6e40166e20 con 0x7f6e40152920 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 0x7f6e4009adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 0x7f6e4009adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41360/0 (socket says 172.21.15.88:41360) 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 -- 172.21.15.88:0/3359238317 learned_addr learned my addr 172.21.15.88:0/3359238317 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e45800640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e400aebc0 0x7f6e4009c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e44e00640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e40152920 0x7f6e40150c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 -- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e40152920 msgr2=0x7f6e40150c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e40152920 0x7f6e40150c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 -- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e400aebc0 msgr2=0x7f6e4009c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e400aebc0 0x7f6e4009c970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e46200640 1 -- 172.21.15.88:0/3359238317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e401515b0 con 0x7f6e401511a0 2024-09-21T07:52:32.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.679+0000 7f6e45800640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e400aebc0 0x7f6e4009c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:32.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.680+0000 7f6e46200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 0x7f6e4009adc0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6e38035040 tx=0x7f6e38032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:32.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.680+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38037040 con 0x7f6e401511a0 2024-09-21T07:52:32.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.680+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6e38042070 con 0x7f6e401511a0 2024-09-21T07:52:32.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.680+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e401547f0 con 0x7f6e401511a0 2024-09-21T07:52:32.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.680+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e38003e60 con 0x7f6e401511a0 2024-09-21T07:52:32.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.680+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e40154db0 con 0x7f6e401511a0 2024-09-21T07:52:32.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.681+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e4009d1a0 con 0x7f6e401511a0 2024-09-21T07:52:32.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.682+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f6e38004260 con 0x7f6e401511a0 2024-09-21T07:52:32.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.683+0000 7f6e3d600640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6e20079040 0x7f6e2007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:32.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.683+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7f6e380bc020 con 0x7f6e401511a0 2024-09-21T07:52:32.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.683+0000 7f6e45800640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6e20079040 0x7f6e2007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:32.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.684+0000 7f6e45800640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6e20079040 0x7f6e2007b550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e34006fd0 tx=0x7f6e34008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:32.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.688+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6e401515b0 con 0x7f6e401511a0 2024-09-21T07:52:32.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.943+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]} v 0) -- 0x7f6e400b2750 con 0x7f6e401511a0 2024-09-21T07:52:32.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.962+0000 7f6e3d600640 1 -- 172.21.15.88:0/3359238317 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_sanity"]}]=0 v361) ==== 149+0+0 (secure 0 0 0) 0x7f6e38084150 con 0x7f6e401511a0 2024-09-21T07:52:32.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.964+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6e20079040 msgr2=0x7f6e2007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.964+0000 7f6e48200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6e20079040 0x7f6e2007b550 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6e34006fd0 tx=0x7f6e34008040 comp rx=0 tx=0).stop 2024-09-21T07:52:32.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.964+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 msgr2=0x7f6e4009adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:32.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.964+0000 7f6e48200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 0x7f6e4009adc0 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7f6e38035040 tx=0x7f6e38032d10 comp rx=0 tx=0).stop 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 shutdown_connections 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6e20079040 0x7f6e2007b550 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6e40152920 0x7f6e40150c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e401511a0 0x7f6e4009adc0 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 --2- 172.21.15.88:0/3359238317 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6e400aebc0 0x7f6e4009c970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 >> 172.21.15.88:0/3359238317 conn(0x7f6e4006d730 msgr2=0x7f6e4015cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 shutdown_connections 2024-09-21T07:52:32.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:32.965+0000 7f6e48200640 1 -- 172.21.15.88:0/3359238317 wait complete. 2024-09-21T07:52:32.993 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-21T07:52:32.993 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-21T07:52:32.993 INFO:tasks.cephfs_test_runner:test_subvolume_no_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:52:32.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:52:33.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 -- 172.21.15.88:0/1374105506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 msgr2=0x7f57c81651b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 --2- 172.21.15.88:0/1374105506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81651b0 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7f57c80adbc0 tx=0x7f57c002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 -- 172.21.15.88:0/1374105506 shutdown_connections 2024-09-21T07:52:33.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 --2- 172.21.15.88:0/1374105506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81651b0 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 --2- 172.21.15.88:0/1374105506 >> v2:172.21.15.175:3300/0 conn(0x7f57c80ae080 0x7f57c8070000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 -- 172.21.15.88:0/1374105506 >> 172.21.15.88:0/1374105506 conn(0x7f57c80b1e40 msgr2=0x7f57c80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 -- 172.21.15.88:0/1374105506 shutdown_connections 2024-09-21T07:52:33.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.213+0000 7f57c7200640 1 -- 172.21.15.88:0/1374105506 wait complete. 2024-09-21T07:52:33.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.215+0000 7f57c7200640 1 Processor -- start 2024-09-21T07:52:33.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.215+0000 7f57c7200640 1 -- start start 2024-09-21T07:52:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81492b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80ae080 0x7f57c8152cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80af050 0x7f57c81531f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57c8165870 con 0x7f57c8070540 2024-09-21T07:52:33.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57c81656f0 con 0x7f57c80af050 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57c81659f0 con 0x7f57c80ae080 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81492b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81492b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41378/0 (socket says 172.21.15.88:41378) 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 -- 172.21.15.88:0/2319042949 learned_addr learned my addr 172.21.15.88:0/2319042949 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c6800640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80af050 0x7f57c81531f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5400640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80ae080 0x7f57c8152cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 -- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80ae080 msgr2=0x7f57c8152cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80ae080 0x7f57c8152cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 -- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80af050 msgr2=0x7f57c81531f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80af050 0x7f57c81531f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5e00640 1 -- 172.21.15.88:0/2319042949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57c814b7a0 con 0x7f57c8070540 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.217+0000 7f57c5400640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80ae080 0x7f57c8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.218+0000 7f57c5e00640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81492b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c3a0 tx=0x7f57b800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.218+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8011790 con 0x7f57c8070540 2024-09-21T07:52:33.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.218+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57b8017070 con 0x7f57c8070540 2024-09-21T07:52:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.218+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57b8011cc0 con 0x7f57c8070540 2024-09-21T07:52:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.218+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57c8163c70 con 0x7f57c8070540 2024-09-21T07:52:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.218+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57c8163f10 con 0x7f57c8070540 2024-09-21T07:52:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.219+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57c8070000 con 0x7f57c8070540 2024-09-21T07:52:33.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.220+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f57b801c460 con 0x7f57c8070540 2024-09-21T07:52:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.222+0000 7f57be000640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57a8079010 0x7f57a807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.222+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7f57b809b090 con 0x7f57c8070540 2024-09-21T07:52:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.222+0000 7f57c5400640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57a8079010 0x7f57a807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.223+0000 7f57c5400640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57a8079010 0x7f57a807b520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57c80733f0 tx=0x7f57b4005ef0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.226+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57b80630f0 con 0x7f57c8070540 2024-09-21T07:52:33.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.412+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f57c8069800 con 0x7f57c8070540 2024-09-21T07:52:33.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.413+0000 7f57be000640 1 -- 172.21.15.88:0/2319042949 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v112) ==== 74+0+13054 (secure 0 0 0) 0x7f57b80629f0 con 0x7f57c8070540 2024-09-21T07:52:33.414 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:33.414 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":112,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:52:31.954401+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":106,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":103,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/3744288878":"2024-09-22T07:52:28.943183+0000","172.21.15.88:6832/3744288878":"2024-09-22T07:52:28.943183+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57a8079010 msgr2=0x7f57a807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57a8079010 0x7f57a807b520 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f57c80733f0 tx=0x7f57b4005ef0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 msgr2=0x7f57c81492b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81492b0 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7f57b800c3a0 tx=0x7f57b800c860 comp rx=0 tx=0).stop 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 shutdown_connections 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57a8079010 0x7f57a807b520 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57c80ae080 0x7f57c8152cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57c80af050 0x7f57c81531f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 --2- 172.21.15.88:0/2319042949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57c8070540 0x7f57c81492b0 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 >> 172.21.15.88:0/2319042949 conn(0x7f57c80b1e40 msgr2=0x7f57c8164ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 shutdown_connections 2024-09-21T07:52:33.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.416+0000 7f57c7200640 1 -- 172.21.15.88:0/2319042949 wait complete. 2024-09-21T07:52:33.439 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:52:33.439 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 -- 172.21.15.88:0/1109093570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780d3d10 msgr2=0x7f60780b9680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 --2- 172.21.15.88:0/1109093570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780d3d10 0x7f60780b9680 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f60780cf9a0 tx=0x7f606402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 -- 172.21.15.88:0/1109093570 shutdown_connections 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 --2- 172.21.15.88:0/1109093570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780d3d10 0x7f60780b9680 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 -- 172.21.15.88:0/1109093570 >> 172.21.15.88:0/1109093570 conn(0x7f607809d8e0 msgr2=0x7f607808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 -- 172.21.15.88:0/1109093570 shutdown_connections 2024-09-21T07:52:33.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.660+0000 7f6077200640 1 -- 172.21.15.88:0/1109093570 wait complete. 2024-09-21T07:52:33.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.663+0000 7f6077200640 1 Processor -- start 2024-09-21T07:52:33.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.663+0000 7f6077200640 1 -- start start 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6077200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60780d0c80 0x7f607815bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6077200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60780d3d10 0x7f60780cc960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6077200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 0x7f607815eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6077200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60780d2e40 con 0x7f60780ccea0 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6077200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f60780d2cc0 con 0x7f60780d0c80 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6077200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f60780d2fc0 con 0x7f60780d3d10 2024-09-21T07:52:33.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6076800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 0x7f607815eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6076800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 0x7f607815eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41390/0 (socket says 172.21.15.88:41390) 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.664+0000 7f6076800640 1 -- 172.21.15.88:0/987972863 learned_addr learned my addr 172.21.15.88:0/987972863 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6075e00640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60780d0c80 0x7f607815bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6075400640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60780d3d10 0x7f60780cc960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6076800640 1 -- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60780d3d10 msgr2=0x7f60780cc960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6076800640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60780d3d10 0x7f60780cc960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6076800640 1 -- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60780d0c80 msgr2=0x7f607815bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6076800640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60780d0c80 0x7f607815bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6076800640 1 -- 172.21.15.88:0/987972863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f607815e270 con 0x7f60780ccea0 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6075e00640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60780d0c80 0x7f607815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:33.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6076800640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 0x7f607815eb80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f607000cb10 tx=0x7f607000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070016020 con 0x7f60780ccea0 2024-09-21T07:52:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f607000ad80 con 0x7f60780ccea0 2024-09-21T07:52:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.665+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60780cdc30 con 0x7f60780ccea0 2024-09-21T07:52:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.666+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6070012680 con 0x7f60780ccea0 2024-09-21T07:52:33.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.666+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60780cd950 con 0x7f60780ccea0 2024-09-21T07:52:33.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.667+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60780d1890 con 0x7f60780ccea0 2024-09-21T07:52:33.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.668+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f6070019020 con 0x7f60780ccea0 2024-09-21T07:52:33.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.669+0000 7f606e000640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6058079010 0x7f605807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:33.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.669+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7f607009b9b0 con 0x7f60780ccea0 2024-09-21T07:52:33.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.669+0000 7f6075e00640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6058079010 0x7f605807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:33.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.670+0000 7f6075e00640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6058079010 0x7f605807b520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6068001670 tx=0x7f6068009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:33.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.673+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6070020020 con 0x7f60780ccea0 2024-09-21T07:52:33.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.863+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f6078069190 con 0x7f60780ccea0 2024-09-21T07:52:33.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.865+0000 7f606e000640 1 -- 172.21.15.88:0/987972863 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v112) ==== 93+0+10 (secure 0 0 0) 0x7f60780cdc30 con 0x7f60780ccea0 2024-09-21T07:52:33.866 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:52:33.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6058079010 msgr2=0x7f605807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6058079010 0x7f605807b520 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f6068001670 tx=0x7f6068009040 comp rx=0 tx=0).stop 2024-09-21T07:52:33.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 msgr2=0x7f607815eb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:33.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 0x7f607815eb80 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f607000cb10 tx=0x7f607000a060 comp rx=0 tx=0).stop 2024-09-21T07:52:33.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 shutdown_connections 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6058079010 0x7f605807b520 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f60780d3d10 0x7f60780cc960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60780ccea0 0x7f607815eb80 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 --2- 172.21.15.88:0/987972863 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f60780d0c80 0x7f607815bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 >> 172.21.15.88:0/987972863 conn(0x7f607809d8e0 msgr2=0x7f60780b1910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 shutdown_connections 2024-09-21T07:52:33.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:33.868+0000 7f6077200640 1 -- 172.21.15.88:0/987972863 wait complete. 2024-09-21T07:52:33.893 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.116+0000 7f59ffe00640 1 -- 172.21.15.88:0/1706547350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000d29e0 msgr2=0x7f5a000d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.116+0000 7f59ffe00640 1 --2- 172.21.15.88:0/1706547350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000d29e0 0x7f5a000d1580 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f5a0008d440 tx=0x7f59f802a980 comp rx=0 tx=0).stop 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 -- 172.21.15.88:0/1706547350 shutdown_connections 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 --2- 172.21.15.88:0/1706547350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000d29e0 0x7f5a000d1580 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 --2- 172.21.15.88:0/1706547350 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 0x7f5a000d24a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 --2- 172.21.15.88:0/1706547350 >> v2:172.21.15.175:3300/0 conn(0x7f5a000b9e00 0x7f5a00162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 -- 172.21.15.88:0/1706547350 >> 172.21.15.88:0/1706547350 conn(0x7f5a0008bfe0 msgr2=0x7f5a0009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:34.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 -- 172.21.15.88:0/1706547350 shutdown_connections 2024-09-21T07:52:34.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.117+0000 7f59ffe00640 1 -- 172.21.15.88:0/1706547350 wait complete. 2024-09-21T07:52:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.119+0000 7f59ffe00640 1 Processor -- start 2024-09-21T07:52:34.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.119+0000 7f59ffe00640 1 -- start start 2024-09-21T07:52:34.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.120+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 0x7f5a000cd980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:34.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000b9e00 0x7f5a0009f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59ffe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a000d29e0 0x7f5a000a17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59ffe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000d4330 con 0x7f5a000b9e00 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59ffe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5a000d1e10 con 0x7f5a000d29e0 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59ffe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5a000d44b0 con 0x7f5a000b3160 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59fea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 0x7f5a000cd980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59fe000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000b9e00 0x7f5a0009f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59fea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 0x7f5a000cd980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3301/0 says I am v2:172.21.15.88:49726/0 (socket says 172.21.15.88:49726) 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59fea00640 1 -- 172.21.15.88:0/4184571360 learned_addr learned my addr 172.21.15.88:0/4184571360 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.121+0000 7f59ff400640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a000d29e0 0x7f5a000a17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59fe000640 1 -- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 msgr2=0x7f5a000cd980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59fe000640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 0x7f5a000cd980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59fe000640 1 -- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a000d29e0 msgr2=0x7f5a000a17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59fe000640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a000d29e0 0x7f5a000a17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59fe000640 1 -- 172.21.15.88:0/4184571360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a000a00a0 con 0x7f5a000b9e00 2024-09-21T07:52:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59fe000640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000b9e00 0x7f5a0009f370 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f59f000d680 tx=0x7f59f000db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0017070 con 0x7f5a000b9e00 2024-09-21T07:52:34.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f59f0005310 con 0x7f5a000b9e00 2024-09-21T07:52:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a000a0380 con 0x7f5a000b9e00 2024-09-21T07:52:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f00097c0 con 0x7f5a000b9e00 2024-09-21T07:52:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.122+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a000a0940 con 0x7f5a000b9e00 2024-09-21T07:52:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.123+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a0009fe10 con 0x7f5a000b9e00 2024-09-21T07:52:34.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.125+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f59f0009960 con 0x7f5a000b9e00 2024-09-21T07:52:34.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.125+0000 7f59f6a00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59dc079010 0x7f59dc07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:34.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.126+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7f59f009b380 con 0x7f5a000b9e00 2024-09-21T07:52:34.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.126+0000 7f59fea00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59dc079010 0x7f59dc07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:34.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.126+0000 7f59fea00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59dc079010 0x7f59dc07b520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f59e80058c0 tx=0x7f59e800b410 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:34.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.130+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f59f0062c60 con 0x7f5a000b9e00 2024-09-21T07:52:34.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.389+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f5a000a0d20 con 0x7f5a000b9e00 2024-09-21T07:52:34.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.979+0000 7f59f6a00640 1 -- 172.21.15.88:0/4184571360 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v363) ==== 151+0+0 (secure 0 0 0) 0x7f5a000a0d20 con 0x7f5a000b9e00 2024-09-21T07:52:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59dc079010 msgr2=0x7f59dc07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59dc079010 0x7f59dc07b520 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f59e80058c0 tx=0x7f59e800b410 comp rx=0 tx=0).stop 2024-09-21T07:52:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000b9e00 msgr2=0x7f5a0009f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:34.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000b9e00 0x7f5a0009f370 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f59f000d680 tx=0x7f59f000db40 comp rx=0 tx=0).stop 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 shutdown_connections 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59dc079010 0x7f59dc07b520 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5a000d29e0 0x7f5a000a17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a000b9e00 0x7f5a0009f370 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 --2- 172.21.15.88:0/4184571360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5a000b3160 0x7f5a000cd980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 >> 172.21.15.88:0/4184571360 conn(0x7f5a0008bfe0 msgr2=0x7f5a0006f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 shutdown_connections 2024-09-21T07:52:34.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:34.982+0000 7f59ffe00640 1 -- 172.21.15.88:0/4184571360 wait complete. 2024-09-21T07:52:35.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:52:35.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea14a00640 1 -- 172.21.15.88:0/3357113364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9f4003c10 msgr2=0x7fe9f4004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea14a00640 1 --2- 172.21.15.88:0/3357113364 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe9f4003c10 0x7fe9f4004060 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7fea00001830 tx=0x7fe9f4005830 comp rx=0 tx=0).stop 2024-09-21T07:52:35.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea0e800640 1 -- 172.21.15.88:0/3357113364 reap_dead start 2024-09-21T07:52:35.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea14a00640 1 -- 172.21.15.88:0/3357113364 shutdown_connections 2024-09-21T07:52:35.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea14a00640 1 -- 172.21.15.88:0/3357113364 >> 172.21.15.88:0/3357113364 conn(0x7fea1006d730 msgr2=0x7fea100b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:35.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea14a00640 1 -- 172.21.15.88:0/3357113364 shutdown_connections 2024-09-21T07:52:35.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.228+0000 7fea14a00640 1 -- 172.21.15.88:0/3357113364 wait complete. 2024-09-21T07:52:35.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.230+0000 7fea14a00640 1 Processor -- start 2024-09-21T07:52:35.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.231+0000 7fea14a00640 1 -- start start 2024-09-21T07:52:35.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea14a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9f4003c10 0x7fea101f9a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea14a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 0x7fea1015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea14a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea10148eb0 0x7fea1015bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea14a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea100b4da0 con 0x7fea10148aa0 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea14a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fea100b4c20 con 0x7fea10148eb0 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea14a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fea100ad4f0 con 0x7fe9f4003c10 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea05400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 0x7fea1015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea05400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 0x7fea1015b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41432/0 (socket says 172.21.15.88:41432) 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.232+0000 7fea05400640 1 -- 172.21.15.88:0/2189257272 learned_addr learned my addr 172.21.15.88:0/2189257272 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea0e800640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea10148eb0 0x7fea1015bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea0de00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9f4003c10 0x7fea101f9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea05400640 1 -- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9f4003c10 msgr2=0x7fea101f9a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea05400640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9f4003c10 0x7fea101f9a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea05400640 1 -- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea10148eb0 msgr2=0x7fea1015bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea05400640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea10148eb0 0x7fea1015bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea05400640 1 -- 172.21.15.88:0/2189257272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea10156d50 con 0x7fea10148aa0 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea05400640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 0x7fea1015b7f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8011ab0 tx=0x7fe9f8011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f801e070 con 0x7fea10148aa0 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe9f800fc70 con 0x7fea10148aa0 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea101496b0 con 0x7fea10148aa0 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9f8019670 con 0x7fea10148aa0 2024-09-21T07:52:35.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.233+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea10149c70 con 0x7fea10148aa0 2024-09-21T07:52:35.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.235+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea101535c0 con 0x7fea10148aa0 2024-09-21T07:52:35.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.236+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fe9f8021070 con 0x7fea10148aa0 2024-09-21T07:52:35.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.237+0000 7fea06a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9ec079010 0x7fe9ec07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.237+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7fe9f809bbc0 con 0x7fea10148aa0 2024-09-21T07:52:35.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.237+0000 7fea0de00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9ec079010 0x7fe9ec07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.238+0000 7fea0de00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9ec079010 0x7fe9ec07b520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea00032000 tx=0x7fea00001360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:35.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.241+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe9f800fe60 con 0x7fea10148aa0 2024-09-21T07:52:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.427+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fea101f9f70 con 0x7fea10148aa0 2024-09-21T07:52:35.428 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.427+0000 7fea06a00640 1 -- 172.21.15.88:0/2189257272 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v43) ==== 59+0+2294 (secure 0 0 0) 0x7fe9f80634e0 con 0x7fea10148aa0 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:52:35.429 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:52:35.430 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:52:35.431 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9ec079010 msgr2=0x7fe9ec07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9ec079010 0x7fe9ec07b520 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fea00032000 tx=0x7fea00001360 comp rx=0 tx=0).stop 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 msgr2=0x7fea1015b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 0x7fea1015b7f0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7fe9f8011ab0 tx=0x7fe9f8011f70 comp rx=0 tx=0).stop 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 shutdown_connections 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe9ec079010 0x7fe9ec07b520 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fea10148eb0 0x7fea1015bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea10148aa0 0x7fea1015b7f0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 --2- 172.21.15.88:0/2189257272 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9f4003c10 0x7fea101f9a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.432 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 >> 172.21.15.88:0/2189257272 conn(0x7fea1006d730 msgr2=0x7fea1015e760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 shutdown_connections 2024-09-21T07:52:35.433 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.430+0000 7fea14a00640 1 -- 172.21.15.88:0/2189257272 wait complete. 2024-09-21T07:52:35.453 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.677+0000 7f6bcf400640 1 -- 172.21.15.88:0/115445744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145a40 msgr2=0x7f6bc8073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.677+0000 7f6bcf400640 1 --2- 172.21.15.88:0/115445744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145a40 0x7f6bc8073540 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f6bb4005fd0 tx=0x7f6bb402a660 comp rx=0 tx=0).stop 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.677+0000 7f6bcf400640 1 -- 172.21.15.88:0/115445744 shutdown_connections 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.677+0000 7f6bcf400640 1 --2- 172.21.15.88:0/115445744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145a40 0x7f6bc8073540 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.677+0000 7f6bcf400640 1 --2- 172.21.15.88:0/115445744 >> v2:172.21.15.175:3300/0 conn(0x7f6bc8145550 0x7f6bc806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.677+0000 7f6bcf400640 1 -- 172.21.15.88:0/115445744 >> 172.21.15.88:0/115445744 conn(0x7f6bc806d1c0 msgr2=0x7f6bc806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.678+0000 7f6bcf400640 1 -- 172.21.15.88:0/115445744 shutdown_connections 2024-09-21T07:52:35.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.678+0000 7f6bcf400640 1 -- 172.21.15.88:0/115445744 wait complete. 2024-09-21T07:52:35.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.680+0000 7f6bcf400640 1 Processor -- start 2024-09-21T07:52:35.681 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.680+0000 7f6bcf400640 1 -- start start 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.681+0000 7f6bcf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bc8073a80 0x7f6bc813e7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.681+0000 7f6bcf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145550 0x7f6bc80c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.681+0000 7f6bcf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 0x7f6bc815ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.681+0000 7f6bcf400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc8160a30 con 0x7f6bc8145a40 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.681+0000 7f6bcf400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc80ce940 con 0x7f6bc8073a80 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.681+0000 7f6bcf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6bc8160bb0 con 0x7f6bc8145550 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 0x7f6bc815ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 0x7f6bc815ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:41436/0 (socket says 172.21.15.88:41436) 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 -- 172.21.15.88:0/536075599 learned_addr learned my addr 172.21.15.88:0/536075599 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bc7e00640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145550 0x7f6bc80c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.683 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcca00640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bc8073a80 0x7f6bc813e7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 -- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145550 msgr2=0x7f6bc80c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145550 0x7f6bc80c53f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.684 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 -- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bc8073a80 msgr2=0x7f6bc813e7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bc8073a80 0x7f6bc813e7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 -- 172.21.15.88:0/536075599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc81f95c0 con 0x7f6bc8145a40 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.682+0000 7f6bcd400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 0x7f6bc815ef30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000c730 tx=0x7f6bc000cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.683+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0003b70 con 0x7f6bc8145a40 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.683+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6bc0003d10 con 0x7f6bc8145a40 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.683+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc81f98a0 con 0x7f6bc8145a40 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.683+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bc0005330 con 0x7f6bc8145a40 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.683+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc81f9e60 con 0x7f6bc8145a40 2024-09-21T07:52:35.685 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.684+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc81610d0 con 0x7f6bc8145a40 2024-09-21T07:52:35.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.685+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f6bc00054d0 con 0x7f6bc8145a40 2024-09-21T07:52:35.686 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.686+0000 7f6bc4c00640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ba8079010 0x7f6ba807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:35.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.686+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7f6bc009afb0 con 0x7f6bc8145a40 2024-09-21T07:52:35.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.686+0000 7f6bcca00640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ba8079010 0x7f6ba807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:35.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.687+0000 7f6bcca00640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ba8079010 0x7f6ba807b520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009b30 tx=0x7f6bbc005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:35.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.690+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6bc8073e90 con 0x7f6bc8145a40 2024-09-21T07:52:35.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.880+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f6bc80b3020 con 0x7f6bc8145a40 2024-09-21T07:52:35.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.884+0000 7f6bc4c00640 1 -- 172.21.15.88:0/536075599 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f6bc0062940 con 0x7f6bc8145a40 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.886+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ba8079010 msgr2=0x7f6ba807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.886+0000 7f6bcf400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ba8079010 0x7f6ba807b520 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc009b30 tx=0x7f6bbc005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.886+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 msgr2=0x7f6bc815ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.886+0000 7f6bcf400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 0x7f6bc815ef30 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000c730 tx=0x7f6bc000cb30 comp rx=0 tx=0).stop 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 shutdown_connections 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ba8079010 0x7f6ba807b520 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6bc8145550 0x7f6bc80c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8145a40 0x7f6bc815ef30 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 --2- 172.21.15.88:0/536075599 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6bc8073a80 0x7f6bc813e7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:35.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 >> 172.21.15.88:0/536075599 conn(0x7f6bc806d1c0 msgr2=0x7f6bc8143440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 shutdown_connections 2024-09-21T07:52:35.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:35.887+0000 7f6bcf400640 1 -- 172.21.15.88:0/536075599 wait complete. 2024-09-21T07:52:35.912 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:52:35.912 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:52:36.009 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:52:36.111 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:52:36.111 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:36.111 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:36.147 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:52:36.147 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:52:36.147 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:36.184 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:52:36.184 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:36.184 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:36.222 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:52:36.222 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:52:36.222 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:36.260 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:52:36.261 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:52:36.261 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1a00640 1 -- 172.21.15.88:0/4263645551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2c0003a30 msgr2=0x7fb2c0003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/4263645551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2c0003a30 0x7fb2c0003ea0 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc009b50 tx=0x7fb2c0005830 comp rx=0 tx=0).stop 2024-09-21T07:52:36.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1000640 1 -- 172.21.15.88:0/4263645551 reap_dead start 2024-09-21T07:52:36.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1a00640 1 -- 172.21.15.88:0/4263645551 shutdown_connections 2024-09-21T07:52:36.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1a00640 1 -- 172.21.15.88:0/4263645551 >> 172.21.15.88:0/4263645551 conn(0x7fb2dc0b1e40 msgr2=0x7fb2dc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:36.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1a00640 1 -- 172.21.15.88:0/4263645551 shutdown_connections 2024-09-21T07:52:36.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.481+0000 7fb2e1a00640 1 -- 172.21.15.88:0/4263645551 wait complete. 2024-09-21T07:52:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.483+0000 7fb2e1a00640 1 Processor -- start 2024-09-21T07:52:36.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.483+0000 7fb2e1a00640 1 -- start start 2024-09-21T07:52:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c0003a30 0x7fb2dc1f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 0x7fb2dc154220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2dc154760 0x7fb2dc1f7ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc06f690 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2dc0b4ab0 con 0x7fb2c0003a30 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2dc06f810 con 0x7fb2dc154760 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 0x7fb2dc154220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2dbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c0003a30 0x7fb2dc1f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2e1000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2dc154760 0x7fb2dc1f7ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 0x7fb2dc154220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55722/0 (socket says 172.21.15.88:55722) 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 -- 172.21.15.88:0/2575025778 learned_addr learned my addr 172.21.15.88:0/2575025778 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 -- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2dc154760 msgr2=0x7fb2dc1f7ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2dc154760 0x7fb2dc1f7ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 -- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c0003a30 msgr2=0x7fb2dc1f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.485+0000 7fb2d3400640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c0003a30 0x7fb2dc1f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2d3400640 1 -- 172.21.15.88:0/2575025778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2dc0b0f90 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2e1000640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2dc154760 0x7fb2dc1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2d3400640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 0x7fb2dc154220 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc09e160 tx=0x7fb2c4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4015070 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2c4011840 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2c4011c90 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2dc0b1270 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.486+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2dc09add0 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.487+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2dc155780 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.488+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fb2c4018070 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.490+0000 7fb2d8c00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2b8079010 0x7fb2b807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:36.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.490+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7fb2c409bf50 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.490+0000 7fb2dbe00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2b8079010 0x7fb2b807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:36.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.491+0000 7fb2dbe00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2b8079010 0x7fb2b807b520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc035000 tx=0x7fb2cc02d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:36.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.495+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2c40638c0 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.719+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb2dc163260 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.719+0000 7fb2d8c00640 1 -- 172.21.15.88:0/2575025778 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) ==== 94+0+3908 (secure 0 0 0) 0x7fb2c40682f0 con 0x7fb2dc09b4d0 2024-09-21T07:52:36.720 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:36.721 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":73,"btime":"2024-09-21T07:52:29:945652+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6098,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/1601109764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":1601109764},{"type":"v1","addr":"172.21.15.175:6835","nonce":1601109764}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6104,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/51838836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":51838836},{"type":"v1","addr":"172.21.15.88:6834","nonce":51838836}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":6640,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/3939669519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":3939669519},{"type":"v1","addr":"172.21.15.175:6834","nonce":3939669519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":65},{"gid":7425,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/112137546","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":112137546},{"type":"v1","addr":"172.21.15.88:6835","nonce":112137546}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":73}],"filesystems":[]} 2024-09-21T07:52:36.721 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 73 2024-09-21T07:52:36.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2b8079010 msgr2=0x7fb2b807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2b8079010 0x7fb2b807b520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fb2cc035000 tx=0x7fb2cc02d040 comp rx=0 tx=0).stop 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 msgr2=0x7fb2dc154220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 0x7fb2dc154220 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc09e160 tx=0x7fb2c4001670 comp rx=0 tx=0).stop 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 shutdown_connections 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2b8079010 0x7fb2b807b520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2dc154760 0x7fb2dc1f7ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2dc09b4d0 0x7fb2dc154220 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 --2- 172.21.15.88:0/2575025778 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2c0003a30 0x7fb2dc1f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 >> 172.21.15.88:0/2575025778 conn(0x7fb2dc0b1e40 msgr2=0x7fb2dc0d00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 shutdown_connections 2024-09-21T07:52:36.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:36.722+0000 7fb2e1a00640 1 -- 172.21.15.88:0/2575025778 wait complete. 2024-09-21T07:52:36.750 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:52:36.750 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:52:36.750 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:52:36.750 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:52:36.751 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:52:36.751 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:52:36.751 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:52:36.751 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:52:36.751 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:52:36.751 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:52:36.751 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:52:36.752 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:52:36.752 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:52:36.752 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:52:36.752 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:52:36.752 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:52:36.775 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:52:36.776 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:52:36.776 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:52:36.777 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:52:36.789 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:52:36.789 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:52:36.790 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:52:36.790 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:52:36.820 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:52:36.820 DEBUG:teuthology.parallel:result is None 2024-09-21T07:52:36.822 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:52:36.823 DEBUG:teuthology.parallel:result is None 2024-09-21T07:52:36.834 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:52:36.834 DEBUG:teuthology.parallel:result is None 2024-09-21T07:52:36.836 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:52:36.836 DEBUG:teuthology.parallel:result is None 2024-09-21T07:52:36.836 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:52:37.017 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:52:37.017 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:52:37.028 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:52:37.029 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:52:37.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 -- 172.21.15.88:0/1415664127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba80b5f70 msgr2=0x7f8ba80b6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 --2- 172.21.15.88:0/1415664127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba80b5f70 0x7f8ba80b6380 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8ba0009be0 tx=0x7f8ba002a890 comp rx=0 tx=0).stop 2024-09-21T07:52:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 -- 172.21.15.88:0/1415664127 shutdown_connections 2024-09-21T07:52:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 --2- 172.21.15.88:0/1415664127 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba80b5f70 0x7f8ba80b6380 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 -- 172.21.15.88:0/1415664127 >> 172.21.15.88:0/1415664127 conn(0x7f8ba80d44f0 msgr2=0x7f8ba80d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 -- 172.21.15.88:0/1415664127 shutdown_connections 2024-09-21T07:52:37.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.069+0000 7f8baf000640 1 -- 172.21.15.88:0/1415664127 wait complete. 2024-09-21T07:52:37.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.071+0000 7f8baf000640 1 Processor -- start 2024-09-21T07:52:37.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.072+0000 7f8baf000640 1 -- start start 2024-09-21T07:52:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8baf000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 0x7f8ba80ca6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8baf000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba815e770 0x7f8ba80ccf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8baf000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba81f8a10 0x7f8ba81f8e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8baf000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80d3bb0 con 0x7f8ba80b5f70 2024-09-21T07:52:37.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8baf000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ba80d3a30 con 0x7f8ba81f8a10 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8baf000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ba80d3d30 con 0x7f8ba815e770 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8badc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 0x7f8ba80ca6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8badc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 0x7f8ba80ca6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55750/0 (socket says 172.21.15.88:55750) 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8badc00640 1 -- 172.21.15.88:0/1148838203 learned_addr learned my addr 172.21.15.88:0/1148838203 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8bae600640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba81f8a10 0x7f8ba81f8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.073+0000 7f8bad200640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba815e770 0x7f8ba80ccf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8badc00640 1 -- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba815e770 msgr2=0x7f8ba80ccf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8badc00640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba815e770 0x7f8ba80ccf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8badc00640 1 -- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba81f8a10 msgr2=0x7f8ba81f8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8badc00640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba81f8a10 0x7f8ba81f8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8badc00640 1 -- 172.21.15.88:0/1148838203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ba81f9360 con 0x7f8ba80b5f70 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8badc00640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 0x7f8ba80ca6b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c460 tx=0x7f8b9c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c0052d0 con 0x7f8ba80b5f70 2024-09-21T07:52:37.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8b9c005490 con 0x7f8ba80b5f70 2024-09-21T07:52:37.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b9c012870 con 0x7f8ba80b5f70 2024-09-21T07:52:37.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ba80cdb70 con 0x7f8ba80b5f70 2024-09-21T07:52:37.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.074+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ba8154d90 con 0x7f8ba80b5f70 2024-09-21T07:52:37.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.075+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8ba80698a0 con 0x7f8ba80b5f70 2024-09-21T07:52:37.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.077+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f8b9c012a10 con 0x7f8ba80b5f70 2024-09-21T07:52:37.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.078+0000 7f8ba5600640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8b88079010 0x7f8b8807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:37.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.078+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(112..112 src has 1..112) ==== 4943+0+0 (secure 0 0 0) 0x7f8b9c09ac20 con 0x7f8ba80b5f70 2024-09-21T07:52:37.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.078+0000 7f8bad200640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8b88079010 0x7f8b8807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:37.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.078+0000 7f8bad200640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8b88079010 0x7f8b8807b520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000c00 tx=0x7f8b94007ef0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:37.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.082+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8ba81f9360 con 0x7f8ba80b5f70 2024-09-21T07:52:37.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:37.280+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f8ba8162180 con 0x7f8ba80b5f70 2024-09-21T07:52:38.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.000+0000 7f8ba5600640 1 -- 172.21.15.88:0/1148838203 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v114) ==== 97+0+0 (secure 0 0 0) 0x7f8b9c062700 con 0x7f8ba80b5f70 2024-09-21T07:52:38.001 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8b88079010 msgr2=0x7f8b8807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8b88079010 0x7f8b8807b520 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f8b94000c00 tx=0x7f8b94007ef0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 msgr2=0x7f8ba80ca6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 0x7f8ba80ca6b0 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f8b9c00c460 tx=0x7f8b9c00c920 comp rx=0 tx=0).stop 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 shutdown_connections 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8b88079010 0x7f8b8807b520 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ba815e770 0x7f8ba80ccf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ba81f8a10 0x7f8ba81f8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 --2- 172.21.15.88:0/1148838203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ba80b5f70 0x7f8ba80ca6b0 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.003+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 >> 172.21.15.88:0/1148838203 conn(0x7f8ba80d44f0 msgr2=0x7f8ba80d4900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:38.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.004+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 shutdown_connections 2024-09-21T07:52:38.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.004+0000 7f8baf000640 1 -- 172.21.15.88:0/1148838203 wait complete. 2024-09-21T07:52:38.029 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:52:38.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 -- 172.21.15.88:0/2353314599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 msgr2=0x7f19d415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 --2- 172.21.15.88:0/2353314599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d415f3d0 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f19bc005fd0 tx=0x7f19bc02a540 comp rx=0 tx=0).stop 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 -- 172.21.15.88:0/2353314599 shutdown_connections 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 --2- 172.21.15.88:0/2353314599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d415f3d0 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 --2- 172.21.15.88:0/2353314599 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d40aebc0 0x7f19d4152920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 -- 172.21.15.88:0/2353314599 >> 172.21.15.88:0/2353314599 conn(0x7f19d406d730 msgr2=0x7f19d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 -- 172.21.15.88:0/2353314599 shutdown_connections 2024-09-21T07:52:38.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.248+0000 7f19dba00640 1 -- 172.21.15.88:0/2353314599 wait complete. 2024-09-21T07:52:38.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.251+0000 7f19dba00640 1 Processor -- start 2024-09-21T07:52:38.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.251+0000 7f19dba00640 1 -- start start 2024-09-21T07:52:38.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19dba00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d40aebc0 0x7f19d4158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19dba00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d4158bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19dba00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d415f910 0x7f19d40ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19dba00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f19d415c4a0 con 0x7f19d40b2750 2024-09-21T07:52:38.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19dba00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f19d40b4a20 con 0x7f19d40aebc0 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19dba00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f19d415c620 con 0x7f19d415f910 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.252+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d4158bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d4158bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55774/0 (socket says 172.21.15.88:55774) 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 -- 172.21.15.88:0/1056587458 learned_addr learned my addr 172.21.15.88:0/1056587458 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19d9000640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d40aebc0 0x7f19d4158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19d9a00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d415f910 0x7f19d40ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 -- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d415f910 msgr2=0x7f19d40ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d415f910 0x7f19d40ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 -- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d40aebc0 msgr2=0x7f19d4158670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d40aebc0 0x7f19d4158670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 -- 172.21.15.88:0/1056587458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19d4155bf0 con 0x7f19d40b2750 2024-09-21T07:52:38.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.253+0000 7f19cbe00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d4158bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006010 tx=0x7f19bc0038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.254+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc03e030 con 0x7f19d40b2750 2024-09-21T07:52:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.254+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f19bc003d90 con 0x7f19d40b2750 2024-09-21T07:52:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.254+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19bc035040 con 0x7f19d40b2750 2024-09-21T07:52:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.254+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19d4153a00 con 0x7f19d40b2750 2024-09-21T07:52:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.254+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19d4155560 con 0x7f19d40b2750 2024-09-21T07:52:38.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.255+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19d415cb40 con 0x7f19d40b2750 2024-09-21T07:52:38.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.256+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f19bc041030 con 0x7f19d40b2750 2024-09-21T07:52:38.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.257+0000 7f19d1600640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f19b0079010 0x7f19b007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.257+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4857+0+0 (secure 0 0 0) 0x7f19bc0bb9c0 con 0x7f19d40b2750 2024-09-21T07:52:38.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.257+0000 7f19d9000640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f19b0079010 0x7f19b007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.258+0000 7f19d9000640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f19b0079010 0x7f19b007b520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009b30 tx=0x7f19c4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:38.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.261+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f19bc0833c0 con 0x7f19d40b2750 2024-09-21T07:52:38.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.516+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f19d406f590 con 0x7f19d40b2750 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.517+0000 7f19d1600640 1 -- 172.21.15.88:0/1056587458 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) ==== 80+0+5304 (secure 0 0 0) 0x7f19bc087df0 con 0x7f19d40b2750 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:52:38.518 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:52:38.519 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.520 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.521 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.522 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.523 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:52:38.524 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.525 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.526 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.527 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.528 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.529 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:52:38.530 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:52:38.531 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:38.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f19b0079010 msgr2=0x7f19b007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f19b0079010 0x7f19b007b520 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009b30 tx=0x7f19c4005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:38.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 msgr2=0x7f19d4158bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d4158bb0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7f19bc006010 tx=0x7f19bc0038a0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 shutdown_connections 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f19b0079010 0x7f19b007b520 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f19d415f910 0x7f19d40ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f19d40b2750 0x7f19d4158bb0 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 --2- 172.21.15.88:0/1056587458 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f19d40aebc0 0x7f19d4158670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 >> 172.21.15.88:0/1056587458 conn(0x7f19d406d730 msgr2=0x7f19d40702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.519+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 shutdown_connections 2024-09-21T07:52:38.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.520+0000 7f19dba00640 1 -- 172.21.15.88:0/1056587458 wait complete. 2024-09-21T07:52:38.542 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:52:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 -- 172.21.15.88:0/3407896549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40b3550 msgr2=0x7f00d40b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 --2- 172.21.15.88:0/3407896549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40b3550 0x7f00d40b3960 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f00cc005fd0 tx=0x7f00cc02aa00 comp rx=0 tx=0).stop 2024-09-21T07:52:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 -- 172.21.15.88:0/3407896549 shutdown_connections 2024-09-21T07:52:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 --2- 172.21.15.88:0/3407896549 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d40b3550 0x7f00d40b3960 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 --2- 172.21.15.88:0/3407896549 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d415d760 0x7f00d41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 -- 172.21.15.88:0/3407896549 >> 172.21.15.88:0/3407896549 conn(0x7f00d406d730 msgr2=0x7f00d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 -- 172.21.15.88:0/3407896549 shutdown_connections 2024-09-21T07:52:38.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.761+0000 7f00daa00640 1 -- 172.21.15.88:0/3407896549 wait complete. 2024-09-21T07:52:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.763+0000 7f00daa00640 1 Processor -- start 2024-09-21T07:52:38.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.763+0000 7f00daa00640 1 -- start start 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00daa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d40ad0e0 0x7f00d4159b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00daa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 0x7f00d414c950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00daa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d415d760 0x7f00d41602a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00daa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00d40b4da0 con 0x7f00d40b3550 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00daa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00d40b4c20 con 0x7f00d415d760 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00daa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00d40b3ea0 con 0x7f00d40ad0e0 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 0x7f00d414c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 0x7f00d414c950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55790/0 (socket says 172.21.15.88:55790) 2024-09-21T07:52:38.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3400640 1 -- 172.21.15.88:0/1008292552 learned_addr learned my addr 172.21.15.88:0/1008292552 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:38.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3e00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d40ad0e0 0x7f00d4159b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d8a00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d415d760 0x7f00d41602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3400640 1 -- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d40ad0e0 msgr2=0x7f00d4159b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3400640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d40ad0e0 0x7f00d4159b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.765+0000 7f00d3400640 1 -- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d415d760 msgr2=0x7f00d41602a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00d3400640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d415d760 0x7f00d41602a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00d3400640 1 -- 172.21.15.88:0/1008292552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00d41f8df0 con 0x7f00d40b3550 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00d8a00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d415d760 0x7f00d41602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00d3400640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 0x7f00d414c950 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c370 tx=0x7f00c000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0017070 con 0x7f00d40b3550 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00c00035f0 con 0x7f00d40b3550 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0003a40 con 0x7f00d40b3550 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00d41f90b0 con 0x7f00d40b3550 2024-09-21T07:52:38.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.766+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00d41f9670 con 0x7f00d40b3550 2024-09-21T07:52:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.767+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00d40b4330 con 0x7f00d40b3550 2024-09-21T07:52:38.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.768+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f00c001a070 con 0x7f00d40b3550 2024-09-21T07:52:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.769+0000 7f00cbe00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00b4079010 0x7f00b407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:38.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.770+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4857+0+0 (secure 0 0 0) 0x7f00c009a510 con 0x7f00d40b3550 2024-09-21T07:52:38.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.770+0000 7f00d3e00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00b4079010 0x7f00b407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:38.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.770+0000 7f00d3e00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00b4079010 0x7f00b407b520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00c40095f0 tx=0x7f00c4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:38.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.774+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00c0061ed0 con 0x7f00d40b3550 2024-09-21T07:52:38.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.965+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f00d415b000 con 0x7f00d40b3550 2024-09-21T07:52:38.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.966+0000 7f00cbe00640 1 -- 172.21.15.88:0/1008292552 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v116) ==== 84+0+65 (secure 0 0 0) 0x7f00c0066900 con 0x7f00d40b3550 2024-09-21T07:52:38.967 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:38.967 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:52:38.967 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:38.968 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:52:38.968 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:52:38.968 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:38.968 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00b4079010 msgr2=0x7f00b407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00b4079010 0x7f00b407b520 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f00c40095f0 tx=0x7f00c4005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 msgr2=0x7f00d414c950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 0x7f00d414c950 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7f00c000c370 tx=0x7f00c000c830 comp rx=0 tx=0).stop 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 shutdown_connections 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00b4079010 0x7f00b407b520 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00d415d760 0x7f00d41602a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00d40b3550 0x7f00d414c950 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 --2- 172.21.15.88:0/1008292552 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00d40ad0e0 0x7f00d4159b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:38.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 >> 172.21.15.88:0/1008292552 conn(0x7f00d406d730 msgr2=0x7f00d40cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.968+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 shutdown_connections 2024-09-21T07:52:38.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:38.969+0000 7f00daa00640 1 -- 172.21.15.88:0/1008292552 wait complete. 2024-09-21T07:52:38.995 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:52:38.995 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:52:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 -- 172.21.15.88:0/201876339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80cc830 msgr2=0x7f48f80b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 --2- 172.21.15.88:0/201876339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80cc830 0x7f48f80b4620 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009830 tx=0x7f48e802a510 comp rx=0 tx=0).stop 2024-09-21T07:52:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 -- 172.21.15.88:0/201876339 shutdown_connections 2024-09-21T07:52:39.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 --2- 172.21.15.88:0/201876339 >> v2:172.21.15.175:3300/0 conn(0x7f48f806cda0 0x7f48f806d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 --2- 172.21.15.88:0/201876339 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80cc830 0x7f48f80b4620 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 -- 172.21.15.88:0/201876339 >> 172.21.15.88:0/201876339 conn(0x7f48f80b2540 msgr2=0x7f48f80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 -- 172.21.15.88:0/201876339 shutdown_connections 2024-09-21T07:52:39.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.218+0000 7f48fd600640 1 -- 172.21.15.88:0/201876339 wait complete. 2024-09-21T07:52:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.220+0000 7f48fd600640 1 Processor -- start 2024-09-21T07:52:39.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.220+0000 7f48fd600640 1 -- start start 2024-09-21T07:52:39.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48fd600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 0x7f48f81f5820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48fd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80af100 0x7f48f8151c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48fd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f80cc830 0x7f48f809da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48fd600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80b3600 con 0x7f48f806cda0 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48fd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48f80b3480 con 0x7f48f80cc830 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48fd600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48f80b3780 con 0x7f48f80af100 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48f6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 0x7f48f81f5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48f6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 0x7f48f81f5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55800/0 (socket says 172.21.15.88:55800) 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48f6a00640 1 -- 172.21.15.88:0/3473759955 learned_addr learned my addr 172.21.15.88:0/3473759955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48f6000640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80af100 0x7f48f8151c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.222+0000 7f48f7400640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f80cc830 0x7f48f809da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48f6a00640 1 -- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80af100 msgr2=0x7f48f8151c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48f6a00640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80af100 0x7f48f8151c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48f6a00640 1 -- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f80cc830 msgr2=0x7f48f809da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48f6a00640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f80cc830 0x7f48f809da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48f6a00640 1 -- 172.21.15.88:0/3473759955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f8157250 con 0x7f48f806cda0 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48f6a00640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 0x7f48f81f5820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009950 tx=0x7f48e8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8031d60 con 0x7f48f806cda0 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f809c980 con 0x7f48f806cda0 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48e8033e10 con 0x7f48f806cda0 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e8033310 con 0x7f48f806cda0 2024-09-21T07:52:39.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.223+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f809cf40 con 0x7f48f806cda0 2024-09-21T07:52:39.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.224+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f80b3ca0 con 0x7f48f806cda0 2024-09-21T07:52:39.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.226+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f48e80334b0 con 0x7f48f806cda0 2024-09-21T07:52:39.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.226+0000 7f48eea00640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48d8079010 0x7f48d807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:39.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.227+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(116..116 src has 1..116) ==== 4857+0+0 (secure 0 0 0) 0x7f48e80bc620 con 0x7f48f806cda0 2024-09-21T07:52:39.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.227+0000 7f48f6000640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48d8079010 0x7f48d807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:39.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.228+0000 7f48f6000640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48d8079010 0x7f48d807b520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48f80981c0 tx=0x7f48e4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:39.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.232+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f48f8157250 con 0x7f48f806cda0 2024-09-21T07:52:39.421 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:39.420+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f48f80d1030 con 0x7f48f806cda0 2024-09-21T07:52:40.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.013+0000 7f48eea00640 1 -- 172.21.15.88:0/3473759955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v117) ==== 138+0+0 (secure 0 0 0) 0x7f48e803f030 con 0x7f48f806cda0 2024-09-21T07:52:40.014 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:52:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48d8079010 msgr2=0x7f48d807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48d8079010 0x7f48d807b520 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f48f80981c0 tx=0x7f48e4005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 msgr2=0x7f48f81f5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 0x7f48f81f5820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009950 tx=0x7f48e8033c20 comp rx=0 tx=0).stop 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 shutdown_connections 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48d8079010 0x7f48d807b520 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48f80af100 0x7f48f8151c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48f80cc830 0x7f48f809da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 --2- 172.21.15.88:0/3473759955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48f806cda0 0x7f48f81f5820 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 >> 172.21.15.88:0/3473759955 conn(0x7f48f80b2540 msgr2=0x7f48f80b05c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 shutdown_connections 2024-09-21T07:52:40.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.016+0000 7f48fd600640 1 -- 172.21.15.88:0/3473759955 wait complete. 2024-09-21T07:52:40.048 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 -- 172.21.15.88:0/1092922239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160370 msgr2=0x7fe0d00ccb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 --2- 172.21.15.88:0/1092922239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160370 0x7fe0d00ccb30 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fe0d0072980 tx=0x7fe0bc02a560 comp rx=0 tx=0).stop 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 -- 172.21.15.88:0/1092922239 shutdown_connections 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 --2- 172.21.15.88:0/1092922239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160370 0x7fe0d00ccb30 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 --2- 172.21.15.88:0/1092922239 >> v2:172.21.15.175:3300/0 conn(0x7fe0d00af4e0 0x7fe0d015d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 -- 172.21.15.88:0/1092922239 >> 172.21.15.88:0/1092922239 conn(0x7fe0d00b1dd0 msgr2=0x7fe0d00b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 -- 172.21.15.88:0/1092922239 shutdown_connections 2024-09-21T07:52:40.276 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.274+0000 7fe0d6800640 1 -- 172.21.15.88:0/1092922239 wait complete. 2024-09-21T07:52:40.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.276+0000 7fe0d6800640 1 Processor -- start 2024-09-21T07:52:40.277 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.277+0000 7fe0d6800640 1 -- start start 2024-09-21T07:52:40.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d6800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00af4e0 0x7fe0d0156700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d6800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0160370 0x7fe0d01f5180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 0x7fe0d015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d6800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00cd400 con 0x7fe0d0160780 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d6800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d00cd280 con 0x7fe0d0160370 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d6800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe0d0070c80 con 0x7fe0d00af4e0 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 0x7fe0d015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 0x7fe0d015f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55820/0 (socket says 172.21.15.88:55820) 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.278+0000 7fe0d5e00640 1 -- 172.21.15.88:0/2195805186 learned_addr learned my addr 172.21.15.88:0/2195805186 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d4a00640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0160370 0x7fe0d01f5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5400640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00af4e0 0x7fe0d0156700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5e00640 1 -- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00af4e0 msgr2=0x7fe0d0156700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5e00640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00af4e0 0x7fe0d0156700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5e00640 1 -- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0160370 msgr2=0x7fe0d01f5180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5e00640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0160370 0x7fe0d01f5180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5e00640 1 -- 172.21.15.88:0/2195805186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d0151740 con 0x7fe0d0160780 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d4a00640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0160370 0x7fe0d01f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d5e00640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 0x7fe0d015f390 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c6b0 tx=0x7fe0c800cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8003b70 con 0x7fe0d0160780 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe0c8003d30 con 0x7fe0d0160780 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d0150f10 con 0x7fe0d0160780 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.279+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe0c8005330 con 0x7fe0d0160780 2024-09-21T07:52:40.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.280+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d01f97f0 con 0x7fe0d0160780 2024-09-21T07:52:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.281+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d01f9280 con 0x7fe0d0160780 2024-09-21T07:52:40.283 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.282+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fe0c80054d0 con 0x7fe0d0160780 2024-09-21T07:52:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.283+0000 7fe0cd600640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe0b0079040 0x7fe0b007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.283+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(117..117 src has 1..117) ==== 5227+0+0 (secure 0 0 0) 0x7fe0c809b1b0 con 0x7fe0d0160780 2024-09-21T07:52:40.284 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.283+0000 7fe0d5400640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe0b0079040 0x7fe0b007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:40.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.284+0000 7fe0d5400640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe0b0079040 0x7fe0b007b550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009ac0 tx=0x7fe0c4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:40.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.288+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe0d015dcd0 con 0x7fe0d0160780 2024-09-21T07:52:40.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:40.475+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fe0d015cb60 con 0x7fe0d0160780 2024-09-21T07:52:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.021+0000 7fe0cd600640 1 -- 172.21.15.88:0/2195805186 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v118) ==== 170+0+0 (secure 0 0 0) 0x7fe0c80630f0 con 0x7fe0d0160780 2024-09-21T07:52:41.022 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:52:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe0b0079040 msgr2=0x7fe0b007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe0b0079040 0x7fe0b007b550 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4009ac0 tx=0x7fe0c4005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 msgr2=0x7fe0d015f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 0x7fe0d015f390 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fe0c800c6b0 tx=0x7fe0c800cb70 comp rx=0 tx=0).stop 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 shutdown_connections 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe0b0079040 0x7fe0b007b550 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe0d0160780 0x7fe0d015f390 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe0d0160370 0x7fe0d01f5180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 --2- 172.21.15.88:0/2195805186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe0d00af4e0 0x7fe0d0156700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.024+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 >> 172.21.15.88:0/2195805186 conn(0x7fe0d00b1dd0 msgr2=0x7fe0d00b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.025+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 shutdown_connections 2024-09-21T07:52:41.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.025+0000 7fe0d6800640 1 -- 172.21.15.88:0/2195805186 wait complete. 2024-09-21T07:52:41.065 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 -- 172.21.15.88:0/1917680457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c80cd000 msgr2=0x7f67c80af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 --2- 172.21.15.88:0/1917680457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c80cd000 0x7f67c80af0d0 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7f67c80b3160 tx=0x7f67c002aac0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 -- 172.21.15.88:0/1917680457 shutdown_connections 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 --2- 172.21.15.88:0/1917680457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c80cd000 0x7f67c80af0d0 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 --2- 172.21.15.88:0/1917680457 >> v2:172.21.15.175:3300/0 conn(0x7f67c8156ca0 0x7f67c80ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 -- 172.21.15.88:0/1917680457 >> 172.21.15.88:0/1917680457 conn(0x7f67c806d620 msgr2=0x7f67c80b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 -- 172.21.15.88:0/1917680457 shutdown_connections 2024-09-21T07:52:41.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.288+0000 7f67c7200640 1 -- 172.21.15.88:0/1917680457 wait complete. 2024-09-21T07:52:41.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.291+0000 7f67c7200640 1 Processor -- start 2024-09-21T07:52:41.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.291+0000 7f67c7200640 1 -- start start 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c80b3f30 0x7f67c81f9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67c80cd000 0x7f67c8145a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 0x7f67c81f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80af9a0 con 0x7f67c8156ca0 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f67c80af820 con 0x7f67c80b3f30 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f67c80afb20 con 0x7f67c80cd000 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.292+0000 7f67c6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 0x7f67c81f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 0x7f67c81f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55840/0 (socket says 172.21.15.88:55840) 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 -- 172.21.15.88:0/144725191 learned_addr learned my addr 172.21.15.88:0/144725191 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c5400640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67c80cd000 0x7f67c8145a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c5e00640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c80b3f30 0x7f67c81f9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 -- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67c80cd000 msgr2=0x7f67c8145a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67c80cd000 0x7f67c8145a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 -- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c80b3f30 msgr2=0x7f67c81f9bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c80b3f30 0x7f67c81f9bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 -- 172.21.15.88:0/144725191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67c8145fd0 con 0x7f67c8156ca0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.293+0000 7f67c6800640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 0x7f67c81f5040 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009d60 tx=0x7f67c0003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.294+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0044070 con 0x7f67c8156ca0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.294+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f67c0003e50 con 0x7f67c8156ca0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.294+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c00044b0 con 0x7f67c8156ca0 2024-09-21T07:52:41.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.294+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67c81f5b60 con 0x7f67c8156ca0 2024-09-21T07:52:41.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.294+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67c81f6120 con 0x7f67c8156ca0 2024-09-21T07:52:41.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.295+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67c81461a0 con 0x7f67c8156ca0 2024-09-21T07:52:41.298 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.296+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f67c0004650 con 0x7f67c8156ca0 2024-09-21T07:52:41.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.297+0000 7f67be000640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f67a8079010 0x7f67a807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:41.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.298+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(118..118 src has 1..118) ==== 5609+0+0 (secure 0 0 0) 0x7f67c00bbed0 con 0x7f67c8156ca0 2024-09-21T07:52:41.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.298+0000 7f67c5e00640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f67a8079010 0x7f67a807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:41.299 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.298+0000 7f67c5e00640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f67a8079010 0x7f67a807b520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f67c80cf590 tx=0x7f67b8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:41.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.301+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f67c0032030 con 0x7f67c8156ca0 2024-09-21T07:52:41.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:41.523+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f67c80af0d0 con 0x7f67c8156ca0 2024-09-21T07:52:42.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.037+0000 7f67be000640 1 -- 172.21.15.88:0/144725191 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v76) ==== 174+0+0 (secure 0 0 0) 0x7f67c0083650 con 0x7f67c8156ca0 2024-09-21T07:52:42.039 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 18 and data pool 19 2024-09-21T07:52:42.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.044+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f67a8079010 msgr2=0x7f67a807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.044+0000 7f67c7200640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f67a8079010 0x7f67a807b520 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f67c80cf590 tx=0x7f67b8009040 comp rx=0 tx=0).stop 2024-09-21T07:52:42.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.045+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 msgr2=0x7f67c81f5040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.045+0000 7f67c7200640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 0x7f67c81f5040 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f67c0009d60 tx=0x7f67c0003ce0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.046+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 shutdown_connections 2024-09-21T07:52:42.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.046+0000 7f67c7200640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f67a8079010 0x7f67a807b520 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.046+0000 7f67c7200640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f67c80cd000 0x7f67c8145a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.046+0000 7f67c7200640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67c8156ca0 0x7f67c81f5040 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.046+0000 7f67c7200640 1 --2- 172.21.15.88:0/144725191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f67c80b3f30 0x7f67c81f9bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.047 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.046+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 >> 172.21.15.88:0/144725191 conn(0x7f67c806d620 msgr2=0x7f67c8068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.047+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 shutdown_connections 2024-09-21T07:52:42.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.048+0000 7f67c7200640 1 -- 172.21.15.88:0/144725191 wait complete. 2024-09-21T07:52:42.091 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:52:42.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.312+0000 7f33bf600640 1 -- 172.21.15.88:0/2747936073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 msgr2=0x7f339c004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.312+0000 7f33bf600640 1 --2- 172.21.15.88:0/2747936073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f339c004800 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7f33b815cd60 tx=0x7f339c005830 comp rx=0 tx=0).stop 2024-09-21T07:52:42.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.312+0000 7f33bd600640 1 -- 172.21.15.88:0/2747936073 reap_dead start 2024-09-21T07:52:42.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.312+0000 7f33bf600640 1 -- 172.21.15.88:0/2747936073 shutdown_connections 2024-09-21T07:52:42.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.312+0000 7f33bf600640 1 -- 172.21.15.88:0/2747936073 >> 172.21.15.88:0/2747936073 conn(0x7f33b806d730 msgr2=0x7f33b80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.312+0000 7f33bf600640 1 -- 172.21.15.88:0/2747936073 shutdown_connections 2024-09-21T07:52:42.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.313+0000 7f33bf600640 1 -- 172.21.15.88:0/2747936073 wait complete. 2024-09-21T07:52:42.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.315+0000 7f33bf600640 1 Processor -- start 2024-09-21T07:52:42.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.315+0000 7f33bf600640 1 -- start start 2024-09-21T07:52:42.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.316+0000 7f33bf600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f33b81559f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.316+0000 7f33bf600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b814f900 0x7f33b815dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.316+0000 7f33bf600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8073d70 0x7f33b815e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.316+0000 7f33bf600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80b4da0 con 0x7f339c003fd0 2024-09-21T07:52:42.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.316+0000 7f33bf600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f33b80b4c20 con 0x7f33b8073d70 2024-09-21T07:52:42.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.316+0000 7f33bf600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f33b80b3960 con 0x7f33b814f900 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f33b81559f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f33b81559f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55864/0 (socket says 172.21.15.88:55864) 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 -- 172.21.15.88:0/498312499 learned_addr learned my addr 172.21.15.88:0/498312499 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bd600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8073d70 0x7f33b815e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33b7e00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b814f900 0x7f33b815dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 -- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b814f900 msgr2=0x7f33b815dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b814f900 0x7f33b815dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 -- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8073d70 msgr2=0x7f33b815e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8073d70 0x7f33b815e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 -- 172.21.15.88:0/498312499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33b8156360 con 0x7f339c003fd0 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33b7e00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b814f900 0x7f33b815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bd600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8073d70 0x7f33b815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.317+0000 7f33bcc00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f33b81559f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f33a8001db0 tx=0x7f33a8006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.318+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a800fd40 con 0x7f339c003fd0 2024-09-21T07:52:42.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.318+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f33a8006910 con 0x7f339c003fd0 2024-09-21T07:52:42.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.318+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33a8011690 con 0x7f339c003fd0 2024-09-21T07:52:42.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.318+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33b8073290 con 0x7f339c003fd0 2024-09-21T07:52:42.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.318+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33b8154a30 con 0x7f339c003fd0 2024-09-21T07:52:42.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.319+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33b8154cb0 con 0x7f339c003fd0 2024-09-21T07:52:42.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.320+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f33a8018020 con 0x7f339c003fd0 2024-09-21T07:52:42.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.321+0000 7f33b4c00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3398079040 0x7f339807b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.321+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f33a809c310 con 0x7f339c003fd0 2024-09-21T07:52:42.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.321+0000 7f33b7e00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3398079040 0x7f339807b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.322+0000 7f33b7e00640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3398079040 0x7f339807b550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f33a4035000 tx=0x7f33a40046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.325+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f33a8015030 con 0x7f339c003fd0 2024-09-21T07:52:42.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.512+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f33b80cfdf0 con 0x7f339c003fd0 2024-09-21T07:52:42.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.514+0000 7f33b4c00640 1 -- 172.21.15.88:0/498312499 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16712 (secure 0 0 0) 0x7f33a807de20 con 0x7f339c003fd0 2024-09-21T07:52:42.515 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:42.515 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":119,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:52:41.524327+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:52:39.432702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T07:52:40.488474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:52:42.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3398079040 msgr2=0x7f339807b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3398079040 0x7f339807b550 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f33a4035000 tx=0x7f33a40046c0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 msgr2=0x7f33b81559f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f33b81559f0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7f33a8001db0 tx=0x7f33a8006720 comp rx=0 tx=0).stop 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 shutdown_connections 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3398079040 0x7f339807b550 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f33b814f900 0x7f33b815dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f33b8073d70 0x7f33b815e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 --2- 172.21.15.88:0/498312499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c003fd0 0x7f33b81559f0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 >> 172.21.15.88:0/498312499 conn(0x7f33b806d730 msgr2=0x7f33b81602a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 shutdown_connections 2024-09-21T07:52:42.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.516+0000 7f33bf600640 1 -- 172.21.15.88:0/498312499 wait complete. 2024-09-21T07:52:42.543 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 -- 172.21.15.88:0/355734832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 msgr2=0x7f4644160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 --2- 172.21.15.88:0/355734832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f4644160720 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7f4644075440 tx=0x7f463c02a980 comp rx=0 tx=0).stop 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 -- 172.21.15.88:0/355734832 shutdown_connections 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 --2- 172.21.15.88:0/355734832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f4644160720 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 --2- 172.21.15.88:0/355734832 >> v2:172.21.15.175:3300/0 conn(0x7f46440ad0e0 0x7f46440ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 --2- 172.21.15.88:0/355734832 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f464415d760 0x7f46441535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.761+0000 7f464b200640 1 -- 172.21.15.88:0/355734832 >> 172.21.15.88:0/355734832 conn(0x7f464406d730 msgr2=0x7f46440b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.762+0000 7f464b200640 1 -- 172.21.15.88:0/355734832 shutdown_connections 2024-09-21T07:52:42.764 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.762+0000 7f464b200640 1 -- 172.21.15.88:0/355734832 wait complete. 2024-09-21T07:52:42.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.764+0000 7f464b200640 1 Processor -- start 2024-09-21T07:52:42.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.764+0000 7f464b200640 1 -- start start 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f464b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440ad0e0 0x7f4644154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f464b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464415d760 0x7f4644151980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f464b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f46441595b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f464b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46440b4da0 con 0x7f4644160310 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f464b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46440b4c20 con 0x7f464415d760 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f464b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4644160c60 con 0x7f46440ad0e0 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.765+0000 7f4649200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f46441595b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f46441595b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55880/0 (socket says 172.21.15.88:55880) 2024-09-21T07:52:42.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 -- 172.21.15.88:0/2276587710 learned_addr learned my addr 172.21.15.88:0/2276587710 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f463be00640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464415d760 0x7f4644151980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4643e00640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440ad0e0 0x7f4644154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 -- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440ad0e0 msgr2=0x7f4644154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440ad0e0 0x7f4644154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 -- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464415d760 msgr2=0x7f4644151980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464415d760 0x7f4644151980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 -- 172.21.15.88:0/2276587710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4644151ec0 con 0x7f4644160310 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.766+0000 7f4649200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f46441595b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f463c009d60 tx=0x7f463c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.767+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c034040 con 0x7f4644160310 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.767+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f463c004540 con 0x7f4644160310 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.767+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4644154f30 con 0x7f4644160310 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.767+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f463c045480 con 0x7f4644160310 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.767+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f464409c230 con 0x7f4644160310 2024-09-21T07:52:42.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.768+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46441535c0 con 0x7f4644160310 2024-09-21T07:52:42.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.769+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f463c037020 con 0x7f4644160310 2024-09-21T07:52:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.771+0000 7f4640c00640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4620079040 0x7f462007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.771+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(119..119 src has 1..119) ==== 5709+0+0 (secure 0 0 0) 0x7f463c0bce10 con 0x7f4644160310 2024-09-21T07:52:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.771+0000 7f4643e00640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4620079040 0x7f462007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:42.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.771+0000 7f4643e00640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4620079040 0x7f462007b550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4634009a40 tx=0x7f4634005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:42.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.775+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f463c084a90 con 0x7f4644160310 2024-09-21T07:52:42.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.963+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f46440702c0 con 0x7f4644160310 2024-09-21T07:52:42.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.964+0000 7f4640c00640 1 -- 172.21.15.88:0/2276587710 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) ==== 74+0+16712 (secure 0 0 0) 0x7f463c084390 con 0x7f4644160310 2024-09-21T07:52:42.966 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:42.966 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":119,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:52:41.524327+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:52:39.432702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T07:52:40.488474+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4620079040 msgr2=0x7f462007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4620079040 0x7f462007b550 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f4634009a40 tx=0x7f4634005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 msgr2=0x7f46441595b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f46441595b0 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7f463c009d60 tx=0x7f463c003f10 comp rx=0 tx=0).stop 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 shutdown_connections 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4620079040 0x7f462007b550 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4644160310 0x7f46441595b0 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f464415d760 0x7f4644151980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 --2- 172.21.15.88:0/2276587710 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46440ad0e0 0x7f4644154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:42.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 >> 172.21.15.88:0/2276587710 conn(0x7f464406d730 msgr2=0x7f46441527a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:42.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 shutdown_connections 2024-09-21T07:52:42.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:42.967+0000 7f464b200640 1 -- 172.21.15.88:0/2276587710 wait complete. 2024-09-21T07:52:42.991 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:52:43.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 -- 172.21.15.88:0/2370466312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c147b90 msgr2=0x7f4e2c073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 --2- 172.21.15.88:0/2370466312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c147b90 0x7f4e2c073540 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f4e18005fd0 tx=0x7f4e1802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 -- 172.21.15.88:0/2370466312 shutdown_connections 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 --2- 172.21.15.88:0/2370466312 >> v2:172.21.15.175:3300/0 conn(0x7f4e2c073a80 0x7f4e2c06f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 --2- 172.21.15.88:0/2370466312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c147b90 0x7f4e2c073540 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 -- 172.21.15.88:0/2370466312 >> 172.21.15.88:0/2370466312 conn(0x7f4e2c06d1c0 msgr2=0x7f4e2c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 -- 172.21.15.88:0/2370466312 shutdown_connections 2024-09-21T07:52:43.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.219+0000 7f4e33e00640 1 -- 172.21.15.88:0/2370466312 wait complete. 2024-09-21T07:52:43.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.221+0000 7f4e33e00640 1 Processor -- start 2024-09-21T07:52:43.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.222+0000 7f4e33e00640 1 -- start start 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c073a80 0x7f4e2c1463b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 0x7f4e2c1468f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e33e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c15f9e0 0x7f4e2c141600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e33e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c148470 con 0x7f4e2c147b90 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e33e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e2c1482f0 con 0x7f4e2c073a80 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e33e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e2c06fd10 con 0x7f4e2c15f9e0 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e30a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 0x7f4e2c1468f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e30a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 0x7f4e2c1468f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55896/0 (socket says 172.21.15.88:55896) 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.223+0000 7f4e30a00640 1 -- 172.21.15.88:0/1927581817 learned_addr learned my addr 172.21.15.88:0/1927581817 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e31400640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c073a80 0x7f4e2c1463b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e31e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c15f9e0 0x7f4e2c141600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e30a00640 1 -- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c15f9e0 msgr2=0x7f4e2c141600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e30a00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c15f9e0 0x7f4e2c141600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e30a00640 1 -- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c073a80 msgr2=0x7f4e2c1463b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e30a00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c073a80 0x7f4e2c1463b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e30a00640 1 -- 172.21.15.88:0/1927581817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e2c1f6860 con 0x7f4e2c147b90 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e31e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c15f9e0 0x7f4e2c141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:43.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e31400640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c073a80 0x7f4e2c1463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.224+0000 7f4e30a00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 0x7f4e2c1468f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000c00 tx=0x7f4e180038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.225+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e1803e030 con 0x7f4e2c147b90 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.225+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4e18003db0 con 0x7f4e2c147b90 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.225+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18035040 con 0x7f4e2c147b90 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.225+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e2c13dd30 con 0x7f4e2c147b90 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.225+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e2c1f5aa0 con 0x7f4e2c147b90 2024-09-21T07:52:43.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.226+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e2c13f740 con 0x7f4e2c147b90 2024-09-21T07:52:43.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.227+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f4e18041030 con 0x7f4e2c147b90 2024-09-21T07:52:43.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.228+0000 7f4e29600640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e0c079010 0x7f4e0c07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.228+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f4e180bbe20 con 0x7f4e2c147b90 2024-09-21T07:52:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.228+0000 7f4e31400640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e0c079010 0x7f4e0c07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:43.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.229+0000 7f4e31400640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e0c079010 0x7f4e0c07b520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4e20007fd0 tx=0x7f4e20009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:43.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.233+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e180bf030 con 0x7f4e2c147b90 2024-09-21T07:52:43.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:43.457+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4e2c06f7d0 con 0x7f4e2c147b90 2024-09-21T07:52:44.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.059+0000 7f4e29600640 1 -- 172.21.15.88:0/1927581817 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v79) ==== 118+0+0 (secure 0 0 0) 0x7f4e18083a90 con 0x7f4e2c147b90 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e0c079010 msgr2=0x7f4e0c07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e0c079010 0x7f4e0c07b520 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4e20007fd0 tx=0x7f4e20009040 comp rx=0 tx=0).stop 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 msgr2=0x7f4e2c1468f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 0x7f4e2c1468f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f4e18000c00 tx=0x7f4e180038a0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 shutdown_connections 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4e0c079010 0x7f4e0c07b520 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e2c15f9e0 0x7f4e2c141600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e2c147b90 0x7f4e2c1468f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 --2- 172.21.15.88:0/1927581817 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e2c073a80 0x7f4e2c1463b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 >> 172.21.15.88:0/1927581817 conn(0x7f4e2c06d1c0 msgr2=0x7f4e2c144b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:44.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 shutdown_connections 2024-09-21T07:52:44.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.061+0000 7f4e33e00640 1 -- 172.21.15.88:0/1927581817 wait complete. 2024-09-21T07:52:44.090 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:52:44.090 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:52:44.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.311+0000 7f0cc8000640 1 -- 172.21.15.88:0/3024753666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0073c50 msgr2=0x7f0cc01674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.311+0000 7f0cc8000640 1 --2- 172.21.15.88:0/3024753666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0073c50 0x7f0cc01674a0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f0cb8005fd0 tx=0x7f0cb802a940 comp rx=0 tx=0).stop 2024-09-21T07:52:44.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.312+0000 7f0cc8000640 1 -- 172.21.15.88:0/3024753666 shutdown_connections 2024-09-21T07:52:44.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.312+0000 7f0cc8000640 1 --2- 172.21.15.88:0/3024753666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0073c50 0x7f0cc01674a0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.312+0000 7f0cc8000640 1 -- 172.21.15.88:0/3024753666 >> 172.21.15.88:0/3024753666 conn(0x7f0cc006d1c0 msgr2=0x7f0cc006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:44.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.312+0000 7f0cc8000640 1 -- 172.21.15.88:0/3024753666 shutdown_connections 2024-09-21T07:52:44.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.312+0000 7f0cc8000640 1 -- 172.21.15.88:0/3024753666 wait complete. 2024-09-21T07:52:44.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.314+0000 7f0cc8000640 1 Processor -- start 2024-09-21T07:52:44.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.314+0000 7f0cc8000640 1 -- start start 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.315+0000 7f0cc8000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc0145550 0x7f0cc015edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.315+0000 7f0cc8000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 0x7f0cc015f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.315+0000 7f0cc8000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc015c340 0x7f0cc01f5c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.315+0000 7f0cc8000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc0167d70 con 0x7f0cc0145a40 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.315+0000 7f0cc8000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0cc0167bf0 con 0x7f0cc015c340 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.315+0000 7f0cc8000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0cc0167ef0 con 0x7f0cc0145550 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 0x7f0cc015f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 0x7f0cc015f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55910/0 (socket says 172.21.15.88:55910) 2024-09-21T07:52:44.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 -- 172.21.15.88:0/2208493401 learned_addr learned my addr 172.21.15.88:0/2208493401 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc5600640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc0145550 0x7f0cc015edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc6000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc015c340 0x7f0cc01f5c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 -- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc0145550 msgr2=0x7f0cc015edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc0145550 0x7f0cc015edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 -- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc015c340 msgr2=0x7f0cc01f5c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc015c340 0x7f0cc01f5c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc4c00640 1 -- 172.21.15.88:0/2208493401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0cc013e5c0 con 0x7f0cc0145a40 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.316+0000 7f0cc6000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc015c340 0x7f0cc01f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.317+0000 7f0cc4c00640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 0x7f0cc015f2f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00c3c0 tx=0x7f0cac00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.317+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac017070 con 0x7f0cc0145a40 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.317+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0cac0035f0 con 0x7f0cc0145a40 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.317+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0cc013e8a0 con 0x7f0cc0145a40 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.317+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cac003a20 con 0x7f0cc0145a40 2024-09-21T07:52:44.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.317+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0cc01f5700 con 0x7f0cc0145a40 2024-09-21T07:52:44.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.318+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0cc006fb40 con 0x7f0cc0145a40 2024-09-21T07:52:44.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.319+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f0cac01a070 con 0x7f0cc0145a40 2024-09-21T07:52:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.321+0000 7f0cbd600640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ca0079010 0x7f0ca007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.321+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f0cac09a700 con 0x7f0cc0145a40 2024-09-21T07:52:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.321+0000 7f0cc5600640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ca0079010 0x7f0ca007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:44.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.322+0000 7f0cc5600640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ca0079010 0x7f0ca007b520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4006fd0 tx=0x7f0cb4009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:44.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.325+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0cac061ec0 con 0x7f0cc0145a40 2024-09-21T07:52:44.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:44.546+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7f0cc013fad0 con 0x7f0cc0145a40 2024-09-21T07:52:45.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.067+0000 7f0cbd600640 1 -- 172.21.15.88:0/2208493401 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v80) ==== 122+0+0 (secure 0 0 0) 0x7f0cac0668f0 con 0x7f0cc0145a40 2024-09-21T07:52:45.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ca0079010 msgr2=0x7f0ca007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ca0079010 0x7f0ca007b520 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4006fd0 tx=0x7f0cb4009060 comp rx=0 tx=0).stop 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 msgr2=0x7f0cc015f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 0x7f0cc015f2f0 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f0cac00c3c0 tx=0x7f0cac00c880 comp rx=0 tx=0).stop 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 shutdown_connections 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ca0079010 0x7f0ca007b520 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0cc015c340 0x7f0cc01f5c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0cc0145a40 0x7f0cc015f2f0 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 --2- 172.21.15.88:0/2208493401 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0cc0145550 0x7f0cc015edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 >> 172.21.15.88:0/2208493401 conn(0x7f0cc006d1c0 msgr2=0x7f0cc013b620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 shutdown_connections 2024-09-21T07:52:45.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.070+0000 7f0cc8000640 1 -- 172.21.15.88:0/2208493401 wait complete. 2024-09-21T07:52:45.095 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:52:45.095 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:45.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 -- 172.21.15.88:0/944520485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 msgr2=0x7f97401535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 --2- 172.21.15.88:0/944520485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401535c0 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f97400b3100 tx=0x7f973002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 -- 172.21.15.88:0/944520485 shutdown_connections 2024-09-21T07:52:45.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 --2- 172.21.15.88:0/944520485 >> v2:172.21.15.175:3300/0 conn(0x7f97400b3550 0x7f97400b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 --2- 172.21.15.88:0/944520485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401535c0 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 -- 172.21.15.88:0/944520485 >> 172.21.15.88:0/944520485 conn(0x7f974006d730 msgr2=0x7f97400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:45.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 -- 172.21.15.88:0/944520485 shutdown_connections 2024-09-21T07:52:45.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.313+0000 7f9745600640 1 -- 172.21.15.88:0/944520485 wait complete. 2024-09-21T07:52:45.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.315+0000 7f9745600640 1 Processor -- start 2024-09-21T07:52:45.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.316+0000 7f9745600640 1 -- start start 2024-09-21T07:52:45.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f9745600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97400ad0e0 0x7f974014c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f9745600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97400b3550 0x7f97401f99f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f9745600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401569a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f9745600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97400b4da0 con 0x7f974015d760 2024-09-21T07:52:45.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f9745600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f97400b4c20 con 0x7f97400ad0e0 2024-09-21T07:52:45.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f9745600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f97400b3ea0 con 0x7f97400b3550 2024-09-21T07:52:45.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f973f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401569a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:45.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f973f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401569a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55928/0 (socket says 172.21.15.88:55928) 2024-09-21T07:52:45.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.317+0000 7f973f400640 1 -- 172.21.15.88:0/2263778945 learned_addr learned my addr 172.21.15.88:0/2263778945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f973f400640 1 -- 172.21.15.88:0/2263778945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97400b3550 msgr2=0x7f97401f99f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f973f400640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97400b3550 0x7f97401f99f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f973f400640 1 -- 172.21.15.88:0/2263778945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97400ad0e0 msgr2=0x7f974014c6a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f973f400640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97400ad0e0 0x7f974014c6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f973f400640 1 -- 172.21.15.88:0/2263778945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9740074830 con 0x7f974015d760 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f973f400640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401569a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f973800b9e0 tx=0x7f973800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738019070 con 0x7f974015d760 2024-09-21T07:52:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f973800cbd0 con 0x7f974015d760 2024-09-21T07:52:45.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9738014440 con 0x7f974015d760 2024-09-21T07:52:45.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9740074b10 con 0x7f974015d760 2024-09-21T07:52:45.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.318+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97401f91f0 con 0x7f974015d760 2024-09-21T07:52:45.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.320+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97400745a0 con 0x7f974015d760 2024-09-21T07:52:45.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.321+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f973801c070 con 0x7f974015d760 2024-09-21T07:52:45.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.322+0000 7f9736a00640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9720079040 0x7f972007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.322+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f973809b330 con 0x7f974015d760 2024-09-21T07:52:45.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.322+0000 7f973ea00640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9720079040 0x7f972007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.323+0000 7f973ea00640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9720079040 0x7f972007b550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9730033000 tx=0x7f9730004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:45.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.327+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9738062ff0 con 0x7f974015d760 2024-09-21T07:52:45.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.547+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f97400b4420 con 0x7f974015d760 2024-09-21T07:52:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.548+0000 7f9736a00640 1 -- 172.21.15.88:0/2263778945 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5228 (secure 0 0 0) 0x7f97380628f0 con 0x7f974015d760 2024-09-21T07:52:45.550 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:45.550 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":80,"btime":"2024-09-21T07:52:45:060798+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:52:41.524287+0000","modified":"2024-09-21T07:52:44.548337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6834/3361463212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3361463212},{"type":"v1","addr":"172.21.15.88:6834","nonce":3361463212}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":9}]} 2024-09-21T07:52:45.550 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 80 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9720079040 msgr2=0x7f972007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9720079040 0x7f972007b550 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f9730033000 tx=0x7f9730004730 comp rx=0 tx=0).stop 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 msgr2=0x7f97401569a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401569a0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f973800b9e0 tx=0x7f973800bea0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 shutdown_connections 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9720079040 0x7f972007b550 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f97400b3550 0x7f97401f99f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974015d760 0x7f97401569a0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 --2- 172.21.15.88:0/2263778945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f97400ad0e0 0x7f974014c6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 >> 172.21.15.88:0/2263778945 conn(0x7f974006d730 msgr2=0x7f974015dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 shutdown_connections 2024-09-21T07:52:45.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.550+0000 7f9745600640 1 -- 172.21.15.88:0/2263778945 wait complete. 2024-09-21T07:52:45.573 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 -- 172.21.15.88:0/3144897187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600b3550 msgr2=0x7f98600b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 --2- 172.21.15.88:0/3144897187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600b3550 0x7f98600b3960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9858005fd0 tx=0x7f985802aa10 comp rx=0 tx=0).stop 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 -- 172.21.15.88:0/3144897187 shutdown_connections 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 --2- 172.21.15.88:0/3144897187 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600b3550 0x7f98600b3960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 --2- 172.21.15.88:0/3144897187 >> v2:172.21.15.175:3300/0 conn(0x7f98600ad0e0 0x7f98600ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 -- 172.21.15.88:0/3144897187 >> 172.21.15.88:0/3144897187 conn(0x7f986006d730 msgr2=0x7f98600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 -- 172.21.15.88:0/3144897187 shutdown_connections 2024-09-21T07:52:45.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.792+0000 7f9866c00640 1 -- 172.21.15.88:0/3144897187 wait complete. 2024-09-21T07:52:45.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.794+0000 7f9866c00640 1 Processor -- start 2024-09-21T07:52:45.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.795+0000 7f9866c00640 1 -- start start 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9866c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600ad0e0 0x7f98600cec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9866c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98600b3550 0x7f9860073160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9866c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 0x7f98601516e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9866c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98600b4da0 con 0x7f986015d760 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9866c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98600b4c20 con 0x7f98600b3550 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9866c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98600b3ea0 con 0x7f98600ad0e0 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9864c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 0x7f98601516e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9864c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 0x7f98601516e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57194/0 (socket says 172.21.15.88:57194) 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f9864c00640 1 -- 172.21.15.88:0/583650799 learned_addr learned my addr 172.21.15.88:0/583650799 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:45.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f985f400640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98600b3550 0x7f9860073160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.796+0000 7f985fe00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600ad0e0 0x7f98600cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9864c00640 1 -- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600ad0e0 msgr2=0x7f98600cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9864c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600ad0e0 0x7f98600cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9864c00640 1 -- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98600b3550 msgr2=0x7f9860073160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9864c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98600b3550 0x7f9860073160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9864c00640 1 -- 172.21.15.88:0/583650799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f986015a5f0 con 0x7f986015d760 2024-09-21T07:52:45.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9864c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 0x7f98601516e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9858006010 tx=0x7f9858009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858046070 con 0x7f986015d760 2024-09-21T07:52:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.797+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9858004540 con 0x7f986015d760 2024-09-21T07:52:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.798+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9860159d50 con 0x7f986015d760 2024-09-21T07:52:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.798+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9858033070 con 0x7f986015d760 2024-09-21T07:52:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.798+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f986015a310 con 0x7f986015d760 2024-09-21T07:52:45.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.799+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98600b4370 con 0x7f986015d760 2024-09-21T07:52:45.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.800+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f9858049070 con 0x7f986015d760 2024-09-21T07:52:45.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.802+0000 7f9857e00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9840079010 0x7f984007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:45.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.802+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f98580bc670 con 0x7f986015d760 2024-09-21T07:52:45.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.802+0000 7f985fe00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9840079010 0x7f984007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:45.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.802+0000 7f985fe00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9840079010 0x7f984007b520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9850000f60 tx=0x7f9850009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:45.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.806+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f986015a5f0 con 0x7f986015d760 2024-09-21T07:52:45.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.995+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f986015fcd0 con 0x7f986015d760 2024-09-21T07:52:45.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.997+0000 7f9857e00640 1 -- 172.21.15.88:0/583650799 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v120) ==== 74+0+16699 (secure 0 0 0) 0x7f985803e070 con 0x7f986015d760 2024-09-21T07:52:45.999 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:45.999 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":120,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:52:43.036695+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:52:39.432702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T07:52:40.488474+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"120","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9840079010 msgr2=0x7f984007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9840079010 0x7f984007b520 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f9850000f60 tx=0x7f9850009040 comp rx=0 tx=0).stop 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 msgr2=0x7f98601516e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 0x7f98601516e0 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f9858006010 tx=0x7f9858009360 comp rx=0 tx=0).stop 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 shutdown_connections 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9840079010 0x7f984007b520 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f986015d760 0x7f98601516e0 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98600b3550 0x7f9860073160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 --2- 172.21.15.88:0/583650799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98600ad0e0 0x7f98600cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 >> 172.21.15.88:0/583650799 conn(0x7f986006d730 msgr2=0x7f98600cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 shutdown_connections 2024-09-21T07:52:46.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:45.999+0000 7f9866c00640 1 -- 172.21.15.88:0/583650799 wait complete. 2024-09-21T07:52:46.024 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:52:46.025 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f655b600640 1 -- 172.21.15.88:0/491878407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65380039e0 msgr2=0x7f6538003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f655b600640 1 --2- 172.21.15.88:0/491878407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65380039e0 0x7f6538003e10 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7f65440017c0 tx=0x7f6538005830 comp rx=0 tx=0).stop 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f6559600640 1 -- 172.21.15.88:0/491878407 reap_dead start 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f655b600640 1 -- 172.21.15.88:0/491878407 shutdown_connections 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f655b600640 1 -- 172.21.15.88:0/491878407 >> 172.21.15.88:0/491878407 conn(0x7f655406d730 msgr2=0x7f65540b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f655b600640 1 -- 172.21.15.88:0/491878407 shutdown_connections 2024-09-21T07:52:46.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.243+0000 7f655b600640 1 -- 172.21.15.88:0/491878407 wait complete. 2024-09-21T07:52:46.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.246+0000 7f655b600640 1 Processor -- start 2024-09-21T07:52:46.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.246+0000 7f655b600640 1 -- start start 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.247+0000 7f655b600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380039e0 0x7f655414f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.247+0000 7f655b600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655414b930 0x7f65541f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.247+0000 7f655b600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 0x7f65541f7a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.247+0000 7f655b600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f65540b4da0 con 0x7f65541f7600 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.247+0000 7f655b600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f65540b4c20 con 0x7f655414b930 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.247+0000 7f655b600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f65540ad4f0 con 0x7f65380039e0 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 0x7f65541f7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 0x7f65541f7a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57216/0 (socket says 172.21.15.88:57216) 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 -- 172.21.15.88:0/2929204264 learned_addr learned my addr 172.21.15.88:0/2929204264 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6558c00640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380039e0 0x7f655414f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6553e00640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655414b930 0x7f65541f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 -- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380039e0 msgr2=0x7f655414f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380039e0 0x7f655414f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 -- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655414b930 msgr2=0x7f65541f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655414b930 0x7f65541f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 -- 172.21.15.88:0/2929204264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f655414d160 con 0x7f65541f7600 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.248+0000 7f6559600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 0x7f65541f7a10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f65540ae430 tx=0x7f654c006430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.249+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c011900 con 0x7f65541f7600 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.249+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f654c019070 con 0x7f65541f7600 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.249+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f654c011d20 con 0x7f65541f7600 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.249+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f655414d440 con 0x7f65541f7600 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.249+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f655414da00 con 0x7f65541f7600 2024-09-21T07:52:46.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.250+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65541535c0 con 0x7f65541f7600 2024-09-21T07:52:46.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.251+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f654c01e460 con 0x7f65541f7600 2024-09-21T07:52:46.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.252+0000 7f6550c00640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6534079010 0x7f653407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.252+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f654c09bc70 con 0x7f65541f7600 2024-09-21T07:52:46.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.253+0000 7f6558c00640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6534079010 0x7f653407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.253+0000 7f6558c00640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6534079010 0x7f653407b520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6544032000 tx=0x7f654402c000 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:46.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.257+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f65540ada30 con 0x7f65541f7600 2024-09-21T07:52:46.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.441+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6554153ab0 con 0x7f6534079010 2024-09-21T07:52:46.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.446+0000 7f6550c00640 1 -- 172.21.15.88:0/2929204264 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247113 (secure 0 0 0) 0x7f6554153ab0 con 0x7f6534079010 2024-09-21T07:52:46.451 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:46.452 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6534079010 msgr2=0x7f653407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6534079010 0x7f653407b520 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f6544032000 tx=0x7f654402c000 comp rx=0 tx=0).stop 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 msgr2=0x7f65541f7a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 0x7f65541f7a10 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7f65540ae430 tx=0x7f654c006430 comp rx=0 tx=0).stop 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 shutdown_connections 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6534079010 0x7f653407b520 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65541f7600 0x7f65541f7a10 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f655414b930 0x7f65541f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 --2- 172.21.15.88:0/2929204264 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f65380039e0 0x7f655414f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 >> 172.21.15.88:0/2929204264 conn(0x7f655406d730 msgr2=0x7f6554070480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:46.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 shutdown_connections 2024-09-21T07:52:46.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.450+0000 7f655b600640 1 -- 172.21.15.88:0/2929204264 wait complete. 2024-09-21T07:52:46.478 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-21T07:52:45.723271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262092,"kb_used_data":12548,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712628,"statfs":{"total":773094113280,"available":772825731072,"internally_reserved":0,"allocated":12849152,"data_stored":6295727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.706306"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027993+0000","last_change":"2024-09-21T07:52:41.027993+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027993+0000","last_undegraded":"2024-09-21T07:52:41.027993+0000","last_fullsized":"2024-09-21T07:52:41.027993+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031021+0000","last_change":"2024-09-21T07:52:41.031021+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031021+0000","last_undegraded":"2024-09-21T07:52:41.031021+0000","last_fullsized":"2024-09-21T07:52:41.031021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035228+0000","last_change":"2024-09-21T07:52:41.035228+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035228+0000","last_undegraded":"2024-09-21T07:52:41.035228+0000","last_fullsized":"2024-09-21T07:52:41.035228+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027064+0000","last_change":"2024-09-21T07:52:41.027064+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027064+0000","last_undegraded":"2024-09-21T07:52:41.027064+0000","last_fullsized":"2024-09-21T07:52:41.027064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035220+0000","last_change":"2024-09-21T07:52:41.035220+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035220+0000","last_undegraded":"2024-09-21T07:52:41.035220+0000","last_fullsized":"2024-09-21T07:52:41.035220+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.040021+0000","last_change":"2024-09-21T07:52:41.040021+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.040021+0000","last_undegraded":"2024-09-21T07:52:41.040021+0000","last_fullsized":"2024-09-21T07:52:41.040021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.036420+0000","last_change":"2024-09-21T07:52:41.036420+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.036420+0000","last_undegraded":"2024-09-21T07:52:41.036420+0000","last_fullsized":"2024-09-21T07:52:41.036420+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032595+0000","last_change":"2024-09-21T07:52:41.032595+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032595+0000","last_undegraded":"2024-09-21T07:52:41.032595+0000","last_fullsized":"2024-09-21T07:52:41.032595+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.039205+0000","last_change":"2024-09-21T07:52:41.039205+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.039205+0000","last_undegraded":"2024-09-21T07:52:41.039205+0000","last_fullsized":"2024-09-21T07:52:41.039205+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030391+0000","last_change":"2024-09-21T07:52:41.030391+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030391+0000","last_undegraded":"2024-09-21T07:52:41.030391+0000","last_fullsized":"2024-09-21T07:52:41.030391+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027094+0000","last_change":"2024-09-21T07:52:41.027094+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027094+0000","last_undegraded":"2024-09-21T07:52:41.027094+0000","last_fullsized":"2024-09-21T07:52:41.027094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034190+0000","last_change":"2024-09-21T07:52:41.034190+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034190+0000","last_undegraded":"2024-09-21T07:52:41.034190+0000","last_fullsized":"2024-09-21T07:52:41.034190+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.026540+0000","last_change":"2024-09-21T07:52:41.026540+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.026540+0000","last_undegraded":"2024-09-21T07:52:41.026540+0000","last_fullsized":"2024-09-21T07:52:41.026540+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035819+0000","last_change":"2024-09-21T07:52:41.035819+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035819+0000","last_undegraded":"2024-09-21T07:52:41.035819+0000","last_fullsized":"2024-09-21T07:52:41.035819+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033553+0000","last_change":"2024-09-21T07:52:41.033553+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033553+0000","last_undegraded":"2024-09-21T07:52:41.033553+0000","last_fullsized":"2024-09-21T07:52:41.033553+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.038549+0000","last_change":"2024-09-21T07:52:41.038549+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.038549+0000","last_undegraded":"2024-09-21T07:52:41.038549+0000","last_fullsized":"2024-09-21T07:52:41.038549+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033229+0000","last_change":"2024-09-21T07:52:41.033229+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033229+0000","last_undegraded":"2024-09-21T07:52:41.033229+0000","last_fullsized":"2024-09-21T07:52:41.033229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035039+0000","last_change":"2024-09-21T07:52:41.035039+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035039+0000","last_undegraded":"2024-09-21T07:52:41.035039+0000","last_fullsized":"2024-09-21T07:52:41.035039+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029611+0000","last_change":"2024-09-21T07:52:41.029611+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029611+0000","last_undegraded":"2024-09-21T07:52:41.029611+0000","last_fullsized":"2024-09-21T07:52:41.029611+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.036505+0000","last_change":"2024-09-21T07:52:41.036505+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.036505+0000","last_undegraded":"2024-09-21T07:52:41.036505+0000","last_fullsized":"2024-09-21T07:52:41.036505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034977+0000","last_change":"2024-09-21T07:52:41.034977+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034977+0000","last_undegraded":"2024-09-21T07:52:41.034977+0000","last_fullsized":"2024-09-21T07:52:41.034977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030494+0000","last_change":"2024-09-21T07:52:41.030494+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030494+0000","last_undegraded":"2024-09-21T07:52:41.030494+0000","last_fullsized":"2024-09-21T07:52:41.030494+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037052+0000","last_change":"2024-09-21T07:52:41.037052+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037052+0000","last_undegraded":"2024-09-21T07:52:41.037052+0000","last_fullsized":"2024-09-21T07:52:41.037052+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035828+0000","last_change":"2024-09-21T07:52:41.035828+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035828+0000","last_undegraded":"2024-09-21T07:52:41.035828+0000","last_fullsized":"2024-09-21T07:52:41.035828+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027380+0000","last_change":"2024-09-21T07:52:41.027380+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027380+0000","last_undegraded":"2024-09-21T07:52:41.027380+0000","last_fullsized":"2024-09-21T07:52:41.027380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029802+0000","last_change":"2024-09-21T07:52:41.029802+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029802+0000","last_undegraded":"2024-09-21T07:52:41.029802+0000","last_fullsized":"2024-09-21T07:52:41.029802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033847+0000","last_change":"2024-09-21T07:52:41.033847+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033847+0000","last_undegraded":"2024-09-21T07:52:41.033847+0000","last_fullsized":"2024-09-21T07:52:41.033847+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030093+0000","last_change":"2024-09-21T07:52:41.030093+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030093+0000","last_undegraded":"2024-09-21T07:52:41.030093+0000","last_fullsized":"2024-09-21T07:52:41.030093+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.028483+0000","last_change":"2024-09-21T07:52:41.028483+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.028483+0000","last_undegraded":"2024-09-21T07:52:41.028483+0000","last_fullsized":"2024-09-21T07:52:41.028483+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029992+0000","last_change":"2024-09-21T07:52:41.029992+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029992+0000","last_undegraded":"2024-09-21T07:52:41.029992+0000","last_fullsized":"2024-09-21T07:52:41.029992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030493+0000","last_change":"2024-09-21T07:52:41.030493+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030493+0000","last_undegraded":"2024-09-21T07:52:41.030493+0000","last_fullsized":"2024-09-21T07:52:41.030493+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033920+0000","last_change":"2024-09-21T07:52:41.033920+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033920+0000","last_undegraded":"2024-09-21T07:52:41.033920+0000","last_fullsized":"2024-09-21T07:52:41.033920+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035011+0000","last_change":"2024-09-21T07:52:41.035011+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035011+0000","last_undegraded":"2024-09-21T07:52:41.035011+0000","last_fullsized":"2024-09-21T07:52:41.035011+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054190+0000","last_change":"2024-09-21T07:52:41.052517+0000","last_active":"2024-09-21T07:52:41.054190+0000","last_peered":"2024-09-21T07:52:41.054190+0000","last_clean":"2024-09-21T07:52:41.054190+0000","last_became_active":"2024-09-21T07:52:41.052152+0000","last_became_peered":"2024-09-21T07:52:41.052152+0000","last_unstale":"2024-09-21T07:52:41.054190+0000","last_undegraded":"2024-09-21T07:52:41.054190+0000","last_fullsized":"2024-09-21T07:52:41.054190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:43.545555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.038710+0000","last_change":"2024-09-21T07:52:41.038710+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.038710+0000","last_undegraded":"2024-09-21T07:52:41.038710+0000","last_fullsized":"2024-09-21T07:52:41.038710+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054932+0000","last_change":"2024-09-21T07:52:41.051644+0000","last_active":"2024-09-21T07:52:41.054932+0000","last_peered":"2024-09-21T07:52:41.054932+0000","last_clean":"2024-09-21T07:52:41.054932+0000","last_became_active":"2024-09-21T07:52:41.051270+0000","last_became_peered":"2024-09-21T07:52:41.051270+0000","last_unstale":"2024-09-21T07:52:41.054932+0000","last_undegraded":"2024-09-21T07:52:41.054932+0000","last_fullsized":"2024-09-21T07:52:41.054932+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:19.575593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031733+0000","last_change":"2024-09-21T07:52:41.031733+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031733+0000","last_undegraded":"2024-09-21T07:52:41.031733+0000","last_fullsized":"2024-09-21T07:52:41.031733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045732+0000","last_change":"2024-09-21T07:52:41.040236+0000","last_active":"2024-09-21T07:52:41.045732+0000","last_peered":"2024-09-21T07:52:41.045732+0000","last_clean":"2024-09-21T07:52:41.045732+0000","last_became_active":"2024-09-21T07:52:41.039288+0000","last_became_peered":"2024-09-21T07:52:41.039288+0000","last_unstale":"2024-09-21T07:52:41.045732+0000","last_undegraded":"2024-09-21T07:52:41.045732+0000","last_fullsized":"2024-09-21T07:52:41.045732+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:09.577922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035750+0000","last_change":"2024-09-21T07:52:41.035750+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035750+0000","last_undegraded":"2024-09-21T07:52:41.035750+0000","last_fullsized":"2024-09-21T07:52:41.035750+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053006+0000","last_change":"2024-09-21T07:52:41.049531+0000","last_active":"2024-09-21T07:52:41.053006+0000","last_peered":"2024-09-21T07:52:41.053006+0000","last_clean":"2024-09-21T07:52:41.053006+0000","last_became_active":"2024-09-21T07:52:41.048245+0000","last_became_peered":"2024-09-21T07:52:41.048245+0000","last_unstale":"2024-09-21T07:52:41.053006+0000","last_undegraded":"2024-09-21T07:52:41.053006+0000","last_fullsized":"2024-09-21T07:52:41.053006+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:44.807993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037343+0000","last_change":"2024-09-21T07:52:41.037343+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037343+0000","last_undegraded":"2024-09-21T07:52:41.037343+0000","last_fullsized":"2024-09-21T07:52:41.037343+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045753+0000","last_change":"2024-09-21T07:52:41.039897+0000","last_active":"2024-09-21T07:52:41.045753+0000","last_peered":"2024-09-21T07:52:41.045753+0000","last_clean":"2024-09-21T07:52:41.045753+0000","last_became_active":"2024-09-21T07:52:41.039535+0000","last_became_peered":"2024-09-21T07:52:41.039535+0000","last_unstale":"2024-09-21T07:52:41.045753+0000","last_undegraded":"2024-09-21T07:52:41.045753+0000","last_fullsized":"2024-09-21T07:52:41.045753+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:22.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034529+0000","last_change":"2024-09-21T07:52:41.034529+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034529+0000","last_undegraded":"2024-09-21T07:52:41.034529+0000","last_fullsized":"2024-09-21T07:52:41.034529+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052555+0000","last_change":"2024-09-21T07:52:41.048831+0000","last_active":"2024-09-21T07:52:41.052555+0000","last_peered":"2024-09-21T07:52:41.052555+0000","last_clean":"2024-09-21T07:52:41.052555+0000","last_became_active":"2024-09-21T07:52:41.048477+0000","last_became_peered":"2024-09-21T07:52:41.048477+0000","last_unstale":"2024-09-21T07:52:41.052555+0000","last_undegraded":"2024-09-21T07:52:41.052555+0000","last_fullsized":"2024-09-21T07:52:41.052555+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:46:09.004978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031583+0000","last_change":"2024-09-21T07:52:41.031583+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031583+0000","last_undegraded":"2024-09-21T07:52:41.031583+0000","last_fullsized":"2024-09-21T07:52:41.031583+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054414+0000","last_change":"2024-09-21T07:52:41.046640+0000","last_active":"2024-09-21T07:52:41.054414+0000","last_peered":"2024-09-21T07:52:41.054414+0000","last_clean":"2024-09-21T07:52:41.054414+0000","last_became_active":"2024-09-21T07:52:41.046298+0000","last_became_peered":"2024-09-21T07:52:41.046298+0000","last_unstale":"2024-09-21T07:52:41.054414+0000","last_undegraded":"2024-09-21T07:52:41.054414+0000","last_fullsized":"2024-09-21T07:52:41.054414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:49.783011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029388+0000","last_change":"2024-09-21T07:52:41.029388+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029388+0000","last_undegraded":"2024-09-21T07:52:41.029388+0000","last_fullsized":"2024-09-21T07:52:41.029388+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047618+0000","last_change":"2024-09-21T07:52:41.043692+0000","last_active":"2024-09-21T07:52:41.047618+0000","last_peered":"2024-09-21T07:52:41.047618+0000","last_clean":"2024-09-21T07:52:41.047618+0000","last_became_active":"2024-09-21T07:52:41.043389+0000","last_became_peered":"2024-09-21T07:52:41.043389+0000","last_unstale":"2024-09-21T07:52:41.047618+0000","last_undegraded":"2024-09-21T07:52:41.047618+0000","last_fullsized":"2024-09-21T07:52:41.047618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:30.458696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029191+0000","last_change":"2024-09-21T07:52:41.029191+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029191+0000","last_undegraded":"2024-09-21T07:52:41.029191+0000","last_fullsized":"2024-09-21T07:52:41.029191+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052502+0000","last_change":"2024-09-21T07:52:41.049633+0000","last_active":"2024-09-21T07:52:41.052502+0000","last_peered":"2024-09-21T07:52:41.052502+0000","last_clean":"2024-09-21T07:52:41.052502+0000","last_became_active":"2024-09-21T07:52:41.047189+0000","last_became_peered":"2024-09-21T07:52:41.047189+0000","last_unstale":"2024-09-21T07:52:41.052502+0000","last_undegraded":"2024-09-21T07:52:41.052502+0000","last_fullsized":"2024-09-21T07:52:41.052502+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:13.359032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.041579+0000","last_change":"2024-09-21T07:52:41.041579+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.041579+0000","last_undegraded":"2024-09-21T07:52:41.041579+0000","last_fullsized":"2024-09-21T07:52:41.041579+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052501+0000","last_change":"2024-09-21T07:52:41.048436+0000","last_active":"2024-09-21T07:52:41.052501+0000","last_peered":"2024-09-21T07:52:41.052501+0000","last_clean":"2024-09-21T07:52:41.052501+0000","last_became_active":"2024-09-21T07:52:41.047717+0000","last_became_peered":"2024-09-21T07:52:41.047717+0000","last_unstale":"2024-09-21T07:52:41.052501+0000","last_undegraded":"2024-09-21T07:52:41.052501+0000","last_fullsized":"2024-09-21T07:52:41.052501+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:28.786853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033729+0000","last_change":"2024-09-21T07:52:41.033729+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033729+0000","last_undegraded":"2024-09-21T07:52:41.033729+0000","last_fullsized":"2024-09-21T07:52:41.033729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054694+0000","last_change":"2024-09-21T07:52:41.051335+0000","last_active":"2024-09-21T07:52:41.054694+0000","last_peered":"2024-09-21T07:52:41.054694+0000","last_clean":"2024-09-21T07:52:41.054694+0000","last_became_active":"2024-09-21T07:52:41.051017+0000","last_became_peered":"2024-09-21T07:52:41.051017+0000","last_unstale":"2024-09-21T07:52:41.054694+0000","last_undegraded":"2024-09-21T07:52:41.054694+0000","last_fullsized":"2024-09-21T07:52:41.054694+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:38.414746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030074+0000","last_change":"2024-09-21T07:52:41.030074+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030074+0000","last_undegraded":"2024-09-21T07:52:41.030074+0000","last_fullsized":"2024-09-21T07:52:41.030074+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045672+0000","last_change":"2024-09-21T07:52:41.042291+0000","last_active":"2024-09-21T07:52:41.045672+0000","last_peered":"2024-09-21T07:52:41.045672+0000","last_clean":"2024-09-21T07:52:41.045672+0000","last_became_active":"2024-09-21T07:52:41.041543+0000","last_became_peered":"2024-09-21T07:52:41.041543+0000","last_unstale":"2024-09-21T07:52:41.045672+0000","last_undegraded":"2024-09-21T07:52:41.045672+0000","last_fullsized":"2024-09-21T07:52:41.045672+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:22.388273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031136+0000","last_change":"2024-09-21T07:52:41.031136+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031136+0000","last_undegraded":"2024-09-21T07:52:41.031136+0000","last_fullsized":"2024-09-21T07:52:41.031136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048030+0000","last_change":"2024-09-21T07:52:41.043893+0000","last_active":"2024-09-21T07:52:41.048030+0000","last_peered":"2024-09-21T07:52:41.048030+0000","last_clean":"2024-09-21T07:52:41.048030+0000","last_became_active":"2024-09-21T07:52:41.043513+0000","last_became_peered":"2024-09-21T07:52:41.043513+0000","last_unstale":"2024-09-21T07:52:41.048030+0000","last_undegraded":"2024-09-21T07:52:41.048030+0000","last_fullsized":"2024-09-21T07:52:41.048030+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:26.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037809+0000","last_change":"2024-09-21T07:52:41.037809+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037809+0000","last_undegraded":"2024-09-21T07:52:41.037809+0000","last_fullsized":"2024-09-21T07:52:41.037809+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047619+0000","last_change":"2024-09-21T07:52:41.043649+0000","last_active":"2024-09-21T07:52:41.047619+0000","last_peered":"2024-09-21T07:52:41.047619+0000","last_clean":"2024-09-21T07:52:41.047619+0000","last_became_active":"2024-09-21T07:52:41.043360+0000","last_became_peered":"2024-09-21T07:52:41.043360+0000","last_unstale":"2024-09-21T07:52:41.047619+0000","last_undegraded":"2024-09-21T07:52:41.047619+0000","last_fullsized":"2024-09-21T07:52:41.047619+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:51.651416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033514+0000","last_change":"2024-09-21T07:52:41.033514+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033514+0000","last_undegraded":"2024-09-21T07:52:41.033514+0000","last_fullsized":"2024-09-21T07:52:41.033514+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052214+0000","last_change":"2024-09-21T07:52:41.049536+0000","last_active":"2024-09-21T07:52:41.052214+0000","last_peered":"2024-09-21T07:52:41.052214+0000","last_clean":"2024-09-21T07:52:41.052214+0000","last_became_active":"2024-09-21T07:52:41.045025+0000","last_became_peered":"2024-09-21T07:52:41.045025+0000","last_unstale":"2024-09-21T07:52:41.052214+0000","last_undegraded":"2024-09-21T07:52:41.052214+0000","last_fullsized":"2024-09-21T07:52:41.052214+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:26.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.039796+0000","last_change":"2024-09-21T07:52:41.039796+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.039796+0000","last_undegraded":"2024-09-21T07:52:41.039796+0000","last_fullsized":"2024-09-21T07:52:41.039796+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053707+0000","last_change":"2024-09-21T07:52:41.050888+0000","last_active":"2024-09-21T07:52:41.053707+0000","last_peered":"2024-09-21T07:52:41.053707+0000","last_clean":"2024-09-21T07:52:41.053707+0000","last_became_active":"2024-09-21T07:52:41.050131+0000","last_became_peered":"2024-09-21T07:52:41.050131+0000","last_unstale":"2024-09-21T07:52:41.053707+0000","last_undegraded":"2024-09-21T07:52:41.053707+0000","last_fullsized":"2024-09-21T07:52:41.053707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:32.028954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.041080+0000","last_change":"2024-09-21T07:52:41.041080+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.041080+0000","last_undegraded":"2024-09-21T07:52:41.041080+0000","last_fullsized":"2024-09-21T07:52:41.041080+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045546+0000","last_change":"2024-09-21T07:52:41.036154+0000","last_active":"2024-09-21T07:52:41.045546+0000","last_peered":"2024-09-21T07:52:41.045546+0000","last_clean":"2024-09-21T07:52:41.045546+0000","last_became_active":"2024-09-21T07:52:41.035812+0000","last_became_peered":"2024-09-21T07:52:41.035812+0000","last_unstale":"2024-09-21T07:52:41.045546+0000","last_undegraded":"2024-09-21T07:52:41.045546+0000","last_fullsized":"2024-09-21T07:52:41.045546+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:37.178689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027661+0000","last_change":"2024-09-21T07:52:41.027661+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027661+0000","last_undegraded":"2024-09-21T07:52:41.027661+0000","last_fullsized":"2024-09-21T07:52:41.027661+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053420+0000","last_change":"2024-09-21T07:52:41.046956+0000","last_active":"2024-09-21T07:52:41.053420+0000","last_peered":"2024-09-21T07:52:41.053420+0000","last_clean":"2024-09-21T07:52:41.053420+0000","last_became_active":"2024-09-21T07:52:41.046644+0000","last_became_peered":"2024-09-21T07:52:41.046644+0000","last_unstale":"2024-09-21T07:52:41.053420+0000","last_undegraded":"2024-09-21T07:52:41.053420+0000","last_fullsized":"2024-09-21T07:52:41.053420+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:36.785804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029014+0000","last_change":"2024-09-21T07:52:41.029014+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029014+0000","last_undegraded":"2024-09-21T07:52:41.029014+0000","last_fullsized":"2024-09-21T07:52:41.029014+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.049922+0000","last_change":"2024-09-21T07:52:41.047727+0000","last_active":"2024-09-21T07:52:41.049922+0000","last_peered":"2024-09-21T07:52:41.049922+0000","last_clean":"2024-09-21T07:52:41.049922+0000","last_became_active":"2024-09-21T07:52:41.047444+0000","last_became_peered":"2024-09-21T07:52:41.047444+0000","last_unstale":"2024-09-21T07:52:41.049922+0000","last_undegraded":"2024-09-21T07:52:41.049922+0000","last_fullsized":"2024-09-21T07:52:41.049922+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:21.613841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029439+0000","last_change":"2024-09-21T07:52:41.029439+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029439+0000","last_undegraded":"2024-09-21T07:52:41.029439+0000","last_fullsized":"2024-09-21T07:52:41.029439+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048348+0000","last_change":"2024-09-21T07:52:41.045986+0000","last_active":"2024-09-21T07:52:41.048348+0000","last_peered":"2024-09-21T07:52:41.048348+0000","last_clean":"2024-09-21T07:52:41.048348+0000","last_became_active":"2024-09-21T07:52:41.045666+0000","last_became_peered":"2024-09-21T07:52:41.045666+0000","last_unstale":"2024-09-21T07:52:41.048348+0000","last_undegraded":"2024-09-21T07:52:41.048348+0000","last_fullsized":"2024-09-21T07:52:41.048348+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:21.170296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029347+0000","last_change":"2024-09-21T07:52:41.029347+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029347+0000","last_undegraded":"2024-09-21T07:52:41.029347+0000","last_fullsized":"2024-09-21T07:52:41.029347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053445+0000","last_change":"2024-09-21T07:52:41.051206+0000","last_active":"2024-09-21T07:52:41.053445+0000","last_peered":"2024-09-21T07:52:41.053445+0000","last_clean":"2024-09-21T07:52:41.053445+0000","last_became_active":"2024-09-21T07:52:41.050919+0000","last_became_peered":"2024-09-21T07:52:41.050919+0000","last_unstale":"2024-09-21T07:52:41.053445+0000","last_undegraded":"2024-09-21T07:52:41.053445+0000","last_fullsized":"2024-09-21T07:52:41.053445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:19.295030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037219+0000","last_change":"2024-09-21T07:52:41.037219+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037219+0000","last_undegraded":"2024-09-21T07:52:41.037219+0000","last_fullsized":"2024-09-21T07:52:41.037219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048075+0000","last_change":"2024-09-21T07:52:41.044491+0000","last_active":"2024-09-21T07:52:41.048075+0000","last_peered":"2024-09-21T07:52:41.048075+0000","last_clean":"2024-09-21T07:52:41.048075+0000","last_became_active":"2024-09-21T07:52:41.043914+0000","last_became_peered":"2024-09-21T07:52:41.043914+0000","last_unstale":"2024-09-21T07:52:41.048075+0000","last_undegraded":"2024-09-21T07:52:41.048075+0000","last_fullsized":"2024-09-21T07:52:41.048075+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:13.359032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032466+0000","last_change":"2024-09-21T07:52:41.032466+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032466+0000","last_undegraded":"2024-09-21T07:52:41.032466+0000","last_fullsized":"2024-09-21T07:52:41.032466+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045792+0000","last_change":"2024-09-21T07:52:41.040325+0000","last_active":"2024-09-21T07:52:41.045792+0000","last_peered":"2024-09-21T07:52:41.045792+0000","last_clean":"2024-09-21T07:52:41.045792+0000","last_became_active":"2024-09-21T07:52:41.039761+0000","last_became_peered":"2024-09-21T07:52:41.039761+0000","last_unstale":"2024-09-21T07:52:41.045792+0000","last_undegraded":"2024-09-21T07:52:41.045792+0000","last_fullsized":"2024-09-21T07:52:41.045792+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:27.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029377+0000","last_change":"2024-09-21T07:52:41.029377+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029377+0000","last_undegraded":"2024-09-21T07:52:41.029377+0000","last_fullsized":"2024-09-21T07:52:41.029377+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.056786+0000","last_change":"2024-09-21T07:52:41.056134+0000","last_active":"2024-09-21T07:52:41.056786+0000","last_peered":"2024-09-21T07:52:41.056786+0000","last_clean":"2024-09-21T07:52:41.056786+0000","last_became_active":"2024-09-21T07:52:41.055763+0000","last_became_peered":"2024-09-21T07:52:41.055763+0000","last_unstale":"2024-09-21T07:52:41.056786+0000","last_undegraded":"2024-09-21T07:52:41.056786+0000","last_fullsized":"2024-09-21T07:52:41.056786+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:15.231881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.026772+0000","last_change":"2024-09-21T07:52:41.026772+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.026772+0000","last_undegraded":"2024-09-21T07:52:41.026772+0000","last_fullsized":"2024-09-21T07:52:41.026772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054453+0000","last_change":"2024-09-21T07:52:41.053277+0000","last_active":"2024-09-21T07:52:41.054453+0000","last_peered":"2024-09-21T07:52:41.054453+0000","last_clean":"2024-09-21T07:52:41.054453+0000","last_became_active":"2024-09-21T07:52:41.052592+0000","last_became_peered":"2024-09-21T07:52:41.052592+0000","last_unstale":"2024-09-21T07:52:41.054453+0000","last_undegraded":"2024-09-21T07:52:41.054453+0000","last_fullsized":"2024-09-21T07:52:41.054453+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.806940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034577+0000","last_change":"2024-09-21T07:52:41.034577+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034577+0000","last_undegraded":"2024-09-21T07:52:41.034577+0000","last_fullsized":"2024-09-21T07:52:41.034577+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.055973+0000","last_change":"2024-09-21T07:52:41.053679+0000","last_active":"2024-09-21T07:52:41.055973+0000","last_peered":"2024-09-21T07:52:41.055973+0000","last_clean":"2024-09-21T07:52:41.055973+0000","last_became_active":"2024-09-21T07:52:41.053337+0000","last_became_peered":"2024-09-21T07:52:41.053337+0000","last_unstale":"2024-09-21T07:52:41.055973+0000","last_undegraded":"2024-09-21T07:52:41.055973+0000","last_fullsized":"2024-09-21T07:52:41.055973+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:03.857276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034863+0000","last_change":"2024-09-21T07:52:41.034863+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034863+0000","last_undegraded":"2024-09-21T07:52:41.034863+0000","last_fullsized":"2024-09-21T07:52:41.034863+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.050242+0000","last_change":"2024-09-21T07:52:41.044918+0000","last_active":"2024-09-21T07:52:41.050242+0000","last_peered":"2024-09-21T07:52:41.050242+0000","last_clean":"2024-09-21T07:52:41.050242+0000","last_became_active":"2024-09-21T07:52:41.044501+0000","last_became_peered":"2024-09-21T07:52:41.044501+0000","last_unstale":"2024-09-21T07:52:41.050242+0000","last_undegraded":"2024-09-21T07:52:41.050242+0000","last_fullsized":"2024-09-21T07:52:41.050242+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:32.028954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033987+0000","last_change":"2024-09-21T07:52:41.033987+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033987+0000","last_undegraded":"2024-09-21T07:52:41.033987+0000","last_fullsized":"2024-09-21T07:52:41.033987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047581+0000","last_change":"2024-09-21T07:52:41.043000+0000","last_active":"2024-09-21T07:52:41.047581+0000","last_peered":"2024-09-21T07:52:41.047581+0000","last_clean":"2024-09-21T07:52:41.047581+0000","last_became_active":"2024-09-21T07:52:41.042576+0000","last_became_peered":"2024-09-21T07:52:41.042576+0000","last_unstale":"2024-09-21T07:52:41.047581+0000","last_undegraded":"2024-09-21T07:52:41.047581+0000","last_fullsized":"2024-09-21T07:52:41.047581+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:19.481187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034058+0000","last_change":"2024-09-21T07:52:41.034058+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034058+0000","last_undegraded":"2024-09-21T07:52:41.034058+0000","last_fullsized":"2024-09-21T07:52:41.034058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053262+0000","last_change":"2024-09-21T07:52:41.049196+0000","last_active":"2024-09-21T07:52:41.053262+0000","last_peered":"2024-09-21T07:52:41.053262+0000","last_clean":"2024-09-21T07:52:41.053262+0000","last_became_active":"2024-09-21T07:52:41.048712+0000","last_became_peered":"2024-09-21T07:52:41.048712+0000","last_unstale":"2024-09-21T07:52:41.053262+0000","last_undegraded":"2024-09-21T07:52:41.053262+0000","last_fullsized":"2024-09-21T07:52:41.053262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:03.580681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034036+0000","last_change":"2024-09-21T07:52:41.034036+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034036+0000","last_undegraded":"2024-09-21T07:52:41.034036+0000","last_fullsized":"2024-09-21T07:52:41.034036+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047590+0000","last_change":"2024-09-21T07:52:41.043466+0000","last_active":"2024-09-21T07:52:41.047590+0000","last_peered":"2024-09-21T07:52:41.047590+0000","last_clean":"2024-09-21T07:52:41.047590+0000","last_became_active":"2024-09-21T07:52:41.042404+0000","last_became_peered":"2024-09-21T07:52:41.042404+0000","last_unstale":"2024-09-21T07:52:41.047590+0000","last_undegraded":"2024-09-21T07:52:41.047590+0000","last_fullsized":"2024-09-21T07:52:41.047590+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:37.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032079+0000","last_change":"2024-09-21T07:52:41.032079+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032079+0000","last_undegraded":"2024-09-21T07:52:41.032079+0000","last_fullsized":"2024-09-21T07:52:41.032079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045488+0000","last_change":"2024-09-21T07:52:41.041714+0000","last_active":"2024-09-21T07:52:41.045488+0000","last_peered":"2024-09-21T07:52:41.045488+0000","last_clean":"2024-09-21T07:52:41.045488+0000","last_became_active":"2024-09-21T07:52:41.041407+0000","last_became_peered":"2024-09-21T07:52:41.041407+0000","last_unstale":"2024-09-21T07:52:41.045488+0000","last_undegraded":"2024-09-21T07:52:41.045488+0000","last_fullsized":"2024-09-21T07:52:41.045488+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:03.580681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031023+0000","last_change":"2024-09-21T07:52:41.031023+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031023+0000","last_undegraded":"2024-09-21T07:52:41.031023+0000","last_fullsized":"2024-09-21T07:52:41.031023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.051789+0000","last_change":"2024-09-21T07:52:41.045768+0000","last_active":"2024-09-21T07:52:41.051789+0000","last_peered":"2024-09-21T07:52:41.051789+0000","last_clean":"2024-09-21T07:52:41.051789+0000","last_became_active":"2024-09-21T07:52:41.045435+0000","last_became_peered":"2024-09-21T07:52:41.045435+0000","last_unstale":"2024-09-21T07:52:41.051789+0000","last_undegraded":"2024-09-21T07:52:41.051789+0000","last_fullsized":"2024-09-21T07:52:41.051789+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:05.057723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":320,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.026190+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:52:41.026190+0000","last_peered":"2024-09-21T07:52:41.026190+0000","last_clean":"2024-09-21T07:52:41.026190+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:52:41.026190+0000","last_undegraded":"2024-09-21T07:52:41.026190+0000","last_fullsized":"2024-09-21T07:52:41.026190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2043904,"data_stored":1225820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":635452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476823,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476823,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2056192,"data_stored":1232095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476820,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":635452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:52:46.483 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77f5200640 1 -- 172.21.15.88:0/558813901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 msgr2=0x7f77d400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77f5200640 1 --2- 172.21.15.88:0/558813901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77d400b9c0 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7f77e8009ed0 tx=0x7f77d4005830 comp rx=0 tx=0).stop 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77ee800640 1 -- 172.21.15.88:0/558813901 reap_dead start 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77f5200640 1 -- 172.21.15.88:0/558813901 shutdown_connections 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77f5200640 1 -- 172.21.15.88:0/558813901 >> 172.21.15.88:0/558813901 conn(0x7f77f006d730 msgr2=0x7f77f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77f5200640 1 -- 172.21.15.88:0/558813901 shutdown_connections 2024-09-21T07:52:46.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.740+0000 7f77f5200640 1 -- 172.21.15.88:0/558813901 wait complete. 2024-09-21T07:52:46.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.742+0000 7f77f5200640 1 Processor -- start 2024-09-21T07:52:46.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.743+0000 7f77f5200640 1 -- start start 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77f5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77f01542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77f5200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f00ceeb0 0x7f77f00997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77f5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f01f6510 0x7f77f01f6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77f5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77f00b3750 con 0x7f77d4005150 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77f5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77f015fd40 con 0x7f77f01f6510 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77f5200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77f00b38d0 con 0x7f77f00ceeb0 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77ede00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77f01542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77ede00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77f01542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57226/0 (socket says 172.21.15.88:57226) 2024-09-21T07:52:46.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.744+0000 7f77ede00640 1 -- 172.21.15.88:0/724374896 learned_addr learned my addr 172.21.15.88:0/724374896 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77e5400640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f00ceeb0 0x7f77f00997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ee800640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f01f6510 0x7f77f01f6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ede00640 1 -- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f00ceeb0 msgr2=0x7f77f00997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ede00640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f00ceeb0 0x7f77f00997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ede00640 1 -- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f01f6510 msgr2=0x7f77f01f6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ede00640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f01f6510 0x7f77f01f6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ede00640 1 -- 172.21.15.88:0/724374896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77f01f6e60 con 0x7f77d4005150 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ee800640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f01f6510 0x7f77f01f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77e5400640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f00ceeb0 0x7f77f00997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77ede00640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77f01542b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f77e0011b10 tx=0x7f77e0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e000fc80 con 0x7f77d4005150 2024-09-21T07:52:46.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77e000fe20 con 0x7f77d4005150 2024-09-21T07:52:46.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77f0157590 con 0x7f77d4005150 2024-09-21T07:52:46.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0004280 con 0x7f77d4005150 2024-09-21T07:52:46.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.745+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77f0157b50 con 0x7f77d4005150 2024-09-21T07:52:46.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.746+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77f015bb60 con 0x7f77d4005150 2024-09-21T07:52:46.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.748+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f77e00035f0 con 0x7f77d4005150 2024-09-21T07:52:46.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.749+0000 7f77e6a00640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77cc079010 0x7f77cc07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:46.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.749+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f77e009c0e0 con 0x7f77d4005150 2024-09-21T07:52:46.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.749+0000 7f77e5400640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77cc079010 0x7f77cc07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:46.752 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.749+0000 7f77e5400640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77cc079010 0x7f77cc07b520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f77d8000c00 tx=0x7f77d80067a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:46.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.753+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f77f015bb60 con 0x7f77d4005150 2024-09-21T07:52:46.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.936+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f77f0152920 con 0x7f77cc079010 2024-09-21T07:52:46.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.940+0000 7f77e6a00640 1 -- 172.21.15.88:0/724374896 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247113 (secure 0 0 0) 0x7f77f0152920 con 0x7f77cc079010 2024-09-21T07:52:46.946 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:46.947 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:52:46.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.944+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77cc079010 msgr2=0x7f77cc07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.944+0000 7f77f5200640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77cc079010 0x7f77cc07b520 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f77d8000c00 tx=0x7f77d80067a0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.944+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 msgr2=0x7f77f01542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.944+0000 7f77f5200640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77f01542b0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f77e0011b10 tx=0x7f77e0009f10 comp rx=0 tx=0).stop 2024-09-21T07:52:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 shutdown_connections 2024-09-21T07:52:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77cc079010 0x7f77cc07b520 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77f00ceeb0 0x7f77f00997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77f01f6510 0x7f77f01f6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 --2- 172.21.15.88:0/724374896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d4005150 0x7f77f01542b0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:46.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 >> 172.21.15.88:0/724374896 conn(0x7f77f006d730 msgr2=0x7f77f0068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:46.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 shutdown_connections 2024-09-21T07:52:46.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:46.945+0000 7f77f5200640 1 -- 172.21.15.88:0/724374896 wait complete. 2024-09-21T07:52:46.968 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-21T07:52:45.723271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262092,"kb_used_data":12548,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712628,"statfs":{"total":773094113280,"available":772825731072,"internally_reserved":0,"allocated":12849152,"data_stored":6295727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.706306"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027993+0000","last_change":"2024-09-21T07:52:41.027993+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027993+0000","last_undegraded":"2024-09-21T07:52:41.027993+0000","last_fullsized":"2024-09-21T07:52:41.027993+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031021+0000","last_change":"2024-09-21T07:52:41.031021+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031021+0000","last_undegraded":"2024-09-21T07:52:41.031021+0000","last_fullsized":"2024-09-21T07:52:41.031021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035228+0000","last_change":"2024-09-21T07:52:41.035228+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035228+0000","last_undegraded":"2024-09-21T07:52:41.035228+0000","last_fullsized":"2024-09-21T07:52:41.035228+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027064+0000","last_change":"2024-09-21T07:52:41.027064+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027064+0000","last_undegraded":"2024-09-21T07:52:41.027064+0000","last_fullsized":"2024-09-21T07:52:41.027064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035220+0000","last_change":"2024-09-21T07:52:41.035220+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035220+0000","last_undegraded":"2024-09-21T07:52:41.035220+0000","last_fullsized":"2024-09-21T07:52:41.035220+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.040021+0000","last_change":"2024-09-21T07:52:41.040021+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.040021+0000","last_undegraded":"2024-09-21T07:52:41.040021+0000","last_fullsized":"2024-09-21T07:52:41.040021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.036420+0000","last_change":"2024-09-21T07:52:41.036420+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.036420+0000","last_undegraded":"2024-09-21T07:52:41.036420+0000","last_fullsized":"2024-09-21T07:52:41.036420+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032595+0000","last_change":"2024-09-21T07:52:41.032595+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032595+0000","last_undegraded":"2024-09-21T07:52:41.032595+0000","last_fullsized":"2024-09-21T07:52:41.032595+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.039205+0000","last_change":"2024-09-21T07:52:41.039205+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.039205+0000","last_undegraded":"2024-09-21T07:52:41.039205+0000","last_fullsized":"2024-09-21T07:52:41.039205+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030391+0000","last_change":"2024-09-21T07:52:41.030391+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030391+0000","last_undegraded":"2024-09-21T07:52:41.030391+0000","last_fullsized":"2024-09-21T07:52:41.030391+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027094+0000","last_change":"2024-09-21T07:52:41.027094+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027094+0000","last_undegraded":"2024-09-21T07:52:41.027094+0000","last_fullsized":"2024-09-21T07:52:41.027094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034190+0000","last_change":"2024-09-21T07:52:41.034190+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034190+0000","last_undegraded":"2024-09-21T07:52:41.034190+0000","last_fullsized":"2024-09-21T07:52:41.034190+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.026540+0000","last_change":"2024-09-21T07:52:41.026540+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.026540+0000","last_undegraded":"2024-09-21T07:52:41.026540+0000","last_fullsized":"2024-09-21T07:52:41.026540+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035819+0000","last_change":"2024-09-21T07:52:41.035819+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035819+0000","last_undegraded":"2024-09-21T07:52:41.035819+0000","last_fullsized":"2024-09-21T07:52:41.035819+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033553+0000","last_change":"2024-09-21T07:52:41.033553+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033553+0000","last_undegraded":"2024-09-21T07:52:41.033553+0000","last_fullsized":"2024-09-21T07:52:41.033553+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.038549+0000","last_change":"2024-09-21T07:52:41.038549+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.038549+0000","last_undegraded":"2024-09-21T07:52:41.038549+0000","last_fullsized":"2024-09-21T07:52:41.038549+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033229+0000","last_change":"2024-09-21T07:52:41.033229+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033229+0000","last_undegraded":"2024-09-21T07:52:41.033229+0000","last_fullsized":"2024-09-21T07:52:41.033229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035039+0000","last_change":"2024-09-21T07:52:41.035039+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035039+0000","last_undegraded":"2024-09-21T07:52:41.035039+0000","last_fullsized":"2024-09-21T07:52:41.035039+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029611+0000","last_change":"2024-09-21T07:52:41.029611+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029611+0000","last_undegraded":"2024-09-21T07:52:41.029611+0000","last_fullsized":"2024-09-21T07:52:41.029611+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.036505+0000","last_change":"2024-09-21T07:52:41.036505+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.036505+0000","last_undegraded":"2024-09-21T07:52:41.036505+0000","last_fullsized":"2024-09-21T07:52:41.036505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034977+0000","last_change":"2024-09-21T07:52:41.034977+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034977+0000","last_undegraded":"2024-09-21T07:52:41.034977+0000","last_fullsized":"2024-09-21T07:52:41.034977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030494+0000","last_change":"2024-09-21T07:52:41.030494+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030494+0000","last_undegraded":"2024-09-21T07:52:41.030494+0000","last_fullsized":"2024-09-21T07:52:41.030494+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037052+0000","last_change":"2024-09-21T07:52:41.037052+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037052+0000","last_undegraded":"2024-09-21T07:52:41.037052+0000","last_fullsized":"2024-09-21T07:52:41.037052+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035828+0000","last_change":"2024-09-21T07:52:41.035828+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035828+0000","last_undegraded":"2024-09-21T07:52:41.035828+0000","last_fullsized":"2024-09-21T07:52:41.035828+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027380+0000","last_change":"2024-09-21T07:52:41.027380+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027380+0000","last_undegraded":"2024-09-21T07:52:41.027380+0000","last_fullsized":"2024-09-21T07:52:41.027380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029802+0000","last_change":"2024-09-21T07:52:41.029802+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029802+0000","last_undegraded":"2024-09-21T07:52:41.029802+0000","last_fullsized":"2024-09-21T07:52:41.029802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033847+0000","last_change":"2024-09-21T07:52:41.033847+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033847+0000","last_undegraded":"2024-09-21T07:52:41.033847+0000","last_fullsized":"2024-09-21T07:52:41.033847+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030093+0000","last_change":"2024-09-21T07:52:41.030093+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030093+0000","last_undegraded":"2024-09-21T07:52:41.030093+0000","last_fullsized":"2024-09-21T07:52:41.030093+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.028483+0000","last_change":"2024-09-21T07:52:41.028483+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.028483+0000","last_undegraded":"2024-09-21T07:52:41.028483+0000","last_fullsized":"2024-09-21T07:52:41.028483+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029992+0000","last_change":"2024-09-21T07:52:41.029992+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029992+0000","last_undegraded":"2024-09-21T07:52:41.029992+0000","last_fullsized":"2024-09-21T07:52:41.029992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030493+0000","last_change":"2024-09-21T07:52:41.030493+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030493+0000","last_undegraded":"2024-09-21T07:52:41.030493+0000","last_fullsized":"2024-09-21T07:52:41.030493+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033920+0000","last_change":"2024-09-21T07:52:41.033920+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033920+0000","last_undegraded":"2024-09-21T07:52:41.033920+0000","last_fullsized":"2024-09-21T07:52:41.033920+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035011+0000","last_change":"2024-09-21T07:52:41.035011+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035011+0000","last_undegraded":"2024-09-21T07:52:41.035011+0000","last_fullsized":"2024-09-21T07:52:41.035011+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054190+0000","last_change":"2024-09-21T07:52:41.052517+0000","last_active":"2024-09-21T07:52:41.054190+0000","last_peered":"2024-09-21T07:52:41.054190+0000","last_clean":"2024-09-21T07:52:41.054190+0000","last_became_active":"2024-09-21T07:52:41.052152+0000","last_became_peered":"2024-09-21T07:52:41.052152+0000","last_unstale":"2024-09-21T07:52:41.054190+0000","last_undegraded":"2024-09-21T07:52:41.054190+0000","last_fullsized":"2024-09-21T07:52:41.054190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:43.545555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.038710+0000","last_change":"2024-09-21T07:52:41.038710+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.038710+0000","last_undegraded":"2024-09-21T07:52:41.038710+0000","last_fullsized":"2024-09-21T07:52:41.038710+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054932+0000","last_change":"2024-09-21T07:52:41.051644+0000","last_active":"2024-09-21T07:52:41.054932+0000","last_peered":"2024-09-21T07:52:41.054932+0000","last_clean":"2024-09-21T07:52:41.054932+0000","last_became_active":"2024-09-21T07:52:41.051270+0000","last_became_peered":"2024-09-21T07:52:41.051270+0000","last_unstale":"2024-09-21T07:52:41.054932+0000","last_undegraded":"2024-09-21T07:52:41.054932+0000","last_fullsized":"2024-09-21T07:52:41.054932+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:19.575593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031733+0000","last_change":"2024-09-21T07:52:41.031733+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031733+0000","last_undegraded":"2024-09-21T07:52:41.031733+0000","last_fullsized":"2024-09-21T07:52:41.031733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045732+0000","last_change":"2024-09-21T07:52:41.040236+0000","last_active":"2024-09-21T07:52:41.045732+0000","last_peered":"2024-09-21T07:52:41.045732+0000","last_clean":"2024-09-21T07:52:41.045732+0000","last_became_active":"2024-09-21T07:52:41.039288+0000","last_became_peered":"2024-09-21T07:52:41.039288+0000","last_unstale":"2024-09-21T07:52:41.045732+0000","last_undegraded":"2024-09-21T07:52:41.045732+0000","last_fullsized":"2024-09-21T07:52:41.045732+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:09.577922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035750+0000","last_change":"2024-09-21T07:52:41.035750+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035750+0000","last_undegraded":"2024-09-21T07:52:41.035750+0000","last_fullsized":"2024-09-21T07:52:41.035750+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053006+0000","last_change":"2024-09-21T07:52:41.049531+0000","last_active":"2024-09-21T07:52:41.053006+0000","last_peered":"2024-09-21T07:52:41.053006+0000","last_clean":"2024-09-21T07:52:41.053006+0000","last_became_active":"2024-09-21T07:52:41.048245+0000","last_became_peered":"2024-09-21T07:52:41.048245+0000","last_unstale":"2024-09-21T07:52:41.053006+0000","last_undegraded":"2024-09-21T07:52:41.053006+0000","last_fullsized":"2024-09-21T07:52:41.053006+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:44.807993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037343+0000","last_change":"2024-09-21T07:52:41.037343+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037343+0000","last_undegraded":"2024-09-21T07:52:41.037343+0000","last_fullsized":"2024-09-21T07:52:41.037343+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045753+0000","last_change":"2024-09-21T07:52:41.039897+0000","last_active":"2024-09-21T07:52:41.045753+0000","last_peered":"2024-09-21T07:52:41.045753+0000","last_clean":"2024-09-21T07:52:41.045753+0000","last_became_active":"2024-09-21T07:52:41.039535+0000","last_became_peered":"2024-09-21T07:52:41.039535+0000","last_unstale":"2024-09-21T07:52:41.045753+0000","last_undegraded":"2024-09-21T07:52:41.045753+0000","last_fullsized":"2024-09-21T07:52:41.045753+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:22.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034529+0000","last_change":"2024-09-21T07:52:41.034529+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034529+0000","last_undegraded":"2024-09-21T07:52:41.034529+0000","last_fullsized":"2024-09-21T07:52:41.034529+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052555+0000","last_change":"2024-09-21T07:52:41.048831+0000","last_active":"2024-09-21T07:52:41.052555+0000","last_peered":"2024-09-21T07:52:41.052555+0000","last_clean":"2024-09-21T07:52:41.052555+0000","last_became_active":"2024-09-21T07:52:41.048477+0000","last_became_peered":"2024-09-21T07:52:41.048477+0000","last_unstale":"2024-09-21T07:52:41.052555+0000","last_undegraded":"2024-09-21T07:52:41.052555+0000","last_fullsized":"2024-09-21T07:52:41.052555+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:46:09.004978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031583+0000","last_change":"2024-09-21T07:52:41.031583+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031583+0000","last_undegraded":"2024-09-21T07:52:41.031583+0000","last_fullsized":"2024-09-21T07:52:41.031583+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054414+0000","last_change":"2024-09-21T07:52:41.046640+0000","last_active":"2024-09-21T07:52:41.054414+0000","last_peered":"2024-09-21T07:52:41.054414+0000","last_clean":"2024-09-21T07:52:41.054414+0000","last_became_active":"2024-09-21T07:52:41.046298+0000","last_became_peered":"2024-09-21T07:52:41.046298+0000","last_unstale":"2024-09-21T07:52:41.054414+0000","last_undegraded":"2024-09-21T07:52:41.054414+0000","last_fullsized":"2024-09-21T07:52:41.054414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:49.783011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029388+0000","last_change":"2024-09-21T07:52:41.029388+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029388+0000","last_undegraded":"2024-09-21T07:52:41.029388+0000","last_fullsized":"2024-09-21T07:52:41.029388+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047618+0000","last_change":"2024-09-21T07:52:41.043692+0000","last_active":"2024-09-21T07:52:41.047618+0000","last_peered":"2024-09-21T07:52:41.047618+0000","last_clean":"2024-09-21T07:52:41.047618+0000","last_became_active":"2024-09-21T07:52:41.043389+0000","last_became_peered":"2024-09-21T07:52:41.043389+0000","last_unstale":"2024-09-21T07:52:41.047618+0000","last_undegraded":"2024-09-21T07:52:41.047618+0000","last_fullsized":"2024-09-21T07:52:41.047618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:30.458696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029191+0000","last_change":"2024-09-21T07:52:41.029191+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029191+0000","last_undegraded":"2024-09-21T07:52:41.029191+0000","last_fullsized":"2024-09-21T07:52:41.029191+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052502+0000","last_change":"2024-09-21T07:52:41.049633+0000","last_active":"2024-09-21T07:52:41.052502+0000","last_peered":"2024-09-21T07:52:41.052502+0000","last_clean":"2024-09-21T07:52:41.052502+0000","last_became_active":"2024-09-21T07:52:41.047189+0000","last_became_peered":"2024-09-21T07:52:41.047189+0000","last_unstale":"2024-09-21T07:52:41.052502+0000","last_undegraded":"2024-09-21T07:52:41.052502+0000","last_fullsized":"2024-09-21T07:52:41.052502+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:13.359032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.041579+0000","last_change":"2024-09-21T07:52:41.041579+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.041579+0000","last_undegraded":"2024-09-21T07:52:41.041579+0000","last_fullsized":"2024-09-21T07:52:41.041579+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052501+0000","last_change":"2024-09-21T07:52:41.048436+0000","last_active":"2024-09-21T07:52:41.052501+0000","last_peered":"2024-09-21T07:52:41.052501+0000","last_clean":"2024-09-21T07:52:41.052501+0000","last_became_active":"2024-09-21T07:52:41.047717+0000","last_became_peered":"2024-09-21T07:52:41.047717+0000","last_unstale":"2024-09-21T07:52:41.052501+0000","last_undegraded":"2024-09-21T07:52:41.052501+0000","last_fullsized":"2024-09-21T07:52:41.052501+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:28.786853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033729+0000","last_change":"2024-09-21T07:52:41.033729+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033729+0000","last_undegraded":"2024-09-21T07:52:41.033729+0000","last_fullsized":"2024-09-21T07:52:41.033729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054694+0000","last_change":"2024-09-21T07:52:41.051335+0000","last_active":"2024-09-21T07:52:41.054694+0000","last_peered":"2024-09-21T07:52:41.054694+0000","last_clean":"2024-09-21T07:52:41.054694+0000","last_became_active":"2024-09-21T07:52:41.051017+0000","last_became_peered":"2024-09-21T07:52:41.051017+0000","last_unstale":"2024-09-21T07:52:41.054694+0000","last_undegraded":"2024-09-21T07:52:41.054694+0000","last_fullsized":"2024-09-21T07:52:41.054694+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:38.414746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030074+0000","last_change":"2024-09-21T07:52:41.030074+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030074+0000","last_undegraded":"2024-09-21T07:52:41.030074+0000","last_fullsized":"2024-09-21T07:52:41.030074+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045672+0000","last_change":"2024-09-21T07:52:41.042291+0000","last_active":"2024-09-21T07:52:41.045672+0000","last_peered":"2024-09-21T07:52:41.045672+0000","last_clean":"2024-09-21T07:52:41.045672+0000","last_became_active":"2024-09-21T07:52:41.041543+0000","last_became_peered":"2024-09-21T07:52:41.041543+0000","last_unstale":"2024-09-21T07:52:41.045672+0000","last_undegraded":"2024-09-21T07:52:41.045672+0000","last_fullsized":"2024-09-21T07:52:41.045672+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:22.388273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031136+0000","last_change":"2024-09-21T07:52:41.031136+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031136+0000","last_undegraded":"2024-09-21T07:52:41.031136+0000","last_fullsized":"2024-09-21T07:52:41.031136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048030+0000","last_change":"2024-09-21T07:52:41.043893+0000","last_active":"2024-09-21T07:52:41.048030+0000","last_peered":"2024-09-21T07:52:41.048030+0000","last_clean":"2024-09-21T07:52:41.048030+0000","last_became_active":"2024-09-21T07:52:41.043513+0000","last_became_peered":"2024-09-21T07:52:41.043513+0000","last_unstale":"2024-09-21T07:52:41.048030+0000","last_undegraded":"2024-09-21T07:52:41.048030+0000","last_fullsized":"2024-09-21T07:52:41.048030+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:26.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037809+0000","last_change":"2024-09-21T07:52:41.037809+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037809+0000","last_undegraded":"2024-09-21T07:52:41.037809+0000","last_fullsized":"2024-09-21T07:52:41.037809+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047619+0000","last_change":"2024-09-21T07:52:41.043649+0000","last_active":"2024-09-21T07:52:41.047619+0000","last_peered":"2024-09-21T07:52:41.047619+0000","last_clean":"2024-09-21T07:52:41.047619+0000","last_became_active":"2024-09-21T07:52:41.043360+0000","last_became_peered":"2024-09-21T07:52:41.043360+0000","last_unstale":"2024-09-21T07:52:41.047619+0000","last_undegraded":"2024-09-21T07:52:41.047619+0000","last_fullsized":"2024-09-21T07:52:41.047619+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:51.651416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033514+0000","last_change":"2024-09-21T07:52:41.033514+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033514+0000","last_undegraded":"2024-09-21T07:52:41.033514+0000","last_fullsized":"2024-09-21T07:52:41.033514+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052214+0000","last_change":"2024-09-21T07:52:41.049536+0000","last_active":"2024-09-21T07:52:41.052214+0000","last_peered":"2024-09-21T07:52:41.052214+0000","last_clean":"2024-09-21T07:52:41.052214+0000","last_became_active":"2024-09-21T07:52:41.045025+0000","last_became_peered":"2024-09-21T07:52:41.045025+0000","last_unstale":"2024-09-21T07:52:41.052214+0000","last_undegraded":"2024-09-21T07:52:41.052214+0000","last_fullsized":"2024-09-21T07:52:41.052214+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:26.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.039796+0000","last_change":"2024-09-21T07:52:41.039796+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.039796+0000","last_undegraded":"2024-09-21T07:52:41.039796+0000","last_fullsized":"2024-09-21T07:52:41.039796+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053707+0000","last_change":"2024-09-21T07:52:41.050888+0000","last_active":"2024-09-21T07:52:41.053707+0000","last_peered":"2024-09-21T07:52:41.053707+0000","last_clean":"2024-09-21T07:52:41.053707+0000","last_became_active":"2024-09-21T07:52:41.050131+0000","last_became_peered":"2024-09-21T07:52:41.050131+0000","last_unstale":"2024-09-21T07:52:41.053707+0000","last_undegraded":"2024-09-21T07:52:41.053707+0000","last_fullsized":"2024-09-21T07:52:41.053707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:32.028954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.041080+0000","last_change":"2024-09-21T07:52:41.041080+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.041080+0000","last_undegraded":"2024-09-21T07:52:41.041080+0000","last_fullsized":"2024-09-21T07:52:41.041080+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045546+0000","last_change":"2024-09-21T07:52:41.036154+0000","last_active":"2024-09-21T07:52:41.045546+0000","last_peered":"2024-09-21T07:52:41.045546+0000","last_clean":"2024-09-21T07:52:41.045546+0000","last_became_active":"2024-09-21T07:52:41.035812+0000","last_became_peered":"2024-09-21T07:52:41.035812+0000","last_unstale":"2024-09-21T07:52:41.045546+0000","last_undegraded":"2024-09-21T07:52:41.045546+0000","last_fullsized":"2024-09-21T07:52:41.045546+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:37.178689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027661+0000","last_change":"2024-09-21T07:52:41.027661+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027661+0000","last_undegraded":"2024-09-21T07:52:41.027661+0000","last_fullsized":"2024-09-21T07:52:41.027661+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053420+0000","last_change":"2024-09-21T07:52:41.046956+0000","last_active":"2024-09-21T07:52:41.053420+0000","last_peered":"2024-09-21T07:52:41.053420+0000","last_clean":"2024-09-21T07:52:41.053420+0000","last_became_active":"2024-09-21T07:52:41.046644+0000","last_became_peered":"2024-09-21T07:52:41.046644+0000","last_unstale":"2024-09-21T07:52:41.053420+0000","last_undegraded":"2024-09-21T07:52:41.053420+0000","last_fullsized":"2024-09-21T07:52:41.053420+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:36.785804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029014+0000","last_change":"2024-09-21T07:52:41.029014+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029014+0000","last_undegraded":"2024-09-21T07:52:41.029014+0000","last_fullsized":"2024-09-21T07:52:41.029014+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.049922+0000","last_change":"2024-09-21T07:52:41.047727+0000","last_active":"2024-09-21T07:52:41.049922+0000","last_peered":"2024-09-21T07:52:41.049922+0000","last_clean":"2024-09-21T07:52:41.049922+0000","last_became_active":"2024-09-21T07:52:41.047444+0000","last_became_peered":"2024-09-21T07:52:41.047444+0000","last_unstale":"2024-09-21T07:52:41.049922+0000","last_undegraded":"2024-09-21T07:52:41.049922+0000","last_fullsized":"2024-09-21T07:52:41.049922+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:21.613841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029439+0000","last_change":"2024-09-21T07:52:41.029439+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029439+0000","last_undegraded":"2024-09-21T07:52:41.029439+0000","last_fullsized":"2024-09-21T07:52:41.029439+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048348+0000","last_change":"2024-09-21T07:52:41.045986+0000","last_active":"2024-09-21T07:52:41.048348+0000","last_peered":"2024-09-21T07:52:41.048348+0000","last_clean":"2024-09-21T07:52:41.048348+0000","last_became_active":"2024-09-21T07:52:41.045666+0000","last_became_peered":"2024-09-21T07:52:41.045666+0000","last_unstale":"2024-09-21T07:52:41.048348+0000","last_undegraded":"2024-09-21T07:52:41.048348+0000","last_fullsized":"2024-09-21T07:52:41.048348+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:21.170296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029347+0000","last_change":"2024-09-21T07:52:41.029347+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029347+0000","last_undegraded":"2024-09-21T07:52:41.029347+0000","last_fullsized":"2024-09-21T07:52:41.029347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053445+0000","last_change":"2024-09-21T07:52:41.051206+0000","last_active":"2024-09-21T07:52:41.053445+0000","last_peered":"2024-09-21T07:52:41.053445+0000","last_clean":"2024-09-21T07:52:41.053445+0000","last_became_active":"2024-09-21T07:52:41.050919+0000","last_became_peered":"2024-09-21T07:52:41.050919+0000","last_unstale":"2024-09-21T07:52:41.053445+0000","last_undegraded":"2024-09-21T07:52:41.053445+0000","last_fullsized":"2024-09-21T07:52:41.053445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:19.295030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037219+0000","last_change":"2024-09-21T07:52:41.037219+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037219+0000","last_undegraded":"2024-09-21T07:52:41.037219+0000","last_fullsized":"2024-09-21T07:52:41.037219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048075+0000","last_change":"2024-09-21T07:52:41.044491+0000","last_active":"2024-09-21T07:52:41.048075+0000","last_peered":"2024-09-21T07:52:41.048075+0000","last_clean":"2024-09-21T07:52:41.048075+0000","last_became_active":"2024-09-21T07:52:41.043914+0000","last_became_peered":"2024-09-21T07:52:41.043914+0000","last_unstale":"2024-09-21T07:52:41.048075+0000","last_undegraded":"2024-09-21T07:52:41.048075+0000","last_fullsized":"2024-09-21T07:52:41.048075+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:13.359032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032466+0000","last_change":"2024-09-21T07:52:41.032466+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032466+0000","last_undegraded":"2024-09-21T07:52:41.032466+0000","last_fullsized":"2024-09-21T07:52:41.032466+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045792+0000","last_change":"2024-09-21T07:52:41.040325+0000","last_active":"2024-09-21T07:52:41.045792+0000","last_peered":"2024-09-21T07:52:41.045792+0000","last_clean":"2024-09-21T07:52:41.045792+0000","last_became_active":"2024-09-21T07:52:41.039761+0000","last_became_peered":"2024-09-21T07:52:41.039761+0000","last_unstale":"2024-09-21T07:52:41.045792+0000","last_undegraded":"2024-09-21T07:52:41.045792+0000","last_fullsized":"2024-09-21T07:52:41.045792+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:27.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029377+0000","last_change":"2024-09-21T07:52:41.029377+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029377+0000","last_undegraded":"2024-09-21T07:52:41.029377+0000","last_fullsized":"2024-09-21T07:52:41.029377+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.056786+0000","last_change":"2024-09-21T07:52:41.056134+0000","last_active":"2024-09-21T07:52:41.056786+0000","last_peered":"2024-09-21T07:52:41.056786+0000","last_clean":"2024-09-21T07:52:41.056786+0000","last_became_active":"2024-09-21T07:52:41.055763+0000","last_became_peered":"2024-09-21T07:52:41.055763+0000","last_unstale":"2024-09-21T07:52:41.056786+0000","last_undegraded":"2024-09-21T07:52:41.056786+0000","last_fullsized":"2024-09-21T07:52:41.056786+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:15.231881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.026772+0000","last_change":"2024-09-21T07:52:41.026772+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.026772+0000","last_undegraded":"2024-09-21T07:52:41.026772+0000","last_fullsized":"2024-09-21T07:52:41.026772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054453+0000","last_change":"2024-09-21T07:52:41.053277+0000","last_active":"2024-09-21T07:52:41.054453+0000","last_peered":"2024-09-21T07:52:41.054453+0000","last_clean":"2024-09-21T07:52:41.054453+0000","last_became_active":"2024-09-21T07:52:41.052592+0000","last_became_peered":"2024-09-21T07:52:41.052592+0000","last_unstale":"2024-09-21T07:52:41.054453+0000","last_undegraded":"2024-09-21T07:52:41.054453+0000","last_fullsized":"2024-09-21T07:52:41.054453+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.806940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034577+0000","last_change":"2024-09-21T07:52:41.034577+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034577+0000","last_undegraded":"2024-09-21T07:52:41.034577+0000","last_fullsized":"2024-09-21T07:52:41.034577+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.055973+0000","last_change":"2024-09-21T07:52:41.053679+0000","last_active":"2024-09-21T07:52:41.055973+0000","last_peered":"2024-09-21T07:52:41.055973+0000","last_clean":"2024-09-21T07:52:41.055973+0000","last_became_active":"2024-09-21T07:52:41.053337+0000","last_became_peered":"2024-09-21T07:52:41.053337+0000","last_unstale":"2024-09-21T07:52:41.055973+0000","last_undegraded":"2024-09-21T07:52:41.055973+0000","last_fullsized":"2024-09-21T07:52:41.055973+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:03.857276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034863+0000","last_change":"2024-09-21T07:52:41.034863+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034863+0000","last_undegraded":"2024-09-21T07:52:41.034863+0000","last_fullsized":"2024-09-21T07:52:41.034863+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.050242+0000","last_change":"2024-09-21T07:52:41.044918+0000","last_active":"2024-09-21T07:52:41.050242+0000","last_peered":"2024-09-21T07:52:41.050242+0000","last_clean":"2024-09-21T07:52:41.050242+0000","last_became_active":"2024-09-21T07:52:41.044501+0000","last_became_peered":"2024-09-21T07:52:41.044501+0000","last_unstale":"2024-09-21T07:52:41.050242+0000","last_undegraded":"2024-09-21T07:52:41.050242+0000","last_fullsized":"2024-09-21T07:52:41.050242+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:32.028954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033987+0000","last_change":"2024-09-21T07:52:41.033987+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033987+0000","last_undegraded":"2024-09-21T07:52:41.033987+0000","last_fullsized":"2024-09-21T07:52:41.033987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047581+0000","last_change":"2024-09-21T07:52:41.043000+0000","last_active":"2024-09-21T07:52:41.047581+0000","last_peered":"2024-09-21T07:52:41.047581+0000","last_clean":"2024-09-21T07:52:41.047581+0000","last_became_active":"2024-09-21T07:52:41.042576+0000","last_became_peered":"2024-09-21T07:52:41.042576+0000","last_unstale":"2024-09-21T07:52:41.047581+0000","last_undegraded":"2024-09-21T07:52:41.047581+0000","last_fullsized":"2024-09-21T07:52:41.047581+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:19.481187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034058+0000","last_change":"2024-09-21T07:52:41.034058+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034058+0000","last_undegraded":"2024-09-21T07:52:41.034058+0000","last_fullsized":"2024-09-21T07:52:41.034058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053262+0000","last_change":"2024-09-21T07:52:41.049196+0000","last_active":"2024-09-21T07:52:41.053262+0000","last_peered":"2024-09-21T07:52:41.053262+0000","last_clean":"2024-09-21T07:52:41.053262+0000","last_became_active":"2024-09-21T07:52:41.048712+0000","last_became_peered":"2024-09-21T07:52:41.048712+0000","last_unstale":"2024-09-21T07:52:41.053262+0000","last_undegraded":"2024-09-21T07:52:41.053262+0000","last_fullsized":"2024-09-21T07:52:41.053262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:03.580681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034036+0000","last_change":"2024-09-21T07:52:41.034036+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034036+0000","last_undegraded":"2024-09-21T07:52:41.034036+0000","last_fullsized":"2024-09-21T07:52:41.034036+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047590+0000","last_change":"2024-09-21T07:52:41.043466+0000","last_active":"2024-09-21T07:52:41.047590+0000","last_peered":"2024-09-21T07:52:41.047590+0000","last_clean":"2024-09-21T07:52:41.047590+0000","last_became_active":"2024-09-21T07:52:41.042404+0000","last_became_peered":"2024-09-21T07:52:41.042404+0000","last_unstale":"2024-09-21T07:52:41.047590+0000","last_undegraded":"2024-09-21T07:52:41.047590+0000","last_fullsized":"2024-09-21T07:52:41.047590+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:37.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032079+0000","last_change":"2024-09-21T07:52:41.032079+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032079+0000","last_undegraded":"2024-09-21T07:52:41.032079+0000","last_fullsized":"2024-09-21T07:52:41.032079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045488+0000","last_change":"2024-09-21T07:52:41.041714+0000","last_active":"2024-09-21T07:52:41.045488+0000","last_peered":"2024-09-21T07:52:41.045488+0000","last_clean":"2024-09-21T07:52:41.045488+0000","last_became_active":"2024-09-21T07:52:41.041407+0000","last_became_peered":"2024-09-21T07:52:41.041407+0000","last_unstale":"2024-09-21T07:52:41.045488+0000","last_undegraded":"2024-09-21T07:52:41.045488+0000","last_fullsized":"2024-09-21T07:52:41.045488+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:03.580681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031023+0000","last_change":"2024-09-21T07:52:41.031023+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031023+0000","last_undegraded":"2024-09-21T07:52:41.031023+0000","last_fullsized":"2024-09-21T07:52:41.031023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.051789+0000","last_change":"2024-09-21T07:52:41.045768+0000","last_active":"2024-09-21T07:52:41.051789+0000","last_peered":"2024-09-21T07:52:41.051789+0000","last_clean":"2024-09-21T07:52:41.051789+0000","last_became_active":"2024-09-21T07:52:41.045435+0000","last_became_peered":"2024-09-21T07:52:41.045435+0000","last_unstale":"2024-09-21T07:52:41.051789+0000","last_undegraded":"2024-09-21T07:52:41.051789+0000","last_fullsized":"2024-09-21T07:52:41.051789+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:05.057723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":320,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.026190+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:52:41.026190+0000","last_peered":"2024-09-21T07:52:41.026190+0000","last_clean":"2024-09-21T07:52:41.026190+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:52:41.026190+0000","last_undegraded":"2024-09-21T07:52:41.026190+0000","last_fullsized":"2024-09-21T07:52:41.026190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2043904,"data_stored":1225820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":635452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476823,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476823,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2056192,"data_stored":1232095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476820,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":635452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:52:46.973 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 -- 172.21.15.88:0/3064843967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 msgr2=0x7fd2741535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 --2- 172.21.15.88:0/3064843967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd2741535c0 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7fd264005fd0 tx=0x7fd26402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 -- 172.21.15.88:0/3064843967 shutdown_connections 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 --2- 172.21.15.88:0/3064843967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2740ad0e0 0x7fd2740ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 --2- 172.21.15.88:0/3064843967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd2741535c0 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 -- 172.21.15.88:0/3064843967 >> 172.21.15.88:0/3064843967 conn(0x7fd27406d730 msgr2=0x7fd2740b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 -- 172.21.15.88:0/3064843967 shutdown_connections 2024-09-21T07:52:47.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.228+0000 7fd27a200640 1 -- 172.21.15.88:0/3064843967 wait complete. 2024-09-21T07:52:47.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.231+0000 7fd27a200640 1 Processor -- start 2024-09-21T07:52:47.233 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.231+0000 7fd27a200640 1 -- start start 2024-09-21T07:52:47.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd27a200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2740ad0e0 0x7fd2741f5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd27a200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd27414d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd27a200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd274160310 0x7fd27414dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd27a200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2740b4c20 con 0x7fd27415d760 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd27a200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd274153c70 con 0x7fd2740ad0e0 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd27a200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2740b4da0 con 0x7fd274160310 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.232+0000 7fd26aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd27414d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd27414d860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57258/0 (socket says 172.21.15.88:57258) 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 -- 172.21.15.88:0/432374581 learned_addr learned my addr 172.21.15.88:0/432374581 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:47.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd273400640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2740ad0e0 0x7fd2741f5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 -- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd274160310 msgr2=0x7fd27414dda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd274160310 0x7fd27414dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 -- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2740ad0e0 msgr2=0x7fd2741f5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2740ad0e0 0x7fd2741f5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 -- 172.21.15.88:0/432374581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd27414e3c0 con 0x7fd27415d760 2024-09-21T07:52:47.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26aa00640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd27414d860 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b410 tx=0x7fd25c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.233+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c015020 con 0x7fd27415d760 2024-09-21T07:52:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.234+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd25c011920 con 0x7fd27415d760 2024-09-21T07:52:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.234+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd27414e6a0 con 0x7fd27415d760 2024-09-21T07:52:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.234+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd25c011d70 con 0x7fd27415d760 2024-09-21T07:52:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.234+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd27414ec60 con 0x7fd27415d760 2024-09-21T07:52:47.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.235+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd274160fa0 con 0x7fd27415d760 2024-09-21T07:52:47.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.236+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fd25c018020 con 0x7fd27415d760 2024-09-21T07:52:47.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.237+0000 7fd26be00640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd250079010 0x7fd25007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.237+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fd25c09b530 con 0x7fd27415d760 2024-09-21T07:52:47.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.238+0000 7fd273400640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd250079010 0x7fd25007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.240 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.238+0000 7fd273400640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd250079010 0x7fd25007b520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd264033000 tx=0x7fd264004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:47.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.242+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd25c01f020 con 0x7fd27415d760 2024-09-21T07:52:47.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.423+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd27406e060 con 0x7fd250079010 2024-09-21T07:52:47.430 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.428+0000 7fd26be00640 1 -- 172.21.15.88:0/432374581 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+247113 (secure 0 0 0) 0x7fd27406e060 con 0x7fd250079010 2024-09-21T07:52:47.432 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:47.433 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd250079010 msgr2=0x7fd25007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd250079010 0x7fd25007b520 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd264033000 tx=0x7fd264004730 comp rx=0 tx=0).stop 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 msgr2=0x7fd27414d860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd27414d860 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7fd25c00b410 tx=0x7fd25c00b8d0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 shutdown_connections 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd250079010 0x7fd25007b520 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd274160310 0x7fd27414dda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd27415d760 0x7fd27414d860 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 --2- 172.21.15.88:0/432374581 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2740ad0e0 0x7fd2741f5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 >> 172.21.15.88:0/432374581 conn(0x7fd27406d730 msgr2=0x7fd274160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:47.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 shutdown_connections 2024-09-21T07:52:47.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.432+0000 7fd27a200640 1 -- 172.21.15.88:0/432374581 wait complete. 2024-09-21T07:52:47.458 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":115,"stamp":"2024-09-21T07:52:45.723271+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":21,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262092,"kb_used_data":12548,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712628,"statfs":{"total":773094113280,"available":772825731072,"internally_reserved":0,"allocated":12849152,"data_stored":6295727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.706306"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027993+0000","last_change":"2024-09-21T07:52:41.027993+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027993+0000","last_undegraded":"2024-09-21T07:52:41.027993+0000","last_fullsized":"2024-09-21T07:52:41.027993+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031021+0000","last_change":"2024-09-21T07:52:41.031021+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031021+0000","last_undegraded":"2024-09-21T07:52:41.031021+0000","last_fullsized":"2024-09-21T07:52:41.031021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035228+0000","last_change":"2024-09-21T07:52:41.035228+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035228+0000","last_undegraded":"2024-09-21T07:52:41.035228+0000","last_fullsized":"2024-09-21T07:52:41.035228+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027064+0000","last_change":"2024-09-21T07:52:41.027064+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027064+0000","last_undegraded":"2024-09-21T07:52:41.027064+0000","last_fullsized":"2024-09-21T07:52:41.027064+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035220+0000","last_change":"2024-09-21T07:52:41.035220+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035220+0000","last_undegraded":"2024-09-21T07:52:41.035220+0000","last_fullsized":"2024-09-21T07:52:41.035220+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.040021+0000","last_change":"2024-09-21T07:52:41.040021+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.040021+0000","last_undegraded":"2024-09-21T07:52:41.040021+0000","last_fullsized":"2024-09-21T07:52:41.040021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.036420+0000","last_change":"2024-09-21T07:52:41.036420+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.036420+0000","last_undegraded":"2024-09-21T07:52:41.036420+0000","last_fullsized":"2024-09-21T07:52:41.036420+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032595+0000","last_change":"2024-09-21T07:52:41.032595+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032595+0000","last_undegraded":"2024-09-21T07:52:41.032595+0000","last_fullsized":"2024-09-21T07:52:41.032595+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.039205+0000","last_change":"2024-09-21T07:52:41.039205+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.039205+0000","last_undegraded":"2024-09-21T07:52:41.039205+0000","last_fullsized":"2024-09-21T07:52:41.039205+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030391+0000","last_change":"2024-09-21T07:52:41.030391+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030391+0000","last_undegraded":"2024-09-21T07:52:41.030391+0000","last_fullsized":"2024-09-21T07:52:41.030391+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027094+0000","last_change":"2024-09-21T07:52:41.027094+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027094+0000","last_undegraded":"2024-09-21T07:52:41.027094+0000","last_fullsized":"2024-09-21T07:52:41.027094+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034190+0000","last_change":"2024-09-21T07:52:41.034190+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034190+0000","last_undegraded":"2024-09-21T07:52:41.034190+0000","last_fullsized":"2024-09-21T07:52:41.034190+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.026540+0000","last_change":"2024-09-21T07:52:41.026540+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.026540+0000","last_undegraded":"2024-09-21T07:52:41.026540+0000","last_fullsized":"2024-09-21T07:52:41.026540+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035819+0000","last_change":"2024-09-21T07:52:41.035819+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035819+0000","last_undegraded":"2024-09-21T07:52:41.035819+0000","last_fullsized":"2024-09-21T07:52:41.035819+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033553+0000","last_change":"2024-09-21T07:52:41.033553+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033553+0000","last_undegraded":"2024-09-21T07:52:41.033553+0000","last_fullsized":"2024-09-21T07:52:41.033553+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.038549+0000","last_change":"2024-09-21T07:52:41.038549+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.038549+0000","last_undegraded":"2024-09-21T07:52:41.038549+0000","last_fullsized":"2024-09-21T07:52:41.038549+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033229+0000","last_change":"2024-09-21T07:52:41.033229+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033229+0000","last_undegraded":"2024-09-21T07:52:41.033229+0000","last_fullsized":"2024-09-21T07:52:41.033229+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035039+0000","last_change":"2024-09-21T07:52:41.035039+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035039+0000","last_undegraded":"2024-09-21T07:52:41.035039+0000","last_fullsized":"2024-09-21T07:52:41.035039+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029611+0000","last_change":"2024-09-21T07:52:41.029611+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029611+0000","last_undegraded":"2024-09-21T07:52:41.029611+0000","last_fullsized":"2024-09-21T07:52:41.029611+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.036505+0000","last_change":"2024-09-21T07:52:41.036505+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.036505+0000","last_undegraded":"2024-09-21T07:52:41.036505+0000","last_fullsized":"2024-09-21T07:52:41.036505+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034977+0000","last_change":"2024-09-21T07:52:41.034977+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034977+0000","last_undegraded":"2024-09-21T07:52:41.034977+0000","last_fullsized":"2024-09-21T07:52:41.034977+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030494+0000","last_change":"2024-09-21T07:52:41.030494+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030494+0000","last_undegraded":"2024-09-21T07:52:41.030494+0000","last_fullsized":"2024-09-21T07:52:41.030494+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037052+0000","last_change":"2024-09-21T07:52:41.037052+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037052+0000","last_undegraded":"2024-09-21T07:52:41.037052+0000","last_fullsized":"2024-09-21T07:52:41.037052+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035828+0000","last_change":"2024-09-21T07:52:41.035828+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035828+0000","last_undegraded":"2024-09-21T07:52:41.035828+0000","last_fullsized":"2024-09-21T07:52:41.035828+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027380+0000","last_change":"2024-09-21T07:52:41.027380+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027380+0000","last_undegraded":"2024-09-21T07:52:41.027380+0000","last_fullsized":"2024-09-21T07:52:41.027380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029802+0000","last_change":"2024-09-21T07:52:41.029802+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029802+0000","last_undegraded":"2024-09-21T07:52:41.029802+0000","last_fullsized":"2024-09-21T07:52:41.029802+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033847+0000","last_change":"2024-09-21T07:52:41.033847+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033847+0000","last_undegraded":"2024-09-21T07:52:41.033847+0000","last_fullsized":"2024-09-21T07:52:41.033847+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030093+0000","last_change":"2024-09-21T07:52:41.030093+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030093+0000","last_undegraded":"2024-09-21T07:52:41.030093+0000","last_fullsized":"2024-09-21T07:52:41.030093+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.028483+0000","last_change":"2024-09-21T07:52:41.028483+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.028483+0000","last_undegraded":"2024-09-21T07:52:41.028483+0000","last_fullsized":"2024-09-21T07:52:41.028483+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029992+0000","last_change":"2024-09-21T07:52:41.029992+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029992+0000","last_undegraded":"2024-09-21T07:52:41.029992+0000","last_fullsized":"2024-09-21T07:52:41.029992+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030493+0000","last_change":"2024-09-21T07:52:41.030493+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030493+0000","last_undegraded":"2024-09-21T07:52:41.030493+0000","last_fullsized":"2024-09-21T07:52:41.030493+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033920+0000","last_change":"2024-09-21T07:52:41.033920+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033920+0000","last_undegraded":"2024-09-21T07:52:41.033920+0000","last_fullsized":"2024-09-21T07:52:41.033920+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035011+0000","last_change":"2024-09-21T07:52:41.035011+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035011+0000","last_undegraded":"2024-09-21T07:52:41.035011+0000","last_fullsized":"2024-09-21T07:52:41.035011+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054190+0000","last_change":"2024-09-21T07:52:41.052517+0000","last_active":"2024-09-21T07:52:41.054190+0000","last_peered":"2024-09-21T07:52:41.054190+0000","last_clean":"2024-09-21T07:52:41.054190+0000","last_became_active":"2024-09-21T07:52:41.052152+0000","last_became_peered":"2024-09-21T07:52:41.052152+0000","last_unstale":"2024-09-21T07:52:41.054190+0000","last_undegraded":"2024-09-21T07:52:41.054190+0000","last_fullsized":"2024-09-21T07:52:41.054190+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:43.545555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.038710+0000","last_change":"2024-09-21T07:52:41.038710+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.038710+0000","last_undegraded":"2024-09-21T07:52:41.038710+0000","last_fullsized":"2024-09-21T07:52:41.038710+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054932+0000","last_change":"2024-09-21T07:52:41.051644+0000","last_active":"2024-09-21T07:52:41.054932+0000","last_peered":"2024-09-21T07:52:41.054932+0000","last_clean":"2024-09-21T07:52:41.054932+0000","last_became_active":"2024-09-21T07:52:41.051270+0000","last_became_peered":"2024-09-21T07:52:41.051270+0000","last_unstale":"2024-09-21T07:52:41.054932+0000","last_undegraded":"2024-09-21T07:52:41.054932+0000","last_fullsized":"2024-09-21T07:52:41.054932+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:19.575593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031733+0000","last_change":"2024-09-21T07:52:41.031733+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031733+0000","last_undegraded":"2024-09-21T07:52:41.031733+0000","last_fullsized":"2024-09-21T07:52:41.031733+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045732+0000","last_change":"2024-09-21T07:52:41.040236+0000","last_active":"2024-09-21T07:52:41.045732+0000","last_peered":"2024-09-21T07:52:41.045732+0000","last_clean":"2024-09-21T07:52:41.045732+0000","last_became_active":"2024-09-21T07:52:41.039288+0000","last_became_peered":"2024-09-21T07:52:41.039288+0000","last_unstale":"2024-09-21T07:52:41.045732+0000","last_undegraded":"2024-09-21T07:52:41.045732+0000","last_fullsized":"2024-09-21T07:52:41.045732+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:09.577922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.035750+0000","last_change":"2024-09-21T07:52:41.035750+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.035750+0000","last_undegraded":"2024-09-21T07:52:41.035750+0000","last_fullsized":"2024-09-21T07:52:41.035750+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053006+0000","last_change":"2024-09-21T07:52:41.049531+0000","last_active":"2024-09-21T07:52:41.053006+0000","last_peered":"2024-09-21T07:52:41.053006+0000","last_clean":"2024-09-21T07:52:41.053006+0000","last_became_active":"2024-09-21T07:52:41.048245+0000","last_became_peered":"2024-09-21T07:52:41.048245+0000","last_unstale":"2024-09-21T07:52:41.053006+0000","last_undegraded":"2024-09-21T07:52:41.053006+0000","last_fullsized":"2024-09-21T07:52:41.053006+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:44.807993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037343+0000","last_change":"2024-09-21T07:52:41.037343+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037343+0000","last_undegraded":"2024-09-21T07:52:41.037343+0000","last_fullsized":"2024-09-21T07:52:41.037343+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045753+0000","last_change":"2024-09-21T07:52:41.039897+0000","last_active":"2024-09-21T07:52:41.045753+0000","last_peered":"2024-09-21T07:52:41.045753+0000","last_clean":"2024-09-21T07:52:41.045753+0000","last_became_active":"2024-09-21T07:52:41.039535+0000","last_became_peered":"2024-09-21T07:52:41.039535+0000","last_unstale":"2024-09-21T07:52:41.045753+0000","last_undegraded":"2024-09-21T07:52:41.045753+0000","last_fullsized":"2024-09-21T07:52:41.045753+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:22.942512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034529+0000","last_change":"2024-09-21T07:52:41.034529+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034529+0000","last_undegraded":"2024-09-21T07:52:41.034529+0000","last_fullsized":"2024-09-21T07:52:41.034529+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052555+0000","last_change":"2024-09-21T07:52:41.048831+0000","last_active":"2024-09-21T07:52:41.052555+0000","last_peered":"2024-09-21T07:52:41.052555+0000","last_clean":"2024-09-21T07:52:41.052555+0000","last_became_active":"2024-09-21T07:52:41.048477+0000","last_became_peered":"2024-09-21T07:52:41.048477+0000","last_unstale":"2024-09-21T07:52:41.052555+0000","last_undegraded":"2024-09-21T07:52:41.052555+0000","last_fullsized":"2024-09-21T07:52:41.052555+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:46:09.004978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031583+0000","last_change":"2024-09-21T07:52:41.031583+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031583+0000","last_undegraded":"2024-09-21T07:52:41.031583+0000","last_fullsized":"2024-09-21T07:52:41.031583+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054414+0000","last_change":"2024-09-21T07:52:41.046640+0000","last_active":"2024-09-21T07:52:41.054414+0000","last_peered":"2024-09-21T07:52:41.054414+0000","last_clean":"2024-09-21T07:52:41.054414+0000","last_became_active":"2024-09-21T07:52:41.046298+0000","last_became_peered":"2024-09-21T07:52:41.046298+0000","last_unstale":"2024-09-21T07:52:41.054414+0000","last_undegraded":"2024-09-21T07:52:41.054414+0000","last_fullsized":"2024-09-21T07:52:41.054414+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:49.783011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029388+0000","last_change":"2024-09-21T07:52:41.029388+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029388+0000","last_undegraded":"2024-09-21T07:52:41.029388+0000","last_fullsized":"2024-09-21T07:52:41.029388+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047618+0000","last_change":"2024-09-21T07:52:41.043692+0000","last_active":"2024-09-21T07:52:41.047618+0000","last_peered":"2024-09-21T07:52:41.047618+0000","last_clean":"2024-09-21T07:52:41.047618+0000","last_became_active":"2024-09-21T07:52:41.043389+0000","last_became_peered":"2024-09-21T07:52:41.043389+0000","last_unstale":"2024-09-21T07:52:41.047618+0000","last_undegraded":"2024-09-21T07:52:41.047618+0000","last_fullsized":"2024-09-21T07:52:41.047618+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:00:30.458696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029191+0000","last_change":"2024-09-21T07:52:41.029191+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029191+0000","last_undegraded":"2024-09-21T07:52:41.029191+0000","last_fullsized":"2024-09-21T07:52:41.029191+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052502+0000","last_change":"2024-09-21T07:52:41.049633+0000","last_active":"2024-09-21T07:52:41.052502+0000","last_peered":"2024-09-21T07:52:41.052502+0000","last_clean":"2024-09-21T07:52:41.052502+0000","last_became_active":"2024-09-21T07:52:41.047189+0000","last_became_peered":"2024-09-21T07:52:41.047189+0000","last_unstale":"2024-09-21T07:52:41.052502+0000","last_undegraded":"2024-09-21T07:52:41.052502+0000","last_fullsized":"2024-09-21T07:52:41.052502+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:13.359032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.041579+0000","last_change":"2024-09-21T07:52:41.041579+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.041579+0000","last_undegraded":"2024-09-21T07:52:41.041579+0000","last_fullsized":"2024-09-21T07:52:41.041579+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052501+0000","last_change":"2024-09-21T07:52:41.048436+0000","last_active":"2024-09-21T07:52:41.052501+0000","last_peered":"2024-09-21T07:52:41.052501+0000","last_clean":"2024-09-21T07:52:41.052501+0000","last_became_active":"2024-09-21T07:52:41.047717+0000","last_became_peered":"2024-09-21T07:52:41.047717+0000","last_unstale":"2024-09-21T07:52:41.052501+0000","last_undegraded":"2024-09-21T07:52:41.052501+0000","last_fullsized":"2024-09-21T07:52:41.052501+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:37:28.786853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033729+0000","last_change":"2024-09-21T07:52:41.033729+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033729+0000","last_undegraded":"2024-09-21T07:52:41.033729+0000","last_fullsized":"2024-09-21T07:52:41.033729+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054694+0000","last_change":"2024-09-21T07:52:41.051335+0000","last_active":"2024-09-21T07:52:41.054694+0000","last_peered":"2024-09-21T07:52:41.054694+0000","last_clean":"2024-09-21T07:52:41.054694+0000","last_became_active":"2024-09-21T07:52:41.051017+0000","last_became_peered":"2024-09-21T07:52:41.051017+0000","last_unstale":"2024-09-21T07:52:41.054694+0000","last_undegraded":"2024-09-21T07:52:41.054694+0000","last_fullsized":"2024-09-21T07:52:41.054694+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:45:38.414746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.030074+0000","last_change":"2024-09-21T07:52:41.030074+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.030074+0000","last_undegraded":"2024-09-21T07:52:41.030074+0000","last_fullsized":"2024-09-21T07:52:41.030074+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045672+0000","last_change":"2024-09-21T07:52:41.042291+0000","last_active":"2024-09-21T07:52:41.045672+0000","last_peered":"2024-09-21T07:52:41.045672+0000","last_clean":"2024-09-21T07:52:41.045672+0000","last_became_active":"2024-09-21T07:52:41.041543+0000","last_became_peered":"2024-09-21T07:52:41.041543+0000","last_unstale":"2024-09-21T07:52:41.045672+0000","last_undegraded":"2024-09-21T07:52:41.045672+0000","last_fullsized":"2024-09-21T07:52:41.045672+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:22.388273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031136+0000","last_change":"2024-09-21T07:52:41.031136+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031136+0000","last_undegraded":"2024-09-21T07:52:41.031136+0000","last_fullsized":"2024-09-21T07:52:41.031136+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048030+0000","last_change":"2024-09-21T07:52:41.043893+0000","last_active":"2024-09-21T07:52:41.048030+0000","last_peered":"2024-09-21T07:52:41.048030+0000","last_clean":"2024-09-21T07:52:41.048030+0000","last_became_active":"2024-09-21T07:52:41.043513+0000","last_became_peered":"2024-09-21T07:52:41.043513+0000","last_unstale":"2024-09-21T07:52:41.048030+0000","last_undegraded":"2024-09-21T07:52:41.048030+0000","last_fullsized":"2024-09-21T07:52:41.048030+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:26.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037809+0000","last_change":"2024-09-21T07:52:41.037809+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037809+0000","last_undegraded":"2024-09-21T07:52:41.037809+0000","last_fullsized":"2024-09-21T07:52:41.037809+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047619+0000","last_change":"2024-09-21T07:52:41.043649+0000","last_active":"2024-09-21T07:52:41.047619+0000","last_peered":"2024-09-21T07:52:41.047619+0000","last_clean":"2024-09-21T07:52:41.047619+0000","last_became_active":"2024-09-21T07:52:41.043360+0000","last_became_peered":"2024-09-21T07:52:41.043360+0000","last_unstale":"2024-09-21T07:52:41.047619+0000","last_undegraded":"2024-09-21T07:52:41.047619+0000","last_fullsized":"2024-09-21T07:52:41.047619+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:51.651416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033514+0000","last_change":"2024-09-21T07:52:41.033514+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033514+0000","last_undegraded":"2024-09-21T07:52:41.033514+0000","last_fullsized":"2024-09-21T07:52:41.033514+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.052214+0000","last_change":"2024-09-21T07:52:41.049536+0000","last_active":"2024-09-21T07:52:41.052214+0000","last_peered":"2024-09-21T07:52:41.052214+0000","last_clean":"2024-09-21T07:52:41.052214+0000","last_became_active":"2024-09-21T07:52:41.045025+0000","last_became_peered":"2024-09-21T07:52:41.045025+0000","last_unstale":"2024-09-21T07:52:41.052214+0000","last_undegraded":"2024-09-21T07:52:41.052214+0000","last_fullsized":"2024-09-21T07:52:41.052214+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:48:26.157870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.039796+0000","last_change":"2024-09-21T07:52:41.039796+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.039796+0000","last_undegraded":"2024-09-21T07:52:41.039796+0000","last_fullsized":"2024-09-21T07:52:41.039796+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053707+0000","last_change":"2024-09-21T07:52:41.050888+0000","last_active":"2024-09-21T07:52:41.053707+0000","last_peered":"2024-09-21T07:52:41.053707+0000","last_clean":"2024-09-21T07:52:41.053707+0000","last_became_active":"2024-09-21T07:52:41.050131+0000","last_became_peered":"2024-09-21T07:52:41.050131+0000","last_unstale":"2024-09-21T07:52:41.053707+0000","last_undegraded":"2024-09-21T07:52:41.053707+0000","last_fullsized":"2024-09-21T07:52:41.053707+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:32.028954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.041080+0000","last_change":"2024-09-21T07:52:41.041080+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.041080+0000","last_undegraded":"2024-09-21T07:52:41.041080+0000","last_fullsized":"2024-09-21T07:52:41.041080+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045546+0000","last_change":"2024-09-21T07:52:41.036154+0000","last_active":"2024-09-21T07:52:41.045546+0000","last_peered":"2024-09-21T07:52:41.045546+0000","last_clean":"2024-09-21T07:52:41.045546+0000","last_became_active":"2024-09-21T07:52:41.035812+0000","last_became_peered":"2024-09-21T07:52:41.035812+0000","last_unstale":"2024-09-21T07:52:41.045546+0000","last_undegraded":"2024-09-21T07:52:41.045546+0000","last_fullsized":"2024-09-21T07:52:41.045546+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:37.178689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.027661+0000","last_change":"2024-09-21T07:52:41.027661+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.027661+0000","last_undegraded":"2024-09-21T07:52:41.027661+0000","last_fullsized":"2024-09-21T07:52:41.027661+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053420+0000","last_change":"2024-09-21T07:52:41.046956+0000","last_active":"2024-09-21T07:52:41.053420+0000","last_peered":"2024-09-21T07:52:41.053420+0000","last_clean":"2024-09-21T07:52:41.053420+0000","last_became_active":"2024-09-21T07:52:41.046644+0000","last_became_peered":"2024-09-21T07:52:41.046644+0000","last_unstale":"2024-09-21T07:52:41.053420+0000","last_undegraded":"2024-09-21T07:52:41.053420+0000","last_fullsized":"2024-09-21T07:52:41.053420+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:36.785804+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029014+0000","last_change":"2024-09-21T07:52:41.029014+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029014+0000","last_undegraded":"2024-09-21T07:52:41.029014+0000","last_fullsized":"2024-09-21T07:52:41.029014+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.049922+0000","last_change":"2024-09-21T07:52:41.047727+0000","last_active":"2024-09-21T07:52:41.049922+0000","last_peered":"2024-09-21T07:52:41.049922+0000","last_clean":"2024-09-21T07:52:41.049922+0000","last_became_active":"2024-09-21T07:52:41.047444+0000","last_became_peered":"2024-09-21T07:52:41.047444+0000","last_unstale":"2024-09-21T07:52:41.049922+0000","last_undegraded":"2024-09-21T07:52:41.049922+0000","last_fullsized":"2024-09-21T07:52:41.049922+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:21.613841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029439+0000","last_change":"2024-09-21T07:52:41.029439+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029439+0000","last_undegraded":"2024-09-21T07:52:41.029439+0000","last_fullsized":"2024-09-21T07:52:41.029439+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048348+0000","last_change":"2024-09-21T07:52:41.045986+0000","last_active":"2024-09-21T07:52:41.048348+0000","last_peered":"2024-09-21T07:52:41.048348+0000","last_clean":"2024-09-21T07:52:41.048348+0000","last_became_active":"2024-09-21T07:52:41.045666+0000","last_became_peered":"2024-09-21T07:52:41.045666+0000","last_unstale":"2024-09-21T07:52:41.048348+0000","last_undegraded":"2024-09-21T07:52:41.048348+0000","last_fullsized":"2024-09-21T07:52:41.048348+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:21.170296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029347+0000","last_change":"2024-09-21T07:52:41.029347+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029347+0000","last_undegraded":"2024-09-21T07:52:41.029347+0000","last_fullsized":"2024-09-21T07:52:41.029347+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053445+0000","last_change":"2024-09-21T07:52:41.051206+0000","last_active":"2024-09-21T07:52:41.053445+0000","last_peered":"2024-09-21T07:52:41.053445+0000","last_clean":"2024-09-21T07:52:41.053445+0000","last_became_active":"2024-09-21T07:52:41.050919+0000","last_became_peered":"2024-09-21T07:52:41.050919+0000","last_unstale":"2024-09-21T07:52:41.053445+0000","last_undegraded":"2024-09-21T07:52:41.053445+0000","last_fullsized":"2024-09-21T07:52:41.053445+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:55:19.295030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.037219+0000","last_change":"2024-09-21T07:52:41.037219+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.037219+0000","last_undegraded":"2024-09-21T07:52:41.037219+0000","last_fullsized":"2024-09-21T07:52:41.037219+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.048075+0000","last_change":"2024-09-21T07:52:41.044491+0000","last_active":"2024-09-21T07:52:41.048075+0000","last_peered":"2024-09-21T07:52:41.048075+0000","last_clean":"2024-09-21T07:52:41.048075+0000","last_became_active":"2024-09-21T07:52:41.043914+0000","last_became_peered":"2024-09-21T07:52:41.043914+0000","last_unstale":"2024-09-21T07:52:41.048075+0000","last_undegraded":"2024-09-21T07:52:41.048075+0000","last_fullsized":"2024-09-21T07:52:41.048075+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:13.359032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032466+0000","last_change":"2024-09-21T07:52:41.032466+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032466+0000","last_undegraded":"2024-09-21T07:52:41.032466+0000","last_fullsized":"2024-09-21T07:52:41.032466+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045792+0000","last_change":"2024-09-21T07:52:41.040325+0000","last_active":"2024-09-21T07:52:41.045792+0000","last_peered":"2024-09-21T07:52:41.045792+0000","last_clean":"2024-09-21T07:52:41.045792+0000","last_became_active":"2024-09-21T07:52:41.039761+0000","last_became_peered":"2024-09-21T07:52:41.039761+0000","last_unstale":"2024-09-21T07:52:41.045792+0000","last_undegraded":"2024-09-21T07:52:41.045792+0000","last_fullsized":"2024-09-21T07:52:41.045792+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:27.901371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.029377+0000","last_change":"2024-09-21T07:52:41.029377+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.029377+0000","last_undegraded":"2024-09-21T07:52:41.029377+0000","last_fullsized":"2024-09-21T07:52:41.029377+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.056786+0000","last_change":"2024-09-21T07:52:41.056134+0000","last_active":"2024-09-21T07:52:41.056786+0000","last_peered":"2024-09-21T07:52:41.056786+0000","last_clean":"2024-09-21T07:52:41.056786+0000","last_became_active":"2024-09-21T07:52:41.055763+0000","last_became_peered":"2024-09-21T07:52:41.055763+0000","last_unstale":"2024-09-21T07:52:41.056786+0000","last_undegraded":"2024-09-21T07:52:41.056786+0000","last_fullsized":"2024-09-21T07:52:41.056786+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:15.231881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.026772+0000","last_change":"2024-09-21T07:52:41.026772+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.026772+0000","last_undegraded":"2024-09-21T07:52:41.026772+0000","last_fullsized":"2024-09-21T07:52:41.026772+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.054453+0000","last_change":"2024-09-21T07:52:41.053277+0000","last_active":"2024-09-21T07:52:41.054453+0000","last_peered":"2024-09-21T07:52:41.054453+0000","last_clean":"2024-09-21T07:52:41.054453+0000","last_became_active":"2024-09-21T07:52:41.052592+0000","last_became_peered":"2024-09-21T07:52:41.052592+0000","last_unstale":"2024-09-21T07:52:41.054453+0000","last_undegraded":"2024-09-21T07:52:41.054453+0000","last_fullsized":"2024-09-21T07:52:41.054453+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:14.806940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034577+0000","last_change":"2024-09-21T07:52:41.034577+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034577+0000","last_undegraded":"2024-09-21T07:52:41.034577+0000","last_fullsized":"2024-09-21T07:52:41.034577+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.055973+0000","last_change":"2024-09-21T07:52:41.053679+0000","last_active":"2024-09-21T07:52:41.055973+0000","last_peered":"2024-09-21T07:52:41.055973+0000","last_clean":"2024-09-21T07:52:41.055973+0000","last_became_active":"2024-09-21T07:52:41.053337+0000","last_became_peered":"2024-09-21T07:52:41.053337+0000","last_unstale":"2024-09-21T07:52:41.055973+0000","last_undegraded":"2024-09-21T07:52:41.055973+0000","last_fullsized":"2024-09-21T07:52:41.055973+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:03:03.857276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034863+0000","last_change":"2024-09-21T07:52:41.034863+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034863+0000","last_undegraded":"2024-09-21T07:52:41.034863+0000","last_fullsized":"2024-09-21T07:52:41.034863+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.050242+0000","last_change":"2024-09-21T07:52:41.044918+0000","last_active":"2024-09-21T07:52:41.050242+0000","last_peered":"2024-09-21T07:52:41.050242+0000","last_clean":"2024-09-21T07:52:41.050242+0000","last_became_active":"2024-09-21T07:52:41.044501+0000","last_became_peered":"2024-09-21T07:52:41.044501+0000","last_unstale":"2024-09-21T07:52:41.050242+0000","last_undegraded":"2024-09-21T07:52:41.050242+0000","last_fullsized":"2024-09-21T07:52:41.050242+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:38:32.028954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.033987+0000","last_change":"2024-09-21T07:52:41.033987+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.033987+0000","last_undegraded":"2024-09-21T07:52:41.033987+0000","last_fullsized":"2024-09-21T07:52:41.033987+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047581+0000","last_change":"2024-09-21T07:52:41.043000+0000","last_active":"2024-09-21T07:52:41.047581+0000","last_peered":"2024-09-21T07:52:41.047581+0000","last_clean":"2024-09-21T07:52:41.047581+0000","last_became_active":"2024-09-21T07:52:41.042576+0000","last_became_peered":"2024-09-21T07:52:41.042576+0000","last_unstale":"2024-09-21T07:52:41.047581+0000","last_undegraded":"2024-09-21T07:52:41.047581+0000","last_fullsized":"2024-09-21T07:52:41.047581+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:28:19.481187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034058+0000","last_change":"2024-09-21T07:52:41.034058+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034058+0000","last_undegraded":"2024-09-21T07:52:41.034058+0000","last_fullsized":"2024-09-21T07:52:41.034058+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.053262+0000","last_change":"2024-09-21T07:52:41.049196+0000","last_active":"2024-09-21T07:52:41.053262+0000","last_peered":"2024-09-21T07:52:41.053262+0000","last_clean":"2024-09-21T07:52:41.053262+0000","last_became_active":"2024-09-21T07:52:41.048712+0000","last_became_peered":"2024-09-21T07:52:41.048712+0000","last_unstale":"2024-09-21T07:52:41.053262+0000","last_undegraded":"2024-09-21T07:52:41.053262+0000","last_fullsized":"2024-09-21T07:52:41.053262+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:03.580681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.034036+0000","last_change":"2024-09-21T07:52:41.034036+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.034036+0000","last_undegraded":"2024-09-21T07:52:41.034036+0000","last_fullsized":"2024-09-21T07:52:41.034036+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.047590+0000","last_change":"2024-09-21T07:52:41.043466+0000","last_active":"2024-09-21T07:52:41.047590+0000","last_peered":"2024-09-21T07:52:41.047590+0000","last_clean":"2024-09-21T07:52:41.047590+0000","last_became_active":"2024-09-21T07:52:41.042404+0000","last_became_peered":"2024-09-21T07:52:41.042404+0000","last_unstale":"2024-09-21T07:52:41.047590+0000","last_undegraded":"2024-09-21T07:52:41.047590+0000","last_fullsized":"2024-09-21T07:52:41.047590+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:37.942446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.032079+0000","last_change":"2024-09-21T07:52:41.032079+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.032079+0000","last_undegraded":"2024-09-21T07:52:41.032079+0000","last_fullsized":"2024-09-21T07:52:41.032079+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.045488+0000","last_change":"2024-09-21T07:52:41.041714+0000","last_active":"2024-09-21T07:52:41.045488+0000","last_peered":"2024-09-21T07:52:41.045488+0000","last_clean":"2024-09-21T07:52:41.045488+0000","last_became_active":"2024-09-21T07:52:41.041407+0000","last_became_peered":"2024-09-21T07:52:41.041407+0000","last_unstale":"2024-09-21T07:52:41.045488+0000","last_undegraded":"2024-09-21T07:52:41.045488+0000","last_fullsized":"2024-09-21T07:52:41.045488+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:03.580681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":1,"reported_epoch":118,"state":"creating+peering","last_fresh":"2024-09-21T07:52:41.031023+0000","last_change":"2024-09-21T07:52:41.031023+0000","last_active":"2024-09-21T07:52:41.014053+0000","last_peered":"2024-09-21T07:52:41.014053+0000","last_clean":"2024-09-21T07:52:41.014053+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:52:41.031023+0000","last_undegraded":"2024-09-21T07:52:41.031023+0000","last_fullsized":"2024-09-21T07:52:41.031023+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"0'0","reported_seq":10,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.051789+0000","last_change":"2024-09-21T07:52:41.045768+0000","last_active":"2024-09-21T07:52:41.051789+0000","last_peered":"2024-09-21T07:52:41.051789+0000","last_clean":"2024-09-21T07:52:41.051789+0000","last_became_active":"2024-09-21T07:52:41.045435+0000","last_became_peered":"2024-09-21T07:52:41.045435+0000","last_unstale":"2024-09-21T07:52:41.051789+0000","last_undegraded":"2024-09-21T07:52:41.051789+0000","last_fullsized":"2024-09-21T07:52:41.051789+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:05.057723+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":320,"reported_epoch":118,"state":"active+clean","last_fresh":"2024-09-21T07:52:41.026190+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:52:41.026190+0000","last_peered":"2024-09-21T07:52:41.026190+0000","last_clean":"2024-09-21T07:52:41.026190+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:52:41.026190+0000","last_undegraded":"2024-09-21T07:52:41.026190+0000","last_fullsized":"2024-09-21T07:52:41.026190+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542229,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509524,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509525,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33184,"kb_used_data":1996,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338656,"statfs":{"total":96636764160,"available":96602783744,"internally_reserved":0,"allocated":2043904,"data_stored":1225820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509525,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32604,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339236,"statfs":{"total":96636764160,"available":96603377664,"internally_reserved":0,"allocated":1449984,"data_stored":635452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476823,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32624,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339216,"statfs":{"total":96636764160,"available":96603357184,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476823,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32620,"kb_used_data":1428,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339220,"statfs":{"total":96636764160,"available":96603361280,"internally_reserved":0,"allocated":1462272,"data_stored":641727,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476820,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33204,"kb_used_data":2008,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338636,"statfs":{"total":96636764160,"available":96602763264,"internally_reserved":0,"allocated":2056192,"data_stored":1232095,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476820,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32608,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339232,"statfs":{"total":96636764160,"available":96603373568,"internally_reserved":0,"allocated":1449984,"data_stored":635452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:52:47.462 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3f is not active+clean 2024-09-21T07:52:47.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.027993+0000', 'last_change': '2024-09-21T07:52:41.027993+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.027993+0000', 'last_undegraded': '2024-09-21T07:52:41.027993+0000', 'last_fullsized': '2024-09-21T07:52:41.027993+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.462 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3e is not active+clean 2024-09-21T07:52:47.462 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.031021+0000', 'last_change': '2024-09-21T07:52:41.031021+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.031021+0000', 'last_undegraded': '2024-09-21T07:52:41.031021+0000', 'last_fullsized': '2024-09-21T07:52:41.031021+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.462 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3d is not active+clean 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035228+0000', 'last_change': '2024-09-21T07:52:41.035228+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035228+0000', 'last_undegraded': '2024-09-21T07:52:41.035228+0000', 'last_fullsized': '2024-09-21T07:52:41.035228+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3c is not active+clean 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.027064+0000', 'last_change': '2024-09-21T07:52:41.027064+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.027064+0000', 'last_undegraded': '2024-09-21T07:52:41.027064+0000', 'last_fullsized': '2024-09-21T07:52:41.027064+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 0], 'acting': [5, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3b is not active+clean 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035220+0000', 'last_change': '2024-09-21T07:52:41.035220+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035220+0000', 'last_undegraded': '2024-09-21T07:52:41.035220+0000', 'last_fullsized': '2024-09-21T07:52:41.035220+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 0], 'acting': [2, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3a is not active+clean 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.040021+0000', 'last_change': '2024-09-21T07:52:41.040021+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.040021+0000', 'last_undegraded': '2024-09-21T07:52:41.040021+0000', 'last_fullsized': '2024-09-21T07:52:41.040021+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 4], 'acting': [0, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:PG 19.39 is not active+clean 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.39', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.036420+0000', 'last_change': '2024-09-21T07:52:41.036420+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.036420+0000', 'last_undegraded': '2024-09-21T07:52:41.036420+0000', 'last_fullsized': '2024-09-21T07:52:41.036420+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 3], 'acting': [2, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:PG 19.38 is not active+clean 2024-09-21T07:52:47.463 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.38', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.032595+0000', 'last_change': '2024-09-21T07:52:41.032595+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.032595+0000', 'last_undegraded': '2024-09-21T07:52:41.032595+0000', 'last_fullsized': '2024-09-21T07:52:41.032595+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 4], 'acting': [2, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:PG 19.37 is not active+clean 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.37', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.039205+0000', 'last_change': '2024-09-21T07:52:41.039205+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.039205+0000', 'last_undegraded': '2024-09-21T07:52:41.039205+0000', 'last_fullsized': '2024-09-21T07:52:41.039205+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:PG 19.36 is not active+clean 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.36', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.030391+0000', 'last_change': '2024-09-21T07:52:41.030391+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.030391+0000', 'last_undegraded': '2024-09-21T07:52:41.030391+0000', 'last_fullsized': '2024-09-21T07:52:41.030391+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 0], 'acting': [4, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:PG 19.35 is not active+clean 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.35', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.027094+0000', 'last_change': '2024-09-21T07:52:41.027094+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.027094+0000', 'last_undegraded': '2024-09-21T07:52:41.027094+0000', 'last_fullsized': '2024-09-21T07:52:41.027094+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 6], 'acting': [5, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:PG 19.34 is not active+clean 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.34', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034190+0000', 'last_change': '2024-09-21T07:52:41.034190+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034190+0000', 'last_undegraded': '2024-09-21T07:52:41.034190+0000', 'last_fullsized': '2024-09-21T07:52:41.034190+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:PG 19.33 is not active+clean 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.33', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.026540+0000', 'last_change': '2024-09-21T07:52:41.026540+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.026540+0000', 'last_undegraded': '2024-09-21T07:52:41.026540+0000', 'last_fullsized': '2024-09-21T07:52:41.026540+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:PG 19.32 is not active+clean 2024-09-21T07:52:47.464 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.32', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035819+0000', 'last_change': '2024-09-21T07:52:41.035819+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035819+0000', 'last_undegraded': '2024-09-21T07:52:41.035819+0000', 'last_fullsized': '2024-09-21T07:52:41.035819+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:PG 19.31 is not active+clean 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.31', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033553+0000', 'last_change': '2024-09-21T07:52:41.033553+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033553+0000', 'last_undegraded': '2024-09-21T07:52:41.033553+0000', 'last_fullsized': '2024-09-21T07:52:41.033553+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:PG 19.30 is not active+clean 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.30', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.038549+0000', 'last_change': '2024-09-21T07:52:41.038549+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.038549+0000', 'last_undegraded': '2024-09-21T07:52:41.038549+0000', 'last_fullsized': '2024-09-21T07:52:41.038549+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2f is not active+clean 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033229+0000', 'last_change': '2024-09-21T07:52:41.033229+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033229+0000', 'last_undegraded': '2024-09-21T07:52:41.033229+0000', 'last_fullsized': '2024-09-21T07:52:41.033229+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2e is not active+clean 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035039+0000', 'last_change': '2024-09-21T07:52:41.035039+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035039+0000', 'last_undegraded': '2024-09-21T07:52:41.035039+0000', 'last_fullsized': '2024-09-21T07:52:41.035039+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 3], 'acting': [7, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2d is not active+clean 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029611+0000', 'last_change': '2024-09-21T07:52:41.029611+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029611+0000', 'last_undegraded': '2024-09-21T07:52:41.029611+0000', 'last_fullsized': '2024-09-21T07:52:41.029611+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2c is not active+clean 2024-09-21T07:52:47.465 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.036505+0000', 'last_change': '2024-09-21T07:52:41.036505+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.036505+0000', 'last_undegraded': '2024-09-21T07:52:41.036505+0000', 'last_fullsized': '2024-09-21T07:52:41.036505+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 5], 'acting': [7, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2b is not active+clean 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034977+0000', 'last_change': '2024-09-21T07:52:41.034977+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034977+0000', 'last_undegraded': '2024-09-21T07:52:41.034977+0000', 'last_fullsized': '2024-09-21T07:52:41.034977+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2a is not active+clean 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.030494+0000', 'last_change': '2024-09-21T07:52:41.030494+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.030494+0000', 'last_undegraded': '2024-09-21T07:52:41.030494+0000', 'last_fullsized': '2024-09-21T07:52:41.030494+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:PG 19.29 is not active+clean 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.29', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.037052+0000', 'last_change': '2024-09-21T07:52:41.037052+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.037052+0000', 'last_undegraded': '2024-09-21T07:52:41.037052+0000', 'last_fullsized': '2024-09-21T07:52:41.037052+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:PG 19.28 is not active+clean 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.28', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035828+0000', 'last_change': '2024-09-21T07:52:41.035828+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035828+0000', 'last_undegraded': '2024-09-21T07:52:41.035828+0000', 'last_fullsized': '2024-09-21T07:52:41.035828+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:PG 19.27 is not active+clean 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.27', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.027380+0000', 'last_change': '2024-09-21T07:52:41.027380+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.027380+0000', 'last_undegraded': '2024-09-21T07:52:41.027380+0000', 'last_fullsized': '2024-09-21T07:52:41.027380+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 1], 'acting': [5, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2024-09-21T07:52:47.466 INFO:tasks.ceph.ceph_manager.ceph:PG 19.26 is not active+clean 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.26', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029802+0000', 'last_change': '2024-09-21T07:52:41.029802+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029802+0000', 'last_undegraded': '2024-09-21T07:52:41.029802+0000', 'last_fullsized': '2024-09-21T07:52:41.029802+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 7], 'acting': [4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:PG 19.25 is not active+clean 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.25', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033847+0000', 'last_change': '2024-09-21T07:52:41.033847+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033847+0000', 'last_undegraded': '2024-09-21T07:52:41.033847+0000', 'last_fullsized': '2024-09-21T07:52:41.033847+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 0], 'acting': [6, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:PG 19.24 is not active+clean 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.24', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.030093+0000', 'last_change': '2024-09-21T07:52:41.030093+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.030093+0000', 'last_undegraded': '2024-09-21T07:52:41.030093+0000', 'last_fullsized': '2024-09-21T07:52:41.030093+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:PG 19.23 is not active+clean 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.23', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.028483+0000', 'last_change': '2024-09-21T07:52:41.028483+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.028483+0000', 'last_undegraded': '2024-09-21T07:52:41.028483+0000', 'last_fullsized': '2024-09-21T07:52:41.028483+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:PG 19.22 is not active+clean 2024-09-21T07:52:47.467 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.22', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029992+0000', 'last_change': '2024-09-21T07:52:41.029992+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029992+0000', 'last_undegraded': '2024-09-21T07:52:41.029992+0000', 'last_fullsized': '2024-09-21T07:52:41.029992+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:PG 19.21 is not active+clean 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.21', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.030493+0000', 'last_change': '2024-09-21T07:52:41.030493+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.030493+0000', 'last_undegraded': '2024-09-21T07:52:41.030493+0000', 'last_fullsized': '2024-09-21T07:52:41.030493+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 7], 'acting': [1, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:PG 19.20 is not active+clean 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.20', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033920+0000', 'last_change': '2024-09-21T07:52:41.033920+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033920+0000', 'last_undegraded': '2024-09-21T07:52:41.033920+0000', 'last_fullsized': '2024-09-21T07:52:41.033920+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 4], 'acting': [3, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1e is not active+clean 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035011+0000', 'last_change': '2024-09-21T07:52:41.035011+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035011+0000', 'last_undegraded': '2024-09-21T07:52:41.035011+0000', 'last_fullsized': '2024-09-21T07:52:41.035011+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1f is not active+clean 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.038710+0000', 'last_change': '2024-09-21T07:52:41.038710+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.038710+0000', 'last_undegraded': '2024-09-21T07:52:41.038710+0000', 'last_fullsized': '2024-09-21T07:52:41.038710+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1c is not active+clean 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.031733+0000', 'last_change': '2024-09-21T07:52:41.031733+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.031733+0000', 'last_undegraded': '2024-09-21T07:52:41.031733+0000', 'last_fullsized': '2024-09-21T07:52:41.031733+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.468 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1d is not active+clean 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.035750+0000', 'last_change': '2024-09-21T07:52:41.035750+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.035750+0000', 'last_undegraded': '2024-09-21T07:52:41.035750+0000', 'last_fullsized': '2024-09-21T07:52:41.035750+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1a is not active+clean 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.037343+0000', 'last_change': '2024-09-21T07:52:41.037343+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.037343+0000', 'last_undegraded': '2024-09-21T07:52:41.037343+0000', 'last_fullsized': '2024-09-21T07:52:41.037343+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 2], 'acting': [7, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1b is not active+clean 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034529+0000', 'last_change': '2024-09-21T07:52:41.034529+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034529+0000', 'last_undegraded': '2024-09-21T07:52:41.034529+0000', 'last_fullsized': '2024-09-21T07:52:41.034529+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 6], 'acting': [0, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:PG 19.18 is not active+clean 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.031583+0000', 'last_change': '2024-09-21T07:52:41.031583+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.031583+0000', 'last_undegraded': '2024-09-21T07:52:41.031583+0000', 'last_fullsized': '2024-09-21T07:52:41.031583+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:PG 19.19 is not active+clean 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029388+0000', 'last_change': '2024-09-21T07:52:41.029388+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029388+0000', 'last_undegraded': '2024-09-21T07:52:41.029388+0000', 'last_fullsized': '2024-09-21T07:52:41.029388+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:PG 19.16 is not active+clean 2024-09-21T07:52:47.469 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029191+0000', 'last_change': '2024-09-21T07:52:41.029191+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029191+0000', 'last_undegraded': '2024-09-21T07:52:41.029191+0000', 'last_fullsized': '2024-09-21T07:52:41.029191+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:PG 19.17 is not active+clean 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.041579+0000', 'last_change': '2024-09-21T07:52:41.041579+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.041579+0000', 'last_undegraded': '2024-09-21T07:52:41.041579+0000', 'last_fullsized': '2024-09-21T07:52:41.041579+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 7], 'acting': [2, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:PG 19.14 is not active+clean 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033729+0000', 'last_change': '2024-09-21T07:52:41.033729+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033729+0000', 'last_undegraded': '2024-09-21T07:52:41.033729+0000', 'last_fullsized': '2024-09-21T07:52:41.033729+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:PG 19.15 is not active+clean 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.030074+0000', 'last_change': '2024-09-21T07:52:41.030074+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.030074+0000', 'last_undegraded': '2024-09-21T07:52:41.030074+0000', 'last_fullsized': '2024-09-21T07:52:41.030074+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:PG 19.13 is not active+clean 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.031136+0000', 'last_change': '2024-09-21T07:52:41.031136+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.031136+0000', 'last_undegraded': '2024-09-21T07:52:41.031136+0000', 'last_fullsized': '2024-09-21T07:52:41.031136+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:PG 19.10 is not active+clean 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.037809+0000', 'last_change': '2024-09-21T07:52:41.037809+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.037809+0000', 'last_undegraded': '2024-09-21T07:52:41.037809+0000', 'last_fullsized': '2024-09-21T07:52:41.037809+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 3], 'acting': [6, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:PG 19.11 is not active+clean 2024-09-21T07:52:47.470 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033514+0000', 'last_change': '2024-09-21T07:52:41.033514+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033514+0000', 'last_undegraded': '2024-09-21T07:52:41.033514+0000', 'last_fullsized': '2024-09-21T07:52:41.033514+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:PG 19.e is not active+clean 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.039796+0000', 'last_change': '2024-09-21T07:52:41.039796+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.039796+0000', 'last_undegraded': '2024-09-21T07:52:41.039796+0000', 'last_fullsized': '2024-09-21T07:52:41.039796+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 6], 'acting': [2, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:PG 19.f is not active+clean 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.041080+0000', 'last_change': '2024-09-21T07:52:41.041080+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.041080+0000', 'last_undegraded': '2024-09-21T07:52:41.041080+0000', 'last_fullsized': '2024-09-21T07:52:41.041080+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 5], 'acting': [2, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:PG 19.c is not active+clean 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.027661+0000', 'last_change': '2024-09-21T07:52:41.027661+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.027661+0000', 'last_undegraded': '2024-09-21T07:52:41.027661+0000', 'last_fullsized': '2024-09-21T07:52:41.027661+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:PG 19.d is not active+clean 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029014+0000', 'last_change': '2024-09-21T07:52:41.029014+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029014+0000', 'last_undegraded': '2024-09-21T07:52:41.029014+0000', 'last_fullsized': '2024-09-21T07:52:41.029014+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:PG 19.a is not active+clean 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029439+0000', 'last_change': '2024-09-21T07:52:41.029439+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029439+0000', 'last_undegraded': '2024-09-21T07:52:41.029439+0000', 'last_fullsized': '2024-09-21T07:52:41.029439+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:PG 19.b is not active+clean 2024-09-21T07:52:47.471 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029347+0000', 'last_change': '2024-09-21T07:52:41.029347+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029347+0000', 'last_undegraded': '2024-09-21T07:52:41.029347+0000', 'last_fullsized': '2024-09-21T07:52:41.029347+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:PG 19.8 is not active+clean 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.037219+0000', 'last_change': '2024-09-21T07:52:41.037219+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.037219+0000', 'last_undegraded': '2024-09-21T07:52:41.037219+0000', 'last_fullsized': '2024-09-21T07:52:41.037219+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 7], 'acting': [6, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:PG 19.9 is not active+clean 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.032466+0000', 'last_change': '2024-09-21T07:52:41.032466+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.032466+0000', 'last_undegraded': '2024-09-21T07:52:41.032466+0000', 'last_fullsized': '2024-09-21T07:52:41.032466+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:PG 19.6 is not active+clean 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.029377+0000', 'last_change': '2024-09-21T07:52:41.029377+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.029377+0000', 'last_undegraded': '2024-09-21T07:52:41.029377+0000', 'last_fullsized': '2024-09-21T07:52:41.029377+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 0], 'acting': [1, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:PG 19.7 is not active+clean 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.026772+0000', 'last_change': '2024-09-21T07:52:41.026772+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.026772+0000', 'last_undegraded': '2024-09-21T07:52:41.026772+0000', 'last_fullsized': '2024-09-21T07:52:41.026772+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:PG 19.4 is not active+clean 2024-09-21T07:52:47.472 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034577+0000', 'last_change': '2024-09-21T07:52:41.034577+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034577+0000', 'last_undegraded': '2024-09-21T07:52:41.034577+0000', 'last_fullsized': '2024-09-21T07:52:41.034577+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:PG 19.5 is not active+clean 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034863+0000', 'last_change': '2024-09-21T07:52:41.034863+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034863+0000', 'last_undegraded': '2024-09-21T07:52:41.034863+0000', 'last_fullsized': '2024-09-21T07:52:41.034863+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:PG 19.2 is not active+clean 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.033987+0000', 'last_change': '2024-09-21T07:52:41.033987+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.033987+0000', 'last_undegraded': '2024-09-21T07:52:41.033987+0000', 'last_fullsized': '2024-09-21T07:52:41.033987+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 2], 'acting': [0, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:PG 19.3 is not active+clean 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034058+0000', 'last_change': '2024-09-21T07:52:41.034058+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034058+0000', 'last_undegraded': '2024-09-21T07:52:41.034058+0000', 'last_fullsized': '2024-09-21T07:52:41.034058+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:PG 19.0 is not active+clean 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.034036+0000', 'last_change': '2024-09-21T07:52:41.034036+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.034036+0000', 'last_undegraded': '2024-09-21T07:52:41.034036+0000', 'last_fullsized': '2024-09-21T07:52:41.034036+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 0], 'acting': [7, 0], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:PG 19.1 is not active+clean 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.032079+0000', 'last_change': '2024-09-21T07:52:41.032079+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.032079+0000', 'last_undegraded': '2024-09-21T07:52:41.032079+0000', 'last_fullsized': '2024-09-21T07:52:41.032079+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:PG 19.12 is not active+clean 2024-09-21T07:52:47.473 INFO:tasks.ceph.ceph_manager.ceph:{'pgid': '19.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 118, 'state': 'creating+peering', 'last_fresh': '2024-09-21T07:52:41.031023+0000', 'last_change': '2024-09-21T07:52:41.031023+0000', 'last_active': '2024-09-21T07:52:41.014053+0000', 'last_peered': '2024-09-21T07:52:41.014053+0000', 'last_clean': '2024-09-21T07:52:41.014053+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2024-09-21T07:52:41.031023+0000', 'last_undegraded': '2024-09-21T07:52:41.031023+0000', 'last_fullsized': '2024-09-21T07:52:41.031023+0000', 'mapping_epoch': 118, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 118, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'last_clean_scrub_stamp': '2024-09-21T07:52:41.014053+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 5], 'acting': [3, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2024-09-21T07:52:47.474 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-09-21T07:52:47.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 -- 172.21.15.88:0/1053526974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 msgr2=0x7fbda00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 --2- 172.21.15.88:0/1053526974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda00ad4f0 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7fbda0075250 tx=0x7fbd8c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 -- 172.21.15.88:0/1053526974 shutdown_connections 2024-09-21T07:52:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 --2- 172.21.15.88:0/1053526974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda00ad4f0 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 --2- 172.21.15.88:0/1053526974 >> v2:172.21.15.175:3300/0 conn(0x7fbda015d760 0x7fbda01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 -- 172.21.15.88:0/1053526974 >> 172.21.15.88:0/1053526974 conn(0x7fbda006d730 msgr2=0x7fbda00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.716+0000 7fbda7800640 1 -- 172.21.15.88:0/1053526974 shutdown_connections 2024-09-21T07:52:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.717+0000 7fbda7800640 1 -- 172.21.15.88:0/1053526974 wait complete. 2024-09-21T07:52:47.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.719+0000 7fbda7800640 1 Processor -- start 2024-09-21T07:52:47.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.719+0000 7fbda7800640 1 -- start start 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda7800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda0145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda7800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda015d760 0x7fbda0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda7800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda0160310 0x7fbda0073580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda7800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbda00b4c20 con 0x7fbda00ad0e0 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda7800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbda0153c70 con 0x7fbda0160310 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda7800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbda00b4da0 con 0x7fbda015d760 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda0145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.720+0000 7fbda4e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda0145730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57268/0 (socket says 172.21.15.88:57268) 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 -- 172.21.15.88:0/2783887342 learned_addr learned my addr 172.21.15.88:0/2783887342 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbd9fe00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda015d760 0x7fbda0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda5800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda0160310 0x7fbda0073580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 -- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda015d760 msgr2=0x7fbda0073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda015d760 0x7fbda0073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 -- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda0160310 msgr2=0x7fbda0073580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda0160310 0x7fbda0073580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 -- 172.21.15.88:0/2783887342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda014fe70 con 0x7fbda00ad0e0 2024-09-21T07:52:47.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbda4e00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda0145730 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c9a0 tx=0x7fbd9400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:47.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94007bb0 con 0x7fbda00ad0e0 2024-09-21T07:52:47.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.721+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd94007e40 con 0x7fbda00ad0e0 2024-09-21T07:52:47.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.722+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400f690 con 0x7fbda00ad0e0 2024-09-21T07:52:47.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.722+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda01543b0 con 0x7fbda00ad0e0 2024-09-21T07:52:47.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.722+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda0154970 con 0x7fbda00ad0e0 2024-09-21T07:52:47.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.723+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda015e9b0 con 0x7fbda00ad0e0 2024-09-21T07:52:47.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.724+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fbd9400f830 con 0x7fbda00ad0e0 2024-09-21T07:52:47.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.726+0000 7fbd9cc00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd80079040 0x7fbd8007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:47.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.726+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fbd9409bea0 con 0x7fbda00ad0e0 2024-09-21T07:52:47.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.726+0000 7fbd9fe00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd80079040 0x7fbd8007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:47.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.726+0000 7fbd9fe00640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd80079040 0x7fbd8007b550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c033000 tx=0x7fbd8c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:47.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.730+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbda015e9b0 con 0x7fbda00ad0e0 2024-09-21T07:52:47.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.989+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) -- 0x7fbda0161010 con 0x7fbda00ad0e0 2024-09-21T07:52:47.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.990+0000 7fbd9cc00640 1 -- 172.21.15.88:0/2783887342 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) ==== 72+0+1078 (secure 0 0 0) 0x7fbd9409d450 con 0x7fbda00ad0e0 2024-09-21T07:52:47.992 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:47.992 INFO:teuthology.orchestra.run.smithi088.stdout:{"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":429,"monmap":{"epoch":1,"min_mon_release_name":"squid","num_mons":3},"osdmap":{"epoch":120,"num_osds":8,"num_up_osds":8,"osd_up_since":1726904755,"num_in_osds":8,"osd_in_since":1726904744,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"creating+peering","count":64},{"state_name":"active+clean","count":33}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":268382208,"bytes_avail":772825731072,"bytes_total":773094113280,"inactive_pgs_ratio":0.65979379415512085},"fsmap":{"epoch":80,"btime":"2024-09-21T07:52:45:060798+0000","id":9,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":9,"rank":0,"name":"c","status":"up:active","gid":7509}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":14,"modified":"2024-09-21T07:52:29.719469+0000","services":{}},"progress_events":{}} 2024-09-21T07:52:47.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd80079040 msgr2=0x7fbd8007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd80079040 0x7fbd8007b550 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fbd8c033000 tx=0x7fbd8c004730 comp rx=0 tx=0).stop 2024-09-21T07:52:47.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 msgr2=0x7fbda0145730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:47.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda0145730 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fbd9400c9a0 tx=0x7fbd9400ce60 comp rx=0 tx=0).stop 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 shutdown_connections 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd80079040 0x7fbd8007b550 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda015d760 0x7fbda0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbda0160310 0x7fbda0073580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 --2- 172.21.15.88:0/2783887342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda00ad0e0 0x7fbda0145730 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 >> 172.21.15.88:0/2783887342 conn(0x7fbda006d730 msgr2=0x7fbda015f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 shutdown_connections 2024-09-21T07:52:47.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:47.992+0000 7fbda7800640 1 -- 172.21.15.88:0/2783887342 wait complete. 2024-09-21T07:52:48.006 INFO:tasks.ceph.mds.c.smithi088.stderr:2024-09-21T07:52:48.004+0000 7fab2aa00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:52:48.017 INFO:tasks.ceph.ceph_manager.ceph:no progress seen, keeping timeout for now 2024-09-21T07:52:48.017 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 -- 172.21.15.88:0/1211370143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c06f7b0 msgr2=0x7f7f0c0b31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 --2- 172.21.15.88:0/1211370143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c06f7b0 0x7f7f0c0b31e0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c072810 tx=0x7f7f0402aa10 comp rx=0 tx=0).stop 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 -- 172.21.15.88:0/1211370143 shutdown_connections 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 --2- 172.21.15.88:0/1211370143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c06f7b0 0x7f7f0c0b31e0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 --2- 172.21.15.88:0/1211370143 >> v2:172.21.15.175:3300/0 conn(0x7f7f0c0af630 0x7f7f0c073500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 -- 172.21.15.88:0/1211370143 >> 172.21.15.88:0/1211370143 conn(0x7f7f0c06d730 msgr2=0x7f7f0c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 -- 172.21.15.88:0/1211370143 shutdown_connections 2024-09-21T07:52:48.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.235+0000 7f7f13000640 1 -- 172.21.15.88:0/1211370143 wait complete. 2024-09-21T07:52:48.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.237+0000 7f7f13000640 1 Processor -- start 2024-09-21T07:52:48.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.237+0000 7f7f13000640 1 -- start start 2024-09-21T07:52:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f13000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f0c06f7b0 0x7f7f0c1f95c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f13000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f0c073a40 0x7f7f0c14e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f13000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 0x7f7f0c09c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f13000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c0b3b60 con 0x7f7f0c0af630 2024-09-21T07:52:48.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f13000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f0c0b39e0 con 0x7f7f0c073a40 2024-09-21T07:52:48.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f13000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7f0c0b3ce0 con 0x7f7f0c06f7b0 2024-09-21T07:52:48.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f11000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 0x7f7f0c09c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:48.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f11000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 0x7f7f0c09c4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57290/0 (socket says 172.21.15.88:57290) 2024-09-21T07:52:48.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f11000640 1 -- 172.21.15.88:0/4233533517 learned_addr learned my addr 172.21.15.88:0/4233533517 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:48.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f0be00640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f0c06f7b0 0x7f7f0c1f95c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.239+0000 7f7f0b400640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f0c073a40 0x7f7f0c14e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f11000640 1 -- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f0c06f7b0 msgr2=0x7f7f0c1f95c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f11000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f0c06f7b0 0x7f7f0c1f95c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f11000640 1 -- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f0c073a40 msgr2=0x7f7f0c14e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f11000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f0c073a40 0x7f7f0c14e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f11000640 1 -- 172.21.15.88:0/4233533517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0c0ca1d0 con 0x7f7f0c0af630 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f0be00640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f0c06f7b0 0x7f7f0c1f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f0b400640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f0c073a40 0x7f7f0c14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f11000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 0x7f7f0c09c4a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009d60 tx=0x7f7f04003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:48.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04033050 con 0x7f7f0c0af630 2024-09-21T07:52:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f04004550 con 0x7f7f0c0af630 2024-09-21T07:52:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f0c1597b0 con 0x7f7f0c0af630 2024-09-21T07:52:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f04038450 con 0x7f7f0c0af630 2024-09-21T07:52:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.240+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f0c159d70 con 0x7f7f0c0af630 2024-09-21T07:52:48.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.242+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f0c0b4200 con 0x7f7f0c0af630 2024-09-21T07:52:48.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.243+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f7f04036070 con 0x7f7f0c0af630 2024-09-21T07:52:48.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.243+0000 7f7f03e00640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7eec079010 0x7f7eec07b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:48.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.244+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f7f040bcb70 con 0x7f7f0c0af630 2024-09-21T07:52:48.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.244+0000 7f7f0be00640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7eec079010 0x7f7eec07b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:48.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.244+0000 7f7f0be00640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7eec079010 0x7f7eec07b520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00ba60 tx=0x7f7efc005cf0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:48.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.248+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f0c073e50 con 0x7f7f0c0af630 2024-09-21T07:52:48.434 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.432+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7f0c0ca4b0 con 0x7f7eec079010 2024-09-21T07:52:48.438 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.436+0000 7f7f03e00640 1 -- 172.21.15.88:0/4233533517 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252477 (secure 0 0 0) 0x7f7f0c0ca4b0 con 0x7f7eec079010 2024-09-21T07:52:48.441 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:48.443 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.440+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7eec079010 msgr2=0x7f7eec07b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.440+0000 7f7f13000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7eec079010 0x7f7eec07b520 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f7efc00ba60 tx=0x7f7efc005cf0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 msgr2=0x7f7f0c09c4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 0x7f7f0c09c4a0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7f7f04009d60 tx=0x7f7f04003f10 comp rx=0 tx=0).stop 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 shutdown_connections 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7eec079010 0x7f7eec07b520 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f0c0af630 0x7f7f0c09c4a0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f0c073a40 0x7f7f0c14e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.445 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 --2- 172.21.15.88:0/4233533517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f0c06f7b0 0x7f7f0c1f95c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:48.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 >> 172.21.15.88:0/4233533517 conn(0x7f7f0c06d730 msgr2=0x7f7f0c0b0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:48.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 shutdown_connections 2024-09-21T07:52:48.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:48.441+0000 7f7f13000640 1 -- 172.21.15.88:0/4233533517 wait complete. 2024-09-21T07:52:48.464 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":116,"stamp":"2024-09-21T07:52:47.723648+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":175,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262368,"kb_used_data":12840,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712352,"statfs":{"total":773094113280,"available":772825448448,"internally_reserved":0,"allocated":13148160,"data_stored":6423972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.699369"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044601+0000","last_change":"2024-09-21T07:52:42.050752+0000","last_active":"2024-09-21T07:52:43.044601+0000","last_peered":"2024-09-21T07:52:43.044601+0000","last_clean":"2024-09-21T07:52:43.044601+0000","last_became_active":"2024-09-21T07:52:42.050179+0000","last_became_peered":"2024-09-21T07:52:42.050179+0000","last_unstale":"2024-09-21T07:52:43.044601+0000","last_undegraded":"2024-09-21T07:52:43.044601+0000","last_fullsized":"2024-09-21T07:52:43.044601+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:45.206923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044149+0000","last_change":"2024-09-21T07:52:42.053064+0000","last_active":"2024-09-21T07:52:43.044149+0000","last_peered":"2024-09-21T07:52:43.044149+0000","last_clean":"2024-09-21T07:52:43.044149+0000","last_became_active":"2024-09-21T07:52:42.052295+0000","last_became_peered":"2024-09-21T07:52:42.052295+0000","last_unstale":"2024-09-21T07:52:43.044149+0000","last_undegraded":"2024-09-21T07:52:43.044149+0000","last_fullsized":"2024-09-21T07:52:43.044149+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:17.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087707+0000","last_change":"2024-09-21T07:52:42.049236+0000","last_active":"2024-09-21T07:52:43.087707+0000","last_peered":"2024-09-21T07:52:43.087707+0000","last_clean":"2024-09-21T07:52:43.087707+0000","last_became_active":"2024-09-21T07:52:42.048928+0000","last_became_peered":"2024-09-21T07:52:42.048928+0000","last_unstale":"2024-09-21T07:52:43.087707+0000","last_undegraded":"2024-09-21T07:52:43.087707+0000","last_fullsized":"2024-09-21T07:52:43.087707+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:17.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086753+0000","last_change":"2024-09-21T07:52:42.058830+0000","last_active":"2024-09-21T07:52:43.086753+0000","last_peered":"2024-09-21T07:52:43.086753+0000","last_clean":"2024-09-21T07:52:43.086753+0000","last_became_active":"2024-09-21T07:52:42.057357+0000","last_became_peered":"2024-09-21T07:52:42.057357+0000","last_unstale":"2024-09-21T07:52:43.086753+0000","last_undegraded":"2024-09-21T07:52:43.086753+0000","last_fullsized":"2024-09-21T07:52:43.086753+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:31.637465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043780+0000","last_change":"2024-09-21T07:52:42.051708+0000","last_active":"2024-09-21T07:52:43.043780+0000","last_peered":"2024-09-21T07:52:43.043780+0000","last_clean":"2024-09-21T07:52:43.043780+0000","last_became_active":"2024-09-21T07:52:42.051391+0000","last_became_peered":"2024-09-21T07:52:42.051391+0000","last_unstale":"2024-09-21T07:52:43.043780+0000","last_undegraded":"2024-09-21T07:52:43.043780+0000","last_fullsized":"2024-09-21T07:52:43.043780+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:57.517239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050485+0000","last_change":"2024-09-21T07:52:42.050823+0000","last_active":"2024-09-21T07:52:43.050485+0000","last_peered":"2024-09-21T07:52:43.050485+0000","last_clean":"2024-09-21T07:52:43.050485+0000","last_became_active":"2024-09-21T07:52:42.047622+0000","last_became_peered":"2024-09-21T07:52:42.047622+0000","last_unstale":"2024-09-21T07:52:43.050485+0000","last_undegraded":"2024-09-21T07:52:43.050485+0000","last_fullsized":"2024-09-21T07:52:43.050485+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.334232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043566+0000","last_change":"2024-09-21T07:52:42.037357+0000","last_active":"2024-09-21T07:52:43.043566+0000","last_peered":"2024-09-21T07:52:43.043566+0000","last_clean":"2024-09-21T07:52:43.043566+0000","last_became_active":"2024-09-21T07:52:42.037059+0000","last_became_peered":"2024-09-21T07:52:42.037059+0000","last_unstale":"2024-09-21T07:52:43.043566+0000","last_undegraded":"2024-09-21T07:52:43.043566+0000","last_fullsized":"2024-09-21T07:52:43.043566+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.562593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043531+0000","last_change":"2024-09-21T07:52:42.042633+0000","last_active":"2024-09-21T07:52:43.043531+0000","last_peered":"2024-09-21T07:52:43.043531+0000","last_clean":"2024-09-21T07:52:43.043531+0000","last_became_active":"2024-09-21T07:52:42.042339+0000","last_became_peered":"2024-09-21T07:52:42.042339+0000","last_unstale":"2024-09-21T07:52:43.043531+0000","last_undegraded":"2024-09-21T07:52:43.043531+0000","last_fullsized":"2024-09-21T07:52:43.043531+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:45.815588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087021+0000","last_change":"2024-09-21T07:52:42.049256+0000","last_active":"2024-09-21T07:52:43.087021+0000","last_peered":"2024-09-21T07:52:43.087021+0000","last_clean":"2024-09-21T07:52:43.087021+0000","last_became_active":"2024-09-21T07:52:42.048951+0000","last_became_peered":"2024-09-21T07:52:42.048951+0000","last_unstale":"2024-09-21T07:52:43.087021+0000","last_undegraded":"2024-09-21T07:52:43.087021+0000","last_fullsized":"2024-09-21T07:52:43.087021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:24.601213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044243+0000","last_change":"2024-09-21T07:52:42.048121+0000","last_active":"2024-09-21T07:52:43.044243+0000","last_peered":"2024-09-21T07:52:43.044243+0000","last_clean":"2024-09-21T07:52:43.044243+0000","last_became_active":"2024-09-21T07:52:42.047802+0000","last_became_peered":"2024-09-21T07:52:42.047802+0000","last_unstale":"2024-09-21T07:52:43.044243+0000","last_undegraded":"2024-09-21T07:52:43.044243+0000","last_fullsized":"2024-09-21T07:52:43.044243+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:28.809696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087072+0000","last_change":"2024-09-21T07:52:42.056770+0000","last_active":"2024-09-21T07:52:43.087072+0000","last_peered":"2024-09-21T07:52:43.087072+0000","last_clean":"2024-09-21T07:52:43.087072+0000","last_became_active":"2024-09-21T07:52:42.054745+0000","last_became_peered":"2024-09-21T07:52:42.054745+0000","last_unstale":"2024-09-21T07:52:43.087072+0000","last_undegraded":"2024-09-21T07:52:43.087072+0000","last_fullsized":"2024-09-21T07:52:43.087072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:25.430970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051086+0000","last_change":"2024-09-21T07:52:42.057065+0000","last_active":"2024-09-21T07:52:43.051086+0000","last_peered":"2024-09-21T07:52:43.051086+0000","last_clean":"2024-09-21T07:52:43.051086+0000","last_became_active":"2024-09-21T07:52:42.055053+0000","last_became_peered":"2024-09-21T07:52:42.055053+0000","last_unstale":"2024-09-21T07:52:43.051086+0000","last_undegraded":"2024-09-21T07:52:43.051086+0000","last_fullsized":"2024-09-21T07:52:43.051086+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:52.659011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087092+0000","last_change":"2024-09-21T07:52:42.034410+0000","last_active":"2024-09-21T07:52:43.087092+0000","last_peered":"2024-09-21T07:52:43.087092+0000","last_clean":"2024-09-21T07:52:43.087092+0000","last_became_active":"2024-09-21T07:52:42.034074+0000","last_became_peered":"2024-09-21T07:52:42.034074+0000","last_unstale":"2024-09-21T07:52:43.087092+0000","last_undegraded":"2024-09-21T07:52:43.087092+0000","last_fullsized":"2024-09-21T07:52:43.087092+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:22.274012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042677+0000","last_change":"2024-09-21T07:52:42.052363+0000","last_active":"2024-09-21T07:52:43.042677+0000","last_peered":"2024-09-21T07:52:43.042677+0000","last_clean":"2024-09-21T07:52:43.042677+0000","last_became_active":"2024-09-21T07:52:42.050716+0000","last_became_peered":"2024-09-21T07:52:42.050716+0000","last_unstale":"2024-09-21T07:52:43.042677+0000","last_undegraded":"2024-09-21T07:52:43.042677+0000","last_fullsized":"2024-09-21T07:52:43.042677+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:32.733984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087535+0000","last_change":"2024-09-21T07:52:42.059858+0000","last_active":"2024-09-21T07:52:43.087535+0000","last_peered":"2024-09-21T07:52:43.087535+0000","last_clean":"2024-09-21T07:52:43.087535+0000","last_became_active":"2024-09-21T07:52:42.059552+0000","last_became_peered":"2024-09-21T07:52:42.059552+0000","last_unstale":"2024-09-21T07:52:43.087535+0000","last_undegraded":"2024-09-21T07:52:43.087535+0000","last_fullsized":"2024-09-21T07:52:43.087535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:15.045213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043303+0000","last_change":"2024-09-21T07:52:42.058578+0000","last_active":"2024-09-21T07:52:43.043303+0000","last_peered":"2024-09-21T07:52:43.043303+0000","last_clean":"2024-09-21T07:52:43.043303+0000","last_became_active":"2024-09-21T07:52:42.053450+0000","last_became_peered":"2024-09-21T07:52:42.053450+0000","last_unstale":"2024-09-21T07:52:43.043303+0000","last_undegraded":"2024-09-21T07:52:43.043303+0000","last_fullsized":"2024-09-21T07:52:43.043303+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:10.585517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087551+0000","last_change":"2024-09-21T07:52:42.041074+0000","last_active":"2024-09-21T07:52:43.087551+0000","last_peered":"2024-09-21T07:52:43.087551+0000","last_clean":"2024-09-21T07:52:43.087551+0000","last_became_active":"2024-09-21T07:52:42.040748+0000","last_became_peered":"2024-09-21T07:52:42.040748+0000","last_unstale":"2024-09-21T07:52:43.087551+0000","last_undegraded":"2024-09-21T07:52:43.087551+0000","last_fullsized":"2024-09-21T07:52:43.087551+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:17.071991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087560+0000","last_change":"2024-09-21T07:52:42.058605+0000","last_active":"2024-09-21T07:52:43.087560+0000","last_peered":"2024-09-21T07:52:43.087560+0000","last_clean":"2024-09-21T07:52:43.087560+0000","last_became_active":"2024-09-21T07:52:42.058244+0000","last_became_peered":"2024-09-21T07:52:42.058244+0000","last_unstale":"2024-09-21T07:52:43.087560+0000","last_undegraded":"2024-09-21T07:52:43.087560+0000","last_fullsized":"2024-09-21T07:52:43.087560+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:20.377996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044029+0000","last_change":"2024-09-21T07:52:42.054229+0000","last_active":"2024-09-21T07:52:43.044029+0000","last_peered":"2024-09-21T07:52:43.044029+0000","last_clean":"2024-09-21T07:52:43.044029+0000","last_became_active":"2024-09-21T07:52:42.053413+0000","last_became_peered":"2024-09-21T07:52:42.053413+0000","last_unstale":"2024-09-21T07:52:43.044029+0000","last_undegraded":"2024-09-21T07:52:43.044029+0000","last_fullsized":"2024-09-21T07:52:43.044029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:55.002454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086789+0000","last_change":"2024-09-21T07:52:42.056523+0000","last_active":"2024-09-21T07:52:43.086789+0000","last_peered":"2024-09-21T07:52:43.086789+0000","last_clean":"2024-09-21T07:52:43.086789+0000","last_became_active":"2024-09-21T07:52:42.055917+0000","last_became_peered":"2024-09-21T07:52:42.055917+0000","last_unstale":"2024-09-21T07:52:43.086789+0000","last_undegraded":"2024-09-21T07:52:43.086789+0000","last_fullsized":"2024-09-21T07:52:43.086789+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:07.835000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050482+0000","last_change":"2024-09-21T07:52:42.057926+0000","last_active":"2024-09-21T07:52:43.050482+0000","last_peered":"2024-09-21T07:52:43.050482+0000","last_clean":"2024-09-21T07:52:43.050482+0000","last_became_active":"2024-09-21T07:52:42.057630+0000","last_became_peered":"2024-09-21T07:52:42.057630+0000","last_unstale":"2024-09-21T07:52:43.050482+0000","last_undegraded":"2024-09-21T07:52:43.050482+0000","last_fullsized":"2024-09-21T07:52:43.050482+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:05.673571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042672+0000","last_change":"2024-09-21T07:52:42.039105+0000","last_active":"2024-09-21T07:52:43.042672+0000","last_peered":"2024-09-21T07:52:43.042672+0000","last_clean":"2024-09-21T07:52:43.042672+0000","last_became_active":"2024-09-21T07:52:42.038859+0000","last_became_peered":"2024-09-21T07:52:42.038859+0000","last_unstale":"2024-09-21T07:52:43.042672+0000","last_undegraded":"2024-09-21T07:52:43.042672+0000","last_fullsized":"2024-09-21T07:52:43.042672+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:53.772949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086509+0000","last_change":"2024-09-21T07:52:42.039198+0000","last_active":"2024-09-21T07:52:43.086509+0000","last_peered":"2024-09-21T07:52:43.086509+0000","last_clean":"2024-09-21T07:52:43.086509+0000","last_became_active":"2024-09-21T07:52:42.038344+0000","last_became_peered":"2024-09-21T07:52:42.038344+0000","last_unstale":"2024-09-21T07:52:43.086509+0000","last_undegraded":"2024-09-21T07:52:43.086509+0000","last_fullsized":"2024-09-21T07:52:43.086509+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:51.708113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086908+0000","last_change":"2024-09-21T07:52:42.062200+0000","last_active":"2024-09-21T07:52:43.086908+0000","last_peered":"2024-09-21T07:52:43.086908+0000","last_clean":"2024-09-21T07:52:43.086908+0000","last_became_active":"2024-09-21T07:52:42.061735+0000","last_became_peered":"2024-09-21T07:52:42.061735+0000","last_unstale":"2024-09-21T07:52:43.086908+0000","last_undegraded":"2024-09-21T07:52:43.086908+0000","last_fullsized":"2024-09-21T07:52:43.086908+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:37.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087578+0000","last_change":"2024-09-21T07:52:42.043994+0000","last_active":"2024-09-21T07:52:43.087578+0000","last_peered":"2024-09-21T07:52:43.087578+0000","last_clean":"2024-09-21T07:52:43.087578+0000","last_became_active":"2024-09-21T07:52:42.042860+0000","last_became_peered":"2024-09-21T07:52:42.042860+0000","last_unstale":"2024-09-21T07:52:43.087578+0000","last_undegraded":"2024-09-21T07:52:43.087578+0000","last_fullsized":"2024-09-21T07:52:43.087578+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:47.270955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044593+0000","last_change":"2024-09-21T07:52:42.052626+0000","last_active":"2024-09-21T07:52:43.044593+0000","last_peered":"2024-09-21T07:52:43.044593+0000","last_clean":"2024-09-21T07:52:43.044593+0000","last_became_active":"2024-09-21T07:52:42.051816+0000","last_became_peered":"2024-09-21T07:52:42.051816+0000","last_unstale":"2024-09-21T07:52:43.044593+0000","last_undegraded":"2024-09-21T07:52:43.044593+0000","last_fullsized":"2024-09-21T07:52:43.044593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:27.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043252+0000","last_change":"2024-09-21T07:52:42.052220+0000","last_active":"2024-09-21T07:52:43.043252+0000","last_peered":"2024-09-21T07:52:43.043252+0000","last_clean":"2024-09-21T07:52:43.043252+0000","last_became_active":"2024-09-21T07:52:42.051926+0000","last_became_peered":"2024-09-21T07:52:42.051926+0000","last_unstale":"2024-09-21T07:52:43.043252+0000","last_undegraded":"2024-09-21T07:52:43.043252+0000","last_fullsized":"2024-09-21T07:52:43.043252+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:38.186284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086573+0000","last_change":"2024-09-21T07:52:42.053749+0000","last_active":"2024-09-21T07:52:43.086573+0000","last_peered":"2024-09-21T07:52:43.086573+0000","last_clean":"2024-09-21T07:52:43.086573+0000","last_became_active":"2024-09-21T07:52:42.053428+0000","last_became_peered":"2024-09-21T07:52:42.053428+0000","last_unstale":"2024-09-21T07:52:43.086573+0000","last_undegraded":"2024-09-21T07:52:43.086573+0000","last_fullsized":"2024-09-21T07:52:43.086573+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:01.852292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043717+0000","last_change":"2024-09-21T07:52:42.045589+0000","last_active":"2024-09-21T07:52:43.043717+0000","last_peered":"2024-09-21T07:52:43.043717+0000","last_clean":"2024-09-21T07:52:43.043717+0000","last_became_active":"2024-09-21T07:52:42.045307+0000","last_became_peered":"2024-09-21T07:52:42.045307+0000","last_unstale":"2024-09-21T07:52:43.043717+0000","last_undegraded":"2024-09-21T07:52:43.043717+0000","last_fullsized":"2024-09-21T07:52:43.043717+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:34.916044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042937+0000","last_change":"2024-09-21T07:52:42.039461+0000","last_active":"2024-09-21T07:52:43.042937+0000","last_peered":"2024-09-21T07:52:43.042937+0000","last_clean":"2024-09-21T07:52:43.042937+0000","last_became_active":"2024-09-21T07:52:42.036661+0000","last_became_peered":"2024-09-21T07:52:42.036661+0000","last_unstale":"2024-09-21T07:52:43.042937+0000","last_undegraded":"2024-09-21T07:52:43.042937+0000","last_fullsized":"2024-09-21T07:52:43.042937+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:05.704681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086268+0000","last_change":"2024-09-21T07:52:42.046781+0000","last_active":"2024-09-21T07:52:43.086268+0000","last_peered":"2024-09-21T07:52:43.086268+0000","last_clean":"2024-09-21T07:52:43.086268+0000","last_became_active":"2024-09-21T07:52:42.046297+0000","last_became_peered":"2024-09-21T07:52:42.046297+0000","last_unstale":"2024-09-21T07:52:43.086268+0000","last_undegraded":"2024-09-21T07:52:43.086268+0000","last_fullsized":"2024-09-21T07:52:43.086268+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:38:58.850800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086475+0000","last_change":"2024-09-21T07:52:42.043362+0000","last_active":"2024-09-21T07:52:43.086475+0000","last_peered":"2024-09-21T07:52:43.086475+0000","last_clean":"2024-09-21T07:52:43.086475+0000","last_became_active":"2024-09-21T07:52:42.042411+0000","last_became_peered":"2024-09-21T07:52:42.042411+0000","last_unstale":"2024-09-21T07:52:43.086475+0000","last_undegraded":"2024-09-21T07:52:43.086475+0000","last_fullsized":"2024-09-21T07:52:43.086475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:58.791224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042701+0000","last_change":"2024-09-21T07:52:42.048886+0000","last_active":"2024-09-21T07:52:43.042701+0000","last_peered":"2024-09-21T07:52:43.042701+0000","last_clean":"2024-09-21T07:52:43.042701+0000","last_became_active":"2024-09-21T07:52:42.048590+0000","last_became_peered":"2024-09-21T07:52:42.048590+0000","last_unstale":"2024-09-21T07:52:43.042701+0000","last_undegraded":"2024-09-21T07:52:43.042701+0000","last_fullsized":"2024-09-21T07:52:43.042701+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:38.936104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"119'4","reported_seq":19,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044649+0000","last_change":"2024-09-21T07:52:41.052517+0000","last_active":"2024-09-21T07:52:43.044649+0000","last_peered":"2024-09-21T07:52:43.044649+0000","last_clean":"2024-09-21T07:52:43.044649+0000","last_became_active":"2024-09-21T07:52:41.052152+0000","last_became_peered":"2024-09-21T07:52:41.052152+0000","last_unstale":"2024-09-21T07:52:43.044649+0000","last_undegraded":"2024-09-21T07:52:43.044649+0000","last_fullsized":"2024-09-21T07:52:43.044649+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:54.707465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087245+0000","last_change":"2024-09-21T07:52:42.043656+0000","last_active":"2024-09-21T07:52:43.087245+0000","last_peered":"2024-09-21T07:52:43.087245+0000","last_clean":"2024-09-21T07:52:43.087245+0000","last_became_active":"2024-09-21T07:52:42.043323+0000","last_became_peered":"2024-09-21T07:52:42.043323+0000","last_unstale":"2024-09-21T07:52:43.087245+0000","last_undegraded":"2024-09-21T07:52:43.087245+0000","last_fullsized":"2024-09-21T07:52:43.087245+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:25.758736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087028+0000","last_change":"2024-09-21T07:52:41.051644+0000","last_active":"2024-09-21T07:52:43.087028+0000","last_peered":"2024-09-21T07:52:43.087028+0000","last_clean":"2024-09-21T07:52:43.087028+0000","last_became_active":"2024-09-21T07:52:41.051270+0000","last_became_peered":"2024-09-21T07:52:41.051270+0000","last_unstale":"2024-09-21T07:52:43.087028+0000","last_undegraded":"2024-09-21T07:52:43.087028+0000","last_fullsized":"2024-09-21T07:52:43.087028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:41.582792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086798+0000","last_change":"2024-09-21T07:52:42.034262+0000","last_active":"2024-09-21T07:52:43.086798+0000","last_peered":"2024-09-21T07:52:43.086798+0000","last_clean":"2024-09-21T07:52:43.086798+0000","last_became_active":"2024-09-21T07:52:42.033909+0000","last_became_peered":"2024-09-21T07:52:42.033909+0000","last_unstale":"2024-09-21T07:52:43.086798+0000","last_undegraded":"2024-09-21T07:52:43.086798+0000","last_fullsized":"2024-09-21T07:52:43.086798+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:31.439346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087051+0000","last_change":"2024-09-21T07:52:41.040236+0000","last_active":"2024-09-21T07:52:43.087051+0000","last_peered":"2024-09-21T07:52:43.087051+0000","last_clean":"2024-09-21T07:52:43.087051+0000","last_became_active":"2024-09-21T07:52:41.039288+0000","last_became_peered":"2024-09-21T07:52:41.039288+0000","last_unstale":"2024-09-21T07:52:43.087051+0000","last_undegraded":"2024-09-21T07:52:43.087051+0000","last_fullsized":"2024-09-21T07:52:43.087051+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:22.533170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051178+0000","last_change":"2024-09-21T07:52:42.061519+0000","last_active":"2024-09-21T07:52:43.051178+0000","last_peered":"2024-09-21T07:52:43.051178+0000","last_clean":"2024-09-21T07:52:43.051178+0000","last_became_active":"2024-09-21T07:52:42.060171+0000","last_became_peered":"2024-09-21T07:52:42.060171+0000","last_unstale":"2024-09-21T07:52:43.051178+0000","last_undegraded":"2024-09-21T07:52:43.051178+0000","last_fullsized":"2024-09-21T07:52:43.051178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:46.035398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044211+0000","last_change":"2024-09-21T07:52:41.049531+0000","last_active":"2024-09-21T07:52:43.044211+0000","last_peered":"2024-09-21T07:52:43.044211+0000","last_clean":"2024-09-21T07:52:43.044211+0000","last_became_active":"2024-09-21T07:52:41.048245+0000","last_became_peered":"2024-09-21T07:52:41.048245+0000","last_unstale":"2024-09-21T07:52:43.044211+0000","last_undegraded":"2024-09-21T07:52:43.044211+0000","last_fullsized":"2024-09-21T07:52:43.044211+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:51.466641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086927+0000","last_change":"2024-09-21T07:52:42.061398+0000","last_active":"2024-09-21T07:52:43.086927+0000","last_peered":"2024-09-21T07:52:43.086927+0000","last_clean":"2024-09-21T07:52:43.086927+0000","last_became_active":"2024-09-21T07:52:42.059409+0000","last_became_peered":"2024-09-21T07:52:42.059409+0000","last_unstale":"2024-09-21T07:52:43.086927+0000","last_undegraded":"2024-09-21T07:52:43.086927+0000","last_fullsized":"2024-09-21T07:52:43.086927+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:47.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043855+0000","last_change":"2024-09-21T07:52:41.039897+0000","last_active":"2024-09-21T07:52:43.043855+0000","last_peered":"2024-09-21T07:52:43.043855+0000","last_clean":"2024-09-21T07:52:43.043855+0000","last_became_active":"2024-09-21T07:52:41.039535+0000","last_became_peered":"2024-09-21T07:52:41.039535+0000","last_unstale":"2024-09-21T07:52:43.043855+0000","last_undegraded":"2024-09-21T07:52:43.043855+0000","last_fullsized":"2024-09-21T07:52:43.043855+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:44.554998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050598+0000","last_change":"2024-09-21T07:52:42.053025+0000","last_active":"2024-09-21T07:52:43.050598+0000","last_peered":"2024-09-21T07:52:43.050598+0000","last_clean":"2024-09-21T07:52:43.050598+0000","last_became_active":"2024-09-21T07:52:42.051889+0000","last_became_peered":"2024-09-21T07:52:42.051889+0000","last_unstale":"2024-09-21T07:52:43.050598+0000","last_undegraded":"2024-09-21T07:52:43.050598+0000","last_fullsized":"2024-09-21T07:52:43.050598+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:59.475333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087132+0000","last_change":"2024-09-21T07:52:41.048831+0000","last_active":"2024-09-21T07:52:43.087132+0000","last_peered":"2024-09-21T07:52:43.087132+0000","last_clean":"2024-09-21T07:52:43.087132+0000","last_became_active":"2024-09-21T07:52:41.048477+0000","last_became_peered":"2024-09-21T07:52:41.048477+0000","last_unstale":"2024-09-21T07:52:43.087132+0000","last_undegraded":"2024-09-21T07:52:43.087132+0000","last_fullsized":"2024-09-21T07:52:43.087132+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:58.164968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050462+0000","last_change":"2024-09-21T07:52:42.046577+0000","last_active":"2024-09-21T07:52:43.050462+0000","last_peered":"2024-09-21T07:52:43.050462+0000","last_clean":"2024-09-21T07:52:43.050462+0000","last_became_active":"2024-09-21T07:52:42.046251+0000","last_became_peered":"2024-09-21T07:52:42.046251+0000","last_unstale":"2024-09-21T07:52:43.050462+0000","last_undegraded":"2024-09-21T07:52:43.050462+0000","last_fullsized":"2024-09-21T07:52:43.050462+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:41.643928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086688+0000","last_change":"2024-09-21T07:52:41.046640+0000","last_active":"2024-09-21T07:52:43.086688+0000","last_peered":"2024-09-21T07:52:43.086688+0000","last_clean":"2024-09-21T07:52:43.086688+0000","last_became_active":"2024-09-21T07:52:41.046298+0000","last_became_peered":"2024-09-21T07:52:41.046298+0000","last_unstale":"2024-09-21T07:52:43.086688+0000","last_undegraded":"2024-09-21T07:52:43.086688+0000","last_fullsized":"2024-09-21T07:52:43.086688+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.200614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042310+0000","last_change":"2024-09-21T07:52:42.035989+0000","last_active":"2024-09-21T07:52:43.042310+0000","last_peered":"2024-09-21T07:52:43.042310+0000","last_clean":"2024-09-21T07:52:43.042310+0000","last_became_active":"2024-09-21T07:52:42.035671+0000","last_became_peered":"2024-09-21T07:52:42.035671+0000","last_unstale":"2024-09-21T07:52:43.042310+0000","last_undegraded":"2024-09-21T07:52:43.042310+0000","last_fullsized":"2024-09-21T07:52:43.042310+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:50.444405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042143+0000","last_change":"2024-09-21T07:52:41.043692+0000","last_active":"2024-09-21T07:52:43.042143+0000","last_peered":"2024-09-21T07:52:43.042143+0000","last_clean":"2024-09-21T07:52:43.042143+0000","last_became_active":"2024-09-21T07:52:41.043389+0000","last_became_peered":"2024-09-21T07:52:41.043389+0000","last_unstale":"2024-09-21T07:52:43.042143+0000","last_undegraded":"2024-09-21T07:52:43.042143+0000","last_fullsized":"2024-09-21T07:52:43.042143+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:11.668317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044602+0000","last_change":"2024-09-21T07:52:42.043365+0000","last_active":"2024-09-21T07:52:43.044602+0000","last_peered":"2024-09-21T07:52:43.044602+0000","last_clean":"2024-09-21T07:52:43.044602+0000","last_became_active":"2024-09-21T07:52:42.042454+0000","last_became_peered":"2024-09-21T07:52:42.042454+0000","last_unstale":"2024-09-21T07:52:43.044602+0000","last_undegraded":"2024-09-21T07:52:43.044602+0000","last_fullsized":"2024-09-21T07:52:43.044602+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051213+0000","last_change":"2024-09-21T07:52:41.049633+0000","last_active":"2024-09-21T07:52:43.051213+0000","last_peered":"2024-09-21T07:52:43.051213+0000","last_clean":"2024-09-21T07:52:43.051213+0000","last_became_active":"2024-09-21T07:52:41.047189+0000","last_became_peered":"2024-09-21T07:52:41.047189+0000","last_unstale":"2024-09-21T07:52:43.051213+0000","last_undegraded":"2024-09-21T07:52:43.051213+0000","last_fullsized":"2024-09-21T07:52:43.051213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.645138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044479+0000","last_change":"2024-09-21T07:52:42.050674+0000","last_active":"2024-09-21T07:52:43.044479+0000","last_peered":"2024-09-21T07:52:43.044479+0000","last_clean":"2024-09-21T07:52:43.044479+0000","last_became_active":"2024-09-21T07:52:42.050307+0000","last_became_peered":"2024-09-21T07:52:42.050307+0000","last_unstale":"2024-09-21T07:52:43.044479+0000","last_undegraded":"2024-09-21T07:52:43.044479+0000","last_fullsized":"2024-09-21T07:52:43.044479+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:12.675913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087528+0000","last_change":"2024-09-21T07:52:41.048436+0000","last_active":"2024-09-21T07:52:43.087528+0000","last_peered":"2024-09-21T07:52:43.087528+0000","last_clean":"2024-09-21T07:52:43.087528+0000","last_became_active":"2024-09-21T07:52:41.047717+0000","last_became_peered":"2024-09-21T07:52:41.047717+0000","last_unstale":"2024-09-21T07:52:43.087528+0000","last_undegraded":"2024-09-21T07:52:43.087528+0000","last_fullsized":"2024-09-21T07:52:43.087528+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:08.012317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051380+0000","last_change":"2024-09-21T07:52:42.055320+0000","last_active":"2024-09-21T07:52:43.051380+0000","last_peered":"2024-09-21T07:52:43.051380+0000","last_clean":"2024-09-21T07:52:43.051380+0000","last_became_active":"2024-09-21T07:52:42.054564+0000","last_became_peered":"2024-09-21T07:52:42.054564+0000","last_unstale":"2024-09-21T07:52:43.051380+0000","last_undegraded":"2024-09-21T07:52:43.051380+0000","last_fullsized":"2024-09-21T07:52:43.051380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:23.540765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"119'3","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044189+0000","last_change":"2024-09-21T07:52:41.051335+0000","last_active":"2024-09-21T07:52:43.044189+0000","last_peered":"2024-09-21T07:52:43.044189+0000","last_clean":"2024-09-21T07:52:43.044189+0000","last_became_active":"2024-09-21T07:52:41.051017+0000","last_became_peered":"2024-09-21T07:52:41.051017+0000","last_unstale":"2024-09-21T07:52:43.044189+0000","last_undegraded":"2024-09-21T07:52:43.044189+0000","last_fullsized":"2024-09-21T07:52:43.044189+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:54.271329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086243+0000","last_change":"2024-09-21T07:52:42.055630+0000","last_active":"2024-09-21T07:52:43.086243+0000","last_peered":"2024-09-21T07:52:43.086243+0000","last_clean":"2024-09-21T07:52:43.086243+0000","last_became_active":"2024-09-21T07:52:42.055283+0000","last_became_peered":"2024-09-21T07:52:42.055283+0000","last_unstale":"2024-09-21T07:52:43.086243+0000","last_undegraded":"2024-09-21T07:52:43.086243+0000","last_fullsized":"2024-09-21T07:52:43.086243+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.112436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"119'6","reported_seq":21,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086746+0000","last_change":"2024-09-21T07:52:41.042291+0000","last_active":"2024-09-21T07:52:43.086746+0000","last_peered":"2024-09-21T07:52:43.086746+0000","last_clean":"2024-09-21T07:52:43.086746+0000","last_became_active":"2024-09-21T07:52:41.041543+0000","last_became_peered":"2024-09-21T07:52:41.041543+0000","last_unstale":"2024-09-21T07:52:43.086746+0000","last_undegraded":"2024-09-21T07:52:43.086746+0000","last_fullsized":"2024-09-21T07:52:43.086746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:40.701818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042711+0000","last_change":"2024-09-21T07:52:42.047972+0000","last_active":"2024-09-21T07:52:43.042711+0000","last_peered":"2024-09-21T07:52:43.042711+0000","last_clean":"2024-09-21T07:52:43.042711+0000","last_became_active":"2024-09-21T07:52:42.045974+0000","last_became_peered":"2024-09-21T07:52:42.045974+0000","last_unstale":"2024-09-21T07:52:43.042711+0000","last_undegraded":"2024-09-21T07:52:43.042711+0000","last_fullsized":"2024-09-21T07:52:43.042711+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:41.389611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042871+0000","last_change":"2024-09-21T07:52:41.043893+0000","last_active":"2024-09-21T07:52:43.042871+0000","last_peered":"2024-09-21T07:52:43.042871+0000","last_clean":"2024-09-21T07:52:43.042871+0000","last_became_active":"2024-09-21T07:52:41.043513+0000","last_became_peered":"2024-09-21T07:52:41.043513+0000","last_unstale":"2024-09-21T07:52:43.042871+0000","last_undegraded":"2024-09-21T07:52:43.042871+0000","last_fullsized":"2024-09-21T07:52:43.042871+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.645138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042296+0000","last_change":"2024-09-21T07:52:42.058352+0000","last_active":"2024-09-21T07:52:43.042296+0000","last_peered":"2024-09-21T07:52:43.042296+0000","last_clean":"2024-09-21T07:52:43.042296+0000","last_became_active":"2024-09-21T07:52:42.054077+0000","last_became_peered":"2024-09-21T07:52:42.054077+0000","last_unstale":"2024-09-21T07:52:43.042296+0000","last_undegraded":"2024-09-21T07:52:43.042296+0000","last_fullsized":"2024-09-21T07:52:43.042296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:28.908966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086586+0000","last_change":"2024-09-21T07:52:41.043649+0000","last_active":"2024-09-21T07:52:43.086586+0000","last_peered":"2024-09-21T07:52:43.086586+0000","last_clean":"2024-09-21T07:52:43.086586+0000","last_became_active":"2024-09-21T07:52:41.043360+0000","last_became_peered":"2024-09-21T07:52:41.043360+0000","last_unstale":"2024-09-21T07:52:43.086586+0000","last_undegraded":"2024-09-21T07:52:43.086586+0000","last_fullsized":"2024-09-21T07:52:43.086586+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.133141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050600+0000","last_change":"2024-09-21T07:52:42.045364+0000","last_active":"2024-09-21T07:52:43.050600+0000","last_peered":"2024-09-21T07:52:43.050600+0000","last_clean":"2024-09-21T07:52:43.050600+0000","last_became_active":"2024-09-21T07:52:42.045066+0000","last_became_peered":"2024-09-21T07:52:42.045066+0000","last_unstale":"2024-09-21T07:52:43.050600+0000","last_undegraded":"2024-09-21T07:52:43.050600+0000","last_fullsized":"2024-09-21T07:52:43.050600+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:10.585517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"119'8","reported_seq":23,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050320+0000","last_change":"2024-09-21T07:52:41.049536+0000","last_active":"2024-09-21T07:52:43.050320+0000","last_peered":"2024-09-21T07:52:43.050320+0000","last_clean":"2024-09-21T07:52:43.050320+0000","last_became_active":"2024-09-21T07:52:41.045025+0000","last_became_peered":"2024-09-21T07:52:41.045025+0000","last_unstale":"2024-09-21T07:52:43.050320+0000","last_undegraded":"2024-09-21T07:52:43.050320+0000","last_fullsized":"2024-09-21T07:52:43.050320+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:48.662085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044435+0000","last_change":"2024-09-21T07:52:42.054840+0000","last_active":"2024-09-21T07:52:43.044435+0000","last_peered":"2024-09-21T07:52:43.044435+0000","last_clean":"2024-09-21T07:52:43.044435+0000","last_became_active":"2024-09-21T07:52:42.053231+0000","last_became_peered":"2024-09-21T07:52:42.053231+0000","last_unstale":"2024-09-21T07:52:43.044435+0000","last_undegraded":"2024-09-21T07:52:43.044435+0000","last_fullsized":"2024-09-21T07:52:43.044435+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.474236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051283+0000","last_change":"2024-09-21T07:52:41.050888+0000","last_active":"2024-09-21T07:52:43.051283+0000","last_peered":"2024-09-21T07:52:43.051283+0000","last_clean":"2024-09-21T07:52:43.051283+0000","last_became_active":"2024-09-21T07:52:41.050131+0000","last_became_peered":"2024-09-21T07:52:41.050131+0000","last_unstale":"2024-09-21T07:52:43.051283+0000","last_undegraded":"2024-09-21T07:52:43.051283+0000","last_fullsized":"2024-09-21T07:52:43.051283+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:01.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044625+0000","last_change":"2024-09-21T07:52:42.047061+0000","last_active":"2024-09-21T07:52:43.044625+0000","last_peered":"2024-09-21T07:52:43.044625+0000","last_clean":"2024-09-21T07:52:43.044625+0000","last_became_active":"2024-09-21T07:52:42.045929+0000","last_became_peered":"2024-09-21T07:52:42.045929+0000","last_unstale":"2024-09-21T07:52:43.044625+0000","last_undegraded":"2024-09-21T07:52:43.044625+0000","last_fullsized":"2024-09-21T07:52:43.044625+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086955+0000","last_change":"2024-09-21T07:52:41.036154+0000","last_active":"2024-09-21T07:52:43.086955+0000","last_peered":"2024-09-21T07:52:43.086955+0000","last_clean":"2024-09-21T07:52:43.086955+0000","last_became_active":"2024-09-21T07:52:41.035812+0000","last_became_peered":"2024-09-21T07:52:41.035812+0000","last_unstale":"2024-09-21T07:52:43.086955+0000","last_undegraded":"2024-09-21T07:52:43.086955+0000","last_fullsized":"2024-09-21T07:52:43.086955+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:02.282645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044350+0000","last_change":"2024-09-21T07:52:42.047448+0000","last_active":"2024-09-21T07:52:43.044350+0000","last_peered":"2024-09-21T07:52:43.044350+0000","last_clean":"2024-09-21T07:52:43.044350+0000","last_became_active":"2024-09-21T07:52:42.046508+0000","last_became_peered":"2024-09-21T07:52:42.046508+0000","last_unstale":"2024-09-21T07:52:43.044350+0000","last_undegraded":"2024-09-21T07:52:43.044350+0000","last_fullsized":"2024-09-21T07:52:43.044350+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:10.168925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087817+0000","last_change":"2024-09-21T07:52:41.046956+0000","last_active":"2024-09-21T07:52:43.087817+0000","last_peered":"2024-09-21T07:52:43.087817+0000","last_clean":"2024-09-21T07:52:43.087817+0000","last_became_active":"2024-09-21T07:52:41.046644+0000","last_became_peered":"2024-09-21T07:52:41.046644+0000","last_unstale":"2024-09-21T07:52:43.087817+0000","last_undegraded":"2024-09-21T07:52:43.087817+0000","last_fullsized":"2024-09-21T07:52:43.087817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:00.232783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044134+0000","last_change":"2024-09-21T07:52:42.044820+0000","last_active":"2024-09-21T07:52:43.044134+0000","last_peered":"2024-09-21T07:52:43.044134+0000","last_clean":"2024-09-21T07:52:43.044134+0000","last_became_active":"2024-09-21T07:52:42.042422+0000","last_became_peered":"2024-09-21T07:52:42.042422+0000","last_unstale":"2024-09-21T07:52:43.044134+0000","last_undegraded":"2024-09-21T07:52:43.044134+0000","last_fullsized":"2024-09-21T07:52:43.044134+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:00.771472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044116+0000","last_change":"2024-09-21T07:52:41.047727+0000","last_active":"2024-09-21T07:52:43.044116+0000","last_peered":"2024-09-21T07:52:43.044116+0000","last_clean":"2024-09-21T07:52:43.044116+0000","last_became_active":"2024-09-21T07:52:41.047444+0000","last_became_peered":"2024-09-21T07:52:41.047444+0000","last_unstale":"2024-09-21T07:52:43.044116+0000","last_undegraded":"2024-09-21T07:52:43.044116+0000","last_fullsized":"2024-09-21T07:52:43.044116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:36.951099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043195+0000","last_change":"2024-09-21T07:52:42.048923+0000","last_active":"2024-09-21T07:52:43.043195+0000","last_peered":"2024-09-21T07:52:43.043195+0000","last_clean":"2024-09-21T07:52:43.043195+0000","last_became_active":"2024-09-21T07:52:42.046724+0000","last_became_peered":"2024-09-21T07:52:42.046724+0000","last_unstale":"2024-09-21T07:52:43.043195+0000","last_undegraded":"2024-09-21T07:52:43.043195+0000","last_fullsized":"2024-09-21T07:52:43.043195+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:19.619396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086828+0000","last_change":"2024-09-21T07:52:41.045986+0000","last_active":"2024-09-21T07:52:43.086828+0000","last_peered":"2024-09-21T07:52:43.086828+0000","last_clean":"2024-09-21T07:52:43.086828+0000","last_became_active":"2024-09-21T07:52:41.045666+0000","last_became_peered":"2024-09-21T07:52:41.045666+0000","last_unstale":"2024-09-21T07:52:43.086828+0000","last_undegraded":"2024-09-21T07:52:43.086828+0000","last_fullsized":"2024-09-21T07:52:43.086828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:50.971261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086183+0000","last_change":"2024-09-21T07:52:42.038707+0000","last_active":"2024-09-21T07:52:43.086183+0000","last_peered":"2024-09-21T07:52:43.086183+0000","last_clean":"2024-09-21T07:52:43.086183+0000","last_became_active":"2024-09-21T07:52:42.038186+0000","last_became_peered":"2024-09-21T07:52:42.038186+0000","last_unstale":"2024-09-21T07:52:43.086183+0000","last_undegraded":"2024-09-21T07:52:43.086183+0000","last_fullsized":"2024-09-21T07:52:43.086183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.404809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087287+0000","last_change":"2024-09-21T07:52:41.051206+0000","last_active":"2024-09-21T07:52:43.087287+0000","last_peered":"2024-09-21T07:52:43.087287+0000","last_clean":"2024-09-21T07:52:43.087287+0000","last_became_active":"2024-09-21T07:52:41.050919+0000","last_became_peered":"2024-09-21T07:52:41.050919+0000","last_unstale":"2024-09-21T07:52:43.087287+0000","last_undegraded":"2024-09-21T07:52:43.087287+0000","last_fullsized":"2024-09-21T07:52:43.087287+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:24.653709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043585+0000","last_change":"2024-09-21T07:52:42.049986+0000","last_active":"2024-09-21T07:52:43.043585+0000","last_peered":"2024-09-21T07:52:43.043585+0000","last_clean":"2024-09-21T07:52:43.043585+0000","last_became_active":"2024-09-21T07:52:42.049583+0000","last_became_peered":"2024-09-21T07:52:42.049583+0000","last_unstale":"2024-09-21T07:52:43.043585+0000","last_undegraded":"2024-09-21T07:52:43.043585+0000","last_fullsized":"2024-09-21T07:52:43.043585+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:38.950041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042213+0000","last_change":"2024-09-21T07:52:41.044491+0000","last_active":"2024-09-21T07:52:43.042213+0000","last_peered":"2024-09-21T07:52:43.042213+0000","last_clean":"2024-09-21T07:52:43.042213+0000","last_became_active":"2024-09-21T07:52:41.043914+0000","last_became_peered":"2024-09-21T07:52:41.043914+0000","last_unstale":"2024-09-21T07:52:43.042213+0000","last_undegraded":"2024-09-21T07:52:43.042213+0000","last_fullsized":"2024-09-21T07:52:43.042213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:48.662085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051604+0000","last_change":"2024-09-21T07:52:42.054652+0000","last_active":"2024-09-21T07:52:43.051604+0000","last_peered":"2024-09-21T07:52:43.051604+0000","last_clean":"2024-09-21T07:52:43.051604+0000","last_became_active":"2024-09-21T07:52:42.054424+0000","last_became_peered":"2024-09-21T07:52:42.054424+0000","last_unstale":"2024-09-21T07:52:43.051604+0000","last_undegraded":"2024-09-21T07:52:43.051604+0000","last_fullsized":"2024-09-21T07:52:43.051604+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:59.140737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086468+0000","last_change":"2024-09-21T07:52:41.040325+0000","last_active":"2024-09-21T07:52:43.086468+0000","last_peered":"2024-09-21T07:52:43.086468+0000","last_clean":"2024-09-21T07:52:43.086468+0000","last_became_active":"2024-09-21T07:52:41.039761+0000","last_became_peered":"2024-09-21T07:52:41.039761+0000","last_unstale":"2024-09-21T07:52:43.086468+0000","last_undegraded":"2024-09-21T07:52:43.086468+0000","last_fullsized":"2024-09-21T07:52:43.086468+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:45.027803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086586+0000","last_change":"2024-09-21T07:52:42.055657+0000","last_active":"2024-09-21T07:52:43.086586+0000","last_peered":"2024-09-21T07:52:43.086586+0000","last_clean":"2024-09-21T07:52:43.086586+0000","last_became_active":"2024-09-21T07:52:42.055345+0000","last_became_peered":"2024-09-21T07:52:42.055345+0000","last_unstale":"2024-09-21T07:52:43.086586+0000","last_undegraded":"2024-09-21T07:52:43.086586+0000","last_fullsized":"2024-09-21T07:52:43.086586+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:16.311427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087583+0000","last_change":"2024-09-21T07:52:41.056134+0000","last_active":"2024-09-21T07:52:43.087583+0000","last_peered":"2024-09-21T07:52:43.087583+0000","last_clean":"2024-09-21T07:52:43.087583+0000","last_became_active":"2024-09-21T07:52:41.055763+0000","last_became_peered":"2024-09-21T07:52:41.055763+0000","last_unstale":"2024-09-21T07:52:43.087583+0000","last_undegraded":"2024-09-21T07:52:43.087583+0000","last_fullsized":"2024-09-21T07:52:43.087583+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:16.085262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044786+0000","last_change":"2024-09-21T07:52:42.053432+0000","last_active":"2024-09-21T07:52:43.044786+0000","last_peered":"2024-09-21T07:52:43.044786+0000","last_clean":"2024-09-21T07:52:43.044786+0000","last_became_active":"2024-09-21T07:52:42.053124+0000","last_became_peered":"2024-09-21T07:52:42.053124+0000","last_unstale":"2024-09-21T07:52:43.044786+0000","last_undegraded":"2024-09-21T07:52:43.044786+0000","last_fullsized":"2024-09-21T07:52:43.044786+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.711786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"120'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044504+0000","last_change":"2024-09-21T07:52:41.053277+0000","last_active":"2024-09-21T07:52:43.044504+0000","last_peered":"2024-09-21T07:52:43.044504+0000","last_clean":"2024-09-21T07:52:43.044504+0000","last_became_active":"2024-09-21T07:52:41.052592+0000","last_became_peered":"2024-09-21T07:52:41.052592+0000","last_unstale":"2024-09-21T07:52:43.044504+0000","last_undegraded":"2024-09-21T07:52:43.044504+0000","last_fullsized":"2024-09-21T07:52:43.044504+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:10.759040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042318+0000","last_change":"2024-09-21T07:52:42.043177+0000","last_active":"2024-09-21T07:52:43.042318+0000","last_peered":"2024-09-21T07:52:43.042318+0000","last_clean":"2024-09-21T07:52:43.042318+0000","last_became_active":"2024-09-21T07:52:42.042875+0000","last_became_peered":"2024-09-21T07:52:42.042875+0000","last_unstale":"2024-09-21T07:52:43.042318+0000","last_undegraded":"2024-09-21T07:52:43.042318+0000","last_fullsized":"2024-09-21T07:52:43.042318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:43.428494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087293+0000","last_change":"2024-09-21T07:52:41.053679+0000","last_active":"2024-09-21T07:52:43.087293+0000","last_peered":"2024-09-21T07:52:43.087293+0000","last_clean":"2024-09-21T07:52:43.087293+0000","last_became_active":"2024-09-21T07:52:41.053337+0000","last_became_peered":"2024-09-21T07:52:41.053337+0000","last_unstale":"2024-09-21T07:52:43.087293+0000","last_undegraded":"2024-09-21T07:52:43.087293+0000","last_fullsized":"2024-09-21T07:52:43.087293+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:30.788781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087664+0000","last_change":"2024-09-21T07:52:42.050358+0000","last_active":"2024-09-21T07:52:43.087664+0000","last_peered":"2024-09-21T07:52:43.087664+0000","last_clean":"2024-09-21T07:52:43.087664+0000","last_became_active":"2024-09-21T07:52:42.049678+0000","last_became_peered":"2024-09-21T07:52:42.049678+0000","last_unstale":"2024-09-21T07:52:43.087664+0000","last_undegraded":"2024-09-21T07:52:43.087664+0000","last_fullsized":"2024-09-21T07:52:43.087664+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:47.224863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042515+0000","last_change":"2024-09-21T07:52:41.044918+0000","last_active":"2024-09-21T07:52:43.042515+0000","last_peered":"2024-09-21T07:52:43.042515+0000","last_clean":"2024-09-21T07:52:43.042515+0000","last_became_active":"2024-09-21T07:52:41.044501+0000","last_became_peered":"2024-09-21T07:52:41.044501+0000","last_unstale":"2024-09-21T07:52:43.042515+0000","last_undegraded":"2024-09-21T07:52:43.042515+0000","last_fullsized":"2024-09-21T07:52:43.042515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:38.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050503+0000","last_change":"2024-09-21T07:52:42.051661+0000","last_active":"2024-09-21T07:52:43.050503+0000","last_peered":"2024-09-21T07:52:43.050503+0000","last_clean":"2024-09-21T07:52:43.050503+0000","last_became_active":"2024-09-21T07:52:42.051352+0000","last_became_peered":"2024-09-21T07:52:42.051352+0000","last_unstale":"2024-09-21T07:52:43.050503+0000","last_undegraded":"2024-09-21T07:52:43.050503+0000","last_fullsized":"2024-09-21T07:52:43.050503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:43.904498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043976+0000","last_change":"2024-09-21T07:52:41.043000+0000","last_active":"2024-09-21T07:52:43.043976+0000","last_peered":"2024-09-21T07:52:43.043976+0000","last_clean":"2024-09-21T07:52:43.043976+0000","last_became_active":"2024-09-21T07:52:41.042576+0000","last_became_peered":"2024-09-21T07:52:41.042576+0000","last_unstale":"2024-09-21T07:52:43.043976+0000","last_undegraded":"2024-09-21T07:52:43.043976+0000","last_fullsized":"2024-09-21T07:52:43.043976+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:56.509644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050764+0000","last_change":"2024-09-21T07:52:42.060191+0000","last_active":"2024-09-21T07:52:43.050764+0000","last_peered":"2024-09-21T07:52:43.050764+0000","last_clean":"2024-09-21T07:52:43.050764+0000","last_became_active":"2024-09-21T07:52:42.058950+0000","last_became_peered":"2024-09-21T07:52:42.058950+0000","last_unstale":"2024-09-21T07:52:43.050764+0000","last_undegraded":"2024-09-21T07:52:43.050764+0000","last_fullsized":"2024-09-21T07:52:43.050764+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:38.950041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050691+0000","last_change":"2024-09-21T07:52:41.049196+0000","last_active":"2024-09-21T07:52:43.050691+0000","last_peered":"2024-09-21T07:52:43.050691+0000","last_clean":"2024-09-21T07:52:43.050691+0000","last_became_active":"2024-09-21T07:52:41.048712+0000","last_became_peered":"2024-09-21T07:52:41.048712+0000","last_unstale":"2024-09-21T07:52:43.050691+0000","last_undegraded":"2024-09-21T07:52:43.050691+0000","last_fullsized":"2024-09-21T07:52:43.050691+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:38.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087202+0000","last_change":"2024-09-21T07:52:42.063877+0000","last_active":"2024-09-21T07:52:43.087202+0000","last_peered":"2024-09-21T07:52:43.087202+0000","last_clean":"2024-09-21T07:52:43.087202+0000","last_became_active":"2024-09-21T07:52:42.063560+0000","last_became_peered":"2024-09-21T07:52:42.063560+0000","last_unstale":"2024-09-21T07:52:43.087202+0000","last_undegraded":"2024-09-21T07:52:43.087202+0000","last_fullsized":"2024-09-21T07:52:43.087202+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:02.219486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086700+0000","last_change":"2024-09-21T07:52:41.043466+0000","last_active":"2024-09-21T07:52:43.086700+0000","last_peered":"2024-09-21T07:52:43.086700+0000","last_clean":"2024-09-21T07:52:43.086700+0000","last_became_active":"2024-09-21T07:52:41.042404+0000","last_became_peered":"2024-09-21T07:52:41.042404+0000","last_unstale":"2024-09-21T07:52:43.086700+0000","last_undegraded":"2024-09-21T07:52:43.086700+0000","last_fullsized":"2024-09-21T07:52:43.086700+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:42.896903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042988+0000","last_change":"2024-09-21T07:52:42.049152+0000","last_active":"2024-09-21T07:52:43.042988+0000","last_peered":"2024-09-21T07:52:43.042988+0000","last_clean":"2024-09-21T07:52:43.042988+0000","last_became_active":"2024-09-21T07:52:42.048645+0000","last_became_peered":"2024-09-21T07:52:42.048645+0000","last_unstale":"2024-09-21T07:52:43.042988+0000","last_undegraded":"2024-09-21T07:52:43.042988+0000","last_fullsized":"2024-09-21T07:52:43.042988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.334232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043616+0000","last_change":"2024-09-21T07:52:41.041714+0000","last_active":"2024-09-21T07:52:43.043616+0000","last_peered":"2024-09-21T07:52:43.043616+0000","last_clean":"2024-09-21T07:52:43.043616+0000","last_became_active":"2024-09-21T07:52:41.041407+0000","last_became_peered":"2024-09-21T07:52:41.041407+0000","last_unstale":"2024-09-21T07:52:43.043616+0000","last_undegraded":"2024-09-21T07:52:43.043616+0000","last_fullsized":"2024-09-21T07:52:43.043616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:04.036122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087020+0000","last_change":"2024-09-21T07:52:42.043635+0000","last_active":"2024-09-21T07:52:43.087020+0000","last_peered":"2024-09-21T07:52:43.087020+0000","last_clean":"2024-09-21T07:52:43.087020+0000","last_became_active":"2024-09-21T07:52:42.043303+0000","last_became_peered":"2024-09-21T07:52:42.043303+0000","last_unstale":"2024-09-21T07:52:43.087020+0000","last_undegraded":"2024-09-21T07:52:43.087020+0000","last_fullsized":"2024-09-21T07:52:43.087020+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:33.309037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050868+0000","last_change":"2024-09-21T07:52:41.045768+0000","last_active":"2024-09-21T07:52:43.050868+0000","last_peered":"2024-09-21T07:52:43.050868+0000","last_clean":"2024-09-21T07:52:43.050868+0000","last_became_active":"2024-09-21T07:52:41.045435+0000","last_became_peered":"2024-09-21T07:52:41.045435+0000","last_unstale":"2024-09-21T07:52:43.050868+0000","last_undegraded":"2024-09-21T07:52:43.050868+0000","last_fullsized":"2024-09-21T07:52:43.050868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:04.036122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":324,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086072+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:52:43.086072+0000","last_peered":"2024-09-21T07:52:43.086072+0000","last_clean":"2024-09-21T07:52:43.086072+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:52:43.086072+0000","last_undegraded":"2024-09-21T07:52:43.086072+0000","last_fullsized":"2024-09-21T07:52:43.086072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542230,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1490944,"data_stored":654836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509525,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1486848,"data_stored":654802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509526,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2093056,"data_stored":1245912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509526,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1486848,"data_stored":654802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476824,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1490944,"data_stored":654872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476824,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":655474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476821,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":1246804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476821,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":656470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:52:51.468 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e8c00640 1 -- 172.21.15.88:0/294061081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 msgr2=0x7fc7c8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/294061081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7c8003e70 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fc7d4009b50 tx=0x7fc7c8005830 comp rx=0 tx=0).stop 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e2800640 1 -- 172.21.15.88:0/294061081 reap_dead start 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e8c00640 1 -- 172.21.15.88:0/294061081 shutdown_connections 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e8c00640 1 -- 172.21.15.88:0/294061081 >> 172.21.15.88:0/294061081 conn(0x7fc7e406d730 msgr2=0x7fc7e40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e8c00640 1 -- 172.21.15.88:0/294061081 shutdown_connections 2024-09-21T07:52:51.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.697+0000 7fc7e8c00640 1 -- 172.21.15.88:0/294061081 wait complete. 2024-09-21T07:52:51.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.700+0000 7fc7e8c00640 1 Processor -- start 2024-09-21T07:52:51.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.700+0000 7fc7e8c00640 1 -- start start 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.701+0000 7fc7e8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7e414e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.701+0000 7fc7e8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7e414f700 0x7fc7e4151340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7e415ce20 0x7fc7e4151880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e8c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e40712b0 con 0x7fc7c8003a00 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7e4071130 con 0x7fc7e414f700 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7e4073040 con 0x7fc7e415ce20 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7e414e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7e414e8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57306/0 (socket says 172.21.15.88:57306) 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 -- 172.21.15.88:0/3346214822 learned_addr learned my addr 172.21.15.88:0/3346214822 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1400640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7e414f700 0x7fc7e4151340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e2800640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7e415ce20 0x7fc7e4151880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:51.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 -- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7e415ce20 msgr2=0x7fc7e4151880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7e415ce20 0x7fc7e4151880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 -- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7e414f700 msgr2=0x7fc7e4151340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7e414f700 0x7fc7e4151340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 -- 172.21.15.88:0/3346214822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7e41624b0 con 0x7fc7c8003a00 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.702+0000 7fc7e1e00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7e414e8a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc7d402bfd0 tx=0x7fc7d4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.703+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4038050 con 0x7fc7c8003a00 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.703+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7d4039040 con 0x7fc7c8003a00 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.703+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7d4033c50 con 0x7fc7c8003a00 2024-09-21T07:52:51.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.703+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7e414f350 con 0x7fc7c8003a00 2024-09-21T07:52:51.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.703+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7e415b620 con 0x7fc7c8003a00 2024-09-21T07:52:51.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.704+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7e415b130 con 0x7fc7c8003a00 2024-09-21T07:52:51.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.705+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fc7d4046020 con 0x7fc7c8003a00 2024-09-21T07:52:51.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.706+0000 7fc7da000640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7c4079010 0x7fc7c407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:51.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.706+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fc7d40bc1f0 con 0x7fc7c8003a00 2024-09-21T07:52:51.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.706+0000 7fc7e1400640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7c4079010 0x7fc7c407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:51.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.707+0000 7fc7e1400640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7c4079010 0x7fc7c407b520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7e41534f0 tx=0x7fc7d00067a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:51.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.711+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7d404d020 con 0x7fc7c8003a00 2024-09-21T07:52:51.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.898+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7e4073600 con 0x7fc7c4079010 2024-09-21T07:52:51.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.902+0000 7fc7da000640 1 -- 172.21.15.88:0/3346214822 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252366 (secure 0 0 0) 0x7fc7e4073600 con 0x7fc7c4079010 2024-09-21T07:52:51.907 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:51.908 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:52:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7c4079010 msgr2=0x7fc7c407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7c4079010 0x7fc7c407b520 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc7e41534f0 tx=0x7fc7d00067a0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 msgr2=0x7fc7e414e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7e414e8a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fc7d402bfd0 tx=0x7fc7d4033670 comp rx=0 tx=0).stop 2024-09-21T07:52:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 shutdown_connections 2024-09-21T07:52:51.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7c4079010 0x7fc7c407b520 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7e415ce20 0x7fc7e4151880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7e414f700 0x7fc7e4151340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 --2- 172.21.15.88:0/3346214822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7c8003a00 0x7fc7e414e8a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 >> 172.21.15.88:0/3346214822 conn(0x7fc7e406d730 msgr2=0x7fc7e40b05b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 shutdown_connections 2024-09-21T07:52:51.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:51.906+0000 7fc7e8c00640 1 -- 172.21.15.88:0/3346214822 wait complete. 2024-09-21T07:52:51.935 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":118,"stamp":"2024-09-21T07:52:51.724847+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":262368,"kb_used_data":12840,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754712352,"statfs":{"total":773094113280,"available":772825448448,"internally_reserved":0,"allocated":13148160,"data_stored":6423972,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.700500"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044601+0000","last_change":"2024-09-21T07:52:42.050752+0000","last_active":"2024-09-21T07:52:43.044601+0000","last_peered":"2024-09-21T07:52:43.044601+0000","last_clean":"2024-09-21T07:52:43.044601+0000","last_became_active":"2024-09-21T07:52:42.050179+0000","last_became_peered":"2024-09-21T07:52:42.050179+0000","last_unstale":"2024-09-21T07:52:43.044601+0000","last_undegraded":"2024-09-21T07:52:43.044601+0000","last_fullsized":"2024-09-21T07:52:43.044601+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:45.206923+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044149+0000","last_change":"2024-09-21T07:52:42.053064+0000","last_active":"2024-09-21T07:52:43.044149+0000","last_peered":"2024-09-21T07:52:43.044149+0000","last_clean":"2024-09-21T07:52:43.044149+0000","last_became_active":"2024-09-21T07:52:42.052295+0000","last_became_peered":"2024-09-21T07:52:42.052295+0000","last_unstale":"2024-09-21T07:52:43.044149+0000","last_undegraded":"2024-09-21T07:52:43.044149+0000","last_fullsized":"2024-09-21T07:52:43.044149+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:17.315153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087707+0000","last_change":"2024-09-21T07:52:42.049236+0000","last_active":"2024-09-21T07:52:43.087707+0000","last_peered":"2024-09-21T07:52:43.087707+0000","last_clean":"2024-09-21T07:52:43.087707+0000","last_became_active":"2024-09-21T07:52:42.048928+0000","last_became_peered":"2024-09-21T07:52:42.048928+0000","last_unstale":"2024-09-21T07:52:43.087707+0000","last_undegraded":"2024-09-21T07:52:43.087707+0000","last_fullsized":"2024-09-21T07:52:43.087707+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:43:17.360980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086753+0000","last_change":"2024-09-21T07:52:42.058830+0000","last_active":"2024-09-21T07:52:43.086753+0000","last_peered":"2024-09-21T07:52:43.086753+0000","last_clean":"2024-09-21T07:52:43.086753+0000","last_became_active":"2024-09-21T07:52:42.057357+0000","last_became_peered":"2024-09-21T07:52:42.057357+0000","last_unstale":"2024-09-21T07:52:43.086753+0000","last_undegraded":"2024-09-21T07:52:43.086753+0000","last_fullsized":"2024-09-21T07:52:43.086753+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:31.637465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043780+0000","last_change":"2024-09-21T07:52:42.051708+0000","last_active":"2024-09-21T07:52:43.043780+0000","last_peered":"2024-09-21T07:52:43.043780+0000","last_clean":"2024-09-21T07:52:43.043780+0000","last_became_active":"2024-09-21T07:52:42.051391+0000","last_became_peered":"2024-09-21T07:52:42.051391+0000","last_unstale":"2024-09-21T07:52:43.043780+0000","last_undegraded":"2024-09-21T07:52:43.043780+0000","last_fullsized":"2024-09-21T07:52:43.043780+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:57.517239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050485+0000","last_change":"2024-09-21T07:52:42.050823+0000","last_active":"2024-09-21T07:52:43.050485+0000","last_peered":"2024-09-21T07:52:43.050485+0000","last_clean":"2024-09-21T07:52:43.050485+0000","last_became_active":"2024-09-21T07:52:42.047622+0000","last_became_peered":"2024-09-21T07:52:42.047622+0000","last_unstale":"2024-09-21T07:52:43.050485+0000","last_undegraded":"2024-09-21T07:52:43.050485+0000","last_fullsized":"2024-09-21T07:52:43.050485+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.334232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043566+0000","last_change":"2024-09-21T07:52:42.037357+0000","last_active":"2024-09-21T07:52:43.043566+0000","last_peered":"2024-09-21T07:52:43.043566+0000","last_clean":"2024-09-21T07:52:43.043566+0000","last_became_active":"2024-09-21T07:52:42.037059+0000","last_became_peered":"2024-09-21T07:52:42.037059+0000","last_unstale":"2024-09-21T07:52:43.043566+0000","last_undegraded":"2024-09-21T07:52:43.043566+0000","last_fullsized":"2024-09-21T07:52:43.043566+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:45.562593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043531+0000","last_change":"2024-09-21T07:52:42.042633+0000","last_active":"2024-09-21T07:52:43.043531+0000","last_peered":"2024-09-21T07:52:43.043531+0000","last_clean":"2024-09-21T07:52:43.043531+0000","last_became_active":"2024-09-21T07:52:42.042339+0000","last_became_peered":"2024-09-21T07:52:42.042339+0000","last_unstale":"2024-09-21T07:52:43.043531+0000","last_undegraded":"2024-09-21T07:52:43.043531+0000","last_fullsized":"2024-09-21T07:52:43.043531+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:42:45.815588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087021+0000","last_change":"2024-09-21T07:52:42.049256+0000","last_active":"2024-09-21T07:52:43.087021+0000","last_peered":"2024-09-21T07:52:43.087021+0000","last_clean":"2024-09-21T07:52:43.087021+0000","last_became_active":"2024-09-21T07:52:42.048951+0000","last_became_peered":"2024-09-21T07:52:42.048951+0000","last_unstale":"2024-09-21T07:52:43.087021+0000","last_undegraded":"2024-09-21T07:52:43.087021+0000","last_fullsized":"2024-09-21T07:52:43.087021+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:24.601213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044243+0000","last_change":"2024-09-21T07:52:42.048121+0000","last_active":"2024-09-21T07:52:43.044243+0000","last_peered":"2024-09-21T07:52:43.044243+0000","last_clean":"2024-09-21T07:52:43.044243+0000","last_became_active":"2024-09-21T07:52:42.047802+0000","last_became_peered":"2024-09-21T07:52:42.047802+0000","last_unstale":"2024-09-21T07:52:43.044243+0000","last_undegraded":"2024-09-21T07:52:43.044243+0000","last_fullsized":"2024-09-21T07:52:43.044243+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:41:28.809696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087072+0000","last_change":"2024-09-21T07:52:42.056770+0000","last_active":"2024-09-21T07:52:43.087072+0000","last_peered":"2024-09-21T07:52:43.087072+0000","last_clean":"2024-09-21T07:52:43.087072+0000","last_became_active":"2024-09-21T07:52:42.054745+0000","last_became_peered":"2024-09-21T07:52:42.054745+0000","last_unstale":"2024-09-21T07:52:43.087072+0000","last_undegraded":"2024-09-21T07:52:43.087072+0000","last_fullsized":"2024-09-21T07:52:43.087072+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:25.430970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051086+0000","last_change":"2024-09-21T07:52:42.057065+0000","last_active":"2024-09-21T07:52:43.051086+0000","last_peered":"2024-09-21T07:52:43.051086+0000","last_clean":"2024-09-21T07:52:43.051086+0000","last_became_active":"2024-09-21T07:52:42.055053+0000","last_became_peered":"2024-09-21T07:52:42.055053+0000","last_unstale":"2024-09-21T07:52:43.051086+0000","last_undegraded":"2024-09-21T07:52:43.051086+0000","last_fullsized":"2024-09-21T07:52:43.051086+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:26:52.659011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087092+0000","last_change":"2024-09-21T07:52:42.034410+0000","last_active":"2024-09-21T07:52:43.087092+0000","last_peered":"2024-09-21T07:52:43.087092+0000","last_clean":"2024-09-21T07:52:43.087092+0000","last_became_active":"2024-09-21T07:52:42.034074+0000","last_became_peered":"2024-09-21T07:52:42.034074+0000","last_unstale":"2024-09-21T07:52:43.087092+0000","last_undegraded":"2024-09-21T07:52:43.087092+0000","last_fullsized":"2024-09-21T07:52:43.087092+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:22.274012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042677+0000","last_change":"2024-09-21T07:52:42.052363+0000","last_active":"2024-09-21T07:52:43.042677+0000","last_peered":"2024-09-21T07:52:43.042677+0000","last_clean":"2024-09-21T07:52:43.042677+0000","last_became_active":"2024-09-21T07:52:42.050716+0000","last_became_peered":"2024-09-21T07:52:42.050716+0000","last_unstale":"2024-09-21T07:52:43.042677+0000","last_undegraded":"2024-09-21T07:52:43.042677+0000","last_fullsized":"2024-09-21T07:52:43.042677+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:32.733984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087535+0000","last_change":"2024-09-21T07:52:42.059858+0000","last_active":"2024-09-21T07:52:43.087535+0000","last_peered":"2024-09-21T07:52:43.087535+0000","last_clean":"2024-09-21T07:52:43.087535+0000","last_became_active":"2024-09-21T07:52:42.059552+0000","last_became_peered":"2024-09-21T07:52:42.059552+0000","last_unstale":"2024-09-21T07:52:43.087535+0000","last_undegraded":"2024-09-21T07:52:43.087535+0000","last_fullsized":"2024-09-21T07:52:43.087535+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:15.045213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043303+0000","last_change":"2024-09-21T07:52:42.058578+0000","last_active":"2024-09-21T07:52:43.043303+0000","last_peered":"2024-09-21T07:52:43.043303+0000","last_clean":"2024-09-21T07:52:43.043303+0000","last_became_active":"2024-09-21T07:52:42.053450+0000","last_became_peered":"2024-09-21T07:52:42.053450+0000","last_unstale":"2024-09-21T07:52:43.043303+0000","last_undegraded":"2024-09-21T07:52:43.043303+0000","last_fullsized":"2024-09-21T07:52:43.043303+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:10.585517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087551+0000","last_change":"2024-09-21T07:52:42.041074+0000","last_active":"2024-09-21T07:52:43.087551+0000","last_peered":"2024-09-21T07:52:43.087551+0000","last_clean":"2024-09-21T07:52:43.087551+0000","last_became_active":"2024-09-21T07:52:42.040748+0000","last_became_peered":"2024-09-21T07:52:42.040748+0000","last_unstale":"2024-09-21T07:52:43.087551+0000","last_undegraded":"2024-09-21T07:52:43.087551+0000","last_fullsized":"2024-09-21T07:52:43.087551+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:41:17.071991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087560+0000","last_change":"2024-09-21T07:52:42.058605+0000","last_active":"2024-09-21T07:52:43.087560+0000","last_peered":"2024-09-21T07:52:43.087560+0000","last_clean":"2024-09-21T07:52:43.087560+0000","last_became_active":"2024-09-21T07:52:42.058244+0000","last_became_peered":"2024-09-21T07:52:42.058244+0000","last_unstale":"2024-09-21T07:52:43.087560+0000","last_undegraded":"2024-09-21T07:52:43.087560+0000","last_fullsized":"2024-09-21T07:52:43.087560+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:20.377996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044029+0000","last_change":"2024-09-21T07:52:42.054229+0000","last_active":"2024-09-21T07:52:43.044029+0000","last_peered":"2024-09-21T07:52:43.044029+0000","last_clean":"2024-09-21T07:52:43.044029+0000","last_became_active":"2024-09-21T07:52:42.053413+0000","last_became_peered":"2024-09-21T07:52:42.053413+0000","last_unstale":"2024-09-21T07:52:43.044029+0000","last_undegraded":"2024-09-21T07:52:43.044029+0000","last_fullsized":"2024-09-21T07:52:43.044029+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:55.002454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086789+0000","last_change":"2024-09-21T07:52:42.056523+0000","last_active":"2024-09-21T07:52:43.086789+0000","last_peered":"2024-09-21T07:52:43.086789+0000","last_clean":"2024-09-21T07:52:43.086789+0000","last_became_active":"2024-09-21T07:52:42.055917+0000","last_became_peered":"2024-09-21T07:52:42.055917+0000","last_unstale":"2024-09-21T07:52:43.086789+0000","last_undegraded":"2024-09-21T07:52:43.086789+0000","last_fullsized":"2024-09-21T07:52:43.086789+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:30:07.835000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050482+0000","last_change":"2024-09-21T07:52:42.057926+0000","last_active":"2024-09-21T07:52:43.050482+0000","last_peered":"2024-09-21T07:52:43.050482+0000","last_clean":"2024-09-21T07:52:43.050482+0000","last_became_active":"2024-09-21T07:52:42.057630+0000","last_became_peered":"2024-09-21T07:52:42.057630+0000","last_unstale":"2024-09-21T07:52:43.050482+0000","last_undegraded":"2024-09-21T07:52:43.050482+0000","last_fullsized":"2024-09-21T07:52:43.050482+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:05.673571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042672+0000","last_change":"2024-09-21T07:52:42.039105+0000","last_active":"2024-09-21T07:52:43.042672+0000","last_peered":"2024-09-21T07:52:43.042672+0000","last_clean":"2024-09-21T07:52:43.042672+0000","last_became_active":"2024-09-21T07:52:42.038859+0000","last_became_peered":"2024-09-21T07:52:42.038859+0000","last_unstale":"2024-09-21T07:52:43.042672+0000","last_undegraded":"2024-09-21T07:52:43.042672+0000","last_fullsized":"2024-09-21T07:52:43.042672+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:30:53.772949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086509+0000","last_change":"2024-09-21T07:52:42.039198+0000","last_active":"2024-09-21T07:52:43.086509+0000","last_peered":"2024-09-21T07:52:43.086509+0000","last_clean":"2024-09-21T07:52:43.086509+0000","last_became_active":"2024-09-21T07:52:42.038344+0000","last_became_peered":"2024-09-21T07:52:42.038344+0000","last_unstale":"2024-09-21T07:52:43.086509+0000","last_undegraded":"2024-09-21T07:52:43.086509+0000","last_fullsized":"2024-09-21T07:52:43.086509+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:10:51.708113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086908+0000","last_change":"2024-09-21T07:52:42.062200+0000","last_active":"2024-09-21T07:52:43.086908+0000","last_peered":"2024-09-21T07:52:43.086908+0000","last_clean":"2024-09-21T07:52:43.086908+0000","last_became_active":"2024-09-21T07:52:42.061735+0000","last_became_peered":"2024-09-21T07:52:42.061735+0000","last_unstale":"2024-09-21T07:52:43.086908+0000","last_undegraded":"2024-09-21T07:52:43.086908+0000","last_fullsized":"2024-09-21T07:52:43.086908+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:56:37.086382+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087578+0000","last_change":"2024-09-21T07:52:42.043994+0000","last_active":"2024-09-21T07:52:43.087578+0000","last_peered":"2024-09-21T07:52:43.087578+0000","last_clean":"2024-09-21T07:52:43.087578+0000","last_became_active":"2024-09-21T07:52:42.042860+0000","last_became_peered":"2024-09-21T07:52:42.042860+0000","last_unstale":"2024-09-21T07:52:43.087578+0000","last_undegraded":"2024-09-21T07:52:43.087578+0000","last_fullsized":"2024-09-21T07:52:43.087578+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:15:47.270955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044593+0000","last_change":"2024-09-21T07:52:42.052626+0000","last_active":"2024-09-21T07:52:43.044593+0000","last_peered":"2024-09-21T07:52:43.044593+0000","last_clean":"2024-09-21T07:52:43.044593+0000","last_became_active":"2024-09-21T07:52:42.051816+0000","last_became_peered":"2024-09-21T07:52:42.051816+0000","last_unstale":"2024-09-21T07:52:43.044593+0000","last_undegraded":"2024-09-21T07:52:43.044593+0000","last_fullsized":"2024-09-21T07:52:43.044593+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:37:27.840132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043252+0000","last_change":"2024-09-21T07:52:42.052220+0000","last_active":"2024-09-21T07:52:43.043252+0000","last_peered":"2024-09-21T07:52:43.043252+0000","last_clean":"2024-09-21T07:52:43.043252+0000","last_became_active":"2024-09-21T07:52:42.051926+0000","last_became_peered":"2024-09-21T07:52:42.051926+0000","last_unstale":"2024-09-21T07:52:43.043252+0000","last_undegraded":"2024-09-21T07:52:43.043252+0000","last_fullsized":"2024-09-21T07:52:43.043252+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:00:38.186284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086573+0000","last_change":"2024-09-21T07:52:42.053749+0000","last_active":"2024-09-21T07:52:43.086573+0000","last_peered":"2024-09-21T07:52:43.086573+0000","last_clean":"2024-09-21T07:52:43.086573+0000","last_became_active":"2024-09-21T07:52:42.053428+0000","last_became_peered":"2024-09-21T07:52:42.053428+0000","last_unstale":"2024-09-21T07:52:43.086573+0000","last_undegraded":"2024-09-21T07:52:43.086573+0000","last_fullsized":"2024-09-21T07:52:43.086573+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:01.852292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043717+0000","last_change":"2024-09-21T07:52:42.045589+0000","last_active":"2024-09-21T07:52:43.043717+0000","last_peered":"2024-09-21T07:52:43.043717+0000","last_clean":"2024-09-21T07:52:43.043717+0000","last_became_active":"2024-09-21T07:52:42.045307+0000","last_became_peered":"2024-09-21T07:52:42.045307+0000","last_unstale":"2024-09-21T07:52:43.043717+0000","last_undegraded":"2024-09-21T07:52:43.043717+0000","last_fullsized":"2024-09-21T07:52:43.043717+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:54:34.916044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042937+0000","last_change":"2024-09-21T07:52:42.039461+0000","last_active":"2024-09-21T07:52:43.042937+0000","last_peered":"2024-09-21T07:52:43.042937+0000","last_clean":"2024-09-21T07:52:43.042937+0000","last_became_active":"2024-09-21T07:52:42.036661+0000","last_became_peered":"2024-09-21T07:52:42.036661+0000","last_unstale":"2024-09-21T07:52:43.042937+0000","last_undegraded":"2024-09-21T07:52:43.042937+0000","last_fullsized":"2024-09-21T07:52:43.042937+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:01:05.704681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086268+0000","last_change":"2024-09-21T07:52:42.046781+0000","last_active":"2024-09-21T07:52:43.086268+0000","last_peered":"2024-09-21T07:52:43.086268+0000","last_clean":"2024-09-21T07:52:43.086268+0000","last_became_active":"2024-09-21T07:52:42.046297+0000","last_became_peered":"2024-09-21T07:52:42.046297+0000","last_unstale":"2024-09-21T07:52:43.086268+0000","last_undegraded":"2024-09-21T07:52:43.086268+0000","last_fullsized":"2024-09-21T07:52:43.086268+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:38:58.850800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086475+0000","last_change":"2024-09-21T07:52:42.043362+0000","last_active":"2024-09-21T07:52:43.086475+0000","last_peered":"2024-09-21T07:52:43.086475+0000","last_clean":"2024-09-21T07:52:43.086475+0000","last_became_active":"2024-09-21T07:52:42.042411+0000","last_became_peered":"2024-09-21T07:52:42.042411+0000","last_unstale":"2024-09-21T07:52:43.086475+0000","last_undegraded":"2024-09-21T07:52:43.086475+0000","last_fullsized":"2024-09-21T07:52:43.086475+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:58.791224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042701+0000","last_change":"2024-09-21T07:52:42.048886+0000","last_active":"2024-09-21T07:52:43.042701+0000","last_peered":"2024-09-21T07:52:43.042701+0000","last_clean":"2024-09-21T07:52:43.042701+0000","last_became_active":"2024-09-21T07:52:42.048590+0000","last_became_peered":"2024-09-21T07:52:42.048590+0000","last_unstale":"2024-09-21T07:52:43.042701+0000","last_undegraded":"2024-09-21T07:52:43.042701+0000","last_fullsized":"2024-09-21T07:52:43.042701+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:50:38.936104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1f","version":"119'4","reported_seq":19,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044649+0000","last_change":"2024-09-21T07:52:41.052517+0000","last_active":"2024-09-21T07:52:43.044649+0000","last_peered":"2024-09-21T07:52:43.044649+0000","last_clean":"2024-09-21T07:52:43.044649+0000","last_became_active":"2024-09-21T07:52:41.052152+0000","last_became_peered":"2024-09-21T07:52:41.052152+0000","last_unstale":"2024-09-21T07:52:43.044649+0000","last_undegraded":"2024-09-21T07:52:43.044649+0000","last_fullsized":"2024-09-21T07:52:43.044649+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:54.707465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087245+0000","last_change":"2024-09-21T07:52:42.043656+0000","last_active":"2024-09-21T07:52:43.087245+0000","last_peered":"2024-09-21T07:52:43.087245+0000","last_clean":"2024-09-21T07:52:43.087245+0000","last_became_active":"2024-09-21T07:52:42.043323+0000","last_became_peered":"2024-09-21T07:52:42.043323+0000","last_unstale":"2024-09-21T07:52:43.087245+0000","last_undegraded":"2024-09-21T07:52:43.087245+0000","last_fullsized":"2024-09-21T07:52:43.087245+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:25.758736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1e","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087028+0000","last_change":"2024-09-21T07:52:41.051644+0000","last_active":"2024-09-21T07:52:43.087028+0000","last_peered":"2024-09-21T07:52:43.087028+0000","last_clean":"2024-09-21T07:52:43.087028+0000","last_became_active":"2024-09-21T07:52:41.051270+0000","last_became_peered":"2024-09-21T07:52:41.051270+0000","last_unstale":"2024-09-21T07:52:43.087028+0000","last_undegraded":"2024-09-21T07:52:43.087028+0000","last_fullsized":"2024-09-21T07:52:43.087028+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:41.582792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086798+0000","last_change":"2024-09-21T07:52:42.034262+0000","last_active":"2024-09-21T07:52:43.086798+0000","last_peered":"2024-09-21T07:52:43.086798+0000","last_clean":"2024-09-21T07:52:43.086798+0000","last_became_active":"2024-09-21T07:52:42.033909+0000","last_became_peered":"2024-09-21T07:52:42.033909+0000","last_unstale":"2024-09-21T07:52:43.086798+0000","last_undegraded":"2024-09-21T07:52:43.086798+0000","last_fullsized":"2024-09-21T07:52:43.086798+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:53:31.439346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1d","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087051+0000","last_change":"2024-09-21T07:52:41.040236+0000","last_active":"2024-09-21T07:52:43.087051+0000","last_peered":"2024-09-21T07:52:43.087051+0000","last_clean":"2024-09-21T07:52:43.087051+0000","last_became_active":"2024-09-21T07:52:41.039288+0000","last_became_peered":"2024-09-21T07:52:41.039288+0000","last_unstale":"2024-09-21T07:52:43.087051+0000","last_undegraded":"2024-09-21T07:52:43.087051+0000","last_fullsized":"2024-09-21T07:52:43.087051+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:22.533170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051178+0000","last_change":"2024-09-21T07:52:42.061519+0000","last_active":"2024-09-21T07:52:43.051178+0000","last_peered":"2024-09-21T07:52:43.051178+0000","last_clean":"2024-09-21T07:52:43.051178+0000","last_became_active":"2024-09-21T07:52:42.060171+0000","last_became_peered":"2024-09-21T07:52:42.060171+0000","last_unstale":"2024-09-21T07:52:43.051178+0000","last_undegraded":"2024-09-21T07:52:43.051178+0000","last_fullsized":"2024-09-21T07:52:43.051178+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:46.035398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1c","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044211+0000","last_change":"2024-09-21T07:52:41.049531+0000","last_active":"2024-09-21T07:52:43.044211+0000","last_peered":"2024-09-21T07:52:43.044211+0000","last_clean":"2024-09-21T07:52:43.044211+0000","last_became_active":"2024-09-21T07:52:41.048245+0000","last_became_peered":"2024-09-21T07:52:41.048245+0000","last_unstale":"2024-09-21T07:52:43.044211+0000","last_undegraded":"2024-09-21T07:52:43.044211+0000","last_fullsized":"2024-09-21T07:52:43.044211+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:51.466641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086927+0000","last_change":"2024-09-21T07:52:42.061398+0000","last_active":"2024-09-21T07:52:43.086927+0000","last_peered":"2024-09-21T07:52:43.086927+0000","last_clean":"2024-09-21T07:52:43.086927+0000","last_became_active":"2024-09-21T07:52:42.059409+0000","last_became_peered":"2024-09-21T07:52:42.059409+0000","last_unstale":"2024-09-21T07:52:43.086927+0000","last_undegraded":"2024-09-21T07:52:43.086927+0000","last_fullsized":"2024-09-21T07:52:43.086927+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:32:47.123571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1b","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043855+0000","last_change":"2024-09-21T07:52:41.039897+0000","last_active":"2024-09-21T07:52:43.043855+0000","last_peered":"2024-09-21T07:52:43.043855+0000","last_clean":"2024-09-21T07:52:43.043855+0000","last_became_active":"2024-09-21T07:52:41.039535+0000","last_became_peered":"2024-09-21T07:52:41.039535+0000","last_unstale":"2024-09-21T07:52:43.043855+0000","last_undegraded":"2024-09-21T07:52:43.043855+0000","last_fullsized":"2024-09-21T07:52:43.043855+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:56:44.554998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050598+0000","last_change":"2024-09-21T07:52:42.053025+0000","last_active":"2024-09-21T07:52:43.050598+0000","last_peered":"2024-09-21T07:52:43.050598+0000","last_clean":"2024-09-21T07:52:43.050598+0000","last_became_active":"2024-09-21T07:52:42.051889+0000","last_became_peered":"2024-09-21T07:52:42.051889+0000","last_unstale":"2024-09-21T07:52:43.050598+0000","last_undegraded":"2024-09-21T07:52:43.050598+0000","last_fullsized":"2024-09-21T07:52:43.050598+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:59.475333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.1a","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087132+0000","last_change":"2024-09-21T07:52:41.048831+0000","last_active":"2024-09-21T07:52:43.087132+0000","last_peered":"2024-09-21T07:52:43.087132+0000","last_clean":"2024-09-21T07:52:43.087132+0000","last_became_active":"2024-09-21T07:52:41.048477+0000","last_became_peered":"2024-09-21T07:52:41.048477+0000","last_unstale":"2024-09-21T07:52:43.087132+0000","last_undegraded":"2024-09-21T07:52:43.087132+0000","last_fullsized":"2024-09-21T07:52:43.087132+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:58.164968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050462+0000","last_change":"2024-09-21T07:52:42.046577+0000","last_active":"2024-09-21T07:52:43.050462+0000","last_peered":"2024-09-21T07:52:43.050462+0000","last_clean":"2024-09-21T07:52:43.050462+0000","last_became_active":"2024-09-21T07:52:42.046251+0000","last_became_peered":"2024-09-21T07:52:42.046251+0000","last_unstale":"2024-09-21T07:52:43.050462+0000","last_undegraded":"2024-09-21T07:52:43.050462+0000","last_fullsized":"2024-09-21T07:52:43.050462+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:55:41.643928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086688+0000","last_change":"2024-09-21T07:52:41.046640+0000","last_active":"2024-09-21T07:52:43.086688+0000","last_peered":"2024-09-21T07:52:43.086688+0000","last_clean":"2024-09-21T07:52:43.086688+0000","last_became_active":"2024-09-21T07:52:41.046298+0000","last_became_peered":"2024-09-21T07:52:41.046298+0000","last_unstale":"2024-09-21T07:52:43.086688+0000","last_undegraded":"2024-09-21T07:52:43.086688+0000","last_fullsized":"2024-09-21T07:52:43.086688+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:16:47.200614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042310+0000","last_change":"2024-09-21T07:52:42.035989+0000","last_active":"2024-09-21T07:52:43.042310+0000","last_peered":"2024-09-21T07:52:43.042310+0000","last_clean":"2024-09-21T07:52:43.042310+0000","last_became_active":"2024-09-21T07:52:42.035671+0000","last_became_peered":"2024-09-21T07:52:42.035671+0000","last_unstale":"2024-09-21T07:52:43.042310+0000","last_undegraded":"2024-09-21T07:52:43.042310+0000","last_fullsized":"2024-09-21T07:52:43.042310+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:10:50.444405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042143+0000","last_change":"2024-09-21T07:52:41.043692+0000","last_active":"2024-09-21T07:52:43.042143+0000","last_peered":"2024-09-21T07:52:43.042143+0000","last_clean":"2024-09-21T07:52:43.042143+0000","last_became_active":"2024-09-21T07:52:41.043389+0000","last_became_peered":"2024-09-21T07:52:41.043389+0000","last_unstale":"2024-09-21T07:52:43.042143+0000","last_undegraded":"2024-09-21T07:52:43.042143+0000","last_fullsized":"2024-09-21T07:52:43.042143+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:11.668317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044602+0000","last_change":"2024-09-21T07:52:42.043365+0000","last_active":"2024-09-21T07:52:43.044602+0000","last_peered":"2024-09-21T07:52:43.044602+0000","last_clean":"2024-09-21T07:52:43.044602+0000","last_became_active":"2024-09-21T07:52:42.042454+0000","last_became_peered":"2024-09-21T07:52:42.042454+0000","last_unstale":"2024-09-21T07:52:43.044602+0000","last_undegraded":"2024-09-21T07:52:43.044602+0000","last_fullsized":"2024-09-21T07:52:43.044602+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:23.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051213+0000","last_change":"2024-09-21T07:52:41.049633+0000","last_active":"2024-09-21T07:52:43.051213+0000","last_peered":"2024-09-21T07:52:43.051213+0000","last_clean":"2024-09-21T07:52:43.051213+0000","last_became_active":"2024-09-21T07:52:41.047189+0000","last_became_peered":"2024-09-21T07:52:41.047189+0000","last_unstale":"2024-09-21T07:52:43.051213+0000","last_undegraded":"2024-09-21T07:52:43.051213+0000","last_fullsized":"2024-09-21T07:52:43.051213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.645138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044479+0000","last_change":"2024-09-21T07:52:42.050674+0000","last_active":"2024-09-21T07:52:43.044479+0000","last_peered":"2024-09-21T07:52:43.044479+0000","last_clean":"2024-09-21T07:52:43.044479+0000","last_became_active":"2024-09-21T07:52:42.050307+0000","last_became_peered":"2024-09-21T07:52:42.050307+0000","last_unstale":"2024-09-21T07:52:43.044479+0000","last_undegraded":"2024-09-21T07:52:43.044479+0000","last_fullsized":"2024-09-21T07:52:43.044479+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:38:12.675913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087528+0000","last_change":"2024-09-21T07:52:41.048436+0000","last_active":"2024-09-21T07:52:43.087528+0000","last_peered":"2024-09-21T07:52:43.087528+0000","last_clean":"2024-09-21T07:52:43.087528+0000","last_became_active":"2024-09-21T07:52:41.047717+0000","last_became_peered":"2024-09-21T07:52:41.047717+0000","last_unstale":"2024-09-21T07:52:43.087528+0000","last_undegraded":"2024-09-21T07:52:43.087528+0000","last_fullsized":"2024-09-21T07:52:43.087528+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:13:08.012317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051380+0000","last_change":"2024-09-21T07:52:42.055320+0000","last_active":"2024-09-21T07:52:43.051380+0000","last_peered":"2024-09-21T07:52:43.051380+0000","last_clean":"2024-09-21T07:52:43.051380+0000","last_became_active":"2024-09-21T07:52:42.054564+0000","last_became_peered":"2024-09-21T07:52:42.054564+0000","last_unstale":"2024-09-21T07:52:43.051380+0000","last_undegraded":"2024-09-21T07:52:43.051380+0000","last_fullsized":"2024-09-21T07:52:43.051380+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:23.540765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.15","version":"119'3","reported_seq":18,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044189+0000","last_change":"2024-09-21T07:52:41.051335+0000","last_active":"2024-09-21T07:52:43.044189+0000","last_peered":"2024-09-21T07:52:43.044189+0000","last_clean":"2024-09-21T07:52:43.044189+0000","last_became_active":"2024-09-21T07:52:41.051017+0000","last_became_peered":"2024-09-21T07:52:41.051017+0000","last_unstale":"2024-09-21T07:52:43.044189+0000","last_undegraded":"2024-09-21T07:52:43.044189+0000","last_fullsized":"2024-09-21T07:52:43.044189+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:54.271329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086243+0000","last_change":"2024-09-21T07:52:42.055630+0000","last_active":"2024-09-21T07:52:43.086243+0000","last_peered":"2024-09-21T07:52:43.086243+0000","last_clean":"2024-09-21T07:52:43.086243+0000","last_became_active":"2024-09-21T07:52:42.055283+0000","last_became_peered":"2024-09-21T07:52:42.055283+0000","last_unstale":"2024-09-21T07:52:43.086243+0000","last_undegraded":"2024-09-21T07:52:43.086243+0000","last_fullsized":"2024-09-21T07:52:43.086243+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:25.112436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.14","version":"119'6","reported_seq":21,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086746+0000","last_change":"2024-09-21T07:52:41.042291+0000","last_active":"2024-09-21T07:52:43.086746+0000","last_peered":"2024-09-21T07:52:43.086746+0000","last_clean":"2024-09-21T07:52:43.086746+0000","last_became_active":"2024-09-21T07:52:41.041543+0000","last_became_peered":"2024-09-21T07:52:41.041543+0000","last_unstale":"2024-09-21T07:52:43.086746+0000","last_undegraded":"2024-09-21T07:52:43.086746+0000","last_fullsized":"2024-09-21T07:52:43.086746+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:33:40.701818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042711+0000","last_change":"2024-09-21T07:52:42.047972+0000","last_active":"2024-09-21T07:52:43.042711+0000","last_peered":"2024-09-21T07:52:43.042711+0000","last_clean":"2024-09-21T07:52:43.042711+0000","last_became_active":"2024-09-21T07:52:42.045974+0000","last_became_peered":"2024-09-21T07:52:42.045974+0000","last_unstale":"2024-09-21T07:52:43.042711+0000","last_undegraded":"2024-09-21T07:52:43.042711+0000","last_fullsized":"2024-09-21T07:52:43.042711+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:27:41.389611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042871+0000","last_change":"2024-09-21T07:52:41.043893+0000","last_active":"2024-09-21T07:52:43.042871+0000","last_peered":"2024-09-21T07:52:43.042871+0000","last_clean":"2024-09-21T07:52:43.042871+0000","last_became_active":"2024-09-21T07:52:41.043513+0000","last_became_peered":"2024-09-21T07:52:41.043513+0000","last_unstale":"2024-09-21T07:52:43.042871+0000","last_undegraded":"2024-09-21T07:52:43.042871+0000","last_fullsized":"2024-09-21T07:52:43.042871+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:47.645138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042296+0000","last_change":"2024-09-21T07:52:42.058352+0000","last_active":"2024-09-21T07:52:43.042296+0000","last_peered":"2024-09-21T07:52:43.042296+0000","last_clean":"2024-09-21T07:52:43.042296+0000","last_became_active":"2024-09-21T07:52:42.054077+0000","last_became_peered":"2024-09-21T07:52:42.054077+0000","last_unstale":"2024-09-21T07:52:43.042296+0000","last_undegraded":"2024-09-21T07:52:43.042296+0000","last_fullsized":"2024-09-21T07:52:43.042296+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:24:28.908966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086586+0000","last_change":"2024-09-21T07:52:41.043649+0000","last_active":"2024-09-21T07:52:43.086586+0000","last_peered":"2024-09-21T07:52:43.086586+0000","last_clean":"2024-09-21T07:52:43.086586+0000","last_became_active":"2024-09-21T07:52:41.043360+0000","last_became_peered":"2024-09-21T07:52:41.043360+0000","last_unstale":"2024-09-21T07:52:43.086586+0000","last_undegraded":"2024-09-21T07:52:43.086586+0000","last_fullsized":"2024-09-21T07:52:43.086586+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:58.133141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050600+0000","last_change":"2024-09-21T07:52:42.045364+0000","last_active":"2024-09-21T07:52:43.050600+0000","last_peered":"2024-09-21T07:52:43.050600+0000","last_clean":"2024-09-21T07:52:43.050600+0000","last_became_active":"2024-09-21T07:52:42.045066+0000","last_became_peered":"2024-09-21T07:52:42.045066+0000","last_unstale":"2024-09-21T07:52:43.050600+0000","last_undegraded":"2024-09-21T07:52:43.050600+0000","last_fullsized":"2024-09-21T07:52:43.050600+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:10.585517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.10","version":"119'8","reported_seq":23,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050320+0000","last_change":"2024-09-21T07:52:41.049536+0000","last_active":"2024-09-21T07:52:43.050320+0000","last_peered":"2024-09-21T07:52:43.050320+0000","last_clean":"2024-09-21T07:52:43.050320+0000","last_became_active":"2024-09-21T07:52:41.045025+0000","last_became_peered":"2024-09-21T07:52:41.045025+0000","last_unstale":"2024-09-21T07:52:43.050320+0000","last_undegraded":"2024-09-21T07:52:43.050320+0000","last_fullsized":"2024-09-21T07:52:43.050320+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:48.662085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044435+0000","last_change":"2024-09-21T07:52:42.054840+0000","last_active":"2024-09-21T07:52:43.044435+0000","last_peered":"2024-09-21T07:52:43.044435+0000","last_clean":"2024-09-21T07:52:43.044435+0000","last_became_active":"2024-09-21T07:52:42.053231+0000","last_became_peered":"2024-09-21T07:52:42.053231+0000","last_unstale":"2024-09-21T07:52:43.044435+0000","last_undegraded":"2024-09-21T07:52:43.044435+0000","last_fullsized":"2024-09-21T07:52:43.044435+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:59:52.474236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.f","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051283+0000","last_change":"2024-09-21T07:52:41.050888+0000","last_active":"2024-09-21T07:52:43.051283+0000","last_peered":"2024-09-21T07:52:43.051283+0000","last_clean":"2024-09-21T07:52:43.051283+0000","last_became_active":"2024-09-21T07:52:41.050131+0000","last_became_peered":"2024-09-21T07:52:41.050131+0000","last_unstale":"2024-09-21T07:52:43.051283+0000","last_undegraded":"2024-09-21T07:52:43.051283+0000","last_fullsized":"2024-09-21T07:52:43.051283+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:01.295506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044625+0000","last_change":"2024-09-21T07:52:42.047061+0000","last_active":"2024-09-21T07:52:43.044625+0000","last_peered":"2024-09-21T07:52:43.044625+0000","last_clean":"2024-09-21T07:52:43.044625+0000","last_became_active":"2024-09-21T07:52:42.045929+0000","last_became_peered":"2024-09-21T07:52:42.045929+0000","last_unstale":"2024-09-21T07:52:43.044625+0000","last_undegraded":"2024-09-21T07:52:43.044625+0000","last_fullsized":"2024-09-21T07:52:43.044625+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:38:18.469134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.e","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086955+0000","last_change":"2024-09-21T07:52:41.036154+0000","last_active":"2024-09-21T07:52:43.086955+0000","last_peered":"2024-09-21T07:52:43.086955+0000","last_clean":"2024-09-21T07:52:43.086955+0000","last_became_active":"2024-09-21T07:52:41.035812+0000","last_became_peered":"2024-09-21T07:52:41.035812+0000","last_unstale":"2024-09-21T07:52:43.086955+0000","last_undegraded":"2024-09-21T07:52:43.086955+0000","last_fullsized":"2024-09-21T07:52:43.086955+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:10:02.282645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044350+0000","last_change":"2024-09-21T07:52:42.047448+0000","last_active":"2024-09-21T07:52:43.044350+0000","last_peered":"2024-09-21T07:52:43.044350+0000","last_clean":"2024-09-21T07:52:43.044350+0000","last_became_active":"2024-09-21T07:52:42.046508+0000","last_became_peered":"2024-09-21T07:52:42.046508+0000","last_unstale":"2024-09-21T07:52:43.044350+0000","last_undegraded":"2024-09-21T07:52:43.044350+0000","last_fullsized":"2024-09-21T07:52:43.044350+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:10.168925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.d","version":"119'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087817+0000","last_change":"2024-09-21T07:52:41.046956+0000","last_active":"2024-09-21T07:52:43.087817+0000","last_peered":"2024-09-21T07:52:43.087817+0000","last_clean":"2024-09-21T07:52:43.087817+0000","last_became_active":"2024-09-21T07:52:41.046644+0000","last_became_peered":"2024-09-21T07:52:41.046644+0000","last_unstale":"2024-09-21T07:52:43.087817+0000","last_undegraded":"2024-09-21T07:52:43.087817+0000","last_fullsized":"2024-09-21T07:52:43.087817+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:00.232783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044134+0000","last_change":"2024-09-21T07:52:42.044820+0000","last_active":"2024-09-21T07:52:43.044134+0000","last_peered":"2024-09-21T07:52:43.044134+0000","last_clean":"2024-09-21T07:52:43.044134+0000","last_became_active":"2024-09-21T07:52:42.042422+0000","last_became_peered":"2024-09-21T07:52:42.042422+0000","last_unstale":"2024-09-21T07:52:43.044134+0000","last_undegraded":"2024-09-21T07:52:43.044134+0000","last_fullsized":"2024-09-21T07:52:43.044134+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:00.771472+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044116+0000","last_change":"2024-09-21T07:52:41.047727+0000","last_active":"2024-09-21T07:52:43.044116+0000","last_peered":"2024-09-21T07:52:43.044116+0000","last_clean":"2024-09-21T07:52:43.044116+0000","last_became_active":"2024-09-21T07:52:41.047444+0000","last_became_peered":"2024-09-21T07:52:41.047444+0000","last_unstale":"2024-09-21T07:52:43.044116+0000","last_undegraded":"2024-09-21T07:52:43.044116+0000","last_fullsized":"2024-09-21T07:52:43.044116+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:06:36.951099+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043195+0000","last_change":"2024-09-21T07:52:42.048923+0000","last_active":"2024-09-21T07:52:43.043195+0000","last_peered":"2024-09-21T07:52:43.043195+0000","last_clean":"2024-09-21T07:52:43.043195+0000","last_became_active":"2024-09-21T07:52:42.046724+0000","last_became_peered":"2024-09-21T07:52:42.046724+0000","last_unstale":"2024-09-21T07:52:43.043195+0000","last_undegraded":"2024-09-21T07:52:43.043195+0000","last_fullsized":"2024-09-21T07:52:43.043195+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:19.619396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.b","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086828+0000","last_change":"2024-09-21T07:52:41.045986+0000","last_active":"2024-09-21T07:52:43.086828+0000","last_peered":"2024-09-21T07:52:43.086828+0000","last_clean":"2024-09-21T07:52:43.086828+0000","last_became_active":"2024-09-21T07:52:41.045666+0000","last_became_peered":"2024-09-21T07:52:41.045666+0000","last_unstale":"2024-09-21T07:52:43.086828+0000","last_undegraded":"2024-09-21T07:52:43.086828+0000","last_fullsized":"2024-09-21T07:52:43.086828+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:44:50.971261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086183+0000","last_change":"2024-09-21T07:52:42.038707+0000","last_active":"2024-09-21T07:52:43.086183+0000","last_peered":"2024-09-21T07:52:43.086183+0000","last_clean":"2024-09-21T07:52:43.086183+0000","last_became_active":"2024-09-21T07:52:42.038186+0000","last_became_peered":"2024-09-21T07:52:42.038186+0000","last_unstale":"2024-09-21T07:52:43.086183+0000","last_undegraded":"2024-09-21T07:52:43.086183+0000","last_fullsized":"2024-09-21T07:52:43.086183+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:31:35.404809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087287+0000","last_change":"2024-09-21T07:52:41.051206+0000","last_active":"2024-09-21T07:52:43.087287+0000","last_peered":"2024-09-21T07:52:43.087287+0000","last_clean":"2024-09-21T07:52:43.087287+0000","last_became_active":"2024-09-21T07:52:41.050919+0000","last_became_peered":"2024-09-21T07:52:41.050919+0000","last_unstale":"2024-09-21T07:52:43.087287+0000","last_undegraded":"2024-09-21T07:52:43.087287+0000","last_fullsized":"2024-09-21T07:52:43.087287+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:39:24.653709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043585+0000","last_change":"2024-09-21T07:52:42.049986+0000","last_active":"2024-09-21T07:52:43.043585+0000","last_peered":"2024-09-21T07:52:43.043585+0000","last_clean":"2024-09-21T07:52:43.043585+0000","last_became_active":"2024-09-21T07:52:42.049583+0000","last_became_peered":"2024-09-21T07:52:42.049583+0000","last_unstale":"2024-09-21T07:52:43.043585+0000","last_undegraded":"2024-09-21T07:52:43.043585+0000","last_fullsized":"2024-09-21T07:52:43.043585+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:38.950041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042213+0000","last_change":"2024-09-21T07:52:41.044491+0000","last_active":"2024-09-21T07:52:43.042213+0000","last_peered":"2024-09-21T07:52:43.042213+0000","last_clean":"2024-09-21T07:52:43.042213+0000","last_became_active":"2024-09-21T07:52:41.043914+0000","last_became_peered":"2024-09-21T07:52:41.043914+0000","last_unstale":"2024-09-21T07:52:43.042213+0000","last_undegraded":"2024-09-21T07:52:43.042213+0000","last_fullsized":"2024-09-21T07:52:43.042213+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:48.662085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.051604+0000","last_change":"2024-09-21T07:52:42.054652+0000","last_active":"2024-09-21T07:52:43.051604+0000","last_peered":"2024-09-21T07:52:43.051604+0000","last_clean":"2024-09-21T07:52:43.051604+0000","last_became_active":"2024-09-21T07:52:42.054424+0000","last_became_peered":"2024-09-21T07:52:42.054424+0000","last_unstale":"2024-09-21T07:52:43.051604+0000","last_undegraded":"2024-09-21T07:52:43.051604+0000","last_fullsized":"2024-09-21T07:52:43.051604+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:20:59.140737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086468+0000","last_change":"2024-09-21T07:52:41.040325+0000","last_active":"2024-09-21T07:52:43.086468+0000","last_peered":"2024-09-21T07:52:43.086468+0000","last_clean":"2024-09-21T07:52:43.086468+0000","last_became_active":"2024-09-21T07:52:41.039761+0000","last_became_peered":"2024-09-21T07:52:41.039761+0000","last_unstale":"2024-09-21T07:52:43.086468+0000","last_undegraded":"2024-09-21T07:52:43.086468+0000","last_fullsized":"2024-09-21T07:52:43.086468+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:45.027803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086586+0000","last_change":"2024-09-21T07:52:42.055657+0000","last_active":"2024-09-21T07:52:43.086586+0000","last_peered":"2024-09-21T07:52:43.086586+0000","last_clean":"2024-09-21T07:52:43.086586+0000","last_became_active":"2024-09-21T07:52:42.055345+0000","last_became_peered":"2024-09-21T07:52:42.055345+0000","last_unstale":"2024-09-21T07:52:43.086586+0000","last_undegraded":"2024-09-21T07:52:43.086586+0000","last_fullsized":"2024-09-21T07:52:43.086586+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:31:16.311427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.7","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087583+0000","last_change":"2024-09-21T07:52:41.056134+0000","last_active":"2024-09-21T07:52:43.087583+0000","last_peered":"2024-09-21T07:52:43.087583+0000","last_clean":"2024-09-21T07:52:43.087583+0000","last_became_active":"2024-09-21T07:52:41.055763+0000","last_became_peered":"2024-09-21T07:52:41.055763+0000","last_unstale":"2024-09-21T07:52:43.087583+0000","last_undegraded":"2024-09-21T07:52:43.087583+0000","last_fullsized":"2024-09-21T07:52:43.087583+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:16.085262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044786+0000","last_change":"2024-09-21T07:52:42.053432+0000","last_active":"2024-09-21T07:52:43.044786+0000","last_peered":"2024-09-21T07:52:43.044786+0000","last_clean":"2024-09-21T07:52:43.044786+0000","last_became_active":"2024-09-21T07:52:42.053124+0000","last_became_peered":"2024-09-21T07:52:42.053124+0000","last_unstale":"2024-09-21T07:52:43.044786+0000","last_undegraded":"2024-09-21T07:52:43.044786+0000","last_fullsized":"2024-09-21T07:52:43.044786+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:23:05.711786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.6","version":"120'1","reported_seq":16,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.044504+0000","last_change":"2024-09-21T07:52:41.053277+0000","last_active":"2024-09-21T07:52:43.044504+0000","last_peered":"2024-09-21T07:52:43.044504+0000","last_clean":"2024-09-21T07:52:43.044504+0000","last_became_active":"2024-09-21T07:52:41.052592+0000","last_became_peered":"2024-09-21T07:52:41.052592+0000","last_unstale":"2024-09-21T07:52:43.044504+0000","last_undegraded":"2024-09-21T07:52:43.044504+0000","last_fullsized":"2024-09-21T07:52:43.044504+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:58:10.759040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042318+0000","last_change":"2024-09-21T07:52:42.043177+0000","last_active":"2024-09-21T07:52:43.042318+0000","last_peered":"2024-09-21T07:52:43.042318+0000","last_clean":"2024-09-21T07:52:43.042318+0000","last_became_active":"2024-09-21T07:52:42.042875+0000","last_became_peered":"2024-09-21T07:52:42.042875+0000","last_unstale":"2024-09-21T07:52:43.042318+0000","last_undegraded":"2024-09-21T07:52:43.042318+0000","last_fullsized":"2024-09-21T07:52:43.042318+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:43.428494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087293+0000","last_change":"2024-09-21T07:52:41.053679+0000","last_active":"2024-09-21T07:52:43.087293+0000","last_peered":"2024-09-21T07:52:43.087293+0000","last_clean":"2024-09-21T07:52:43.087293+0000","last_became_active":"2024-09-21T07:52:41.053337+0000","last_became_peered":"2024-09-21T07:52:41.053337+0000","last_unstale":"2024-09-21T07:52:43.087293+0000","last_undegraded":"2024-09-21T07:52:43.087293+0000","last_fullsized":"2024-09-21T07:52:43.087293+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:12:30.788781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087664+0000","last_change":"2024-09-21T07:52:42.050358+0000","last_active":"2024-09-21T07:52:43.087664+0000","last_peered":"2024-09-21T07:52:43.087664+0000","last_clean":"2024-09-21T07:52:43.087664+0000","last_became_active":"2024-09-21T07:52:42.049678+0000","last_became_peered":"2024-09-21T07:52:42.049678+0000","last_unstale":"2024-09-21T07:52:43.087664+0000","last_undegraded":"2024-09-21T07:52:43.087664+0000","last_fullsized":"2024-09-21T07:52:43.087664+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:30:47.224863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042515+0000","last_change":"2024-09-21T07:52:41.044918+0000","last_active":"2024-09-21T07:52:43.042515+0000","last_peered":"2024-09-21T07:52:43.042515+0000","last_clean":"2024-09-21T07:52:43.042515+0000","last_became_active":"2024-09-21T07:52:41.044501+0000","last_became_peered":"2024-09-21T07:52:41.044501+0000","last_unstale":"2024-09-21T07:52:43.042515+0000","last_undegraded":"2024-09-21T07:52:43.042515+0000","last_fullsized":"2024-09-21T07:52:43.042515+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:38.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050503+0000","last_change":"2024-09-21T07:52:42.051661+0000","last_active":"2024-09-21T07:52:43.050503+0000","last_peered":"2024-09-21T07:52:43.050503+0000","last_clean":"2024-09-21T07:52:43.050503+0000","last_became_active":"2024-09-21T07:52:42.051352+0000","last_became_peered":"2024-09-21T07:52:42.051352+0000","last_unstale":"2024-09-21T07:52:43.050503+0000","last_undegraded":"2024-09-21T07:52:43.050503+0000","last_fullsized":"2024-09-21T07:52:43.050503+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:43.904498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043976+0000","last_change":"2024-09-21T07:52:41.043000+0000","last_active":"2024-09-21T07:52:43.043976+0000","last_peered":"2024-09-21T07:52:43.043976+0000","last_clean":"2024-09-21T07:52:43.043976+0000","last_became_active":"2024-09-21T07:52:41.042576+0000","last_became_peered":"2024-09-21T07:52:41.042576+0000","last_unstale":"2024-09-21T07:52:43.043976+0000","last_undegraded":"2024-09-21T07:52:43.043976+0000","last_fullsized":"2024-09-21T07:52:43.043976+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:56.509644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050764+0000","last_change":"2024-09-21T07:52:42.060191+0000","last_active":"2024-09-21T07:52:43.050764+0000","last_peered":"2024-09-21T07:52:43.050764+0000","last_clean":"2024-09-21T07:52:43.050764+0000","last_became_active":"2024-09-21T07:52:42.058950+0000","last_became_peered":"2024-09-21T07:52:42.058950+0000","last_unstale":"2024-09-21T07:52:43.050764+0000","last_undegraded":"2024-09-21T07:52:43.050764+0000","last_fullsized":"2024-09-21T07:52:43.050764+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:02:38.950041+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050691+0000","last_change":"2024-09-21T07:52:41.049196+0000","last_active":"2024-09-21T07:52:43.050691+0000","last_peered":"2024-09-21T07:52:43.050691+0000","last_clean":"2024-09-21T07:52:43.050691+0000","last_became_active":"2024-09-21T07:52:41.048712+0000","last_became_peered":"2024-09-21T07:52:41.048712+0000","last_unstale":"2024-09-21T07:52:43.050691+0000","last_undegraded":"2024-09-21T07:52:43.050691+0000","last_fullsized":"2024-09-21T07:52:43.050691+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:38.805848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087202+0000","last_change":"2024-09-21T07:52:42.063877+0000","last_active":"2024-09-21T07:52:43.087202+0000","last_peered":"2024-09-21T07:52:43.087202+0000","last_clean":"2024-09-21T07:52:43.087202+0000","last_became_active":"2024-09-21T07:52:42.063560+0000","last_became_peered":"2024-09-21T07:52:42.063560+0000","last_unstale":"2024-09-21T07:52:43.087202+0000","last_undegraded":"2024-09-21T07:52:43.087202+0000","last_fullsized":"2024-09-21T07:52:43.087202+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:02.219486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086700+0000","last_change":"2024-09-21T07:52:41.043466+0000","last_active":"2024-09-21T07:52:43.086700+0000","last_peered":"2024-09-21T07:52:43.086700+0000","last_clean":"2024-09-21T07:52:43.086700+0000","last_became_active":"2024-09-21T07:52:41.042404+0000","last_became_peered":"2024-09-21T07:52:41.042404+0000","last_unstale":"2024-09-21T07:52:43.086700+0000","last_undegraded":"2024-09-21T07:52:43.086700+0000","last_fullsized":"2024-09-21T07:52:43.086700+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:30:42.896903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.042988+0000","last_change":"2024-09-21T07:52:42.049152+0000","last_active":"2024-09-21T07:52:43.042988+0000","last_peered":"2024-09-21T07:52:43.042988+0000","last_clean":"2024-09-21T07:52:43.042988+0000","last_became_active":"2024-09-21T07:52:42.048645+0000","last_became_peered":"2024-09-21T07:52:42.048645+0000","last_unstale":"2024-09-21T07:52:43.042988+0000","last_undegraded":"2024-09-21T07:52:43.042988+0000","last_fullsized":"2024-09-21T07:52:43.042988+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:41:38.334232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":15,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.043616+0000","last_change":"2024-09-21T07:52:41.041714+0000","last_active":"2024-09-21T07:52:43.043616+0000","last_peered":"2024-09-21T07:52:43.043616+0000","last_clean":"2024-09-21T07:52:43.043616+0000","last_became_active":"2024-09-21T07:52:41.041407+0000","last_became_peered":"2024-09-21T07:52:41.041407+0000","last_unstale":"2024-09-21T07:52:43.043616+0000","last_undegraded":"2024-09-21T07:52:43.043616+0000","last_fullsized":"2024-09-21T07:52:43.043616+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:04.036122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":13,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.087020+0000","last_change":"2024-09-21T07:52:42.043635+0000","last_active":"2024-09-21T07:52:43.087020+0000","last_peered":"2024-09-21T07:52:43.087020+0000","last_clean":"2024-09-21T07:52:43.087020+0000","last_became_active":"2024-09-21T07:52:42.043303+0000","last_became_peered":"2024-09-21T07:52:42.043303+0000","last_unstale":"2024-09-21T07:52:43.087020+0000","last_undegraded":"2024-09-21T07:52:43.087020+0000","last_fullsized":"2024-09-21T07:52:43.087020+0000","mapping_epoch":118,"log_start":"0'0","ondisk_log_start":"0'0","created":118,"last_epoch_clean":119,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:41.014053+0000","last_clean_scrub_stamp":"2024-09-21T07:52:41.014053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:26:33.309037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.13","version":"119'2","reported_seq":17,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.050868+0000","last_change":"2024-09-21T07:52:41.045768+0000","last_active":"2024-09-21T07:52:43.050868+0000","last_peered":"2024-09-21T07:52:43.050868+0000","last_clean":"2024-09-21T07:52:43.050868+0000","last_became_active":"2024-09-21T07:52:41.045435+0000","last_became_peered":"2024-09-21T07:52:41.045435+0000","last_unstale":"2024-09-21T07:52:43.050868+0000","last_undegraded":"2024-09-21T07:52:43.050868+0000","last_fullsized":"2024-09-21T07:52:43.050868+0000","mapping_epoch":117,"log_start":"0'0","ondisk_log_start":"0'0","created":117,"last_epoch_clean":118,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:52:40.006458+0000","last_clean_scrub_stamp":"2024-09-21T07:52:40.006458+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:04.036122+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":324,"reported_epoch":120,"state":"active+clean","last_fresh":"2024-09-21T07:52:43.086072+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:52:43.086072+0000","last_peered":"2024-09-21T07:52:43.086072+0000","last_clean":"2024-09-21T07:52:43.086072+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:52:43.086072+0000","last_undegraded":"2024-09-21T07:52:43.086072+0000","last_fullsized":"2024-09-21T07:52:43.086072+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542231,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1490944,"data_stored":654836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509526,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1486848,"data_stored":654802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509527,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2044,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2093056,"data_stored":1245912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509527,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32644,"kb_used_data":1452,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339196,"statfs":{"total":96636764160,"available":96603336704,"internally_reserved":0,"allocated":1486848,"data_stored":654802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476825,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1490944,"data_stored":654872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476825,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32648,"kb_used_data":1460,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339192,"statfs":{"total":96636764160,"available":96603332608,"internally_reserved":0,"allocated":1495040,"data_stored":655474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476822,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33236,"kb_used_data":2048,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338604,"statfs":{"total":96636764160,"available":96602730496,"internally_reserved":0,"allocated":2097152,"data_stored":1246804,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476822,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32664,"kb_used_data":1472,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339176,"statfs":{"total":96636764160,"available":96603316224,"internally_reserved":0,"allocated":1507328,"data_stored":656470,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":742,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:52:51.939 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:52:51.940 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:52:52.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 -- 172.21.15.88:0/226860670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145550 msgr2=0x7fe19406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 --2- 172.21.15.88:0/226860670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145550 0x7fe19406fb40 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fe184009830 tx=0x7fe18402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 -- 172.21.15.88:0/226860670 shutdown_connections 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 --2- 172.21.15.88:0/226860670 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe194145a40 0x7fe194073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 --2- 172.21.15.88:0/226860670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145550 0x7fe19406fb40 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 -- 172.21.15.88:0/226860670 >> 172.21.15.88:0/226860670 conn(0x7fe19406d1c0 msgr2=0x7fe19406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 -- 172.21.15.88:0/226860670 shutdown_connections 2024-09-21T07:52:52.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.201+0000 7fe19b400640 1 -- 172.21.15.88:0/226860670 wait complete. 2024-09-21T07:52:52.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.203+0000 7fe19b400640 1 Processor -- start 2024-09-21T07:52:52.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.203+0000 7fe19b400640 1 -- start start 2024-09-21T07:52:52.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe19b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe194073a80 0x7fe19415fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe19b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe194145550 0x7fe1941f9610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe19b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 0x7fe19415e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe19b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe194160a30 con 0x7fe194145a40 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe19b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1940ce940 con 0x7fe194145550 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe19b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe194160bb0 con 0x7fe194073a80 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe199400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 0x7fe19415e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe199400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 0x7fe19415e8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57318/0 (socket says 172.21.15.88:57318) 2024-09-21T07:52:52.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe199400640 1 -- 172.21.15.88:0/3921008617 learned_addr learned my addr 172.21.15.88:0/3921008617 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe198a00640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe194073a80 0x7fe19415fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.205+0000 7fe18be00640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe194145550 0x7fe1941f9610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe199400640 1 -- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe194073a80 msgr2=0x7fe19415fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe199400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe194073a80 0x7fe19415fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe199400640 1 -- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe194145550 msgr2=0x7fe1941f9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe199400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe194145550 0x7fe1941f9610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe199400640 1 -- 172.21.15.88:0/3921008617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1941fa340 con 0x7fe194145a40 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe199400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 0x7fe19415e8c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c760 tx=0x7fe18c00cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c003da0 con 0x7fe194145a40 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe18c00ce10 con 0x7fe194145a40 2024-09-21T07:52:52.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe194151f20 con 0x7fe194145a40 2024-09-21T07:52:52.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe18c0052d0 con 0x7fe194145a40 2024-09-21T07:52:52.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.206+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1941524e0 con 0x7fe194145a40 2024-09-21T07:52:52.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.207+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe194151d10 con 0x7fe194145a40 2024-09-21T07:52:52.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.208+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fe18c005470 con 0x7fe194145a40 2024-09-21T07:52:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.209+0000 7fe191600640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe170079040 0x7fe17007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.209+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fe18c09b2f0 con 0x7fe194145a40 2024-09-21T07:52:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.210+0000 7fe198a00640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe170079040 0x7fe17007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.210+0000 7fe198a00640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe170079040 0x7fe17007b550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe184034000 tx=0x7fe184004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:52.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.214+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe194073e90 con 0x7fe194145a40 2024-09-21T07:52:52.468 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.466+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fe19406fb40 con 0x7fe194145a40 2024-09-21T07:52:52.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.470+0000 7fe191600640 1 -- 172.21.15.88:0/3921008617 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) ==== 190+0+147 (secure 0 0 0) 0x7fe18c063050 con 0x7fe194145a40 2024-09-21T07:52:52.472 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:52:52.472 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:52:52.472 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:52:52.472 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:52:52.472 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:52:52.473 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe170079040 msgr2=0x7fe17007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe170079040 0x7fe17007b550 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fe184034000 tx=0x7fe184004730 comp rx=0 tx=0).stop 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 msgr2=0x7fe19415e8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 0x7fe19415e8c0 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fe18c00c760 tx=0x7fe18c00cc20 comp rx=0 tx=0).stop 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 shutdown_connections 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe170079040 0x7fe17007b550 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe194145a40 0x7fe19415e8c0 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe194145550 0x7fe1941f9610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 --2- 172.21.15.88:0/3921008617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe194073a80 0x7fe19415fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 >> 172.21.15.88:0/3921008617 conn(0x7fe19406d1c0 msgr2=0x7fe1941442b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 shutdown_connections 2024-09-21T07:52:52.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.473+0000 7fe19b400640 1 -- 172.21.15.88:0/3921008617 wait complete. 2024-09-21T07:52:52.497 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f424a400640 1 -- 172.21.15.88:0/2174098438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 msgr2=0x7f422800b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f424a400640 1 --2- 172.21.15.88:0/2174098438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f422800b9c0 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7f423c001f30 tx=0x7f4228005830 comp rx=0 tx=0).stop 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f4249a00640 1 -- 172.21.15.88:0/2174098438 reap_dead start 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f424a400640 1 -- 172.21.15.88:0/2174098438 shutdown_connections 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f424a400640 1 -- 172.21.15.88:0/2174098438 >> 172.21.15.88:0/2174098438 conn(0x7f42440b1e40 msgr2=0x7f42440b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f424a400640 1 -- 172.21.15.88:0/2174098438 shutdown_connections 2024-09-21T07:52:52.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.715+0000 7f424a400640 1 -- 172.21.15.88:0/2174098438 wait complete. 2024-09-21T07:52:52.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.718+0000 7f424a400640 1 Processor -- start 2024-09-21T07:52:52.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.718+0000 7f424a400640 1 -- start start 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f424a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f4244158670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f424a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244158bb0 0x7f4244154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f424a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4244154880 0x7f4244154c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f424a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42440b2f60 con 0x7f4228005320 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f424a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f424406fbb0 con 0x7f4244158bb0 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f424a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f42440b30e0 con 0x7f4244154880 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f4249000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f4244158670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f4249000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f4244158670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57324/0 (socket says 172.21.15.88:57324) 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.719+0000 7f4249000640 1 -- 172.21.15.88:0/418923679 learned_addr learned my addr 172.21.15.88:0/418923679 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f423be00640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244158bb0 0x7f4244154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249a00640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4244154880 0x7f4244154c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249000640 1 -- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4244154880 msgr2=0x7f4244154c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249000640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4244154880 0x7f4244154c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249000640 1 -- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244158bb0 msgr2=0x7f4244154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249000640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244158bb0 0x7f4244154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249000640 1 -- 172.21.15.88:0/418923679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42441593c0 con 0x7f4228005320 2024-09-21T07:52:52.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4249000640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f4244158670 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4234000fd0 tx=0x7f42340039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:52.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234015070 con 0x7f4228005320 2024-09-21T07:52:52.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4234011900 con 0x7f4228005320 2024-09-21T07:52:52.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4244153860 con 0x7f4228005320 2024-09-21T07:52:52.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.720+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4234011d50 con 0x7f4228005320 2024-09-21T07:52:52.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.721+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4244153e20 con 0x7f4228005320 2024-09-21T07:52:52.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.722+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f424415d670 con 0x7f4228005320 2024-09-21T07:52:52.725 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.723+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f4234018070 con 0x7f4228005320 2024-09-21T07:52:52.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.724+0000 7f4241600640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4220079010 0x7f422007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:52.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.724+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f423409c3c0 con 0x7f4228005320 2024-09-21T07:52:52.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.724+0000 7f423be00640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4220079010 0x7f422007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:52.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.725+0000 7f423be00640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4220079010 0x7f422007b520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42440cfa30 tx=0x7f422c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:52.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.728+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f42340640d0 con 0x7f4228005320 2024-09-21T07:52:52.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.950+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f42441486d0 con 0x7f4228005320 2024-09-21T07:52:52.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.951+0000 7f4241600640 1 -- 172.21.15.88:0/418923679 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 80 v80) ==== 94+0+5228 (secure 0 0 0) 0x7f42340639d0 con 0x7f4228005320 2024-09-21T07:52:52.953 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:52:52.953 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":80,"btime":"2024-09-21T07:52:45:060798+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:52:41.524287+0000","modified":"2024-09-21T07:52:44.548337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6834/3361463212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3361463212},{"type":"v1","addr":"172.21.15.88:6834","nonce":3361463212}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":9}]} 2024-09-21T07:52:52.954 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 80 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4220079010 msgr2=0x7f422007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4220079010 0x7f422007b520 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f42440cfa30 tx=0x7f422c005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 msgr2=0x7f4244158670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f4244158670 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7f4234000fd0 tx=0x7f42340039c0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 shutdown_connections 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4220079010 0x7f422007b520 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4244154880 0x7f4244154c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4244158bb0 0x7f4244154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 --2- 172.21.15.88:0/418923679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4228005320 0x7f4244158670 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.953+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 >> 172.21.15.88:0/418923679 conn(0x7f42440b1e40 msgr2=0x7f4244152ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:52.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.954+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 shutdown_connections 2024-09-21T07:52:52.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:52.954+0000 7f424a400640 1 -- 172.21.15.88:0/418923679 wait complete. 2024-09-21T07:52:53.380 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 80, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:52:41.524287+0000', 'modified': '2024-09-21T07:52:44.548337+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7509}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7509': {'gid': 7509, 'name': 'c', 'rank': 0, 'incarnation': 76, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6834/3361463212', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 3361463212}, {'type': 'v1', 'addr': '172.21.15.88:6834', 'nonce': 3361463212}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7509, 'qdb_cluster': [7509]} 2024-09-21T07:52:53.381 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:52:53.381 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:52:53.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 -- 172.21.15.88:0/1572836887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe640053a0 msgr2=0x7efe6400ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 --2- 172.21.15.88:0/1572836887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe640053a0 0x7efe6400ba50 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7efe78001d90 tx=0x7efe64005830 comp rx=0 tx=0).stop 2024-09-21T07:52:53.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 -- 172.21.15.88:0/1572836887 shutdown_connections 2024-09-21T07:52:53.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 --2- 172.21.15.88:0/1572836887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe640053a0 0x7efe6400ba50 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 -- 172.21.15.88:0/1572836887 >> 172.21.15.88:0/1572836887 conn(0x7efe800b1e40 msgr2=0x7efe800b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 -- 172.21.15.88:0/1572836887 shutdown_connections 2024-09-21T07:52:53.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.606+0000 7efe84a00640 1 -- 172.21.15.88:0/1572836887 wait complete. 2024-09-21T07:52:53.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.608+0000 7efe84a00640 1 Processor -- start 2024-09-21T07:52:53.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.609+0000 7efe84a00640 1 -- start start 2024-09-21T07:52:53.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe84a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe640053a0 0x7efe80159620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe84a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe801e3ca0 0x7efe801f29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe84a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 0x7efe80156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe84a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe800b3340 con 0x7efe800cd320 2024-09-21T07:52:53.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe84a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe800b31c0 con 0x7efe640053a0 2024-09-21T07:52:53.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe84a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe8015d670 con 0x7efe801e3ca0 2024-09-21T07:52:53.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 0x7efe80156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 0x7efe80156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57332/0 (socket says 172.21.15.88:57332) 2024-09-21T07:52:53.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.610+0000 7efe7fe00640 1 -- 172.21.15.88:0/2160932798 learned_addr learned my addr 172.21.15.88:0/2160932798 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:53.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7f400640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe640053a0 0x7efe80159620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7ea00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe801e3ca0 0x7efe801f29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7fe00640 1 -- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe801e3ca0 msgr2=0x7efe801f29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7fe00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe801e3ca0 0x7efe801f29c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7fe00640 1 -- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe640053a0 msgr2=0x7efe80159620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7fe00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe640053a0 0x7efe80159620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7fe00640 1 -- 172.21.15.88:0/2160932798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe801e2c00 con 0x7efe800cd320 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7f400640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe640053a0 0x7efe80159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7ea00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe801e3ca0 0x7efe801f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe7fe00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 0x7efe80156620 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7efe78001eb0 tx=0x7efe78006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe77400640 1 -- 172.21.15.88:0/2160932798 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78003d80 con 0x7efe800cd320 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe77400640 1 -- 172.21.15.88:0/2160932798 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe78005b00 con 0x7efe800cd320 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe77400640 1 -- 172.21.15.88:0/2160932798 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe78005f30 con 0x7efe800cd320 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.611+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe801e2ee0 con 0x7efe800cd320 2024-09-21T07:52:53.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.612+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe801e34a0 con 0x7efe800cd320 2024-09-21T07:52:53.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.614+0000 7efe77400640 1 -- 172.21.15.88:0/2160932798 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7efe780060d0 con 0x7efe800cd320 2024-09-21T07:52:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.615+0000 7efe77400640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe60079040 0x7efe6007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.615+0000 7efe77400640 1 -- 172.21.15.88:0/2160932798 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7efe780bb160 con 0x7efe800cd320 2024-09-21T07:52:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.615+0000 7efe8d078740 1 Processor -- start 2024-09-21T07:52:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.615+0000 7efe7f400640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe60079040 0x7efe6007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.615+0000 7efe8d078740 1 -- start start 2024-09-21T07:52:53.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.616+0000 7efe7f400640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe60079040 0x7efe6007b550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe70001050 tx=0x7efe700067a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe8d078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700df9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700dfa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700dca150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe8d078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564700d53430 con 0x564700a29270 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564700d435d0 con 0x564700a08590 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564700d32a00 con 0x564700a003b0 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe7f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700df9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe7f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700df9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57348/0 (socket says 172.21.15.88:57348) 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe7f400640 1 -- 172.21.15.88:0/1322498872 learned_addr learned my addr 172.21.15.88:0/1322498872 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe7ea00640 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700dfa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.617+0000 7efe7fe00640 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700dca150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe7f400640 1 -- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 msgr2=0x564700dfa0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe7f400640 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700dfa0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe7f400640 1 -- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 msgr2=0x564700dca150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe7f400640 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700dca150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe7f400640 1 -- 172.21.15.88:0/1322498872 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564700cfb520 con 0x564700a29270 2024-09-21T07:52:53.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe7f400640 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700df9b60 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efe70003a00 tx=0x7efe70013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49928a75c1377887 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe6b400640 1 -- 172.21.15.88:0/1322498872 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70022070 con 0x564700a29270 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe6b400640 1 -- 172.21.15.88:0/1322498872 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe70012c80 con 0x564700a29270 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.618+0000 7efe6b400640 1 -- 172.21.15.88:0/1322498872 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70016070 con 0x564700a29270 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 -- 172.21.15.88:0/1322498872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 msgr2=0x564700df9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700df9b60 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7efe70003a00 tx=0x7efe70013000 comp rx=0 tx=0).stop 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 -- 172.21.15.88:0/1322498872 shutdown_connections 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700dfa0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700dca150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 --2- 172.21.15.88:0/1322498872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700df9b60 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 -- 172.21.15.88:0/1322498872 >> 172.21.15.88:0/1322498872 conn(0x564700a1d3c0 msgr2=0x564700dcfb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 -- 172.21.15.88:0/1322498872 shutdown_connections 2024-09-21T07:52:53.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.619+0000 7efe8d078740 1 -- 172.21.15.88:0/1322498872 wait complete. 2024-09-21T07:52:53.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.620+0000 7efe8d078740 1 -- start start 2024-09-21T07:52:53.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.620+0000 7efe8d078740 1 Processor -- start 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 0x564700df55c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a08590 0x564700e15220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a29270 0x564700df0120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564700cd9330 con 0x564700a003b0 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x564700d36810 con 0x564700a29270 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564700aa3620 con 0x564700a08590 2024-09-21T07:52:53.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe8d078740 10 client.0 _ll_register_callbacks cb 0x564700cde420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:53.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 0x564700df55c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 0x564700df55c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57350/0 (socket says 172.21.15.88:57350) 2024-09-21T07:52:53.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 -- 172.21.15.88:0/1651237477 learned_addr learned my addr 172.21.15.88:0/1651237477 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:53.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 -- 172.21.15.88:0/1651237477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a08590 msgr2=0x564700e15220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a08590 0x564700e15220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 -- 172.21.15.88:0/1651237477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a29270 msgr2=0x564700df0120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a29270 0x564700df0120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.622+0000 7efe7f400640 1 -- 172.21.15.88:0/1651237477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564700a94850 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe7f400640 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 0x564700df55c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efe70001720 tx=0x7efe70027660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1b30fef5f11054 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe5fe00640 1 -- 172.21.15.88:0/1651237477 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe70016070 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe5fe00640 1 -- 172.21.15.88:0/1651237477 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7efe70012a80 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe70012c80 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe5fe00640 1 -- 172.21.15.88:0/1651237477 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7efe70022050 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe8d078740 10 client.7653 fetch_fsmap learned FSMap version 80 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x564700b0b6b0 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.623+0000 7efe5fe00640 1 -- 172.21.15.88:0/1651237477 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe70012c80 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe5fe00640 1 -- 172.21.15.88:0/1651237477 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 80) ==== 3502+0+0 (secure 0 0 0) 0x7efe7003a050 con 0x564700a003b0 2024-09-21T07:52:53.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe8d078740 10 client.7653 fetch_fsmap finished waiting for FSMap version 80 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe8d078740 10 client.7653 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe8d078740 20 client.7653 populate_metadata read hostname 'smithi088' 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe8d078740 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x5647009e7e40 0x564700e24090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe8d078740 4 client.7653 mds_command: new command op to 7509 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 --> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564700d45520 con 0x5647009e7e40 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.624+0000 7efe7ea00640 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x5647009e7e40 0x564700e24090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.625+0000 7efe7ea00640 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x5647009e7e40 0x564700e24090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6e4e437def61d84 server_cookie=17fea8c7bffb7c14 in_seq=0 out_seq=0 2024-09-21T07:52:53.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.625+0000 7efe5fe00640 10 client.7653 ms_handle_connect on v2:172.21.15.88:6832/3361463212 2024-09-21T07:52:53.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.625+0000 7efe5fe00640 1 -- 172.21.15.88:0/1651237477 <== mds.0 v2:172.21.15.88:6832/3361463212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564700d45520 con 0x5647009e7e40 2024-09-21T07:52:53.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.625+0000 7efe5fe00640 10 client.7653 handle_command_reply: tid=0 2024-09-21T07:52:53.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 client.7653 shutdown 2024-09-21T07:52:53.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 msgr2=0x564700df55c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 0x564700df55c0 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7efe70001720 tx=0x7efe70027660 comp rx=0 tx=0).stop 2024-09-21T07:52:53.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 shutdown_connections 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x5647009e7e40 0x564700e24090 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a08590 0x564700e15220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a29270 0x564700df0120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe5fe00640 0 client.7653 ms_handle_reset on v2:172.21.15.88:6832/3361463212 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 --2- 172.21.15.88:0/1651237477 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a003b0 0x564700df55c0 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 >> 172.21.15.88:0/1651237477 conn(0x564700a1d3c0 msgr2=0x564700e17630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 shutdown_connections 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 1 -- 172.21.15.88:0/1651237477 wait complete. 2024-09-21T07:52:53.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.626+0000 7efe8d078740 20 client.7653 trim_cache size 0 max 16384 2024-09-21T07:52:53.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.645+0000 7efe8d078740 1 Processor -- start 2024-09-21T07:52:53.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.645+0000 7efe8d078740 1 -- start start 2024-09-21T07:52:53.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a003b0 0x564700e37a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe8d078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 0x564700e38110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a29270 0x564700dcae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe8d078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe80200440 con 0x564700a08590 2024-09-21T07:52:53.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe801ff480 con 0x564700a003b0 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe801f7490 con 0x564700a29270 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 0x564700e38110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 0x564700e38110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57360/0 (socket says 172.21.15.88:57360) 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 -- 172.21.15.88:0/1063045078 learned_addr learned my addr 172.21.15.88:0/1063045078 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7f400640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a003b0 0x564700e37a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7fe00640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a29270 0x564700dcae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 -- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a29270 msgr2=0x564700dcae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a29270 0x564700dcae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 -- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a003b0 msgr2=0x564700e37a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a003b0 0x564700e37a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 -- 172.21.15.88:0/1063045078 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564700dcd2b0 con 0x564700a08590 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7fe00640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a29270 0x564700dcae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:53.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7f400640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a003b0 0x564700e37a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:52:53.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.647+0000 7efe7ea00640 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 0x564700e38110 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009a40 tx=0x7efe6c0137d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef8347e569e4643 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.648+0000 7efe6aa00640 1 -- 172.21.15.88:0/1063045078 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c024620 con 0x564700a08590 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.648+0000 7efe6aa00640 1 -- 172.21.15.88:0/1063045078 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe6c01f030 con 0x564700a08590 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.648+0000 7efe6aa00640 1 -- 172.21.15.88:0/1063045078 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe6c024b70 con 0x564700a08590 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.648+0000 7efe8d078740 1 -- 172.21.15.88:0/1063045078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 msgr2=0x564700e38110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.648+0000 7efe8d078740 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 0x564700e38110 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7efe6c009a40 tx=0x7efe6c0137d0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 -- 172.21.15.88:0/1063045078 shutdown_connections 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a29270 0x564700dcae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a08590 0x564700e38110 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 --2- 172.21.15.88:0/1063045078 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a003b0 0x564700e37a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 -- 172.21.15.88:0/1063045078 >> 172.21.15.88:0/1063045078 conn(0x564700a1d3c0 msgr2=0x564700dcfae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 -- 172.21.15.88:0/1063045078 shutdown_connections 2024-09-21T07:52:53.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.649+0000 7efe8d078740 1 -- 172.21.15.88:0/1063045078 wait complete. 2024-09-21T07:52:53.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.650+0000 7efe8d078740 1 -- start start 2024-09-21T07:52:53.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.650+0000 7efe8d078740 1 Processor -- start 2024-09-21T07:52:53.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700e163e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700df52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700e98990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564700e0f400 con 0x564700a29270 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe801fee30 con 0x564700a08590 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x564700e0f920 con 0x564700a003b0 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.651+0000 7efe8d078740 10 client.0 _ll_register_callbacks cb 0x7efe70000ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700e98990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700e98990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57366/0 (socket says 172.21.15.88:57366) 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 -- 172.21.15.88:0/264122397 learned_addr learned my addr 172.21.15.88:0/264122397 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7f400640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700e163e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7ea00640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700df52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 -- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 msgr2=0x564700e163e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700e163e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 -- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 msgr2=0x564700df52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700df52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 -- 172.21.15.88:0/264122397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564700e14e70 con 0x564700a29270 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe7fe00640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700e98990 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7efe78006e10 tx=0x7efe78047ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b011ac73f0a7d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe5fe00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.652+0000 7efe5fe00640 1 -- 172.21.15.88:0/264122397 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe780a1060 con 0x564700a29270 2024-09-21T07:52:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe5fe00640 1 -- 172.21.15.88:0/264122397 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7efe78081ad0 con 0x564700a29270 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe6c01f030 con 0x564700a29270 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe5fe00640 1 -- 172.21.15.88:0/264122397 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=80) ==== 24+0+0 (secure 0 0 0) 0x7efe78082f70 con 0x564700a29270 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 10 client.7659 fetch_fsmap learned FSMap version 80 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=80,monmap=2+}) -- 0x564700e37fb0 con 0x564700a29270 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe5fe00640 1 -- 172.21.15.88:0/264122397 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe6c01f030 con 0x564700a29270 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe5fe00640 1 -- 172.21.15.88:0/264122397 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 80) ==== 3502+0+0 (secure 0 0 0) 0x7efe7807e1c0 con 0x564700a29270 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 10 client.7659 fetch_fsmap finished waiting for FSMap version 80 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 10 client.7659 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 20 client.7659 populate_metadata read hostname 'smithi088' 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x564700df8040 0x564700df9780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 4 client.7659 mds_command: new command op to 7509 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 --> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] -- command(tid 0: {"prefix": "status"}) -- 0x564700d40fd0 con 0x564700df8040 2024-09-21T07:52:53.658 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.653+0000 7efe7f400640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x564700df8040 0x564700df9780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe7f400640 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x564700df8040 0x564700df9780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ac79e5763bbef82 server_cookie=d9b8b50074070376 in_seq=0 out_seq=0 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe5fe00640 10 client.7659 ms_handle_connect on v2:172.21.15.88:6832/3361463212 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe5fe00640 1 -- 172.21.15.88:0/264122397 <== mds.0 v2:172.21.15.88:6832/3361463212 1 ==== command_reply(tid 0: 0 ) ==== 8+0+333 (crc 0 0 0) 0x564700d40fd0 con 0x564700df8040 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe5fe00640 10 client.7659 handle_command_reply: tid=0 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 client.7659 shutdown 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 msgr2=0x564700e98990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700e98990 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7efe78006e10 tx=0x7efe78047ff0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 shutdown_connections 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:6832/3361463212,v1:172.21.15.88:6834/3361463212] conn(0x564700df8040 0x564700df9780 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564700a29270 0x564700e98990 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x564700a08590 0x564700df52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 --2- 172.21.15.88:0/264122397 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x564700a003b0 0x564700e163e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe5fe00640 0 client.7659 ms_handle_reset on v2:172.21.15.88:6832/3361463212 2024-09-21T07:52:53.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.654+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 >> 172.21.15.88:0/264122397 conn(0x564700a1d3c0 msgr2=0x564700e15070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 shutdown_connections 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe8d078740 1 -- 172.21.15.88:0/264122397 wait complete. 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe8d078740 20 client.7659 trim_cache size 0 max 16384 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 7509, 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:52:53.660 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 11.616858946000001, 2024-09-21T07:52:53.661 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 80, 2024-09-21T07:52:53.661 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 120, 2024-09-21T07:52:53.661 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 120, 2024-09-21T07:52:53.661 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 16.635138657999999 2024-09-21T07:52:53.661 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:52:53.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe60079040 msgr2=0x7efe6007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe60079040 0x7efe6007b550 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7efe70001050 tx=0x7efe700067a0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 msgr2=0x7efe80156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 0x7efe80156620 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7efe78001eb0 tx=0x7efe78006ca0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 shutdown_connections 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe60079040 0x7efe6007b550 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe801e3ca0 0x7efe801f29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe800cd320 0x7efe80156620 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 --2- 172.21.15.88:0/2160932798 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe640053a0 0x7efe80159620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 >> 172.21.15.88:0/2160932798 conn(0x7efe800b1e40 msgr2=0x7efe8015ab90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.655+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 shutdown_connections 2024-09-21T07:52:53.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.656+0000 7efe84a00640 1 -- 172.21.15.88:0/2160932798 wait complete. 2024-09-21T07:52:53.681 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:52:53.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 -- 172.21.15.88:0/67456879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1838072e60 msgr2=0x7f1838073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 --2- 172.21.15.88:0/67456879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1838072e60 0x7f1838073270 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7f1820009900 tx=0x7f182002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 -- 172.21.15.88:0/67456879 shutdown_connections 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 --2- 172.21.15.88:0/67456879 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 0x7f1838070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 --2- 172.21.15.88:0/67456879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1838072e60 0x7f1838073270 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 --2- 172.21.15.88:0/67456879 >> v2:172.21.15.175:3300/0 conn(0x7f18380af050 0x7f18380ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 -- 172.21.15.88:0/67456879 >> 172.21.15.88:0/67456879 conn(0x7f18380b1e40 msgr2=0x7f18380b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 -- 172.21.15.88:0/67456879 shutdown_connections 2024-09-21T07:52:53.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.902+0000 7f183e000640 1 -- 172.21.15.88:0/67456879 wait complete. 2024-09-21T07:52:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.904+0000 7f183e000640 1 Processor -- start 2024-09-21T07:52:53.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.904+0000 7f183e000640 1 -- start start 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183e000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 0x7f18381f5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183e000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1838072e60 0x7f18381e5760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 0x7f1838155350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183e000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f183815c0e0 con 0x7f18380af050 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183e000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f183815bf60 con 0x7f1838072e60 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183e000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f183815c260 con 0x7f1838070000 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 0x7f1838155350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f182fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1838072e60 0x7f18381e5760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183cc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 0x7f18381f5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 0x7f1838155350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57390/0 (socket says 172.21.15.88:57390) 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.906+0000 7f183d600640 1 -- 172.21.15.88:0/3740067409 learned_addr learned my addr 172.21.15.88:0/3740067409 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:53.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183d600640 1 -- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 msgr2=0x7f18381f5000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183d600640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 0x7f18381f5000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183d600640 1 -- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1838072e60 msgr2=0x7f18381e5760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183d600640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1838072e60 0x7f18381e5760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183d600640 1 -- 172.21.15.88:0/3740067409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18381f5540 con 0x7f18380af050 2024-09-21T07:52:53.910 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183cc00640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 0x7f18381f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183d600640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 0x7f1838155350 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f183000eab0 tx=0x7f1830009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f183000ce40 con 0x7f18380af050 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1830004c10 con 0x7f18380af050 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1838143460 con 0x7f18380af050 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1830005020 con 0x7f18380af050 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.907+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1838143a20 con 0x7f18380af050 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.908+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18380b2780 con 0x7f18380af050 2024-09-21T07:52:53.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.910+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f18300051c0 con 0x7f18380af050 2024-09-21T07:52:53.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.911+0000 7f1835600640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1814079010 0x7f181407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:53.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.911+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f183009c330 con 0x7f18380af050 2024-09-21T07:52:53.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.911+0000 7f183cc00640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1814079010 0x7f181407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:53.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.911+0000 7f183cc00640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1814079010 0x7f181407b520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1828009f20 tx=0x7f1828005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:53.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:53.915+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18380b2780 con 0x7f18380af050 2024-09-21T07:52:54.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.141+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f183815c780 con 0x7f18380af050 2024-09-21T07:52:54.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.141+0000 7f1835600640 1 -- 172.21.15.88:0/3740067409 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) ==== 53+0+73 (secure 0 0 0) 0x7f1830063950 con 0x7f18380af050 2024-09-21T07:52:54.143 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.143+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1814079010 msgr2=0x7f181407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.143+0000 7f183e000640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1814079010 0x7f181407b520 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1828009f20 tx=0x7f1828005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.143+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 msgr2=0x7f1838155350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.143+0000 7f183e000640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 0x7f1838155350 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7f183000eab0 tx=0x7f1830009f70 comp rx=0 tx=0).stop 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 shutdown_connections 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1814079010 0x7f181407b520 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18380af050 0x7f1838155350 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1838072e60 0x7f18381e5760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:54.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 --2- 172.21.15.88:0/3740067409 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1838070000 0x7f18381f5000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:54.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 >> 172.21.15.88:0/3740067409 conn(0x7f18380b1e40 msgr2=0x7f18380b38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:54.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 shutdown_connections 2024-09-21T07:52:54.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54.144+0000 7f183e000640 1 -- 172.21.15.88:0/3740067409 wait complete. 2024-09-21T07:52:54.169 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:52:54.170 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:54.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:54.170 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:52:54.206 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:52:54.206 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:52:54.206 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:52:54.206 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:52:54.207 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42320sec preferred_lft 42320sec 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:52:54.208 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:52:54.209 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:54.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:54.209 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:52:54.300 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:52:54.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:54.303 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:52:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:52:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:52:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:52:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:52:54.339 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:52:54.339 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:52:54.339 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:52:54.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:52:54.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:52:54.340 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:52:54.340 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:52:54.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:52:54.526 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:54 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:54.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:54.535 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:52:54.570 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:52:54.570 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:52:54.571 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:54.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:54.571 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:52:54.666 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:52:54.666 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:54.701 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:52:54.701 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:52:54.701 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:54.775 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:52:54.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:54.775 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:52:54.970 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:52:54.970 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:52:54.970 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:52:54.990 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:52:54.990 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:52:54.990 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:52:54.990 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:52:54.990 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:52:55.023 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:52:55.025 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:52:55.025 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:52:55.025 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:52:55.025 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:52:55.025 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:52:55.028 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:52:55.028 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:55.029 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:52:55.085 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:52:55.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:52:55.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:55.180 INFO:teuthology.orchestra.run.smithi088.stdout:client6560 2024-09-21T07:52:55.181 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:55.181 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:55.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:52:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.472+0000 7fd55e400640 1 -- 172.21.15.88:0/701249655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 msgr2=0x7fd558160720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.472+0000 7fd55e400640 1 --2- 172.21.15.88:0/701249655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd558160720 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fd558075440 tx=0x7fd55002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 -- 172.21.15.88:0/701249655 shutdown_connections 2024-09-21T07:52:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 --2- 172.21.15.88:0/701249655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd558160720 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 --2- 172.21.15.88:0/701249655 >> v2:172.21.15.175:3300/0 conn(0x7fd5580ad0e0 0x7fd5580ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 --2- 172.21.15.88:0/701249655 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd55815d760 0x7fd5581535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 -- 172.21.15.88:0/701249655 >> 172.21.15.88:0/701249655 conn(0x7fd55806d730 msgr2=0x7fd5580b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:55.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 -- 172.21.15.88:0/701249655 shutdown_connections 2024-09-21T07:52:55.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.473+0000 7fd55e400640 1 -- 172.21.15.88:0/701249655 wait complete. 2024-09-21T07:52:55.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.475+0000 7fd55e400640 1 Processor -- start 2024-09-21T07:52:55.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.475+0000 7fd55e400640 1 -- start start 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.476+0000 7fd55e400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5580ad0e0 0x7fd5581f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.476+0000 7fd55e400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55815d760 0x7fd558154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.476+0000 7fd55e400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd5580b1040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.476+0000 7fd55e400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd5580b4da0 con 0x7fd558160310 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.476+0000 7fd55e400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd5580b4c20 con 0x7fd55815d760 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.476+0000 7fd55e400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd558160c60 con 0x7fd5580ad0e0 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd5580b1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd5580b1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57416/0 (socket says 172.21.15.88:57416) 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 -- 172.21.15.88:0/1990623676 learned_addr learned my addr 172.21.15.88:0/1990623676 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd556a00640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55815d760 0x7fd558154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5580ad0e0 0x7fd5581f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 -- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5580ad0e0 msgr2=0x7fd5581f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5580ad0e0 0x7fd5581f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 -- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55815d760 msgr2=0x7fd558154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55815d760 0x7fd558154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 -- 172.21.15.88:0/1990623676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd558146f10 con 0x7fd558160310 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd556a00640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55815d760 0x7fd558154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.477+0000 7fd557e00640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd5580b1040 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd550009d60 tx=0x7fd550003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.478+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550033050 con 0x7fd558160310 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.478+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd550004450 con 0x7fd558160310 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.478+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5581471f0 con 0x7fd558160310 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.478+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd550038450 con 0x7fd558160310 2024-09-21T07:52:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.478+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd5581477b0 con 0x7fd558160310 2024-09-21T07:52:55.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.479+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd5581535c0 con 0x7fd558160310 2024-09-21T07:52:55.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.480+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fd550036070 con 0x7fd558160310 2024-09-21T07:52:55.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.481+0000 7fd54f400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd538079010 0x7fd53807b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:55.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.481+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fd5500bc9f0 con 0x7fd558160310 2024-09-21T07:52:55.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.481+0000 7fd557400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd538079010 0x7fd53807b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:55.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.482+0000 7fd557400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd538079010 0x7fd53807b520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd54800daa0 tx=0x7fd548005f30 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.486+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd550084620 con 0x7fd558160310 2024-09-21T07:52:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.707+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd558146c80 con 0x7fd558160310 2024-09-21T07:52:55.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.707+0000 7fd54f400640 1 -- 172.21.15.88:0/1990623676 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v80) ==== 53+0+73 (secure 0 0 0) 0x7fd550083f20 con 0x7fd558160310 2024-09-21T07:52:55.709 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:52:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd538079010 msgr2=0x7fd53807b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd538079010 0x7fd53807b520 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fd54800daa0 tx=0x7fd548005f30 comp rx=0 tx=0).stop 2024-09-21T07:52:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 msgr2=0x7fd5580b1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd5580b1040 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7fd550009d60 tx=0x7fd550003f10 comp rx=0 tx=0).stop 2024-09-21T07:52:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 shutdown_connections 2024-09-21T07:52:55.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd538079010 0x7fd53807b520 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd558160310 0x7fd5580b1040 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd55815d760 0x7fd558154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 --2- 172.21.15.88:0/1990623676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd5580ad0e0 0x7fd5581f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:55.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 >> 172.21.15.88:0/1990623676 conn(0x7fd55806d730 msgr2=0x7fd55815dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:55.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 shutdown_connections 2024-09-21T07:52:55.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:55.710+0000 7fd55e400640 1 -- 172.21.15.88:0/1990623676 wait complete. 2024-09-21T07:52:55.734 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:52:55.734 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:52:55.734 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:52:55.734 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:52:55.734 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:52:55.734 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:52:55.735 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:52:55.735 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:52:55.735 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:52:55.735 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:55.735 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:55.775 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42269sec preferred_lft 42269sec 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:52:55.776 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:52:55.777 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:52:55.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:55.778 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:52:55.876 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:52:55.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:55.880 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:52:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:52:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:52:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:52:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:52:55.916 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:52:55.917 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:52:55.917 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:52:55.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:52:55.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:52:55.917 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:52:55.917 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:52:56.035 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:52:56 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:52:56.106 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:56.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:56.114 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:52:56.149 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:52:56.149 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:52:56.149 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:52:56.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:56.149 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:52:56.246 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:52:56.246 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:52:56.281 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:52:56.281 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:52:56.281 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:52:56.355 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:52:56.355 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:56.355 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:52:56.548 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:52:56.548 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:52:56.548 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:52:56.568 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:52:56.568 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:52:56.568 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:52:56.568 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:52:56.568 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:52:56.597 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:52:56.597 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:52:56.597 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:52:56.597 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:52:56.597 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:52:56.598 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:52:56.598 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:52:56.598 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:52:56.598 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:52:56.599 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:52:56.599 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:52:56.599 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:52:56.599 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:52:56.602 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:52:56.602 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:56.602 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:52:56.659 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:52:56.742 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:52:56 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:52:56.745 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:52:56 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:52:56.753 INFO:teuthology.orchestra.run.smithi175.stdout:client7240 2024-09-21T07:52:56.753 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:52:56.753 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:52:56.791 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:52:57.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.025+0000 7f48da800640 1 -- 172.21.15.88:0/3578860319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b8003a00 msgr2=0x7f48b8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:57.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.025+0000 7f48da800640 1 --2- 172.21.15.88:0/3578860319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48b8003a00 0x7f48b8003e70 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c290 tx=0x7f48b8005830 comp rx=0 tx=0).stop 2024-09-21T07:52:57.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.025+0000 7f48d3e00640 1 -- 172.21.15.88:0/3578860319 reap_dead start 2024-09-21T07:52:57.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.026+0000 7f48da800640 1 -- 172.21.15.88:0/3578860319 shutdown_connections 2024-09-21T07:52:57.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.026+0000 7f48da800640 1 -- 172.21.15.88:0/3578860319 >> 172.21.15.88:0/3578860319 conn(0x7f48d40b2540 msgr2=0x7f48d40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:57.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.026+0000 7f48da800640 1 -- 172.21.15.88:0/3578860319 shutdown_connections 2024-09-21T07:52:57.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.026+0000 7f48da800640 1 -- 172.21.15.88:0/3578860319 wait complete. 2024-09-21T07:52:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.028+0000 7f48da800640 1 Processor -- start 2024-09-21T07:52:57.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.028+0000 7f48da800640 1 -- start start 2024-09-21T07:52:57.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.029+0000 7f48da800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b8003a00 0x7f48d40c2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:57.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48da800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 0x7f48d4098d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:57.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48da800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d4099270 0x7f48d41f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48da800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f48d40b47a0 con 0x7f48d4152d20 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48da800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f48d40b4620 con 0x7f48b8003a00 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48da800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f48d40b4920 con 0x7f48d4099270 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 0x7f48d4098d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 0x7f48d4098d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52918/0 (socket says 172.21.15.88:52918) 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 -- 172.21.15.88:0/3679474593 learned_addr learned my addr 172.21.15.88:0/3679474593 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d3400640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b8003a00 0x7f48d40c2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d3e00640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d4099270 0x7f48d41f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 -- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d4099270 msgr2=0x7f48d41f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d4099270 0x7f48d41f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 -- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b8003a00 msgr2=0x7f48d40c2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b8003a00 0x7f48d40c2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:57.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.030+0000 7f48d2a00640 1 -- 172.21.15.88:0/3679474593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48d4099680 con 0x7f48d4152d20 2024-09-21T07:52:57.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.031+0000 7f48d2a00640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 0x7f48d4098d30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f48c0011a40 tx=0x7f48c0011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:57.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.031+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c001e070 con 0x7f48d4152d20 2024-09-21T07:52:57.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.031+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f48c000fdc0 con 0x7f48d4152d20 2024-09-21T07:52:57.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.031+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48d41e7ca0 con 0x7f48d4152d20 2024-09-21T07:52:57.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.031+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48c0004290 con 0x7f48d4152d20 2024-09-21T07:52:57.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.031+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48d41e8260 con 0x7f48d4152d20 2024-09-21T07:52:57.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.032+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48d40719b0 con 0x7f48d4152d20 2024-09-21T07:52:57.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.033+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f48c0021070 con 0x7f48d4152d20 2024-09-21T07:52:57.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.034+0000 7f48cb400640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48b4079010 0x7f48b407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:57.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.034+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f48c009bfe0 con 0x7f48d4152d20 2024-09-21T07:52:57.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.035+0000 7f48d3400640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48b4079010 0x7f48b407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:57.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.035+0000 7f48d3400640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48b4079010 0x7f48b407b520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c290 tx=0x7f48c4038040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:57.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.039+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f48d40719b0 con 0x7f48d4152d20 2024-09-21T07:52:57.226 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:57.231 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:52:57.268 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.266+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f48d40af510 con 0x7f48d4152d20 2024-09-21T07:52:57.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.802+0000 7f48cb400640 1 -- 172.21.15.88:0/3679474593 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v81) ==== 136+0+0 (secure 0 0 0) 0x7f48d4099680 con 0x7f48d4152d20 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48b4079010 msgr2=0x7f48b407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48b4079010 0x7f48b407b520 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f48c400c290 tx=0x7f48c4038040 comp rx=0 tx=0).stop 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 msgr2=0x7f48d4098d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 0x7f48d4098d30 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f48c0011a40 tx=0x7f48c0011e40 comp rx=0 tx=0).stop 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 shutdown_connections 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f48b4079010 0x7f48b407b520 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f48d4099270 0x7f48d41f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f48d4152d20 0x7f48d4098d30 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 --2- 172.21.15.88:0/3679474593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f48b8003a00 0x7f48d40c2f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.805+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 >> 172.21.15.88:0/3679474593 conn(0x7f48d40b2540 msgr2=0x7f48d40c9cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.806+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 shutdown_connections 2024-09-21T07:52:57.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:57.806+0000 7f48da800640 1 -- 172.21.15.88:0/3679474593 wait complete. 2024-09-21T07:52:57.835 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:52:58.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 -- 172.21.15.88:0/4196567460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b4040 msgr2=0x7fddd80b4450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 --2- 172.21.15.88:0/4196567460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b4040 0x7fddd80b4450 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fddc0005fd0 tx=0x7fddc002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 -- 172.21.15.88:0/4196567460 shutdown_connections 2024-09-21T07:52:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 --2- 172.21.15.88:0/4196567460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b4040 0x7fddd80b4450 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 --2- 172.21.15.88:0/4196567460 >> v2:172.21.15.175:3300/0 conn(0x7fddd80b6430 0x7fddd80cd1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 -- 172.21.15.88:0/4196567460 >> 172.21.15.88:0/4196567460 conn(0x7fddd80b2d00 msgr2=0x7fddd80b3110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 -- 172.21.15.88:0/4196567460 shutdown_connections 2024-09-21T07:52:58.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.058+0000 7fdddd200640 1 -- 172.21.15.88:0/4196567460 wait complete. 2024-09-21T07:52:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.060+0000 7fdddd200640 1 Processor -- start 2024-09-21T07:52:58.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.061+0000 7fdddd200640 1 -- start start 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fdddd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddd80b4040 0x7fddd815f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fdddd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b6430 0x7fddd814c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fdddd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 0x7fddd81567d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fdddd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80cba90 con 0x7fddd80cdbe0 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fdddd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddd80ce910 con 0x7fddd80b4040 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fdddd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddd80cbc10 con 0x7fddd80b6430 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.062+0000 7fddd7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 0x7fddd81567d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddd7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddd80b4040 0x7fddd815f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b6430 0x7fddd814c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddd7400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddd80b4040 0x7fddd815f370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:42548/0 (socket says 172.21.15.88:42548) 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddd7400640 1 -- 172.21.15.88:0/890848538 learned_addr learned my addr 172.21.15.88:0/890848538 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:58.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddd7e00640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 0x7fddd81567d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52926/0 (socket says 172.21.15.88:52926) 2024-09-21T07:52:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 -- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddd80b4040 msgr2=0x7fddd815f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddd80b4040 0x7fddd815f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 -- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 msgr2=0x7fddd81567d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 0x7fddd81567d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 -- 172.21.15.88:0/890848538 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddd81538f0 con 0x7fddd80b6430 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddd7e00640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 0x7fddd81567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.063+0000 7fddcfe00640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b6430 0x7fddd814c5b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddd80ccd10 tx=0x7fddc0005750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.064+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mon.2 v2:172.21.15.175:3301/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc003e030 con 0x7fddd80b6430 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.064+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mon.2 v2:172.21.15.175:3301/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fddc0003a70 con 0x7fddd80b6430 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.064+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mon.2 v2:172.21.15.175:3301/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddc0035040 con 0x7fddd80b6430 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.064+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddd8154830 con 0x7fddd80b6430 2024-09-21T07:52:58.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.064+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_subscribe({osdmap=0}) -- 0x7fddd8154df0 con 0x7fddd80b6430 2024-09-21T07:52:58.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.065+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddd80d1e40 con 0x7fddd80b6430 2024-09-21T07:52:58.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.067+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mon.2 v2:172.21.15.175:3301/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fddc0041030 con 0x7fddd80b6430 2024-09-21T07:52:58.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.067+0000 7fddcf400640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddb4079010 0x7fddb407b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.068+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mon.2 v2:172.21.15.175:3301/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fddc00bbfa0 con 0x7fddd80b6430 2024-09-21T07:52:58.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.068+0000 7fddd7400640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddb4079010 0x7fddb407b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.068+0000 7fddd7400640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddb4079010 0x7fddb407b520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009bf0 tx=0x7fddc8005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:58.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.071+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mon.2 v2:172.21.15.175:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fddd80d1e40 con 0x7fddd80b6430 2024-09-21T07:52:58.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.278+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fddd80b4450 con 0x7fddb4079010 2024-09-21T07:52:58.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.280+0000 7fddcf400640 1 -- 172.21.15.88:0/890848538 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fddd80b4450 con 0x7fddb4079010 2024-09-21T07:52:58.282 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:52:58.282 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:52:58.282 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:52:58.282 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:52:58.282 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddb4079010 msgr2=0x7fddb407b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddb4079010 0x7fddb407b520 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fddc8009bf0 tx=0x7fddc8005f90 comp rx=0 tx=0).stop 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b6430 msgr2=0x7fddd814c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b6430 0x7fddd814c5b0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fddd80ccd10 tx=0x7fddc0005750 comp rx=0 tx=0).stop 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 shutdown_connections 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddb4079010 0x7fddb407b520 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddd80b6430 0x7fddd814c5b0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd80cdbe0 0x7fddd81567d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 --2- 172.21.15.88:0/890848538 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddd80b4040 0x7fddd815f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 >> 172.21.15.88:0/890848538 conn(0x7fddd80b2d00 msgr2=0x7fddd80b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.282+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 shutdown_connections 2024-09-21T07:52:58.285 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.283+0000 7fdddd200640 1 -- 172.21.15.88:0/890848538 wait complete. 2024-09-21T07:52:58.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:52:58.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 -- 172.21.15.88:0/2358143518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4145a40 msgr2=0x7f62b4073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 --2- 172.21.15.88:0/2358143518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4145a40 0x7f62b4073540 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f62b41464e0 tx=0x7f62a002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 -- 172.21.15.88:0/2358143518 shutdown_connections 2024-09-21T07:52:58.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 --2- 172.21.15.88:0/2358143518 >> v2:172.21.15.175:3300/0 conn(0x7f62b4073a80 0x7f62b4160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 --2- 172.21.15.88:0/2358143518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4145a40 0x7f62b4073540 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 --2- 172.21.15.88:0/2358143518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62b4145550 0x7f62b406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 -- 172.21.15.88:0/2358143518 >> 172.21.15.88:0/2358143518 conn(0x7f62b406d1c0 msgr2=0x7f62b406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:58.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 -- 172.21.15.88:0/2358143518 shutdown_connections 2024-09-21T07:52:58.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.532+0000 7f62ba200640 1 -- 172.21.15.88:0/2358143518 wait complete. 2024-09-21T07:52:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.534+0000 7f62ba200640 1 Processor -- start 2024-09-21T07:52:58.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.534+0000 7f62ba200640 1 -- start start 2024-09-21T07:52:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.535+0000 7f62ba200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 0x7f62b415d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.535+0000 7f62ba200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62b4145550 0x7f62b4152ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.535+0000 7f62ba200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62b4145a40 0x7f62b4143f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62ba200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4160c20 con 0x7f62b4073a80 2024-09-21T07:52:58.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62ba200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62b4160aa0 con 0x7f62b4145550 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62ba200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62b4160da0 con 0x7f62b4145a40 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 0x7f62b415d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 0x7f62b415d770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52948/0 (socket says 172.21.15.88:52948) 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 -- 172.21.15.88:0/883221129 learned_addr learned my addr 172.21.15.88:0/883221129 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3e00640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62b4145a40 0x7f62b4143f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b2a00640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62b4145550 0x7f62b4152ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 -- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62b4145a40 msgr2=0x7f62b4143f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62b4145a40 0x7f62b4143f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 -- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62b4145550 msgr2=0x7f62b4152ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62b4145550 0x7f62b4152ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.536+0000 7f62b3400640 1 -- 172.21.15.88:0/883221129 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62b41533e0 con 0x7f62b4073a80 2024-09-21T07:52:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.537+0000 7f62b3400640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 0x7f62b415d770 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f62a400e7c0 tx=0x7f62a400ebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.537+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a400cab0 con 0x7f62b4073a80 2024-09-21T07:52:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.537+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f62a400cc50 con 0x7f62b4073a80 2024-09-21T07:52:58.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.537+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62a4005020 con 0x7f62b4073a80 2024-09-21T07:52:58.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.537+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62b40aa170 con 0x7f62b4073a80 2024-09-21T07:52:58.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.537+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62b40aa730 con 0x7f62b4073a80 2024-09-21T07:52:58.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.538+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62b406fb40 con 0x7f62b4073a80 2024-09-21T07:52:58.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.539+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f62a40051c0 con 0x7f62b4073a80 2024-09-21T07:52:58.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.540+0000 7f62ab400640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6294079040 0x7f629407b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:52:58.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.540+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f62a409bf20 con 0x7f62b4073a80 2024-09-21T07:52:58.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.541+0000 7f62b2a00640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6294079040 0x7f629407b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:52:58.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.541+0000 7f62b2a00640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6294079040 0x7f629407b550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62a0033000 tx=0x7f62a0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:52:58.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.545+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62a4063b90 con 0x7f62b4073a80 2024-09-21T07:52:58.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.729+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f62b40b0570 con 0x7f62b4073a80 2024-09-21T07:52:58.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.729+0000 7f62ab400640 1 -- 172.21.15.88:0/883221129 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v44)=0 v44) ==== 122+0+0 (secure 0 0 0) 0x7f62a4063490 con 0x7f62b4073a80 2024-09-21T07:52:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6294079040 msgr2=0x7f629407b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6294079040 0x7f629407b550 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f62a0033000 tx=0x7f62a0004730 comp rx=0 tx=0).stop 2024-09-21T07:52:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 msgr2=0x7f62b415d770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:52:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 0x7f62b415d770 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f62a400e7c0 tx=0x7f62a400ebc0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 shutdown_connections 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6294079040 0x7f629407b550 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62b4145a40 0x7f62b4143f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62b4145550 0x7f62b4152ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 --2- 172.21.15.88:0/883221129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f62b4073a80 0x7f62b415d770 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 >> 172.21.15.88:0/883221129 conn(0x7f62b406d1c0 msgr2=0x7f62b40b1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 shutdown_connections 2024-09-21T07:52:58.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:52:58.732+0000 7f62ba200640 1 -- 172.21.15.88:0/883221129 wait complete. 2024-09-21T07:52:58.756 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:58.756 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol62770/9c75e8ac-f338-43fd-998b-3a9b61087a8d) 2024-09-21T07:52:58.821 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:58.822 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol62770/9c75e8ac-f338-43fd-998b-3a9b61087a8d/.snap/fake) 2024-09-21T07:52:58.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:58.906 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:52:58.942 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:52:58.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:58.942 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol62770/9c75e8ac-f338-43fd-998b-3a9b61087a8d) 2024-09-21T07:52:59.042 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:59.042 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol62770/.meta 2024-09-21T07:52:59.105 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.105 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp80740/subvol17521/c2e0f0d8-3cfb-4784-85fe-e6fcf685f3bc) 2024-09-21T07:52:59.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.204 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp80740/subvol17521/c2e0f0d8-3cfb-4784-85fe-e6fcf685f3bc/.snap/fake) 2024-09-21T07:52:59.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.288 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:52:59.324 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:52:59.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.325 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp80740/subvol17521/c2e0f0d8-3cfb-4784-85fe-e6fcf685f3bc) 2024-09-21T07:52:59.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:59.425 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp80740/subvol17521/.meta 2024-09-21T07:52:59.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.482 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol80472/c3be4aee-26eb-4dab-bb74-0c2a25ea087f) 2024-09-21T07:52:59.580 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.580 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:52:59.616 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:52:59.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:52:59.616 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol80472/c3be4aee-26eb-4dab-bb74-0c2a25ea087f) 2024-09-21T07:52:59.717 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:52:59.717 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol80472/.meta 2024-09-21T07:52:59.773 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol62770 2024-09-21T07:53:00.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.034+0000 7f9a59e00640 1 -- 172.21.15.88:0/254218727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 msgr2=0x7f9a5406ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.034+0000 7f9a59e00640 1 --2- 172.21.15.88:0/254218727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 0x7f9a5406ef10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00afd0 tx=0x7f9a4c02b490 comp rx=0 tx=0).stop 2024-09-21T07:53:00.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.035+0000 7f9a59e00640 1 -- 172.21.15.88:0/254218727 shutdown_connections 2024-09-21T07:53:00.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.035+0000 7f9a59e00640 1 --2- 172.21.15.88:0/254218727 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 0x7f9a5406ef10 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.035+0000 7f9a59e00640 1 --2- 172.21.15.88:0/254218727 >> v2:172.21.15.175:3300/0 conn(0x7f9a540af8f0 0x7f9a5415b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.035+0000 7f9a59e00640 1 -- 172.21.15.88:0/254218727 >> 172.21.15.88:0/254218727 conn(0x7f9a5406d620 msgr2=0x7f9a540b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:00.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.035+0000 7f9a59e00640 1 -- 172.21.15.88:0/254218727 shutdown_connections 2024-09-21T07:53:00.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.035+0000 7f9a59e00640 1 -- 172.21.15.88:0/254218727 wait complete. 2024-09-21T07:53:00.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.037+0000 7f9a59e00640 1 Processor -- start 2024-09-21T07:53:00.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.037+0000 7f9a59e00640 1 -- start start 2024-09-21T07:53:00.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.038+0000 7f9a59e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a540af4e0 0x7f9a541f3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.038+0000 7f9a59e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 0x7f9a5409d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.038+0000 7f9a59e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 0x7f9a54154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.038+0000 7f9a59e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5406f450 con 0x7f9a540af8f0 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.038+0000 7f9a59e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9a540711b0 con 0x7f9a540af4e0 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.038+0000 7f9a59e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9a5406f5d0 con 0x7f9a5415bc20 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 0x7f9a5409d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 0x7f9a5409d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:52958/0 (socket says 172.21.15.88:52958) 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 -- 172.21.15.88:0/1313477341 learned_addr learned my addr 172.21.15.88:0/1313477341 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a58a00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a540af4e0 0x7f9a541f3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a59400640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 0x7f9a54154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 -- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 msgr2=0x7f9a54154390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 0x7f9a54154390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 -- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a540af4e0 msgr2=0x7f9a541f3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a540af4e0 0x7f9a541f3470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 -- 172.21.15.88:0/1313477341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a541548d0 con 0x7f9a540af8f0 2024-09-21T07:53:00.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.039+0000 7f9a4be00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 0x7f9a5409d9b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ea20 tx=0x7f9a3c00eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:00.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.040+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c00ce40 con 0x7f9a540af8f0 2024-09-21T07:53:00.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.040+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9a3c004020 con 0x7f9a540af8f0 2024-09-21T07:53:00.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.040+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c004470 con 0x7f9a540af8f0 2024-09-21T07:53:00.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.040+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a54153000 con 0x7f9a540af8f0 2024-09-21T07:53:00.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.040+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a541f3fa0 con 0x7f9a540af8f0 2024-09-21T07:53:00.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.041+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a541f3a30 con 0x7f9a540af8f0 2024-09-21T07:53:00.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.042+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f9a3c0035f0 con 0x7f9a540af8f0 2024-09-21T07:53:00.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.043+0000 7f9a51600640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9a30079040 0x7f9a3007b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.043+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f9a3c09bf00 con 0x7f9a540af8f0 2024-09-21T07:53:00.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.043+0000 7f9a58a00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9a30079040 0x7f9a3007b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.044+0000 7f9a58a00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9a30079040 0x7f9a3007b550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a440015e0 tx=0x7f9a44009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:00.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.048+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9a3c063bc0 con 0x7f9a540af8f0 2024-09-21T07:53:00.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.262+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol62770", "target": ["mon-mgr", ""]}) -- 0x7f9a5415b6e0 con 0x7f9a30079040 2024-09-21T07:53:00.273 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:00.271+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:00.273 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:00.271+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:00.273 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:00.271+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:00.273 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:00.271+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:00.273 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:00.271+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:00.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.311+0000 7f9a51600640 1 -- 172.21.15.88:0/1313477341 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f9a5415b6e0 con 0x7f9a30079040 2024-09-21T07:53:00.313 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol62770/9c75e8ac-f338-43fd-998b-3a9b61087a8d 2024-09-21T07:53:00.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9a30079040 msgr2=0x7f9a3007b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9a30079040 0x7f9a3007b550 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9a440015e0 tx=0x7f9a44009040 comp rx=0 tx=0).stop 2024-09-21T07:53:00.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 msgr2=0x7f9a5409d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 0x7f9a5409d9b0 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00ea20 tx=0x7f9a3c00eee0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 shutdown_connections 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9a30079040 0x7f9a3007b550 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9a5415bc20 0x7f9a54154390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a540af8f0 0x7f9a5409d9b0 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 --2- 172.21.15.88:0/1313477341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9a540af4e0 0x7f9a541f3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 >> 172.21.15.88:0/1313477341 conn(0x7f9a5406d620 msgr2=0x7f9a54073ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 shutdown_connections 2024-09-21T07:53:00.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.314+0000 7f9a59e00640 1 -- 172.21.15.88:0/1313477341 wait complete. 2024-09-21T07:53:00.339 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol17521 subvol_grp80740 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 -- 172.21.15.88:0/3371735412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0b36f0 msgr2=0x7fdbec0cb930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 --2- 172.21.15.88:0/3371735412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0b36f0 0x7fdbec0cb930 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7fdbe4005fd0 tx=0x7fdbe402a980 comp rx=0 tx=0).stop 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 -- 172.21.15.88:0/3371735412 shutdown_connections 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 --2- 172.21.15.88:0/3371735412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0b36f0 0x7fdbec0cb930 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 --2- 172.21.15.88:0/3371735412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec15ba50 0x7fdbec0b3120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 -- 172.21.15.88:0/3371735412 >> 172.21.15.88:0/3371735412 conn(0x7fdbec0b2540 msgr2=0x7fdbec0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 -- 172.21.15.88:0/3371735412 shutdown_connections 2024-09-21T07:53:00.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.561+0000 7fdbf1200640 1 -- 172.21.15.88:0/3371735412 wait complete. 2024-09-21T07:53:00.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.564+0000 7fdbf1200640 1 Processor -- start 2024-09-21T07:53:00.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.564+0000 7fdbf1200640 1 -- start start 2024-09-21T07:53:00.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbf1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b36f0 0x7fdbec1f6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbf1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 0x7fdbec15e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbf1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec15ba50 0x7fdbec0ad610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbf1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec06f8a0 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbf1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbec06f720 con 0x7fdbec15ba50 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbf1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdbec06fa20 con 0x7fdbec0b36f0 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbe9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 0x7fdbec15e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbe9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 0x7fdbec15e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53008/0 (socket says 172.21.15.88:53008) 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.565+0000 7fdbe9400640 1 -- 172.21.15.88:0/587120394 learned_addr learned my addr 172.21.15.88:0/587120394 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe9400640 1 -- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b36f0 msgr2=0x7fdbec1f6080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe9400640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b36f0 0x7fdbec1f6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe9400640 1 -- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec15ba50 msgr2=0x7fdbec0ad610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbea800640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec15ba50 0x7fdbec0ad610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe9400640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec15ba50 0x7fdbec0ad610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe9400640 1 -- 172.21.15.88:0/587120394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbec15d6c0 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbea800640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec15ba50 0x7fdbec0ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe9400640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 0x7fdbec15e760 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800d770 tx=0x7fdbd800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8017070 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbd8009d20 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbec15d9a0 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.566+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd80106c0 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.567+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbec15df60 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.568+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbec06ff40 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.569+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7fdbd801a070 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.571+0000 7fdbe2000640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdbcc079040 0x7fdbcc07b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:00.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.571+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fdbd809b7c0 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.571+0000 7fdbe9e00640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdbcc079040 0x7fdbcc07b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:00.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.572+0000 7fdbe9e00640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdbcc079040 0x7fdbcc07b550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc000f60 tx=0x7fdbdc0079b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:00.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.576+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdbd8063430 con 0x7fdbec0cc7d0 2024-09-21T07:53:00.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.788+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol17521", "group_name": "subvol_grp80740", "target": ["mon-mgr", ""]}) -- 0x7fdbec15d450 con 0x7fdbcc079040 2024-09-21T07:53:00.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.831+0000 7fdbe2000640 1 -- 172.21.15.88:0/587120394 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+74 (secure 0 0 0) 0x7fdbec15d450 con 0x7fdbcc079040 2024-09-21T07:53:00.833 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp80740/subvol17521/c2e0f0d8-3cfb-4784-85fe-e6fcf685f3bc 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.833+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdbcc079040 msgr2=0x7fdbcc07b550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.833+0000 7fdbf1200640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdbcc079040 0x7fdbcc07b550 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc000f60 tx=0x7fdbdc0079b0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.833+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 msgr2=0x7fdbec15e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.833+0000 7fdbf1200640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 0x7fdbec15e760 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7fdbd800d770 tx=0x7fdbd800dc30 comp rx=0 tx=0).stop 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 shutdown_connections 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdbcc079040 0x7fdbcc07b550 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbec15ba50 0x7fdbec0ad610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbec0cc7d0 0x7fdbec15e760 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 --2- 172.21.15.88:0/587120394 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbec0b36f0 0x7fdbec1f6080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:00.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 >> 172.21.15.88:0/587120394 conn(0x7fdbec0b2540 msgr2=0x7fdbec0afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:00.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 shutdown_connections 2024-09-21T07:53:00.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:00.834+0000 7fdbf1200640 1 -- 172.21.15.88:0/587120394 wait complete. 2024-09-21T07:53:00.857 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status cephfs subvol80472 2024-09-21T07:53:01.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 -- 172.21.15.88:0/3201840519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 msgr2=0x7f8c940b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 --2- 172.21.15.88:0/3201840519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940b3a50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f8c84009820 tx=0x7f8c8402a510 comp rx=0 tx=0).stop 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 -- 172.21.15.88:0/3201840519 shutdown_connections 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 --2- 172.21.15.88:0/3201840519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c940d4580 0x7f8c940d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 --2- 172.21.15.88:0/3201840519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940b3a50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 -- 172.21.15.88:0/3201840519 >> 172.21.15.88:0/3201840519 conn(0x7f8c9409d940 msgr2=0x7f8c9408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 -- 172.21.15.88:0/3201840519 shutdown_connections 2024-09-21T07:53:01.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.080+0000 7f8c99a00640 1 -- 172.21.15.88:0/3201840519 wait complete. 2024-09-21T07:53:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.082+0000 7f8c99a00640 1 Processor -- start 2024-09-21T07:53:01.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.083+0000 7f8c99a00640 1 -- start start 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c940d4580 0x7f8c941455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 0x7f8c941e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9406f3a0 con 0x7f8c94161e70 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c9406f220 con 0x7f8c940b9660 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c9406f520 con 0x7f8c940d4580 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 0x7f8c941e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 0x7f8c941e9660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53014/0 (socket says 172.21.15.88:53014) 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c99000640 1 -- 172.21.15.88:0/1523191776 learned_addr learned my addr 172.21.15.88:0/1523191776 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:01.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.084+0000 7f8c8be00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c940d4580 0x7f8c941455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c93e00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99000640 1 -- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c940d4580 msgr2=0x7f8c941455f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99000640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c940d4580 0x7f8c941455f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99000640 1 -- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 msgr2=0x7f8c940cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99000640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940cab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99000640 1 -- 172.21.15.88:0/1523191776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c94157f80 con 0x7f8c94161e70 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c93e00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:01.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99000640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 0x7f8c941e9660 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00ebf0 tx=0x7f8c8c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c003b70 con 0x7f8c94161e70 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c940b4160 con 0x7f8c94161e70 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c8c003d10 con 0x7f8c94161e70 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c8c004280 con 0x7f8c94161e70 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.085+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c941e8e60 con 0x7f8c94161e70 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.087+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c9406fc00 con 0x7f8c94161e70 2024-09-21T07:53:01.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.088+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f8c8c004420 con 0x7f8c94161e70 2024-09-21T07:53:01.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.089+0000 7f8c90c00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c70079010 0x7f8c7007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.089+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f8c8c09c390 con 0x7f8c94161e70 2024-09-21T07:53:01.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.089+0000 7f8c93e00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c70079010 0x7f8c7007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.090+0000 7f8c93e00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c70079010 0x7f8c7007b520 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c84034000 tx=0x7f8c84004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:01.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.093+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c9406fc00 con 0x7f8c94161e70 2024-09-21T07:53:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.315+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "cephfs", "clone_name": "subvol80472", "target": ["mon-mgr", ""]}) -- 0x7f8c9406f980 con 0x7f8c70079010 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.349+0000 7f8c90c00640 1 -- 172.21.15.88:0/1523191776 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+144 (secure 0 0 0) 0x7f8c9406f980 con 0x7f8c70079010 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: "status": { 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "pending", 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: "source": { 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: "volume": "fake", 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: "subvolume": "fake", 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: "snapshot": "fake" 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:01.351 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:01.352 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:53:01.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c70079010 msgr2=0x7f8c7007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c70079010 0x7f8c7007b520 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8c84034000 tx=0x7f8c84004730 comp rx=0 tx=0).stop 2024-09-21T07:53:01.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 msgr2=0x7f8c941e9660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 0x7f8c941e9660 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8c8c00ebf0 tx=0x7f8c8c00c6a0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 shutdown_connections 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c70079010 0x7f8c7007b520 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c940d4580 0x7f8c941455f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c94161e70 0x7f8c941e9660 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 --2- 172.21.15.88:0/1523191776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c940b9660 0x7f8c940cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 >> 172.21.15.88:0/1523191776 conn(0x7f8c9409d940 msgr2=0x7f8c940d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 shutdown_connections 2024-09-21T07:53:01.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.351+0000 7f8c99a00640 1 -- 172.21.15.88:0/1523191776 wait complete. 2024-09-21T07:53:01.379 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol62770 fake 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.602+0000 7f363fe00640 1 -- 172.21.15.88:0/1308174380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 msgr2=0x7f36400b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.602+0000 7f363fe00640 1 --2- 172.21.15.88:0/1308174380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f36400b47c0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3630005fd0 tx=0x7f363002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.602+0000 7f363fe00640 1 -- 172.21.15.88:0/1308174380 shutdown_connections 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.602+0000 7f363fe00640 1 --2- 172.21.15.88:0/1308174380 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36400af3c0 0x7f36401602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.602+0000 7f363fe00640 1 --2- 172.21.15.88:0/1308174380 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f36400b47c0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.602+0000 7f363fe00640 1 -- 172.21.15.88:0/1308174380 >> 172.21.15.88:0/1308174380 conn(0x7f36400b1dd0 msgr2=0x7f36400b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.603+0000 7f363fe00640 1 -- 172.21.15.88:0/1308174380 shutdown_connections 2024-09-21T07:53:01.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.603+0000 7f363fe00640 1 -- 172.21.15.88:0/1308174380 wait complete. 2024-09-21T07:53:01.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.605+0000 7f363fe00640 1 Processor -- start 2024-09-21T07:53:01.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.605+0000 7f363fe00640 1 -- start start 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.606+0000 7f363fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36400705b0 0x7f36401e6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.606+0000 7f363fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36400af3c0 0x7f3640152bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.606+0000 7f363fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f3640153eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.606+0000 7f363fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3640160b80 con 0x7f36400b3ec0 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.606+0000 7f363fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3640160a00 con 0x7f36400705b0 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.606+0000 7f363fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f364015cc60 con 0x7f36400af3c0 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f3640153eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f3640153eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53028/0 (socket says 172.21.15.88:53028) 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 -- 172.21.15.88:0/2954888515 learned_addr learned my addr 172.21.15.88:0/2954888515 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:01.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363e000640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36400af3c0 0x7f3640152bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363ea00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36400705b0 0x7f36401e6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 -- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36400af3c0 msgr2=0x7f3640152bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36400af3c0 0x7f3640152bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 -- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36400705b0 msgr2=0x7f36401e6a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36400705b0 0x7f36401e6a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 -- 172.21.15.88:0/2954888515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36401e7590 con 0x7f36400b3ec0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.607+0000 7f363f400640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f3640153eb0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f363800caa0 tx=0x7f363800cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.608+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800ad90 con 0x7f36400b3ec0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.608+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f363800f540 con 0x7f36400b3ec0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.608+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f363800f990 con 0x7f36400b3ec0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.608+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36401e72b0 con 0x7f36400b3ec0 2024-09-21T07:53:01.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.608+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36401f6e40 con 0x7f36400b3ec0 2024-09-21T07:53:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.609+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36400aed50 con 0x7f36400b3ec0 2024-09-21T07:53:01.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.610+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 21) ==== 101849+0+0 (secure 0 0 0) 0x7f363800fb30 con 0x7f36400b3ec0 2024-09-21T07:53:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.612+0000 7f3636a00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3620079010 0x7f362007b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.612+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f363809bfd0 con 0x7f36400b3ec0 2024-09-21T07:53:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.612+0000 7f363ea00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3620079010 0x7f362007b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:01.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.613+0000 7f363ea00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3620079010 0x7f362007b520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3630033000 tx=0x7f3630004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:01.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.616+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36400aed50 con 0x7f36400b3ec0 2024-09-21T07:53:01.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.829+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f363805f860 con 0x7f36400b3ec0 2024-09-21T07:53:01.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.840+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol62770", "snap_name": "fake", "target": ["mon-mgr", ""]}) -- 0x7f36401602b0 con 0x7f3620079010 2024-09-21T07:53:01.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.882+0000 7f3636a00640 1 -- 172.21.15.88:0/2954888515 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f36401602b0 con 0x7f3620079010 2024-09-21T07:53:01.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3620079010 msgr2=0x7f362007b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3620079010 0x7f362007b520 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3630033000 tx=0x7f3630004730 comp rx=0 tx=0).stop 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 msgr2=0x7f3640153eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f3640153eb0 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f363800caa0 tx=0x7f363800cf60 comp rx=0 tx=0).stop 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 shutdown_connections 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3620079010 0x7f362007b520 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36400af3c0 0x7f3640152bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36400b3ec0 0x7f3640153eb0 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 --2- 172.21.15.88:0/2954888515 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36400705b0 0x7f36401e6a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 >> 172.21.15.88:0/2954888515 conn(0x7f36400b1dd0 msgr2=0x7f36400b0580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 shutdown_connections 2024-09-21T07:53:01.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:01.885+0000 7f363fe00640 1 -- 172.21.15.88:0/2954888515 wait complete. 2024-09-21T07:53:01.916 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm cephfs subvol17521 fake subvol_grp80740 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.144+0000 7f2b76600640 1 -- 172.21.15.88:0/2883348752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b700747e0 msgr2=0x7f2b70153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.144+0000 7f2b76600640 1 --2- 172.21.15.88:0/2883348752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b700747e0 0x7f2b70153950 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f2b58005fd0 tx=0x7f2b5802a540 comp rx=0 tx=0).stop 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.144+0000 7f2b76600640 1 -- 172.21.15.88:0/2883348752 shutdown_connections 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.144+0000 7f2b76600640 1 --2- 172.21.15.88:0/2883348752 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b7006ddb0 0x7f2b7006e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.144+0000 7f2b76600640 1 --2- 172.21.15.88:0/2883348752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b700747e0 0x7f2b70153950 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.144+0000 7f2b76600640 1 -- 172.21.15.88:0/2883348752 >> 172.21.15.88:0/2883348752 conn(0x7f2b70073920 msgr2=0x7f2b70073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.145+0000 7f2b76600640 1 -- 172.21.15.88:0/2883348752 shutdown_connections 2024-09-21T07:53:02.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.145+0000 7f2b76600640 1 -- 172.21.15.88:0/2883348752 wait complete. 2024-09-21T07:53:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.147+0000 7f2b76600640 1 Processor -- start 2024-09-21T07:53:02.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.147+0000 7f2b76600640 1 -- start start 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b76600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 0x7f2b7014a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b76600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7006f960 0x7f2b700b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b76600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b700747e0 0x7f2b7015f210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b76600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7015d4b0 con 0x7f2b7006ddb0 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b76600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b7015d330 con 0x7f2b7006f960 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b76600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b7015d630 con 0x7f2b700747e0 2024-09-21T07:53:02.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.148+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 0x7f2b7014a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 0x7f2b7014a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53038/0 (socket says 172.21.15.88:53038) 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 -- 172.21.15.88:0/3577539835 learned_addr learned my addr 172.21.15.88:0/3577539835 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6fe00640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b700747e0 0x7f2b7015f210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 -- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b700747e0 msgr2=0x7f2b7015f210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b700747e0 0x7f2b7015f210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b66a00640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7006f960 0x7f2b700b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 -- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7006f960 msgr2=0x7f2b700b4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7006f960 0x7f2b700b4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 -- 172.21.15.88:0/3577539835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b7014d810 con 0x7f2b7006ddb0 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b66a00640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7006f960 0x7f2b700b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:02.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b6f400640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 0x7f2b7014a010 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c730 tx=0x7f2b6000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.149+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b60003b70 con 0x7f2b7006ddb0 2024-09-21T07:53:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.150+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b60003d30 con 0x7f2b7006ddb0 2024-09-21T07:53:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.150+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b6000aab0 con 0x7f2b7006ddb0 2024-09-21T07:53:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.150+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b701f5cd0 con 0x7f2b7006ddb0 2024-09-21T07:53:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.150+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b7014daf0 con 0x7f2b7006ddb0 2024-09-21T07:53:02.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.151+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b7015dbb0 con 0x7f2b7006ddb0 2024-09-21T07:53:02.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.152+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f2b6000ac50 con 0x7f2b7006ddb0 2024-09-21T07:53:02.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.153+0000 7f2b67e00640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b4c0790d0 0x7f2b4c07b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.153+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f2b6009b640 con 0x7f2b7006ddb0 2024-09-21T07:53:02.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.153+0000 7f2b66a00640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b4c0790d0 0x7f2b4c07b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:02.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.154+0000 7f2b66a00640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b4c0790d0 0x7f2b4c07b5e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b58004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:02.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.157+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b7006e700 con 0x7f2b7006ddb0 2024-09-21T07:53:02.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.375+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "cephfs", "sub_name": "subvol17521", "snap_name": "fake", "group_name": "subvol_grp80740", "target": ["mon-mgr", ""]}) -- 0x7f2b7015fdb0 con 0x7f2b4c0790d0 2024-09-21T07:53:02.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.418+0000 7f2b67e00640 1 -- 172.21.15.88:0/3577539835 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2b7015fdb0 con 0x7f2b4c0790d0 2024-09-21T07:53:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b4c0790d0 msgr2=0x7f2b4c07b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b4c0790d0 0x7f2b4c07b5e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b58033000 tx=0x7f2b58004730 comp rx=0 tx=0).stop 2024-09-21T07:53:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 msgr2=0x7f2b7014a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 0x7f2b7014a010 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c730 tx=0x7f2b6000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 shutdown_connections 2024-09-21T07:53:02.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b4c0790d0 0x7f2b4c07b5e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b700747e0 0x7f2b7015f210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b7006f960 0x7f2b700b4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 --2- 172.21.15.88:0/3577539835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b7006ddb0 0x7f2b7014a010 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 >> 172.21.15.88:0/3577539835 conn(0x7f2b70073920 msgr2=0x7f2b7014ddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 shutdown_connections 2024-09-21T07:53:02.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.421+0000 7f2b76600640 1 -- 172.21.15.88:0/3577539835 wait complete. 2024-09-21T07:53:02.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:02.446 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol62770/.meta) 2024-09-21T07:53:02.502 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:53:02.502 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:53:02.502 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:53:02.502 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol62770/9c75e8ac-f338-43fd-998b-3a9b61087a8d 2024-09-21T07:53:02.502 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:53:02.502 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:02.505 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:02.505 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp80740/subvol17521/.meta) 2024-09-21T07:53:02.563 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:53:02.563 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:53:02.563 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:53:02.563 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/subvol_grp80740/subvol17521/c2e0f0d8-3cfb-4784-85fe-e6fcf685f3bc 2024-09-21T07:53:02.563 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:53:02.563 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:02.566 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol62770 2024-09-21T07:53:02.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 -- 172.21.15.88:0/1900609985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40b9660 msgr2=0x7f71c40b3a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 --2- 172.21.15.88:0/1900609985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40b9660 0x7f71c40b3a50 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f71b40098e0 tx=0x7f71b402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 -- 172.21.15.88:0/1900609985 shutdown_connections 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 --2- 172.21.15.88:0/1900609985 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c40d4580 0x7f71c40d4990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 --2- 172.21.15.88:0/1900609985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40b9660 0x7f71c40b3a50 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 -- 172.21.15.88:0/1900609985 >> 172.21.15.88:0/1900609985 conn(0x7f71c409d940 msgr2=0x7f71c408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 -- 172.21.15.88:0/1900609985 shutdown_connections 2024-09-21T07:53:02.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.788+0000 7f71ca000640 1 -- 172.21.15.88:0/1900609985 wait complete. 2024-09-21T07:53:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.790+0000 7f71ca000640 1 Processor -- start 2024-09-21T07:53:02.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.790+0000 7f71ca000640 1 -- start start 2024-09-21T07:53:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71ca000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c40b9660 0x7f71c40cab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71ca000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 0x7f71c41455f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71ca000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c4161e70 0x7f71c41e9660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71ca000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71c406f3a0 con 0x7f71c40d4580 2024-09-21T07:53:02.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71ca000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f71c406f220 con 0x7f71c4161e70 2024-09-21T07:53:02.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71ca000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f71c406f520 con 0x7f71c40b9660 2024-09-21T07:53:02.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 0x7f71c41455f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:02.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71bbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 0x7f71c41455f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53060/0 (socket says 172.21.15.88:53060) 2024-09-21T07:53:02.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.792+0000 7f71bbe00640 1 -- 172.21.15.88:0/1752789694 learned_addr learned my addr 172.21.15.88:0/1752789694 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71bbe00640 1 -- 172.21.15.88:0/1752789694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c40b9660 msgr2=0x7f71c40cab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71bbe00640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c40b9660 0x7f71c40cab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71bbe00640 1 -- 172.21.15.88:0/1752789694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c4161e70 msgr2=0x7f71c41e9660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71bbe00640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c4161e70 0x7f71c41e9660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71bbe00640 1 -- 172.21.15.88:0/1752789694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71c4157f80 con 0x7f71c40d4580 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71bbe00640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 0x7f71c41455f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00c3a0 tx=0x7f71ac00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac017070 con 0x7f71c40d4580 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f71ac0053c0 con 0x7f71c40d4580 2024-09-21T07:53:02.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71ac003850 con 0x7f71c40d4580 2024-09-21T07:53:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71c40b4160 con 0x7f71c40d4580 2024-09-21T07:53:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.793+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71c41e8e60 con 0x7f71c40d4580 2024-09-21T07:53:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.794+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f71c406fc00 con 0x7f71c40d4580 2024-09-21T07:53:02.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.796+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f71ac01a070 con 0x7f71c40d4580 2024-09-21T07:53:02.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.797+0000 7f71c1600640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71a00790a0 0x7f71a007b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:02.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.797+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f71ac09ab60 con 0x7f71c40d4580 2024-09-21T07:53:02.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.797+0000 7f71c8c00640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71a00790a0 0x7f71a007b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:02.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.798+0000 7f71c8c00640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71a00790a0 0x7f71a007b5b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034000 tx=0x7f71b40046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:02.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:02.802+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f71c406fc00 con 0x7f71c40d4580 2024-09-21T07:53:03.018 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.016+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol62770", "target": ["mon-mgr", ""]}) -- 0x7f71c406f980 con 0x7f71a00790a0 2024-09-21T07:53:03.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.052+0000 7f71c1600640 1 -- 172.21.15.88:0/1752789694 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f71c406f980 con 0x7f71a00790a0 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71a00790a0 msgr2=0x7f71a007b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71a00790a0 0x7f71a007b5b0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f71b4034000 tx=0x7f71b40046c0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 msgr2=0x7f71c41455f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 0x7f71c41455f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f71ac00c3a0 tx=0x7f71ac00c860 comp rx=0 tx=0).stop 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 shutdown_connections 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f71a00790a0 0x7f71a007b5b0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f71c4161e70 0x7f71c41e9660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71c40d4580 0x7f71c41455f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 --2- 172.21.15.88:0/1752789694 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f71c40b9660 0x7f71c40cab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.054+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 >> 172.21.15.88:0/1752789694 conn(0x7f71c409d940 msgr2=0x7f71c40d1760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.055+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 shutdown_connections 2024-09-21T07:53:03.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.055+0000 7f71ca000640 1 -- 172.21.15.88:0/1752789694 wait complete. 2024-09-21T07:53:03.062 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.060+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.062 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.060+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.062 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.060+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.062 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.060+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.063 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.060+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.080 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.078+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.080 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.078+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.080 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.078+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.080 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.078+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.080 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:03.078+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:03.084 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol17521 subvol_grp80740 2024-09-21T07:53:03.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 -- 172.21.15.88:0/542436279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7041532e0 msgr2=0x7fa7041536f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 --2- 172.21.15.88:0/542436279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7041532e0 0x7fa7041536f0 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7fa70406ddf0 tx=0x7fa6fc02a980 comp rx=0 tx=0).stop 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 -- 172.21.15.88:0/542436279 shutdown_connections 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 --2- 172.21.15.88:0/542436279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7041532e0 0x7fa7041536f0 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 --2- 172.21.15.88:0/542436279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7040705b0 0x7fa7040af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 --2- 172.21.15.88:0/542436279 >> v2:172.21.15.175:3300/0 conn(0x7fa7040b3ec0 0x7fa7040b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 -- 172.21.15.88:0/542436279 >> 172.21.15.88:0/542436279 conn(0x7fa7040b1dd0 msgr2=0x7fa7040b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 -- 172.21.15.88:0/542436279 shutdown_connections 2024-09-21T07:53:03.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.310+0000 7fa70aa00640 1 -- 172.21.15.88:0/542436279 wait complete. 2024-09-21T07:53:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.312+0000 7fa70aa00640 1 Processor -- start 2024-09-21T07:53:03.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.313+0000 7fa70aa00640 1 -- start start 2024-09-21T07:53:03.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa70aa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7040705b0 0x7fa7041511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa70aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 0x7fa7041f5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa70aa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7041532e0 0x7fa704074810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa70aa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa704160860 con 0x7fa7040b3ec0 2024-09-21T07:53:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa70aa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7041606e0 con 0x7fa7040705b0 2024-09-21T07:53:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa70aa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7041609e0 con 0x7fa7041532e0 2024-09-21T07:53:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa708c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 0x7fa7041f5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa708c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 0x7fa7041f5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53120/0 (socket says 172.21.15.88:53120) 2024-09-21T07:53:03.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.314+0000 7fa708c00640 1 -- 172.21.15.88:0/1735202682 learned_addr learned my addr 172.21.15.88:0/1735202682 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa709600640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7040705b0 0x7fa7041511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa708c00640 1 -- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7041532e0 msgr2=0x7fa704074810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa70a000640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7041532e0 0x7fa704074810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa708c00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7041532e0 0x7fa704074810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa708c00640 1 -- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7040705b0 msgr2=0x7fa7041511f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa708c00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7040705b0 0x7fa7041511f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa708c00640 1 -- 172.21.15.88:0/1735202682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa704151730 con 0x7fa7040b3ec0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa709600640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7040705b0 0x7fa7041511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa70a000640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7041532e0 0x7fa704074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa708c00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 0x7fa7041f5ce0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000d660 tx=0x7fa6f000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0017070 con 0x7fa7040b3ec0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f0005310 con 0x7fa7040b3ec0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7041590b0 con 0x7fa7040b3ec0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.315+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f0009820 con 0x7fa7040b3ec0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.316+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa704159670 con 0x7fa7040b3ec0 2024-09-21T07:53:03.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.317+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7040b47c0 con 0x7fa7040b3ec0 2024-09-21T07:53:03.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.318+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7fa6f00099c0 con 0x7fa7040b3ec0 2024-09-21T07:53:03.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.319+0000 7fa701600640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e40790d0 0x7fa6e407b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.319+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fa6f009b6f0 con 0x7fa7040b3ec0 2024-09-21T07:53:03.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.319+0000 7fa709600640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e40790d0 0x7fa6e407b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.320+0000 7fa709600640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e40790d0 0x7fa6e407b5e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009aa0 tx=0x7fa6f8005db0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:03.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.324+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa704151730 con 0x7fa7040b3ec0 2024-09-21T07:53:03.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.534+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol17521", "group_name": "subvol_grp80740", "target": ["mon-mgr", ""]}) -- 0x7fa70409de20 con 0x7fa6e40790d0 2024-09-21T07:53:03.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.572+0000 7fa701600640 1 -- 172.21.15.88:0/1735202682 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa70409de20 con 0x7fa6e40790d0 2024-09-21T07:53:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e40790d0 msgr2=0x7fa6e407b5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e40790d0 0x7fa6e407b5e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa6f8009aa0 tx=0x7fa6f8005db0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 msgr2=0x7fa7041f5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 0x7fa7041f5ce0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7fa6f000d660 tx=0x7fa6f000db20 comp rx=0 tx=0).stop 2024-09-21T07:53:03.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 shutdown_connections 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e40790d0 0x7fa6e407b5e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7041532e0 0x7fa704074810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7040b3ec0 0x7fa7041f5ce0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 --2- 172.21.15.88:0/1735202682 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7040705b0 0x7fa7041511f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 >> 172.21.15.88:0/1735202682 conn(0x7fa7040b1dd0 msgr2=0x7fa70406f9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 shutdown_connections 2024-09-21T07:53:03.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.575+0000 7fa70aa00640 1 -- 172.21.15.88:0/1735202682 wait complete. 2024-09-21T07:53:03.604 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol80472 2024-09-21T07:53:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.828+0000 7fb292a00640 1 -- 172.21.15.88:0/2210488457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 msgr2=0x7fb28c156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.828+0000 7fb292a00640 1 --2- 172.21.15.88:0/2210488457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 0x7fb28c156ca0 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7fb280009830 tx=0x7fb28002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.829+0000 7fb292a00640 1 -- 172.21.15.88:0/2210488457 shutdown_connections 2024-09-21T07:53:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.829+0000 7fb292a00640 1 --2- 172.21.15.88:0/2210488457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 0x7fb28c156ca0 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.829+0000 7fb292a00640 1 -- 172.21.15.88:0/2210488457 >> 172.21.15.88:0/2210488457 conn(0x7fb28c06d620 msgr2=0x7fb28c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.829+0000 7fb292a00640 1 -- 172.21.15.88:0/2210488457 shutdown_connections 2024-09-21T07:53:03.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.829+0000 7fb292a00640 1 -- 172.21.15.88:0/2210488457 wait complete. 2024-09-21T07:53:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.831+0000 7fb292a00640 1 Processor -- start 2024-09-21T07:53:03.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.831+0000 7fb292a00640 1 -- start start 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 0x7fb28c1590e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28c0cd1d0 0x7fb28c1f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28c15d6e0 0x7fb28c1f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c0af7e0 con 0x7fb28c0b3f30 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb28c070370 con 0x7fb28c15d6e0 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb28c0af960 con 0x7fb28c0cd1d0 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb291600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 0x7fb28c1590e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb291600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 0x7fb28c1590e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53140/0 (socket says 172.21.15.88:53140) 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb291600640 1 -- 172.21.15.88:0/2537063688 learned_addr learned my addr 172.21.15.88:0/2537063688 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb290c00640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28c0cd1d0 0x7fb28c1f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.833+0000 7fb292000640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28c15d6e0 0x7fb28c1f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb291600640 1 -- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28c0cd1d0 msgr2=0x7fb28c1f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb291600640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb28c0cd1d0 0x7fb28c1f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb291600640 1 -- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28c15d6e0 msgr2=0x7fb28c1f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb291600640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28c15d6e0 0x7fb28c1f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb291600640 1 -- 172.21.15.88:0/2537063688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb28c1f5600 con 0x7fb28c0b3f30 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb292000640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb28c15d6e0 0x7fb28c1f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:03.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.834+0000 7fb291600640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb28c0b3f30 0x7fb28c1590e0 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7fb280009950 tx=0x7fb280033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.836+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280031d60 con 0x7fb28c0b3f30 2024-09-21T07:53:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.836+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2800330d0 con 0x7fb28c0b3f30 2024-09-21T07:53:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.836+0000 7fb292a00640 1 -- 172.21.15.88:0/2537063688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb28c1f58e0 con 0x7fb28c0b3f30 2024-09-21T07:53:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.836+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb280033520 con 0x7fb28c0b3f30 2024-09-21T07:53:03.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.836+0000 7fb292a00640 1 -- 172.21.15.88:0/2537063688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb28c1f5ea0 con 0x7fb28c0b3f30 2024-09-21T07:53:03.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.837+0000 7fb292a00640 1 -- 172.21.15.88:0/2537063688 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb28c156ca0 con 0x7fb28c0b3f30 2024-09-21T07:53:03.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.838+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7fb2800035f0 con 0x7fb28c0b3f30 2024-09-21T07:53:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.839+0000 7fb289600640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb26c0790a0 0x7fb26c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.839+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fb2800bcc50 con 0x7fb28c0b3f30 2024-09-21T07:53:03.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.839+0000 7fb290c00640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb26c0790a0 0x7fb26c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:03.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.840+0000 7fb290c00640 1 --2- 172.21.15.88:0/2537063688 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb26c0790a0 0x7fb26c07b5b0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fb28c15ead0 tx=0x7fb278009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:03.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:03.844+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb28003f030 con 0x7fb28c0b3f30 2024-09-21T07:53:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.057+0000 7fb292a00640 1 -- 172.21.15.88:0/2537063688 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol80472", "target": ["mon-mgr", ""]}) -- 0x7fb28c1567e0 con 0x7fb26c0790a0 2024-09-21T07:53:04.081 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:04.079+0000 7faca9000640 -1 mgr.server reply reply (11) Resource temporarily unavailable subvolume 'subvol80472' is not ready for operation rm (use --force to override) 2024-09-21T07:53:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.079+0000 7fb289600640 1 -- 172.21.15.88:0/2537063688 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -11 subvolume 'subvol80472' is not ready for operation rm (use --force to override)) ==== 87+0+0 (secure 0 0 0) 0x7fb28c1567e0 con 0x7fb26c0790a0 2024-09-21T07:53:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:Error EAGAIN: subvolume 'subvol80472' is not ready for operation rm (use --force to override) 2024-09-21T07:53:04.097 DEBUG:teuthology.orchestra.run:got remote process result: 11 2024-09-21T07:53:04.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:04.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol80472/.meta) 2024-09-21T07:53:04.152 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:53:04.152 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:53:04.152 INFO:teuthology.orchestra.run.smithi088.stdout:type = clone 2024-09-21T07:53:04.152 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol80472/c3be4aee-26eb-4dab-bb74-0c2a25ea087f 2024-09-21T07:53:04.152 INFO:teuthology.orchestra.run.smithi088.stdout:state = pending 2024-09-21T07:53:04.152 INFO:teuthology.orchestra.run.smithi088.stdout:[source] 2024-09-21T07:53:04.153 INFO:teuthology.orchestra.run.smithi088.stdout:volume = fake 2024-09-21T07:53:04.153 INFO:teuthology.orchestra.run.smithi088.stdout:subvolume = fake 2024-09-21T07:53:04.153 INFO:teuthology.orchestra.run.smithi088.stdout:snapshot = fake 2024-09-21T07:53:04.155 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol80472 --force 2024-09-21T07:53:04.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.377+0000 7fa51a800640 1 -- 172.21.15.88:0/2832978570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 msgr2=0x7fa51406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.377+0000 7fa51a800640 1 --2- 172.21.15.88:0/2832978570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa51406d1b0 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc005fd0 tx=0x7fa4fc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.378+0000 7fa51a800640 1 -- 172.21.15.88:0/2832978570 shutdown_connections 2024-09-21T07:53:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.378+0000 7fa51a800640 1 --2- 172.21.15.88:0/2832978570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa51406d1b0 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.378+0000 7fa51a800640 1 --2- 172.21.15.88:0/2832978570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140cc830 0x7fa5140b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.378+0000 7fa51a800640 1 -- 172.21.15.88:0/2832978570 >> 172.21.15.88:0/2832978570 conn(0x7fa5140b2540 msgr2=0x7fa5140d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.378+0000 7fa51a800640 1 -- 172.21.15.88:0/2832978570 shutdown_connections 2024-09-21T07:53:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.378+0000 7fa51a800640 1 -- 172.21.15.88:0/2832978570 wait complete. 2024-09-21T07:53:04.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.380+0000 7fa51a800640 1 Processor -- start 2024-09-21T07:53:04.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.380+0000 7fa51a800640 1 -- start start 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.381+0000 7fa51a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa5141f5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.381+0000 7fa51a800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140af100 0x7fa5141f9170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.381+0000 7fa51a800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5140cc830 0x7fa5141f96b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.381+0000 7fa51a800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140b3600 con 0x7fa51406cda0 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.381+0000 7fa51a800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa5140b3480 con 0x7fa5140cc830 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.381+0000 7fa51a800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa5140b3780 con 0x7fa5140af100 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa5141f5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa5141f5200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53166/0 (socket says 172.21.15.88:53166) 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 -- 172.21.15.88:0/4058010796 learned_addr learned my addr 172.21.15.88:0/4058010796 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa50aa00640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140af100 0x7fa5141f9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513e00640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5140cc830 0x7fa5141f96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 -- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140af100 msgr2=0x7fa5141f9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140af100 0x7fa5141f9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 -- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5140cc830 msgr2=0x7fa5141f96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5140cc830 0x7fa5141f96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 -- 172.21.15.88:0/4058010796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa51415f9b0 con 0x7fa51406cda0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513e00640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5140cc830 0x7fa5141f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.382+0000 7fa513400640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa5141f5200 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa50400eb30 tx=0x7fa50400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.383+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504010430 con 0x7fa51406cda0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.383+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa5040105d0 con 0x7fa51406cda0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.383+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa504010a20 con 0x7fa51406cda0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.383+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa514154110 con 0x7fa51406cda0 2024-09-21T07:53:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.383+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa51415ebb0 con 0x7fa51406cda0 2024-09-21T07:53:04.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.384+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5140b3ca0 con 0x7fa51406cda0 2024-09-21T07:53:04.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.385+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7fa504010bc0 con 0x7fa51406cda0 2024-09-21T07:53:04.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.386+0000 7fa50be00640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4f00790a0 0x7fa4f007b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.386+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fa50409c580 con 0x7fa51406cda0 2024-09-21T07:53:04.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.386+0000 7fa50aa00640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4f00790a0 0x7fa4f007b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.387+0000 7fa50aa00640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4f00790a0 0x7fa4f007b5b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc033000 tx=0x7fa4fc004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.390+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa51415f9b0 con 0x7fa51406cda0 2024-09-21T07:53:04.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.601+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol80472", "force": true, "target": ["mon-mgr", ""]}) -- 0x7fa51415edf0 con 0x7fa4f00790a0 2024-09-21T07:53:04.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.633+0000 7fa50be00640 1 -- 172.21.15.88:0/4058010796 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa4fc005610 con 0x7fa4f00790a0 2024-09-21T07:53:04.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4f00790a0 msgr2=0x7fa4f007b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4f00790a0 0x7fa4f007b5b0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa4fc033000 tx=0x7fa4fc004730 comp rx=0 tx=0).stop 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 msgr2=0x7fa5141f5200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa5141f5200 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7fa50400eb30 tx=0x7fa50400c2d0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 shutdown_connections 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4f00790a0 0x7fa4f007b5b0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa5140af100 0x7fa5141f9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa5140cc830 0x7fa5141f96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 --2- 172.21.15.88:0/4058010796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa51406cda0 0x7fa5141f5200 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 >> 172.21.15.88:0/4058010796 conn(0x7fa5140b2540 msgr2=0x7fa51406f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 shutdown_connections 2024-09-21T07:53:04.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.636+0000 7fa51a800640 1 -- 172.21.15.88:0/4058010796 wait complete. 2024-09-21T07:53:04.665 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls cephfs 2024-09-21T07:53:04.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 -- 172.21.15.88:0/767815305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104145a40 msgr2=0x7f8104073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 --2- 172.21.15.88:0/767815305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104145a40 0x7f8104073540 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f81041464e0 tx=0x7f80f002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 -- 172.21.15.88:0/767815305 shutdown_connections 2024-09-21T07:53:04.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 --2- 172.21.15.88:0/767815305 >> v2:172.21.15.175:3300/0 conn(0x7f8104073a80 0x7f8104160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 --2- 172.21.15.88:0/767815305 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104145a40 0x7f8104073540 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 --2- 172.21.15.88:0/767815305 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8104145550 0x7f810406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.891+0000 7f810a400640 1 -- 172.21.15.88:0/767815305 >> 172.21.15.88:0/767815305 conn(0x7f810406d1c0 msgr2=0x7f810406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:04.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.892+0000 7f810a400640 1 -- 172.21.15.88:0/767815305 shutdown_connections 2024-09-21T07:53:04.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.892+0000 7f810a400640 1 -- 172.21.15.88:0/767815305 wait complete. 2024-09-21T07:53:04.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.894+0000 7f810a400640 1 Processor -- start 2024-09-21T07:53:04.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.894+0000 7f810a400640 1 -- start start 2024-09-21T07:53:04.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.895+0000 7f810a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 0x7f810413ca30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.895+0000 7f810a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8104145550 0x7f81041f5c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.895+0000 7f810a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8104145a40 0x7f81040b2510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.895+0000 7f810a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8104160c20 con 0x7f8104073a80 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.895+0000 7f810a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8104160aa0 con 0x7f8104145a40 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.895+0000 7f810a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8104160da0 con 0x7f8104145550 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 0x7f810413ca30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 0x7f810413ca30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:53174/0 (socket says 172.21.15.88:53174) 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 -- 172.21.15.88:0/139727998 learned_addr learned my addr 172.21.15.88:0/139727998 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8102a00640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8104145550 0x7f81041f5c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103e00640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8104145a40 0x7f81040b2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 -- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8104145550 msgr2=0x7f81041f5c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8104145550 0x7f81041f5c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 -- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8104145a40 msgr2=0x7f81040b2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8104145a40 0x7f81040b2510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:04.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.896+0000 7f8103400640 1 -- 172.21.15.88:0/139727998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f810413f4a0 con 0x7f8104073a80 2024-09-21T07:53:04.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.897+0000 7f8103400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 0x7f810413ca30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f80f400d740 tx=0x7f80f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:04.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.897+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f4017070 con 0x7f8104073a80 2024-09-21T07:53:04.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.897+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f4009d20 con 0x7f8104073a80 2024-09-21T07:53:04.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.897+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f40106c0 con 0x7f8104073a80 2024-09-21T07:53:04.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.897+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81041f67b0 con 0x7f8104073a80 2024-09-21T07:53:04.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.897+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f810413f760 con 0x7f8104073a80 2024-09-21T07:53:04.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.898+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f810406fb40 con 0x7f8104073a80 2024-09-21T07:53:04.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.899+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7f80f401a070 con 0x7f8104073a80 2024-09-21T07:53:04.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.900+0000 7f80fb400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e40790a0 0x7f80e407b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:04.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.900+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f80f409b670 con 0x7f8104073a80 2024-09-21T07:53:04.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.901+0000 7f8102a00640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e40790a0 0x7f80e407b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:04.903 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.901+0000 7f8102a00640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e40790a0 0x7f80e407b5b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80f0033000 tx=0x7f80f0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:04.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:04.905+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80f4062e10 con 0x7f8104073a80 2024-09-21T07:53:05.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.121+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "cephfs", "target": ["mon-mgr", ""]}) -- 0x7f8104146310 con 0x7f80e40790a0 2024-09-21T07:53:05.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.125+0000 7f80fb400640 1 -- 172.21.15.88:0/139727998 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f8104146310 con 0x7f80e40790a0 2024-09-21T07:53:05.127 INFO:teuthology.orchestra.run.smithi088.stdout:[] 2024-09-21T07:53:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e40790a0 msgr2=0x7f80e407b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e40790a0 0x7f80e407b5b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f80f0033000 tx=0x7f80f0004730 comp rx=0 tx=0).stop 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 msgr2=0x7f810413ca30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 0x7f810413ca30 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f80f400d740 tx=0x7f80f400dc00 comp rx=0 tx=0).stop 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 shutdown_connections 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e40790a0 0x7f80e407b5b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8104145550 0x7f81041f5c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8104145a40 0x7f81040b2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 --2- 172.21.15.88:0/139727998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8104073a80 0x7f810413ca30 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:05.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 >> 172.21.15.88:0/139727998 conn(0x7f810406d1c0 msgr2=0x7f81040b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 shutdown_connections 2024-09-21T07:53:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.127+0000 7f810a400640 1 -- 172.21.15.88:0/139727998 wait complete. 2024-09-21T07:53:05.152 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:53:05.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:53:05.237 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:53:05.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:05.246 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:53:05.246 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:05.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:05.414 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:53:05.414 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:05.414 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:05.414 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:05.449 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:05.450 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:05.525 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:53:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532c00640 1 -- 172.21.15.88:0/3004236619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5140051c0 msgr2=0x7fc51400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532c00640 1 --2- 172.21.15.88:0/3004236619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5140051c0 0x7fc51400b9c0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7fc524005fd0 tx=0x7fc514005830 comp rx=0 tx=0).stop 2024-09-21T07:53:05.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532200640 1 -- 172.21.15.88:0/3004236619 reap_dead start 2024-09-21T07:53:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532c00640 1 -- 172.21.15.88:0/3004236619 shutdown_connections 2024-09-21T07:53:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532c00640 1 -- 172.21.15.88:0/3004236619 >> 172.21.15.88:0/3004236619 conn(0x7fc52c0b1e40 msgr2=0x7fc52c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532c00640 1 -- 172.21.15.88:0/3004236619 shutdown_connections 2024-09-21T07:53:05.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.786+0000 7fc532c00640 1 -- 172.21.15.88:0/3004236619 wait complete. 2024-09-21T07:53:05.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.788+0000 7fc532c00640 1 Processor -- start 2024-09-21T07:53:05.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.788+0000 7fc532c00640 1 -- start start 2024-09-21T07:53:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc532c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140051c0 0x7fc52c155e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc532c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 0x7fc52c1f52f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:05.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc532c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c1f5830 0x7fc52c1f5c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc532c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c162930 con 0x7fc52c1505b0 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc532c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc52c1627b0 con 0x7fc5140051c0 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc532c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc52c162ab0 con 0x7fc52c1f5830 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 0x7fc52c1f52f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 0x7fc52c1f52f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35148/0 (socket says 172.21.15.88:35148) 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 -- 172.21.15.88:0/942324112 learned_addr learned my addr 172.21.15.88:0/942324112 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc531800640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140051c0 0x7fc52c155e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 -- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c1f5830 msgr2=0x7fc52c1f5c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c1f5830 0x7fc52c1f5c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 -- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140051c0 msgr2=0x7fc52c155e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140051c0 0x7fc52c155e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc530e00640 1 -- 172.21.15.88:0/942324112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc52c150c50 con 0x7fc52c1505b0 2024-09-21T07:53:05.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.790+0000 7fc531800640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140051c0 0x7fc52c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.791+0000 7fc530e00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 0x7fc52c1f52f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc52c071280 tx=0x7fc51800fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.791+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc518004230 con 0x7fc52c1505b0 2024-09-21T07:53:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.791+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc518017070 con 0x7fc52c1505b0 2024-09-21T07:53:05.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.791+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc52c1583a0 con 0x7fc52c1505b0 2024-09-21T07:53:05.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.791+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc51800ba50 con 0x7fc52c1505b0 2024-09-21T07:53:05.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.791+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc52c158960 con 0x7fc52c1505b0 2024-09-21T07:53:05.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.792+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc52c155780 con 0x7fc52c1505b0 2024-09-21T07:53:05.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.793+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 22) ==== 101900+0+0 (secure 0 0 0) 0x7fc51800bbf0 con 0x7fc52c1505b0 2024-09-21T07:53:05.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.794+0000 7fc529600640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc50c0790a0 0x7fc50c07b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:05.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.794+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fc51809bf20 con 0x7fc52c1505b0 2024-09-21T07:53:05.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.794+0000 7fc531800640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc50c0790a0 0x7fc50c07b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:05.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.795+0000 7fc531800640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc50c0790a0 0x7fc50c07b5b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc520001a30 tx=0x7fc520000ef0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:05.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.799+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc518063600 con 0x7fc52c1505b0 2024-09-21T07:53:05.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:05.848+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc51805f870 con 0x7fc52c1505b0 2024-09-21T07:53:06.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.002+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fc52c1509c0 con 0x7fc52c1505b0 2024-09-21T07:53:06.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.003+0000 7fc529600640 1 -- 172.21.15.88:0/942324112 <== mon.0 v2:172.21.15.88:3300/0 8 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) ==== 80+0+0 (secure 0 0 0) 0x7fc5180853a0 con 0x7fc52c1505b0 2024-09-21T07:53:06.005 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc50c0790a0 msgr2=0x7fc50c07b5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc50c0790a0 0x7fc50c07b5b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fc520001a30 tx=0x7fc520000ef0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 msgr2=0x7fc52c1f52f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 0x7fc52c1f52f0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7fc52c071280 tx=0x7fc51800fbe0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 shutdown_connections 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc50c0790a0 0x7fc50c07b5b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc52c1f5830 0x7fc52c1f5c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc52c1505b0 0x7fc52c1f52f0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 --2- 172.21.15.88:0/942324112 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5140051c0 0x7fc52c155e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.005+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 >> 172.21.15.88:0/942324112 conn(0x7fc52c0b1e40 msgr2=0x7fc52c160810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:06.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.006+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 shutdown_connections 2024-09-21T07:53:06.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.006+0000 7fc532c00640 1 -- 172.21.15.88:0/942324112 wait complete. 2024-09-21T07:53:06.030 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:53:06.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:06.031 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:06.124 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:53:06.124 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:06.125 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:53:06.174 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:06.174 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:06.174 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:06.211 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:06.211 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:06.286 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 -- 172.21.15.88:0/984485160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e040705b0 msgr2=0x7f0e040709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 --2- 172.21.15.88:0/984485160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e040705b0 0x7f0e040709c0 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f0df0005fd0 tx=0x7f0df002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 -- 172.21.15.88:0/984485160 shutdown_connections 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 --2- 172.21.15.88:0/984485160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e040705b0 0x7f0e040709c0 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 --2- 172.21.15.88:0/984485160 >> v2:172.21.15.175:3300/0 conn(0x7f0e040cfc80 0x7f0e04156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 -- 172.21.15.88:0/984485160 >> 172.21.15.88:0/984485160 conn(0x7f0e0406d3d0 msgr2=0x7f0e040b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.502+0000 7f0e0be00640 1 -- 172.21.15.88:0/984485160 shutdown_connections 2024-09-21T07:53:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.503+0000 7f0e0be00640 1 -- 172.21.15.88:0/984485160 wait complete. 2024-09-21T07:53:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.505+0000 7f0e0be00640 1 Processor -- start 2024-09-21T07:53:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.505+0000 7f0e0be00640 1 -- start start 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e040705b0 0x7f0e040c6ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e040cfc80 0x7f0e040c6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e0be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 0x7f0e040c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e0be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e040cced0 con 0x7f0e0415ce30 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e0be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e040d0170 con 0x7f0e040705b0 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e0be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e040cd050 con 0x7f0e040cfc80 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e09e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 0x7f0e040c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.506+0000 7f0e09e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 0x7f0e040c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35166/0 (socket says 172.21.15.88:35166) 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 -- 172.21.15.88:0/443176844 learned_addr learned my addr 172.21.15.88:0/443176844 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e08a00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e040cfc80 0x7f0e040c6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09400640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e040705b0 0x7f0e040c6ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 -- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e040cfc80 msgr2=0x7f0e040c6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e040cfc80 0x7f0e040c6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 -- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e040705b0 msgr2=0x7f0e040c6ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e040705b0 0x7f0e040c6ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 -- 172.21.15.88:0/443176844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e040c4550 con 0x7f0e0415ce30 2024-09-21T07:53:06.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e09e00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 0x7f0e040c6650 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc002230 tx=0x7f0dfc00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc003b70 con 0x7f0e0415ce30 2024-09-21T07:53:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0dfc003d10 con 0x7f0e0415ce30 2024-09-21T07:53:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0dfc005330 con 0x7f0e0415ce30 2024-09-21T07:53:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.507+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e040c4830 con 0x7f0e0415ce30 2024-09-21T07:53:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.508+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e040c5620 con 0x7f0e0415ce30 2024-09-21T07:53:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.509+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e040cd5b0 con 0x7f0e0415ce30 2024-09-21T07:53:06.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.510+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0dfc0056e0 con 0x7f0e0415ce30 2024-09-21T07:53:06.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.511+0000 7f0e01600640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0de40791c0 0x7f0de407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:06.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.511+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7f0dfc09b960 con 0x7f0e0415ce30 2024-09-21T07:53:06.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.511+0000 7f0e09400640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0de40791c0 0x7f0de407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:06.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.512+0000 7f0e09400640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0de40791c0 0x7f0de407b6d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0df8005fd0 tx=0x7f0df8005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:06.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.515+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e04070f00 con 0x7f0e0415ce30 2024-09-21T07:53:06.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.713+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f0e040afa70 con 0x7f0e0415ce30 2024-09-21T07:53:06.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.714+0000 7f0e01600640 1 -- 172.21.15.88:0/443176844 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v120) ==== 80+0+0 (secure 0 0 0) 0x7f0dfc063310 con 0x7f0e0415ce30 2024-09-21T07:53:06.716 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0de40791c0 msgr2=0x7f0de407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0de40791c0 0x7f0de407b6d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0df8005fd0 tx=0x7f0df8005850 comp rx=0 tx=0).stop 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 msgr2=0x7f0e040c6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 0x7f0e040c6650 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f0dfc002230 tx=0x7f0dfc00cb90 comp rx=0 tx=0).stop 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 shutdown_connections 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0de40791c0 0x7f0de407b6d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e040cfc80 0x7f0e040c6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e0415ce30 0x7f0e040c6650 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 --2- 172.21.15.88:0/443176844 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e040705b0 0x7f0e040c6ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:06.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.716+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 >> 172.21.15.88:0/443176844 conn(0x7f0e0406d3d0 msgr2=0x7f0e040d0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:06.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.717+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 shutdown_connections 2024-09-21T07:53:06.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:06.717+0000 7f0e0be00640 1 -- 172.21.15.88:0/443176844 wait complete. 2024-09-21T07:53:06.745 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:53:06.745 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:06.745 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:53:06.807 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:53:06.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:06.807 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:53:06.843 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:06.843 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:06.843 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:06.879 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:06.879 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:06.879 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:06.879 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:06.919 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:53:06.919 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:07.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.141+0000 7fccca600640 1 -- 172.21.15.88:0/2132666710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc40af630 msgr2=0x7fccc4073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.141+0000 7fccca600640 1 --2- 172.21.15.88:0/2132666710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc40af630 0x7fccc4073500 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7fccc40b2cb0 tx=0x7fccb402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.141+0000 7fccca600640 1 -- 172.21.15.88:0/2132666710 shutdown_connections 2024-09-21T07:53:07.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.141+0000 7fccca600640 1 --2- 172.21.15.88:0/2132666710 >> v2:172.21.15.175:3300/0 conn(0x7fccc4073a40 0x7fccc406f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.141+0000 7fccca600640 1 --2- 172.21.15.88:0/2132666710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc40af630 0x7fccc4073500 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.141+0000 7fccca600640 1 -- 172.21.15.88:0/2132666710 >> 172.21.15.88:0/2132666710 conn(0x7fccc406d730 msgr2=0x7fccc40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:07.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.142+0000 7fccca600640 1 -- 172.21.15.88:0/2132666710 shutdown_connections 2024-09-21T07:53:07.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.142+0000 7fccca600640 1 -- 172.21.15.88:0/2132666710 wait complete. 2024-09-21T07:53:07.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.144+0000 7fccca600640 1 Processor -- start 2024-09-21T07:53:07.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.144+0000 7fccca600640 1 -- start start 2024-09-21T07:53:07.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.145+0000 7fccca600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 0x7fccc414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.145+0000 7fccca600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc4073a40 0x7fccc41610b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.145+0000 7fccca600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccc40af630 0x7fccc41615f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.145+0000 7fccca600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fccc4162c70 con 0x7fccc406f5e0 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.145+0000 7fccca600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fccc40b4400 con 0x7fccc4073a40 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.145+0000 7fccca600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fccc4162df0 con 0x7fccc40af630 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 0x7fccc414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 0x7fccc414d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35184/0 (socket says 172.21.15.88:35184) 2024-09-21T07:53:07.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 -- 172.21.15.88:0/2170117189 learned_addr learned my addr 172.21.15.88:0/2170117189 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 -- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccc40af630 msgr2=0x7fccc41615f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3e00640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccc40af630 0x7fccc41615f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc2a00640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc4073a40 0x7fccc41610b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccc40af630 0x7fccc41615f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 -- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc4073a40 msgr2=0x7fccc41610b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc4073a40 0x7fccc41610b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 -- 172.21.15.88:0/2170117189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fccc414d690 con 0x7fccc406f5e0 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc2a00640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc4073a40 0x7fccc41610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:07.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3e00640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccc40af630 0x7fccc41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:07.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.146+0000 7fccc3400640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 0x7fccc414d150 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fccb4034040 tx=0x7fccb40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:07.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.147+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb403e030 con 0x7fccc406f5e0 2024-09-21T07:53:07.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.147+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fccb40047a0 con 0x7fccc406f5e0 2024-09-21T07:53:07.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.147+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fccb4035040 con 0x7fccc406f5e0 2024-09-21T07:53:07.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.147+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fccc41503d0 con 0x7fccc406f5e0 2024-09-21T07:53:07.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.147+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fccc41f94f0 con 0x7fccc406f5e0 2024-09-21T07:53:07.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.148+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fccc41f9030 con 0x7fccc406f5e0 2024-09-21T07:53:07.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.149+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fccb4041030 con 0x7fccc406f5e0 2024-09-21T07:53:07.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.150+0000 7fccbb400640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcca40791c0 0x7fcca407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.150+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fccb40bc0b0 con 0x7fccc406f5e0 2024-09-21T07:53:07.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.150+0000 7fccc2a00640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcca40791c0 0x7fcca407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.151+0000 7fccc2a00640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcca40791c0 0x7fcca407b6d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fccc409e330 tx=0x7fccb0008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:07.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.155+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fccc41f9030 con 0x7fccc406f5e0 2024-09-21T07:53:07.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.376+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fccc406a7e0 con 0x7fccc406f5e0 2024-09-21T07:53:07.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.377+0000 7fccbb400640 1 -- 172.21.15.88:0/2170117189 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5228 (secure 0 0 0) 0x7fccb4083c60 con 0x7fccc406f5e0 2024-09-21T07:53:07.379 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:07.379 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":81,"btime":"2024-09-21T07:52:57:795632+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:52:41.524287+0000","modified":"2024-09-21T07:52:44.548337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6834/3361463212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3361463212},{"type":"v1","addr":"172.21.15.88:6834","nonce":3361463212}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":9}]} 2024-09-21T07:53:07.379 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 81 2024-09-21T07:53:07.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcca40791c0 msgr2=0x7fcca407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcca40791c0 0x7fcca407b6d0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fccc409e330 tx=0x7fccb0008040 comp rx=0 tx=0).stop 2024-09-21T07:53:07.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 msgr2=0x7fccc414d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 0x7fccc414d150 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fccb4034040 tx=0x7fccb40039e0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 shutdown_connections 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcca40791c0 0x7fcca407b6d0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fccc40af630 0x7fccc41615f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fccc4073a40 0x7fccc41610b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 --2- 172.21.15.88:0/2170117189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fccc406f5e0 0x7fccc414d150 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 >> 172.21.15.88:0/2170117189 conn(0x7fccc406d730 msgr2=0x7fccc40b0670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 shutdown_connections 2024-09-21T07:53:07.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.379+0000 7fccca600640 1 -- 172.21.15.88:0/2170117189 wait complete. 2024-09-21T07:53:07.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 -- 172.21.15.88:0/3469957494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 msgr2=0x7fa4f00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/3469957494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f00ad4f0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fa4f0072d90 tx=0x7fa4dc02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 -- 172.21.15.88:0/3469957494 shutdown_connections 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/3469957494 >> v2:172.21.15.175:3300/0 conn(0x7fa4f0160310 0x7fa4f0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/3469957494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f00ad4f0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/3469957494 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4f015d760 0x7fa4f01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.623+0000 7fa4f7a00640 1 -- 172.21.15.88:0/3469957494 >> 172.21.15.88:0/3469957494 conn(0x7fa4f006d730 msgr2=0x7fa4f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:07.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.624+0000 7fa4f7a00640 1 -- 172.21.15.88:0/3469957494 shutdown_connections 2024-09-21T07:53:07.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.624+0000 7fa4f7a00640 1 -- 172.21.15.88:0/3469957494 wait complete. 2024-09-21T07:53:07.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.626+0000 7fa4f7a00640 1 Processor -- start 2024-09-21T07:53:07.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.626+0000 7fa4f7a00640 1 -- start start 2024-09-21T07:53:07.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.627+0000 7fa4f7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f01f63c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.627+0000 7fa4f7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4f015d760 0x7fa4f01487f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.627+0000 7fa4f7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f0160310 0x7fa4f015b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.627+0000 7fa4f7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00b4da0 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.627+0000 7fa4f7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f00b4c20 con 0x7fa4f0160310 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.627+0000 7fa4f7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa4f0160c60 con 0x7fa4f015d760 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f01f63c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f01f63c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35198/0 (socket says 172.21.15.88:35198) 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 -- 172.21.15.88:0/146714517 learned_addr learned my addr 172.21.15.88:0/146714517 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f0160310 0x7fa4f015b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4efe00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4f015d760 0x7fa4f01487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 -- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4f015d760 msgr2=0x7fa4f01487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4f015d760 0x7fa4f01487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 -- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f0160310 msgr2=0x7fa4f015b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f0160310 0x7fa4f015b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 -- 172.21.15.88:0/146714517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f01f6900 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.628+0000 7fa4f5000640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f01f63c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e7f0 tx=0x7fa4e400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:07.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.629+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e400ccc0 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.629+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa4e400ce80 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.629+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f0156fc0 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.629+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4005280 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.629+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f0074b70 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.630+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f01535c0 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.631+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fa4e4005420 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.632+0000 7fa4ecc00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4d00791c0 0x7fa4d007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:07.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.632+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(120..120 src has 1..120) ==== 5709+0+0 (secure 0 0 0) 0x7fa4e409c1c0 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.632+0000 7fa4efe00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4d00791c0 0x7fa4d007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:07.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.633+0000 7fa4efe00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4d00791c0 0x7fa4d007b6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc033000 tx=0x7fa4dc004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:07.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.637+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa4e40635f0 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.871+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa4f015c760 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.871+0000 7fa4ecc00640 1 -- 172.21.15.88:0/146714517 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5228 (secure 0 0 0) 0x7fa4e406a030 con 0x7fa4f00ad0e0 2024-09-21T07:53:07.874 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:07.874 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":81,"btime":"2024-09-21T07:52:57:795632+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:52:41.524287+0000","modified":"2024-09-21T07:52:44.548337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6834/3361463212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3361463212},{"type":"v1","addr":"172.21.15.88:6834","nonce":3361463212}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":9}]} 2024-09-21T07:53:07.874 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 81 2024-09-21T07:53:07.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4d00791c0 msgr2=0x7fa4d007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4d00791c0 0x7fa4d007b6d0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fa4dc033000 tx=0x7fa4dc004730 comp rx=0 tx=0).stop 2024-09-21T07:53:07.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 msgr2=0x7fa4f01f63c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:07.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f01f63c0 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400e7f0 tx=0x7fa4e400ecb0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 shutdown_connections 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa4d00791c0 0x7fa4d007b6d0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa4f015d760 0x7fa4f01487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa4f0160310 0x7fa4f015b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 --2- 172.21.15.88:0/146714517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4f00ad0e0 0x7fa4f01f63c0 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 >> 172.21.15.88:0/146714517 conn(0x7fa4f006d730 msgr2=0x7fa4f015ecd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 shutdown_connections 2024-09-21T07:53:07.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:07.874+0000 7fa4f7a00640 1 -- 172.21.15.88:0/146714517 wait complete. 2024-09-21T07:53:07.902 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 -- 172.21.15.88:0/2018838222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100ad3c0 msgr2=0x7f5d100ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 --2- 172.21.15.88:0/2018838222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100ad3c0 0x7f5d100ad7d0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7f5cfc005fd0 tx=0x7f5cfc02b110 comp rx=0 tx=0).stop 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 -- 172.21.15.88:0/2018838222 shutdown_connections 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 --2- 172.21.15.88:0/2018838222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100ad3c0 0x7f5d100ad7d0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 --2- 172.21.15.88:0/2018838222 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d100704b0 0x7f5d10073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 -- 172.21.15.88:0/2018838222 >> 172.21.15.88:0/2018838222 conn(0x7f5d1006d730 msgr2=0x7f5d100b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 -- 172.21.15.88:0/2018838222 shutdown_connections 2024-09-21T07:53:08.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.129+0000 7f5d17000640 1 -- 172.21.15.88:0/2018838222 wait complete. 2024-09-21T07:53:08.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.132+0000 7f5d17000640 1 Processor -- start 2024-09-21T07:53:08.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.132+0000 7f5d17000640 1 -- start start 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d17000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d100704b0 0x7f5d1014d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d17000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d100ad3c0 0x7f5d101f9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d17000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 0x7f5d101f9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d17000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5d100712b0 con 0x7f5d100add10 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d17000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d10071130 con 0x7f5d100704b0 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d17000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d100b4520 con 0x7f5d100ad3c0 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d15000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 0x7f5d101f9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d15000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 0x7f5d101f9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35218/0 (socket says 172.21.15.88:35218) 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.133+0000 7f5d15000640 1 -- 172.21.15.88:0/1846574347 learned_addr learned my addr 172.21.15.88:0/1846574347 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d0f400640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d100ad3c0 0x7f5d101f9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d0fe00640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d100704b0 0x7f5d1014d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d15000640 1 -- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d100ad3c0 msgr2=0x7f5d101f9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d15000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d100ad3c0 0x7f5d101f9570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d15000640 1 -- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d100704b0 msgr2=0x7f5d1014d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d15000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d100704b0 0x7f5d1014d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d15000640 1 -- 172.21.15.88:0/1846574347 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d1014d9e0 con 0x7f5d100add10 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d0fe00640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d100704b0 0x7f5d1014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d15000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 0x7f5d101f9ab0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800ba10 tx=0x7f5d0800bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d08019070 con 0x7f5d100add10 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d0800cc30 con 0x7f5d100add10 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d080146a0 con 0x7f5d100add10 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.134+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d1009e310 con 0x7f5d100add10 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.135+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d1009b5f0 con 0x7f5d100add10 2024-09-21T07:53:08.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.136+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d1009e080 con 0x7f5d100add10 2024-09-21T07:53:08.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.137+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f5d0801c070 con 0x7f5d100add10 2024-09-21T07:53:08.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.138+0000 7f5d07e00640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cf00791f0 0x7f5cf007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.139+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5765+0+0 (secure 0 0 0) 0x7f5d0809b5d0 con 0x7f5d100add10 2024-09-21T07:53:08.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.139+0000 7f5d0fe00640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cf00791f0 0x7f5cf007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.139+0000 7f5d0fe00640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cf00791f0 0x7f5cf007b700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d000058c0 tx=0x7f5d00005c90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.143+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5d1014d9e0 con 0x7f5d100add10 2024-09-21T07:53:08.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.339+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f5d100b49d0 con 0x7f5d100add10 2024-09-21T07:53:08.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.340+0000 7f5d07e00640 1 -- 172.21.15.88:0/1846574347 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16810 (secure 0 0 0) 0x7f5d0800ce20 con 0x7f5d100add10 2024-09-21T07:53:08.343 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:08.343 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":121,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:07.843245+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:52:39.432702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T07:52:40.488474+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:08.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cf00791f0 msgr2=0x7f5cf007b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cf00791f0 0x7f5cf007b700 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f5d000058c0 tx=0x7f5d00005c90 comp rx=0 tx=0).stop 2024-09-21T07:53:08.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 msgr2=0x7f5d101f9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 0x7f5d101f9ab0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7f5d0800ba10 tx=0x7f5d0800bed0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 shutdown_connections 2024-09-21T07:53:08.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cf00791f0 0x7f5cf007b700 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d100ad3c0 0x7f5d101f9570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d100add10 0x7f5d101f9ab0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 --2- 172.21.15.88:0/1846574347 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d100704b0 0x7f5d1014d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 >> 172.21.15.88:0/1846574347 conn(0x7f5d1006d730 msgr2=0x7f5d100b1d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 shutdown_connections 2024-09-21T07:53:08.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.343+0000 7f5d17000640 1 -- 172.21.15.88:0/1846574347 wait complete. 2024-09-21T07:53:08.369 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:53:08.369 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4b200640 1 -- 172.21.15.88:0/597670815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f30003a30 msgr2=0x7f4f30003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4b200640 1 --2- 172.21.15.88:0/597670815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f30003a30 0x7f4f30003ea0 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c009b50 tx=0x7f4f30005830 comp rx=0 tx=0).stop 2024-09-21T07:53:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4a800640 1 -- 172.21.15.88:0/597670815 reap_dead start 2024-09-21T07:53:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4b200640 1 -- 172.21.15.88:0/597670815 shutdown_connections 2024-09-21T07:53:08.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4b200640 1 -- 172.21.15.88:0/597670815 >> 172.21.15.88:0/597670815 conn(0x7f4f4c0b1e40 msgr2=0x7f4f4c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4b200640 1 -- 172.21.15.88:0/597670815 shutdown_connections 2024-09-21T07:53:08.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.588+0000 7f4f4b200640 1 -- 172.21.15.88:0/597670815 wait complete. 2024-09-21T07:53:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.590+0000 7f4f4b200640 1 Processor -- start 2024-09-21T07:53:08.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.590+0000 7f4f4b200640 1 -- start start 2024-09-21T07:53:08.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f30003a30 0x7f4f4c073f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 0x7f4f4c0cb7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f4c09c2c0 0x7f4f4c09c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c070cf0 con 0x7f4f4c1f6090 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4f4c06fbb0 con 0x7f4f30003a30 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4f4c0af170 con 0x7f4f4c09c2c0 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 0x7f4f4c0cb7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 0x7f4f4c0cb7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35238/0 (socket says 172.21.15.88:35238) 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 -- 172.21.15.88:0/4153652566 learned_addr learned my addr 172.21.15.88:0/4153652566 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4a800640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f4c09c2c0 0x7f4f4c09c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f49e00640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f30003a30 0x7f4f4c073f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 -- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f4c09c2c0 msgr2=0x7f4f4c09c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f4c09c2c0 0x7f4f4c09c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 -- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f30003a30 msgr2=0x7f4f4c073f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f30003a30 0x7f4f4c073f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f41400640 1 -- 172.21.15.88:0/4153652566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f4c09cc10 con 0x7f4f4c1f6090 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f49e00640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f30003a30 0x7f4f4c073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.592+0000 7f4f4a800640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f4c09c2c0 0x7f4f4c09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.593+0000 7f4f41400640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 0x7f4f4c0cb7d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4f34003f80 tx=0x7f4f340016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.593+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34015070 con 0x7f4f4c1f6090 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.593+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4f34011900 con 0x7f4f4c1f6090 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.593+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f34011d10 con 0x7f4f4c1f6090 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.593+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f4c073000 con 0x7f4f4c1f6090 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.593+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f4c0732a0 con 0x7f4f4c1f6090 2024-09-21T07:53:08.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.594+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f4c0b2370 con 0x7f4f4c1f6090 2024-09-21T07:53:08.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.596+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f4f34018070 con 0x7f4f4c1f6090 2024-09-21T07:53:08.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.597+0000 7f4f42a00640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4f280791c0 0x7f4f2807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:08.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.597+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5765+0+0 (secure 0 0 0) 0x7f4f3409c6e0 con 0x7f4f4c1f6090 2024-09-21T07:53:08.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.597+0000 7f4f49e00640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4f280791c0 0x7f4f2807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:08.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.598+0000 7f4f49e00640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4f280791c0 0x7f4f2807b6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c035000 tx=0x7f4f3c02d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:08.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.601+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4f4c0b2370 con 0x7f4f4c1f6090 2024-09-21T07:53:08.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.825+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f4c15d670 con 0x7f4f4c1f6090 2024-09-21T07:53:08.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.826+0000 7f4f42a00640 1 -- 172.21.15.88:0/4153652566 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5228 (secure 0 0 0) 0x7f4f4c09cc10 con 0x7f4f4c1f6090 2024-09-21T07:53:08.828 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:08.828 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":81,"btime":"2024-09-21T07:52:57:795632+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:52:41.524287+0000","modified":"2024-09-21T07:52:44.548337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6834/3361463212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3361463212},{"type":"v1","addr":"172.21.15.88:6834","nonce":3361463212}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":9}]} 2024-09-21T07:53:08.828 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 81 2024-09-21T07:53:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.828+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4f280791c0 msgr2=0x7f4f2807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.828+0000 7f4f4b200640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4f280791c0 0x7f4f2807b6d0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4f3c035000 tx=0x7f4f3c02d040 comp rx=0 tx=0).stop 2024-09-21T07:53:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.828+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 msgr2=0x7f4f4c0cb7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.828+0000 7f4f4b200640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 0x7f4f4c0cb7d0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x7f4f34003f80 tx=0x7f4f340016f0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 shutdown_connections 2024-09-21T07:53:08.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4f280791c0 0x7f4f2807b6d0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4f4c09c2c0 0x7f4f4c09c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f4c1f6090 0x7f4f4c0cb7d0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 --2- 172.21.15.88:0/4153652566 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4f30003a30 0x7f4f4c073f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 >> 172.21.15.88:0/4153652566 conn(0x7f4f4c0b1e40 msgr2=0x7f4f4c068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 shutdown_connections 2024-09-21T07:53:08.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:08.829+0000 7f4f4b200640 1 -- 172.21.15.88:0/4153652566 wait complete. 2024-09-21T07:53:08.853 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 -- 172.21.15.88:0/1942239348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c00ad0e0 msgr2=0x7fd2c00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/1942239348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c00ad0e0 0x7fd2c00ad4f0 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7fd2a8005fd0 tx=0x7fd2a802a980 comp rx=0 tx=0).stop 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 -- 172.21.15.88:0/1942239348 shutdown_connections 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/1942239348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c00ad0e0 0x7fd2c00ad4f0 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/1942239348 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c015d760 0x7fd2c01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 -- 172.21.15.88:0/1942239348 >> 172.21.15.88:0/1942239348 conn(0x7fd2c006d730 msgr2=0x7fd2c00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 -- 172.21.15.88:0/1942239348 shutdown_connections 2024-09-21T07:53:09.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.072+0000 7fd2c6c00640 1 -- 172.21.15.88:0/1942239348 wait complete. 2024-09-21T07:53:09.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.075+0000 7fd2c6c00640 1 Processor -- start 2024-09-21T07:53:09.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.075+0000 7fd2c6c00640 1 -- start start 2024-09-21T07:53:09.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.076+0000 7fd2c6c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c00ad0e0 0x7fd2c0149b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.076+0000 7fd2c6c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c015d760 0x7fd2c0070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.076+0000 7fd2c6c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 0x7fd2c0070640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.076+0000 7fd2c6c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c006f220 con 0x7fd2c0160310 2024-09-21T07:53:09.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.076+0000 7fd2c6c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2c006f0a0 con 0x7fd2c015d760 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.076+0000 7fd2c6c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2c00b4c20 con 0x7fd2c00ad0e0 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 0x7fd2c0070640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 0x7fd2c0070640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35248/0 (socket says 172.21.15.88:35248) 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 -- 172.21.15.88:0/3397748446 learned_addr learned my addr 172.21.15.88:0/3397748446 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2b7400640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c015d760 0x7fd2c0070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2bfe00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c00ad0e0 0x7fd2c0149b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 -- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c00ad0e0 msgr2=0x7fd2c0149b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c00ad0e0 0x7fd2c0149b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 -- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c015d760 msgr2=0x7fd2c0070100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c015d760 0x7fd2c0070100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 -- 172.21.15.88:0/3397748446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2c0075040 con 0x7fd2c0160310 2024-09-21T07:53:09.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.077+0000 7fd2c4c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 0x7fd2c0070640 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800b410 tx=0x7fd2b800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:09.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.078+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8015030 con 0x7fd2c0160310 2024-09-21T07:53:09.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.078+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2b8004020 con 0x7fd2c0160310 2024-09-21T07:53:09.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.078+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2b8011630 con 0x7fd2c0160310 2024-09-21T07:53:09.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.078+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2c01562d0 con 0x7fd2c0160310 2024-09-21T07:53:09.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.078+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2c0156890 con 0x7fd2c0160310 2024-09-21T07:53:09.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.079+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2c0156040 con 0x7fd2c0160310 2024-09-21T07:53:09.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.080+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd2b8018070 con 0x7fd2c0160310 2024-09-21T07:53:09.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.081+0000 7fd2bcc00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd29c0791f0 0x7fd29c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.081+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5765+0+0 (secure 0 0 0) 0x7fd2b809b230 con 0x7fd2c0160310 2024-09-21T07:53:09.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.081+0000 7fd2bfe00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd29c0791f0 0x7fd29c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.082+0000 7fd2bfe00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd29c0791f0 0x7fd29c07b700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005fd0 tx=0x7fd2b000a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:09.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.086+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd2c0075040 con 0x7fd2c0160310 2024-09-21T07:53:09.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.306+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd2c01573c0 con 0x7fd2c0160310 2024-09-21T07:53:09.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.307+0000 7fd2bcc00640 1 -- 172.21.15.88:0/3397748446 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) ==== 94+0+5228 (secure 0 0 0) 0x7fd2b8062670 con 0x7fd2c0160310 2024-09-21T07:53:09.309 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:09.309 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":81,"btime":"2024-09-21T07:52:57:795632+0000","default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75}],"filesystems":[{"mdsmap":{"epoch":80,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:52:41.524287+0000","modified":"2024-09-21T07:52:44.548337+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7509},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7509":{"gid":7509,"name":"c","rank":0,"incarnation":76,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6834/3361463212","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3361463212},{"type":"v1","addr":"172.21.15.88:6834","nonce":3361463212}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7509,"qdb_cluster":[7509]},"id":9}]} 2024-09-21T07:53:09.309 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 81 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd29c0791f0 msgr2=0x7fd29c07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd29c0791f0 0x7fd29c07b700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fd2b0005fd0 tx=0x7fd2b000a250 comp rx=0 tx=0).stop 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 msgr2=0x7fd2c0070640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 0x7fd2c0070640 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7fd2b800b410 tx=0x7fd2b800b8d0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 shutdown_connections 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd29c0791f0 0x7fd29c07b700 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2c0160310 0x7fd2c0070640 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2c015d760 0x7fd2c0070100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 --2- 172.21.15.88:0/3397748446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd2c00ad0e0 0x7fd2c0149b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.309+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 >> 172.21.15.88:0/3397748446 conn(0x7fd2c006d730 msgr2=0x7fd2c00d0270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.310+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 shutdown_connections 2024-09-21T07:53:09.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.310+0000 7fd2c6c00640 1 -- 172.21.15.88:0/3397748446 wait complete. 2024-09-21T07:53:09.334 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:09.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 -- 172.21.15.88:0/3136509626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb280af630 msgr2=0x7feb28073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 --2- 172.21.15.88:0/3136509626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb280af630 0x7feb28073500 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7feb1c005fd0 tx=0x7feb1c02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 -- 172.21.15.88:0/3136509626 shutdown_connections 2024-09-21T07:53:09.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 --2- 172.21.15.88:0/3136509626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb280af630 0x7feb28073500 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 -- 172.21.15.88:0/3136509626 >> 172.21.15.88:0/3136509626 conn(0x7feb2806d730 msgr2=0x7feb280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 -- 172.21.15.88:0/3136509626 shutdown_connections 2024-09-21T07:53:09.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.553+0000 7feb2f800640 1 -- 172.21.15.88:0/3136509626 wait complete. 2024-09-21T07:53:09.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.555+0000 7feb2f800640 1 Processor -- start 2024-09-21T07:53:09.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.556+0000 7feb2f800640 1 -- start start 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2f800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb2806f7b0 0x7feb2809c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 0x7feb2814f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2f800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb280af630 0x7feb2814fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb280b3df0 con 0x7feb28073a40 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2f800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb280b3c70 con 0x7feb280af630 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2f800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb280b3f70 con 0x7feb2806f7b0 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 0x7feb2814f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 0x7feb2814f6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35262/0 (socket says 172.21.15.88:35262) 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb27e00640 1 -- 172.21.15.88:0/2550079567 learned_addr learned my addr 172.21.15.88:0/2550079567 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.557+0000 7feb2d800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb280af630 0x7feb2814fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb2ce00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb2806f7b0 0x7feb2809c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb27e00640 1 -- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb2806f7b0 msgr2=0x7feb2809c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb27e00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb2806f7b0 0x7feb2809c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb27e00640 1 -- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb280af630 msgr2=0x7feb2814fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb27e00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb280af630 0x7feb2814fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb27e00640 1 -- 172.21.15.88:0/2550079567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb28159620 con 0x7feb28073a40 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb2d800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb280af630 0x7feb2814fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:09.561 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb27e00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 0x7feb2814f6f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7feb1400b3f0 tx=0x7feb1400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:09.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14015020 con 0x7feb28073a40 2024-09-21T07:53:09.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb14011920 con 0x7feb28073a40 2024-09-21T07:53:09.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb28151500 con 0x7feb28073a40 2024-09-21T07:53:09.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.558+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb14011d70 con 0x7feb28073a40 2024-09-21T07:53:09.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.559+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb28150d00 con 0x7feb28073a40 2024-09-21T07:53:09.562 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.560+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb280b44a0 con 0x7feb28073a40 2024-09-21T07:53:09.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.561+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7feb14018020 con 0x7feb28073a40 2024-09-21T07:53:09.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.562+0000 7feb24c00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb080791f0 0x7feb0807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:09.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.562+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5765+0+0 (secure 0 0 0) 0x7feb1409b650 con 0x7feb28073a40 2024-09-21T07:53:09.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.562+0000 7feb2ce00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb080791f0 0x7feb0807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:09.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.563+0000 7feb2ce00640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb080791f0 0x7feb0807b700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb1c033000 tx=0x7feb1c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:09.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.566+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb1401f030 con 0x7feb28073a40 2024-09-21T07:53:09.755 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.753+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb280aea30 con 0x7feb28073a40 2024-09-21T07:53:09.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.754+0000 7feb24c00640 1 -- 172.21.15.88:0/2550079567 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v121) ==== 74+0+16810 (secure 0 0 0) 0x7feb14062fc0 con 0x7feb28073a40 2024-09-21T07:53:09.757 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:09.757 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":121,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:07.843245+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:52:39.432702+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"119","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2024-09-21T07:52:40.488474+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"121","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":121,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:09.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb080791f0 msgr2=0x7feb0807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb080791f0 0x7feb0807b700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7feb1c033000 tx=0x7feb1c004730 comp rx=0 tx=0).stop 2024-09-21T07:53:09.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 msgr2=0x7feb2814f6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:09.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 0x7feb2814f6f0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7feb1400b3f0 tx=0x7feb1400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.759 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 shutdown_connections 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb080791f0 0x7feb0807b700 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb280af630 0x7feb2814fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb28073a40 0x7feb2814f6f0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 --2- 172.21.15.88:0/2550079567 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb2806f7b0 0x7feb2809c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 >> 172.21.15.88:0/2550079567 conn(0x7feb2806d730 msgr2=0x7feb2809d8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 shutdown_connections 2024-09-21T07:53:09.760 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:09.757+0000 7feb2f800640 1 -- 172.21.15.88:0/2550079567 wait complete. 2024-09-21T07:53:09.781 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 -- 172.21.15.88:0/2517429195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8760070ab0 msgr2=0x7f87600b3550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 --2- 172.21.15.88:0/2517429195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8760070ab0 0x7f87600b3550 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f8760070520 tx=0x7f875802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 -- 172.21.15.88:0/2517429195 shutdown_connections 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 --2- 172.21.15.88:0/2517429195 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8760070ab0 0x7f87600b3550 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 --2- 172.21.15.88:0/2517429195 >> v2:172.21.15.175:3300/0 conn(0x7f87600af8d0 0x7f8760070570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 -- 172.21.15.88:0/2517429195 >> 172.21.15.88:0/2517429195 conn(0x7f876006d3d0 msgr2=0x7f87600b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.001+0000 7f8768400640 1 -- 172.21.15.88:0/2517429195 shutdown_connections 2024-09-21T07:53:10.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.002+0000 7f8768400640 1 -- 172.21.15.88:0/2517429195 wait complete. 2024-09-21T07:53:10.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.004+0000 7f8768400640 1 Processor -- start 2024-09-21T07:53:10.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.004+0000 7f8768400640 1 -- start start 2024-09-21T07:53:10.007 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8768400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8760070ab0 0x7f87600984b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8768400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87600af8d0 0x7f87600c73a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8768400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 0x7f87601531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8768400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87600b3e20 con 0x7f87600cbfc0 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8768400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f87600b3ca0 con 0x7f8760070ab0 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8768400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f87600b3fa0 con 0x7f87600af8d0 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8766400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 0x7f87601531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.005+0000 7f8766400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 0x7f87601531e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35276/0 (socket says 172.21.15.88:35276) 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 -- 172.21.15.88:0/3033714974 learned_addr learned my addr 172.21.15.88:0/3033714974 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8765a00640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8760070ab0 0x7f87600984b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.008 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8765000640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87600af8d0 0x7f87600c73a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 -- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87600af8d0 msgr2=0x7f87600c73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87600af8d0 0x7f87600c73a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 -- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8760070ab0 msgr2=0x7f87600984b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8760070ab0 0x7f87600984b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 -- 172.21.15.88:0/3033714974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87600c78e0 con 0x7f87600cbfc0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8765000640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87600af8d0 0x7f87600c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.006+0000 7f8766400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 0x7f87601531e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f87580098f0 tx=0x7f8758033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.007+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758031b50 con 0x7f87600cbfc0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.007+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8758031e00 con 0x7f87600cbfc0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.007+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8758005280 con 0x7f87600cbfc0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.007+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87600c9be0 con 0x7f87600cbfc0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.007+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87600ca1a0 con 0x7f87600cbfc0 2024-09-21T07:53:10.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.008+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87600b3550 con 0x7f87600cbfc0 2024-09-21T07:53:10.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.009+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f8758005420 con 0x7f87600cbfc0 2024-09-21T07:53:10.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.010+0000 7f875d600640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f87400791f0 0x7f874007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.010+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(121..121 src has 1..121) ==== 5765+0+0 (secure 0 0 0) 0x7f87580bc8e0 con 0x7f87600cbfc0 2024-09-21T07:53:10.012 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.010+0000 7f8765a00640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f87400791f0 0x7f874007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.013 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.011+0000 7f8765a00640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f87400791f0 0x7f874007b700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87540012e0 tx=0x7f8754009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:10.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.014+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f875803f030 con 0x7f87600cbfc0 2024-09-21T07:53:10.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.237+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f87600c9970 con 0x7f87600cbfc0 2024-09-21T07:53:10.240 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:53:10.238+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:53:10.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.240+0000 7f875d600640 1 -- 172.21.15.88:0/3033714974 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v82) ==== 161+0+0 (secure 0 0 0) 0x7f8758083e40 con 0x7f87600cbfc0 2024-09-21T07:53:10.242 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:53:10.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f87400791f0 msgr2=0x7f874007b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f87400791f0 0x7f874007b700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f87540012e0 tx=0x7f8754009040 comp rx=0 tx=0).stop 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 msgr2=0x7f87601531e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 0x7f87601531e0 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f87580098f0 tx=0x7f8758033ba0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 shutdown_connections 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f87400791f0 0x7f874007b700 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87600af8d0 0x7f87600c73a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87600cbfc0 0x7f87601531e0 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 --2- 172.21.15.88:0/3033714974 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8760070ab0 0x7f87600984b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 >> 172.21.15.88:0/3033714974 conn(0x7f876006d3d0 msgr2=0x7f8760074cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 shutdown_connections 2024-09-21T07:53:10.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.243+0000 7f8768400640 1 -- 172.21.15.88:0/3033714974 wait complete. 2024-09-21T07:53:10.272 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:53:10.387 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 -- 172.21.15.88:0/2308332868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc073c50 msgr2=0x7f78cc0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 --2- 172.21.15.88:0/2308332868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc073c50 0x7f78cc0cd610 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005fd0 tx=0x7f78c402a960 comp rx=0 tx=0).stop 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 -- 172.21.15.88:0/2308332868 shutdown_connections 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 --2- 172.21.15.88:0/2308332868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc073c50 0x7f78cc0cd610 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 --2- 172.21.15.88:0/2308332868 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78cc145550 0x7f78cc06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 -- 172.21.15.88:0/2308332868 >> 172.21.15.88:0/2308332868 conn(0x7f78cc06d1c0 msgr2=0x7f78cc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 -- 172.21.15.88:0/2308332868 shutdown_connections 2024-09-21T07:53:10.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.496+0000 7f78d4600640 1 -- 172.21.15.88:0/2308332868 wait complete. 2024-09-21T07:53:10.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.498+0000 7f78d4600640 1 Processor -- start 2024-09-21T07:53:10.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.499+0000 7f78d4600640 1 -- start start 2024-09-21T07:53:10.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d4600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78cc145550 0x7f78cc0ab680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d4600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 0x7f78cc0736c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d4600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78cc073c00 0x7f78cc15bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d4600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0ce1e0 con 0x7f78cc145a40 2024-09-21T07:53:10.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d4600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc0ce060 con 0x7f78cc073c00 2024-09-21T07:53:10.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d4600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78cc0ce360 con 0x7f78cc145550 2024-09-21T07:53:10.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 0x7f78cc0736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 0x7f78cc0736c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35310/0 (socket says 172.21.15.88:35310) 2024-09-21T07:53:10.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.500+0000 7f78d1200640 1 -- 172.21.15.88:0/267244191 learned_addr learned my addr 172.21.15.88:0/267244191 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:10.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1c00640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78cc145550 0x7f78cc0ab680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d2600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78cc073c00 0x7f78cc15bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1200640 1 -- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78cc145550 msgr2=0x7f78cc0ab680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1200640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78cc145550 0x7f78cc0ab680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1200640 1 -- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78cc073c00 msgr2=0x7f78cc15bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:10.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1200640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78cc073c00 0x7f78cc15bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1200640 1 -- 172.21.15.88:0/267244191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc0ab2f0 con 0x7f78cc145a40 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d1200640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 0x7f78cc0736c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c3c0 tx=0x7f78b800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8017070 con 0x7f78cc145a40 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78b80035f0 con 0x7f78cc145a40 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b8003a40 con 0x7f78cc145a40 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.501+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1f8db0 con 0x7f78cc145a40 2024-09-21T07:53:10.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.502+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1f9590 con 0x7f78cc145a40 2024-09-21T07:53:10.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.502+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78cc1f9020 con 0x7f78cc145a40 2024-09-21T07:53:10.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.504+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f78b801a070 con 0x7f78cc145a40 2024-09-21T07:53:10.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.505+0000 7f78c9600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78ac0791f0 0x7f78ac07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:10.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.505+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5823+0+0 (secure 0 0 0) 0x7f78b809abb0 con 0x7f78cc145a40 2024-09-21T07:53:10.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.505+0000 7f78d1c00640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78ac0791f0 0x7f78ac07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:10.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.506+0000 7f78d1c00640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78ac0791f0 0x7f78ac07b700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f78c0005fd0 tx=0x7f78c0005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:10.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.509+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f78b8061f60 con 0x7f78cc145a40 2024-09-21T07:53:10.736 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:10.734+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f78cc06a7f0 con 0x7f78cc145a40 2024-09-21T07:53:11.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.242+0000 7f78c9600640 1 -- 172.21.15.88:0/267244191 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v83) ==== 104+0+0 (secure 0 0 0) 0x7f78b8066990 con 0x7f78cc145a40 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78ac0791f0 msgr2=0x7f78ac07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78ac0791f0 0x7f78ac07b700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f78c0005fd0 tx=0x7f78c0005850 comp rx=0 tx=0).stop 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 msgr2=0x7f78cc0736c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 0x7f78cc0736c0 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f78b800c3c0 tx=0x7f78b800c880 comp rx=0 tx=0).stop 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 shutdown_connections 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78ac0791f0 0x7f78ac07b700 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78cc073c00 0x7f78cc15bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78cc145a40 0x7f78cc0736c0 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 --2- 172.21.15.88:0/267244191 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f78cc145550 0x7f78cc0ab680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 >> 172.21.15.88:0/267244191 conn(0x7f78cc06d1c0 msgr2=0x7f78cc142f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.245+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 shutdown_connections 2024-09-21T07:53:11.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.246+0000 7f78d4600640 1 -- 172.21.15.88:0/267244191 wait complete. 2024-09-21T07:53:11.272 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:53:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 -- 172.21.15.88:0/426141721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280af3c0 msgr2=0x7f25281602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 --2- 172.21.15.88:0/426141721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280af3c0 0x7f25281602b0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f2520005fd0 tx=0x7f252002aa10 comp rx=0 tx=0).stop 2024-09-21T07:53:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 -- 172.21.15.88:0/426141721 shutdown_connections 2024-09-21T07:53:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 --2- 172.21.15.88:0/426141721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280af3c0 0x7f25281602b0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 --2- 172.21.15.88:0/426141721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25280b3ec0 0x7f25280b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 -- 172.21.15.88:0/426141721 >> 172.21.15.88:0/426141721 conn(0x7f25280b1dd0 msgr2=0x7f25280b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:11.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 -- 172.21.15.88:0/426141721 shutdown_connections 2024-09-21T07:53:11.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.496+0000 7f2527200640 1 -- 172.21.15.88:0/426141721 wait complete. 2024-09-21T07:53:11.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.498+0000 7f2527200640 1 Processor -- start 2024-09-21T07:53:11.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.498+0000 7f2527200640 1 -- start start 2024-09-21T07:53:11.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.499+0000 7f2527200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25280705b0 0x7f2528145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:11.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2527200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25280af3c0 0x7f2528074380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:11.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2527200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 0x7f25280748c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:11.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2527200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2528160b80 con 0x7f25280b3ec0 2024-09-21T07:53:11.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2527200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2528160a00 con 0x7f25280705b0 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2527200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f252815cc60 con 0x7f25280af3c0 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 0x7f25280748c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 0x7f25280748c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35314/0 (socket says 172.21.15.88:35314) 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 -- 172.21.15.88:0/4049596495 learned_addr learned my addr 172.21.15.88:0/4049596495 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2525e00640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25280705b0 0x7f2528145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2525400640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25280af3c0 0x7f2528074380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 -- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25280af3c0 msgr2=0x7f2528074380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25280af3c0 0x7f2528074380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 -- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25280705b0 msgr2=0x7f2528145a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25280705b0 0x7f2528145a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:11.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.500+0000 7f2526800640 1 -- 172.21.15.88:0/4049596495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25281531b0 con 0x7f25280b3ec0 2024-09-21T07:53:11.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.501+0000 7f2526800640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 0x7f25280748c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2520009d90 tx=0x7f2520003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:11.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.501+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520033050 con 0x7f25280b3ec0 2024-09-21T07:53:11.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.501+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2520004450 con 0x7f25280b3ec0 2024-09-21T07:53:11.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.501+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f252809e1c0 con 0x7f25280b3ec0 2024-09-21T07:53:11.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.501+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2520004860 con 0x7f25280b3ec0 2024-09-21T07:53:11.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.501+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25281f8cb0 con 0x7f25280b3ec0 2024-09-21T07:53:11.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.502+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f25281602b0 con 0x7f25280b3ec0 2024-09-21T07:53:11.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.503+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f2520036070 con 0x7f25280b3ec0 2024-09-21T07:53:11.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.504+0000 7f251e000640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25080791c0 0x7f250807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:11.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.504+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(122..122 src has 1..122) ==== 5823+0+0 (secure 0 0 0) 0x7f25200bcc70 con 0x7f25280b3ec0 2024-09-21T07:53:11.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.505+0000 7f2525e00640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25080791c0 0x7f250807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:11.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.505+0000 7f2525e00640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25080791c0 0x7f250807b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2518009b30 tx=0x7f2518005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:11.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.509+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2520083ff0 con 0x7f25280b3ec0 2024-09-21T07:53:11.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:11.694+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f25280b2590 con 0x7f25280b3ec0 2024-09-21T07:53:12.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.251+0000 7f251e000640 1 -- 172.21.15.88:0/4049596495 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v123) ==== 181+0+0 (secure 0 0 0) 0x7f2520088a20 con 0x7f25280b3ec0 2024-09-21T07:53:12.343 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:53:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25080791c0 msgr2=0x7f250807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25080791c0 0x7f250807b6d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2518009b30 tx=0x7f2518005f90 comp rx=0 tx=0).stop 2024-09-21T07:53:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 msgr2=0x7f25280748c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 0x7f25280748c0 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f2520009d90 tx=0x7f2520003f10 comp rx=0 tx=0).stop 2024-09-21T07:53:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 shutdown_connections 2024-09-21T07:53:12.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f25080791c0 0x7f250807b6d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f25280af3c0 0x7f2528074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f25280b3ec0 0x7f25280748c0 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 --2- 172.21.15.88:0/4049596495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f25280705b0 0x7f2528145a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 >> 172.21.15.88:0/4049596495 conn(0x7f25280b1dd0 msgr2=0x7f252815de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 shutdown_connections 2024-09-21T07:53:12.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.254+0000 7f2527200640 1 -- 172.21.15.88:0/4049596495 wait complete. 2024-09-21T07:53:12.345 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.571+0000 7f0635a00640 1 -- 172.21.15.88:0/2144225948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 msgr2=0x7f0614004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.571+0000 7f0635a00640 1 --2- 172.21.15.88:0/2144225948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f0614004800 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f061800c470 tx=0x7f0614005830 comp rx=0 tx=0).stop 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.571+0000 7f0635000640 1 -- 172.21.15.88:0/2144225948 reap_dead start 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.571+0000 7f0635a00640 1 -- 172.21.15.88:0/2144225948 shutdown_connections 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.571+0000 7f0635a00640 1 -- 172.21.15.88:0/2144225948 >> 172.21.15.88:0/2144225948 conn(0x7f06300b1dd0 msgr2=0x7f06300b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.571+0000 7f0635a00640 1 -- 172.21.15.88:0/2144225948 shutdown_connections 2024-09-21T07:53:12.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.572+0000 7f0635a00640 1 -- 172.21.15.88:0/2144225948 wait complete. 2024-09-21T07:53:12.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.574+0000 7f0635a00640 1 Processor -- start 2024-09-21T07:53:12.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.574+0000 7f0635a00640 1 -- start start 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.575+0000 7f0635a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f06301456a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.575+0000 7f0635a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06301f5350 0x7f0630153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.575+0000 7f0635a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06301f89a0 0x7f06301f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.575+0000 7f0635a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0630160b80 con 0x7f0614003fd0 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.575+0000 7f0635a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0630160a00 con 0x7f06301f89a0 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.575+0000 7f0635a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f06300b47c0 con 0x7f06301f5350 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f06301456a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:12.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f06301456a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35338/0 (socket says 172.21.15.88:35338) 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 -- 172.21.15.88:0/2068908797 learned_addr learned my addr 172.21.15.88:0/2068908797 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f0627400640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06301f5350 0x7f0630153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f0635000640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06301f89a0 0x7f06301f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 -- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06301f5350 msgr2=0x7f0630153cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06301f5350 0x7f0630153cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 -- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06301f89a0 msgr2=0x7f06301f8db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06301f89a0 0x7f06301f8db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:12.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 -- 172.21.15.88:0/2068908797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06301f92f0 con 0x7f0614003fd0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.576+0000 7f062fe00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f06301456a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0620011b10 tx=0x7f0620009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.577+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062001e070 con 0x7f0614003fd0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.577+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0620004050 con 0x7f0614003fd0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.577+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f062000fea0 con 0x7f0614003fd0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.577+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06301f59e0 con 0x7f0614003fd0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.577+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06301f5fa0 con 0x7f0614003fd0 2024-09-21T07:53:12.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.578+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06301602b0 con 0x7f0614003fd0 2024-09-21T07:53:12.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.579+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0620021070 con 0x7f0614003fd0 2024-09-21T07:53:12.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.580+0000 7f062cc00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f060c0791c0 0x7f060c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:12.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.580+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(123..123 src has 1..123) ==== 5385+0+0 (secure 0 0 0) 0x7f062009c280 con 0x7f0614003fd0 2024-09-21T07:53:12.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.580+0000 7f0627400640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f060c0791c0 0x7f060c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:12.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.581+0000 7f0627400640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f060c0791c0 0x7f060c07b6d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0618033000 tx=0x7f06300aece0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:12.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.584+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f06301f92f0 con 0x7f0614003fd0 2024-09-21T07:53:12.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:12.772+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f06300b3240 con 0x7f0614003fd0 2024-09-21T07:53:13.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.252+0000 7f062cc00640 1 -- 172.21.15.88:0/2068908797 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v124) ==== 169+0+0 (secure 0 0 0) 0x7f06200044a0 con 0x7f0614003fd0 2024-09-21T07:53:13.255 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:53:13.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f060c0791c0 msgr2=0x7f060c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f060c0791c0 0x7f060c07b6d0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0618033000 tx=0x7f06300aece0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 msgr2=0x7f06301456a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f06301456a0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f0620011b10 tx=0x7f0620009e90 comp rx=0 tx=0).stop 2024-09-21T07:53:13.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 shutdown_connections 2024-09-21T07:53:13.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f060c0791c0 0x7f060c07b6d0 secure :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f0618033000 tx=0x7f06300aece0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f06301f5350 0x7f0630153cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f06301f89a0 0x7f06301f8db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 --2- 172.21.15.88:0/2068908797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0614003fd0 0x7f06301456a0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.256+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 >> 172.21.15.88:0/2068908797 conn(0x7f06300b1dd0 msgr2=0x7f06300d07d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:13.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.257+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 shutdown_connections 2024-09-21T07:53:13.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.257+0000 7f0635a00640 1 -- 172.21.15.88:0/2068908797 wait complete. 2024-09-21T07:53:13.283 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:53:13.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 -- 172.21.15.88:0/1311887351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0cbfc0 msgr2=0x7f080c0af8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 --2- 172.21.15.88:0/1311887351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0cbfc0 0x7f080c0af8d0 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f080c0b29b0 tx=0x7f080002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 -- 172.21.15.88:0/1311887351 shutdown_connections 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 --2- 172.21.15.88:0/1311887351 >> v2:172.21.15.175:3300/0 conn(0x7f080c070c80 0x7f080c0b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 --2- 172.21.15.88:0/1311887351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c0cbfc0 0x7f080c0af8d0 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 -- 172.21.15.88:0/1311887351 >> 172.21.15.88:0/1311887351 conn(0x7f080c06d3d0 msgr2=0x7f080c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 -- 172.21.15.88:0/1311887351 shutdown_connections 2024-09-21T07:53:13.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.507+0000 7f0814000640 1 -- 172.21.15.88:0/1311887351 wait complete. 2024-09-21T07:53:13.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.509+0000 7f0814000640 1 Processor -- start 2024-09-21T07:53:13.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.509+0000 7f0814000640 1 -- start start 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0814000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 0x7f080c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0814000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f080c0cbfc0 0x7f080c0732c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0814000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080c098e60 0x7f080c073800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0814000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0b3e20 con 0x7f080c06f0d0 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0814000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f080c0b3ca0 con 0x7f080c0cbfc0 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0814000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f080c0b3fa0 con 0x7f080c098e60 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0811600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 0x7f080c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0811600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 0x7f080c153350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35346/0 (socket says 172.21.15.88:35346) 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0811600640 1 -- 172.21.15.88:0/2580982895 learned_addr learned my addr 172.21.15.88:0/2580982895 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:13.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0810c00640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f080c0cbfc0 0x7f080c0732c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.511+0000 7f0812000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080c098e60 0x7f080c073800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0811600640 1 -- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080c098e60 msgr2=0x7f080c073800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0811600640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080c098e60 0x7f080c073800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0811600640 1 -- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f080c0cbfc0 msgr2=0x7f080c0732c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0811600640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f080c0cbfc0 0x7f080c0732c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0811600640 1 -- 172.21.15.88:0/2580982895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f080c1577b0 con 0x7f080c06f0d0 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0810c00640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f080c0cbfc0 0x7f080c0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:13.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0811600640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 0x7f080c153350 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f08000098f0 tx=0x7f0800033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:13.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800031d60 con 0x7f080c06f0d0 2024-09-21T07:53:13.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0800033e10 con 0x7f080c06f0d0 2024-09-21T07:53:13.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f080c157a90 con 0x7f080c06f0d0 2024-09-21T07:53:13.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0800005260 con 0x7f080c06f0d0 2024-09-21T07:53:13.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.512+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f080c0742e0 con 0x7f080c06f0d0 2024-09-21T07:53:13.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.513+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f080c0ca720 con 0x7f080c06f0d0 2024-09-21T07:53:13.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.514+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0800005400 con 0x7f080c06f0d0 2024-09-21T07:53:13.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.515+0000 7f0809600640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f07ec0791c0 0x7f07ec07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.516+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7f08000bc7d0 con 0x7f080c06f0d0 2024-09-21T07:53:13.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.516+0000 7f0810c00640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f07ec0791c0 0x7f07ec07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.516+0000 7f0810c00640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f07ec0791c0 0x7f07ec07b6d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f080c0d0d30 tx=0x7f07f8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:13.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.520+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f080c0ca720 con 0x7f080c06f0d0 2024-09-21T07:53:13.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.709+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f080c0b3550 con 0x7f080c06f0d0 2024-09-21T07:53:13.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.713+0000 7f0809600640 1 -- 172.21.15.88:0/2580982895 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v45) ==== 71+0+0 (secure 0 0 0) 0x7f080003f030 con 0x7f080c06f0d0 2024-09-21T07:53:13.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f07ec0791c0 msgr2=0x7f07ec07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f07ec0791c0 0x7f07ec07b6d0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f080c0d0d30 tx=0x7f07f8009040 comp rx=0 tx=0).stop 2024-09-21T07:53:13.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 msgr2=0x7f080c153350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 0x7f080c153350 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f08000098f0 tx=0x7f0800033c20 comp rx=0 tx=0).stop 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 shutdown_connections 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f07ec0791c0 0x7f07ec07b6d0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f080c098e60 0x7f080c073800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f080c0cbfc0 0x7f080c0732c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 --2- 172.21.15.88:0/2580982895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f080c06f0d0 0x7f080c153350 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 >> 172.21.15.88:0/2580982895 conn(0x7f080c06d3d0 msgr2=0x7f080c074c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 shutdown_connections 2024-09-21T07:53:13.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.716+0000 7f0814000640 1 -- 172.21.15.88:0/2580982895 wait complete. 2024-09-21T07:53:13.743 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2' 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 -- 172.21.15.88:0/3323271518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 msgr2=0x7f5950004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 --2- 172.21.15.88:0/3323271518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f5950004c10 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f5954006f90 tx=0x7f5950005830 comp rx=0 tx=0).stop 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 -- 172.21.15.88:0/3323271518 shutdown_connections 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 --2- 172.21.15.88:0/3323271518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f5950004c10 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 -- 172.21.15.88:0/3323271518 >> 172.21.15.88:0/3323271518 conn(0x7f596c0b1e40 msgr2=0x7f596c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 -- 172.21.15.88:0/3323271518 shutdown_connections 2024-09-21T07:53:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.962+0000 7f596b200640 1 -- 172.21.15.88:0/3323271518 wait complete. 2024-09-21T07:53:13.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.965+0000 7f596b200640 1 Processor -- start 2024-09-21T07:53:13.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.965+0000 7f596b200640 1 -- start start 2024-09-21T07:53:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f596b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f596c1f4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f596b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596c14bed0 0x7f596c1f48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f596b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f596c1485f0 0x7f596c148a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f596b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f596c0af500 con 0x7f5950004800 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f596b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f596c0af380 con 0x7f596c14bed0 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f596b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f596c15d670 con 0x7f596c1485f0 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f5969e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f596c1f4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f5969e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f596c1f4360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35368/0 (socket says 172.21.15.88:35368) 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.966+0000 7f5969e00640 1 -- 172.21.15.88:0/3267380601 learned_addr learned my addr 172.21.15.88:0/3267380601 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5961400640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596c14bed0 0x7f596c1f48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f596a800640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f596c1485f0 0x7f596c148a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5969e00640 1 -- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f596c1485f0 msgr2=0x7f596c148a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5969e00640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f596c1485f0 0x7f596c148a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5969e00640 1 -- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596c14bed0 msgr2=0x7f596c1f48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5969e00640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596c14bed0 0x7f596c1f48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5969e00640 1 -- 172.21.15.88:0/3267380601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f596c1e7070 con 0x7f5950004800 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5961400640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596c14bed0 0x7f596c1f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5969e00640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f596c1f4360 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f595c010ba0 tx=0x7f595c00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c010ea0 con 0x7f5950004800 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f595c004e30 con 0x7f5950004800 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f596c1e7350 con 0x7f5950004800 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.967+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f595c003bb0 con 0x7f5950004800 2024-09-21T07:53:13.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.968+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f596c1e7910 con 0x7f5950004800 2024-09-21T07:53:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.968+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f596c070530 con 0x7f5950004800 2024-09-21T07:53:13.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.970+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f595c004050 con 0x7f5950004800 2024-09-21T07:53:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.971+0000 7f5962a00640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59480791f0 0x7f594807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.971+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7f595c09c060 con 0x7f5950004800 2024-09-21T07:53:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.971+0000 7f5961400640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59480791f0 0x7f594807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:13.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.972+0000 7f5961400640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59480791f0 0x7f594807b700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5954033000 tx=0x7f595402d000 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:13.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:13.975+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f595c063ef0 con 0x7f5950004800 2024-09-21T07:53:14.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.241+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]} v 0) -- 0x7f596c1e7ae0 con 0x7f5950004800 2024-09-21T07:53:14.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.254+0000 7f5962a00640 1 -- 172.21.15.88:0/3267380601 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_no_upgrade_v1_to_v2"]}]=0 v400) ==== 148+0+0 (secure 0 0 0) 0x7f595c0637f0 con 0x7f5950004800 2024-09-21T07:53:14.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59480791f0 msgr2=0x7f594807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.259 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59480791f0 0x7f594807b700 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f5954033000 tx=0x7f595402d000 comp rx=0 tx=0).stop 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 msgr2=0x7f596c1f4360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f596c1f4360 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f595c010ba0 tx=0x7f595c00e720 comp rx=0 tx=0).stop 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 shutdown_connections 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f59480791f0 0x7f594807b700 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f596c1485f0 0x7f596c148a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f596c14bed0 0x7f596c1f48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 --2- 172.21.15.88:0/3267380601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5950004800 0x7f596c1f4360 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 >> 172.21.15.88:0/3267380601 conn(0x7f596c0b1e40 msgr2=0x7f596c0cba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 shutdown_connections 2024-09-21T07:53:14.260 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.257+0000 7f596b200640 1 -- 172.21.15.88:0/3267380601 wait complete. 2024-09-21T07:53:14.284 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-21T07:53:14.284 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:53:14.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:14.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 -- 172.21.15.88:0/2773933007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0ad9c0 msgr2=0x7f538c0af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 --2- 172.21.15.88:0/2773933007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0ad9c0 0x7f538c0af170 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f538c0af6b0 tx=0x7f538002a560 comp rx=0 tx=0).stop 2024-09-21T07:53:14.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 -- 172.21.15.88:0/2773933007 shutdown_connections 2024-09-21T07:53:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 --2- 172.21.15.88:0/2773933007 >> v2:172.21.15.175:3300/0 conn(0x7f538c15a0f0 0x7f538c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 --2- 172.21.15.88:0/2773933007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0ad9c0 0x7f538c0af170 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 -- 172.21.15.88:0/2773933007 >> 172.21.15.88:0/2773933007 conn(0x7f538c0b1e40 msgr2=0x7f538c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 -- 172.21.15.88:0/2773933007 shutdown_connections 2024-09-21T07:53:14.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.503+0000 7f5392c00640 1 -- 172.21.15.88:0/2773933007 wait complete. 2024-09-21T07:53:14.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.505+0000 7f5392c00640 1 Processor -- start 2024-09-21T07:53:14.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.505+0000 7f5392c00640 1 -- start start 2024-09-21T07:53:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538c0ad9c0 0x7f538c150af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 0x7f538c155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538c15a0f0 0x7f538c1556d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f538c06f410 con 0x7f538c0b2f60 2024-09-21T07:53:14.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f538c06f290 con 0x7f538c15a0f0 2024-09-21T07:53:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f538c06f590 con 0x7f538c0ad9c0 2024-09-21T07:53:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 0x7f538c155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 0x7f538c155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35386/0 (socket says 172.21.15.88:35386) 2024-09-21T07:53:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 -- 172.21.15.88:0/1982552288 learned_addr learned my addr 172.21.15.88:0/1982552288 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5391800640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538c0ad9c0 0x7f538c150af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392200640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538c15a0f0 0x7f538c1556d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 -- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538c0ad9c0 msgr2=0x7f538c150af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538c0ad9c0 0x7f538c150af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 -- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538c15a0f0 msgr2=0x7f538c1556d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538c15a0f0 0x7f538c1556d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 -- 172.21.15.88:0/1982552288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538c09e200 con 0x7f538c0b2f60 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5390e00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 0x7f538c155190 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f537800c370 tx=0x7f537800c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378017070 con 0x7f538c0b2f60 2024-09-21T07:53:14.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f53780035f0 con 0x7f538c0b2f60 2024-09-21T07:53:14.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538c0c9710 con 0x7f538c0b2f60 2024-09-21T07:53:14.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5378003a40 con 0x7f538c0b2f60 2024-09-21T07:53:14.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.508+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f538c0c9cd0 con 0x7f538c0b2f60 2024-09-21T07:53:14.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.510+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538c09df70 con 0x7f538c0b2f60 2024-09-21T07:53:14.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.511+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f537801a070 con 0x7f538c0b2f60 2024-09-21T07:53:14.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.511+0000 7f5389600640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f536c0791c0 0x7f536c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.512+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7f537809a960 con 0x7f538c0b2f60 2024-09-21T07:53:14.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.512+0000 7f5391800640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f536c0791c0 0x7f536c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.513+0000 7f5391800640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f536c0791c0 0x7f536c07b6d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5380033000 tx=0x7f5380004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:14.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.516+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5378061fa0 con 0x7f538c0b2f60 2024-09-21T07:53:14.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.703+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f538c0b2250 con 0x7f538c0b2f60 2024-09-21T07:53:14.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.704+0000 7f5389600640 1 -- 172.21.15.88:0/1982552288 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) ==== 74+0+13098 (secure 0 0 0) 0x7f538c0b2250 con 0x7f538c0b2f60 2024-09-21T07:53:14.706 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:14.706 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":124,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:13.252292+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":118,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6834/3361463212":"2024-09-22T07:53:10.239254+0000","172.21.15.88:6832/3361463212":"2024-09-22T07:53:10.239254+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.706+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f536c0791c0 msgr2=0x7f536c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.706+0000 7f5392c00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f536c0791c0 0x7f536c07b6d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5380033000 tx=0x7f5380004730 comp rx=0 tx=0).stop 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 msgr2=0x7f538c155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 0x7f538c155190 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f537800c370 tx=0x7f537800c830 comp rx=0 tx=0).stop 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 shutdown_connections 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f536c0791c0 0x7f536c07b6d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f538c15a0f0 0x7f538c1556d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f538c0b2f60 0x7f538c155190 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 --2- 172.21.15.88:0/1982552288 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f538c0ad9c0 0x7f538c150af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 >> 172.21.15.88:0/1982552288 conn(0x7f538c0b1e40 msgr2=0x7f538c0b3550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:14.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 shutdown_connections 2024-09-21T07:53:14.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.707+0000 7f5392c00640 1 -- 172.21.15.88:0/1982552288 wait complete. 2024-09-21T07:53:14.736 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:53:14.736 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 -- 172.21.15.88:0/3608797880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c072fc0 msgr2=0x7fd04c0733d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 --2- 172.21.15.88:0/3608797880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c072fc0 0x7fd04c0733d0 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd038005fd0 tx=0x7fd03802a530 comp rx=0 tx=0).stop 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 -- 172.21.15.88:0/3608797880 shutdown_connections 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 --2- 172.21.15.88:0/3608797880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd04c152060 0x7fd04c152470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 --2- 172.21.15.88:0/3608797880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c072fc0 0x7fd04c0733d0 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 -- 172.21.15.88:0/3608797880 >> 172.21.15.88:0/3608797880 conn(0x7fd04c06d710 msgr2=0x7fd04c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 -- 172.21.15.88:0/3608797880 shutdown_connections 2024-09-21T07:53:14.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.957+0000 7fd052c00640 1 -- 172.21.15.88:0/3608797880 wait complete. 2024-09-21T07:53:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.959+0000 7fd052c00640 1 Processor -- start 2024-09-21T07:53:14.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.960+0000 7fd052c00640 1 -- start start 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd052c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04c072fc0 0x7fd04c14b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd052c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd04c152060 0x7fd04c0aea50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd052c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 0x7fd04c0aef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd052c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c06f980 con 0x7fd04c15f250 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd052c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd04c06f800 con 0x7fd04c072fc0 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd052c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd04c06fb00 con 0x7fd04c152060 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd050c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 0x7fd04c0aef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd050c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 0x7fd04c0aef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35408/0 (socket says 172.21.15.88:35408) 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.961+0000 7fd050c00640 1 -- 172.21.15.88:0/1116848316 learned_addr learned my addr 172.21.15.88:0/1116848316 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd04b400640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd04c152060 0x7fd04c0aea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd04be00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04c072fc0 0x7fd04c14b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd050c00640 1 -- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd04c152060 msgr2=0x7fd04c0aea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd050c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd04c152060 0x7fd04c0aea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd050c00640 1 -- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04c072fc0 msgr2=0x7fd04c14b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd050c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04c072fc0 0x7fd04c14b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd050c00640 1 -- 172.21.15.88:0/1116848316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd04c153770 con 0x7fd04c15f250 2024-09-21T07:53:14.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd050c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 0x7fd04c0aef90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd04400cad0 tx=0x7fd04400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400adc0 con 0x7fd04c15f250 2024-09-21T07:53:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd04400f540 con 0x7fd04c15f250 2024-09-21T07:53:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.962+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd04400f990 con 0x7fd04c15f250 2024-09-21T07:53:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.963+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd04c153a50 con 0x7fd04c15f250 2024-09-21T07:53:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.963+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd04c154010 con 0x7fd04c15f250 2024-09-21T07:53:14.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.964+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd04c0733d0 con 0x7fd04c15f250 2024-09-21T07:53:14.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.965+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd04400fb30 con 0x7fd04c15f250 2024-09-21T07:53:14.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.966+0000 7fd043e00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd02c0791f0 0x7fd02c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.967+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7fd04409c190 con 0x7fd04c15f250 2024-09-21T07:53:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.967+0000 7fd04be00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd02c0791f0 0x7fd02c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:14.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.967+0000 7fd04be00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd02c0791f0 0x7fd02c07b700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006fd0 tx=0x7fd03c008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:14.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:14.971+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd044014030 con 0x7fd04c15f250 2024-09-21T07:53:15.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.155+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd04c15c560 con 0x7fd04c15f250 2024-09-21T07:53:15.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.156+0000 7fd043e00640 1 -- 172.21.15.88:0/1116848316 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) ==== 93+0+10 (secure 0 0 0) 0x7fd04400fe20 con 0x7fd04c15f250 2024-09-21T07:53:15.158 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd02c0791f0 msgr2=0x7fd02c07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd02c0791f0 0x7fd02c07b700 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fd03c006fd0 tx=0x7fd03c008040 comp rx=0 tx=0).stop 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 msgr2=0x7fd04c0aef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 0x7fd04c0aef90 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7fd04400cad0 tx=0x7fd04400cf90 comp rx=0 tx=0).stop 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 shutdown_connections 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd02c0791f0 0x7fd02c07b700 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd04c152060 0x7fd04c0aea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd04c15f250 0x7fd04c0aef90 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 --2- 172.21.15.88:0/1116848316 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd04c072fc0 0x7fd04c14b930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.158+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 >> 172.21.15.88:0/1116848316 conn(0x7fd04c06d710 msgr2=0x7fd04c15b420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:15.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.159+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 shutdown_connections 2024-09-21T07:53:15.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.159+0000 7fd052c00640 1 -- 172.21.15.88:0/1116848316 wait complete. 2024-09-21T07:53:15.187 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-21T07:53:15.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 -- 172.21.15.88:0/626186920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500ad0e0 msgr2=0x7fd2500ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 --2- 172.21.15.88:0/626186920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500ad0e0 0x7fd2500ad4f0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fd23c005fd0 tx=0x7fd23c02a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 -- 172.21.15.88:0/626186920 shutdown_connections 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 --2- 172.21.15.88:0/626186920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500ad0e0 0x7fd2500ad4f0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 -- 172.21.15.88:0/626186920 >> 172.21.15.88:0/626186920 conn(0x7fd25006d730 msgr2=0x7fd2500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 -- 172.21.15.88:0/626186920 shutdown_connections 2024-09-21T07:53:15.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.405+0000 7fd257e00640 1 -- 172.21.15.88:0/626186920 wait complete. 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.406+0000 7fd257e00640 1 Processor -- start 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 -- start start 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2500ad0e0 0x7fd2501513c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 0x7fd2501f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25015d760 0x7fd250156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500b4da0 con 0x7fd2500b3550 2024-09-21T07:53:15.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd2500b4c20 con 0x7fd2500ad0e0 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd257e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd2500b4070 con 0x7fd25015d760 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd254a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 0x7fd2501f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd254a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 0x7fd2501f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35422/0 (socket says 172.21.15.88:35422) 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.408+0000 7fd254a00640 1 -- 172.21.15.88:0/1852077923 learned_addr learned my addr 172.21.15.88:0/1852077923 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd255400640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2500ad0e0 0x7fd2501513c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd255e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25015d760 0x7fd250156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd254a00640 1 -- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25015d760 msgr2=0x7fd250156010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd254a00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25015d760 0x7fd250156010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd254a00640 1 -- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2500ad0e0 msgr2=0x7fd2501513c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd254a00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2500ad0e0 0x7fd2501513c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd254a00640 1 -- 172.21.15.88:0/1852077923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25009afe0 con 0x7fd2500b3550 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd255400640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2500ad0e0 0x7fd2501513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:15.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd254a00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 0x7fd2501f8ac0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd23c006010 tx=0x7fd23c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c03e030 con 0x7fd2500b3550 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd23c0047a0 con 0x7fd2500b3550 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2501f9700 con 0x7fd2500b3550 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.410+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd23c035040 con 0x7fd2500b3550 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.411+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2501f59b0 con 0x7fd2500b3550 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.412+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2501f9470 con 0x7fd2500b3550 2024-09-21T07:53:15.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.413+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd23c041030 con 0x7fd2500b3550 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.414+0000 7fd24d600640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2300791c0 0x7fd23007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.414+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7fd23c0bbe40 con 0x7fd2500b3550 2024-09-21T07:53:15.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.414+0000 7fd255400640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2300791c0 0x7fd23007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:15.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.415+0000 7fd255400640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2300791c0 0x7fd23007b6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd244007fd0 tx=0x7fd244009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:15.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.418+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd25009afe0 con 0x7fd2500b3550 2024-09-21T07:53:15.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:15.671+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7fd2500ad4f0 con 0x7fd2500b3550 2024-09-21T07:53:16.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.264+0000 7fd24d600640 1 -- 172.21.15.88:0/1852077923 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v402) ==== 154+0+0 (secure 0 0 0) 0x7fd23c083cd0 con 0x7fd2500b3550 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.267+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2300791c0 msgr2=0x7fd23007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.267+0000 7fd257e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2300791c0 0x7fd23007b6d0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fd244007fd0 tx=0x7fd244009040 comp rx=0 tx=0).stop 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.267+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 msgr2=0x7fd2501f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.267+0000 7fd257e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 0x7fd2501f8ac0 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7fd23c006010 tx=0x7fd23c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 shutdown_connections 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2300791c0 0x7fd23007b6d0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd25015d760 0x7fd250156010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2500b3550 0x7fd2501f8ac0 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 --2- 172.21.15.88:0/1852077923 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2500ad0e0 0x7fd2501513c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 >> 172.21.15.88:0/1852077923 conn(0x7fd25006d730 msgr2=0x7fd2500b3960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 shutdown_connections 2024-09-21T07:53:16.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.268+0000 7fd257e00640 1 -- 172.21.15.88:0/1852077923 wait complete. 2024-09-21T07:53:16.292 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 -- 172.21.15.88:0/4218787619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe134005350 msgr2=0x7fe13400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 --2- 172.21.15.88:0/4218787619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe134005350 0x7fe13400b9c0 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7fe148001dc0 tx=0x7fe134005830 comp rx=0 tx=0).stop 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 -- 172.21.15.88:0/4218787619 shutdown_connections 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 --2- 172.21.15.88:0/4218787619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe134005350 0x7fe13400b9c0 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 -- 172.21.15.88:0/4218787619 >> 172.21.15.88:0/4218787619 conn(0x7fe15006d730 msgr2=0x7fe1500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 -- 172.21.15.88:0/4218787619 shutdown_connections 2024-09-21T07:53:16.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.514+0000 7fe157000640 1 -- 172.21.15.88:0/4218787619 wait complete. 2024-09-21T07:53:16.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.517+0000 7fe157000640 1 Processor -- start 2024-09-21T07:53:16.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.517+0000 7fe157000640 1 -- start start 2024-09-21T07:53:16.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe157000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe134005350 0x7fe1501f96d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe157000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe150074460 0x7fe15015d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe157000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 0x7fe1500b0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe157000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe15015f3d0 con 0x7fe15015d970 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe157000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe150152cf0 con 0x7fe150074460 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe157000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe15015f550 con 0x7fe134005350 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.518+0000 7fe155000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 0x7fe1500b0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 0x7fe1500b0a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46466/0 (socket says 172.21.15.88:46466) 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 -- 172.21.15.88:0/3994526002 learned_addr learned my addr 172.21.15.88:0/3994526002 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe14f400640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe150074460 0x7fe15015d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe14fe00640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe134005350 0x7fe1501f96d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 -- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe134005350 msgr2=0x7fe1501f96d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe134005350 0x7fe1501f96d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 -- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe150074460 msgr2=0x7fe15015d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe150074460 0x7fe15015d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 -- 172.21.15.88:0/3994526002 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15015dd80 con 0x7fe15015d970 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.519+0000 7fe155000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 0x7fe1500b0a30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe148001eb0 tx=0x7fe148006ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.520+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148003d80 con 0x7fe15015d970 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.520+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe148005a10 con 0x7fe15015d970 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.520+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1501582d0 con 0x7fe15015d970 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.520+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148005e20 con 0x7fe15015d970 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.520+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1501f5a70 con 0x7fe15015d970 2024-09-21T07:53:16.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.521+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe15015cf80 con 0x7fe15015d970 2024-09-21T07:53:16.523 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.522+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fe148005fc0 con 0x7fe15015d970 2024-09-21T07:53:16.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.523+0000 7fe147e00640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1300791f0 0x7fe13007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.523+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7fe1480bb760 con 0x7fe15015d970 2024-09-21T07:53:16.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.523+0000 7fe14fe00640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1300791f0 0x7fe13007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.524+0000 7fe14fe00640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1300791f0 0x7fe13007b700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe140009460 tx=0x7fe140009390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:16.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.527+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe15015cf80 con 0x7fe15015d970 2024-09-21T07:53:16.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.714+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe15015fa20 con 0x7fe15015d970 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.715+0000 7fe147e00640 1 -- 172.21.15.88:0/3994526002 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v45) ==== 59+0+2294 (secure 0 0 0) 0x7fe148043060 con 0x7fe15015d970 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:53:16.716 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:53:16.717 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:53:16.718 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:53:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1300791f0 msgr2=0x7fe13007b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1300791f0 0x7fe13007b700 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fe140009460 tx=0x7fe140009390 comp rx=0 tx=0).stop 2024-09-21T07:53:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 msgr2=0x7fe1500b0a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 0x7fe1500b0a30 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7fe148001eb0 tx=0x7fe148006ca0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 shutdown_connections 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1300791f0 0x7fe13007b700 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe15015d970 0x7fe1500b0a30 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe150074460 0x7fe15015d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 --2- 172.21.15.88:0/3994526002 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe134005350 0x7fe1501f96d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 >> 172.21.15.88:0/3994526002 conn(0x7fe15006d730 msgr2=0x7fe150151c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 shutdown_connections 2024-09-21T07:53:16.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.717+0000 7fe157000640 1 -- 172.21.15.88:0/3994526002 wait complete. 2024-09-21T07:53:16.740 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:53:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 -- 172.21.15.88:0/1873262905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 msgr2=0x7faae80af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 --2- 172.21.15.88:0/1873262905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae80af0d0 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7faae8075230 tx=0x7faae002aac0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 -- 172.21.15.88:0/1873262905 shutdown_connections 2024-09-21T07:53:16.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 --2- 172.21.15.88:0/1873262905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae80af0d0 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 --2- 172.21.15.88:0/1873262905 >> v2:172.21.15.175:3300/0 conn(0x7faae80b3f30 0x7faae8156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 -- 172.21.15.88:0/1873262905 >> 172.21.15.88:0/1873262905 conn(0x7faae806d620 msgr2=0x7faae80b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:16.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 -- 172.21.15.88:0/1873262905 shutdown_connections 2024-09-21T07:53:16.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.962+0000 7faaece00640 1 -- 172.21.15.88:0/1873262905 wait complete. 2024-09-21T07:53:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.964+0000 7faaece00640 1 Processor -- start 2024-09-21T07:53:16.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.964+0000 7faaece00640 1 -- start start 2024-09-21T07:53:16.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faaece00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae80b3f30 0x7faae8144870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faaece00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae8152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faaece00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae815d6e0 0x7faae808bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faaece00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faae80af9a0 con 0x7faae80cd1d0 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faaece00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faae80af820 con 0x7faae80b3f30 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faaece00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faae80afb20 con 0x7faae815d6e0 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae8152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae8152990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46486/0 (socket says 172.21.15.88:46486) 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 -- 172.21.15.88:0/4258685971 learned_addr learned my addr 172.21.15.88:0/4258685971 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 -- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae815d6e0 msgr2=0x7faae808bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae7e00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae815d6e0 0x7faae808bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae7400640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae80b3f30 0x7faae8144870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae815d6e0 0x7faae808bfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 -- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae80b3f30 msgr2=0x7faae8144870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:16.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae80b3f30 0x7faae8144870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.966+0000 7faae6a00640 1 -- 172.21.15.88:0/4258685971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faae81437d0 con 0x7faae80cd1d0 2024-09-21T07:53:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.967+0000 7faae6a00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae8152990 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faad400c370 tx=0x7faad400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.967+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4017070 con 0x7faae80cd1d0 2024-09-21T07:53:16.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.967+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faad40035f0 con 0x7faae80cd1d0 2024-09-21T07:53:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.967+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faae8143ab0 con 0x7faae80cd1d0 2024-09-21T07:53:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.967+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad4003a40 con 0x7faae80cd1d0 2024-09-21T07:53:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.967+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faae8144070 con 0x7faae80cd1d0 2024-09-21T07:53:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.968+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faae8156ca0 con 0x7faae80cd1d0 2024-09-21T07:53:16.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.969+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7faad401a070 con 0x7faae80cd1d0 2024-09-21T07:53:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.970+0000 7faadf400640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faac80791f0 0x7faac807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.970+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7faad409a950 con 0x7faae80cd1d0 2024-09-21T07:53:16.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.971+0000 7faae7400640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faac80791f0 0x7faac807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:16.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.971+0000 7faae7400640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faac80791f0 0x7faac807b700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faad80058c0 tx=0x7faad8005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:16.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:16.975+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faad40625e0 con 0x7faae80cd1d0 2024-09-21T07:53:17.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.162+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7faae8154410 con 0x7faae80cd1d0 2024-09-21T07:53:17.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.166+0000 7faadf400640 1 -- 172.21.15.88:0/4258685971 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7faad4061ee0 con 0x7faae80cd1d0 2024-09-21T07:53:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faac80791f0 msgr2=0x7faac807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faac80791f0 0x7faac807b700 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faad80058c0 tx=0x7faad8005850 comp rx=0 tx=0).stop 2024-09-21T07:53:17.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 msgr2=0x7faae8152990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae8152990 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7faad400c370 tx=0x7faad400c830 comp rx=0 tx=0).stop 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 shutdown_connections 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faac80791f0 0x7faac807b700 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faae815d6e0 0x7faae808bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faae80cd1d0 0x7faae8152990 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 --2- 172.21.15.88:0/4258685971 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faae80b3f30 0x7faae8144870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 >> 172.21.15.88:0/4258685971 conn(0x7faae806d620 msgr2=0x7faae80b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 shutdown_connections 2024-09-21T07:53:17.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.169+0000 7faaece00640 1 -- 172.21.15.88:0/4258685971 wait complete. 2024-09-21T07:53:17.196 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:53:17.196 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:17.298 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:17.394 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:53:17.394 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:17.394 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:17.430 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:17.430 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:17.430 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:17.468 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:53:17.468 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:17.468 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:17.507 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:17.507 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:17.507 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:17.551 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:53:17.551 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:53:17.551 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:17.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 -- 172.21.15.88:0/2518428638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec81620e0 msgr2=0x7faec80d0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.774 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 --2- 172.21.15.88:0/2518428638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec81620e0 0x7faec80d0ef0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7faeb4009820 tx=0x7faeb402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 -- 172.21.15.88:0/2518428638 shutdown_connections 2024-09-21T07:53:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 --2- 172.21.15.88:0/2518428638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec81620e0 0x7faec80d0ef0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 -- 172.21.15.88:0/2518428638 >> 172.21.15.88:0/2518428638 conn(0x7faec809d820 msgr2=0x7faec808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 -- 172.21.15.88:0/2518428638 shutdown_connections 2024-09-21T07:53:17.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.773+0000 7faec7e00640 1 -- 172.21.15.88:0/2518428638 wait complete. 2024-09-21T07:53:17.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.775+0000 7faec7e00640 1 Processor -- start 2024-09-21T07:53:17.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.775+0000 7faec7e00640 1 -- start start 2024-09-21T07:53:17.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.776+0000 7faec7e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec80b9610 0x7faec80b1210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.776+0000 7faec7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 0x7faec814dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.776+0000 7faec7e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec81620e0 0x7faec80c9900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.776+0000 7faec7e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faec80d1430 con 0x7faec80b9b80 2024-09-21T07:53:17.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.776+0000 7faec7e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faec80b43f0 con 0x7faec80b9610 2024-09-21T07:53:17.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.776+0000 7faec7e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faec80d15b0 con 0x7faec81620e0 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 0x7faec814dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 0x7faec814dcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46504/0 (socket says 172.21.15.88:46504) 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 -- 172.21.15.88:0/2473796905 learned_addr learned my addr 172.21.15.88:0/2473796905 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6a00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec80b9610 0x7faec80b1210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec7400640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec81620e0 0x7faec80c9900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 -- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec81620e0 msgr2=0x7faec80c9900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec81620e0 0x7faec80c9900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 -- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec80b9610 msgr2=0x7faec80b1210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec80b9610 0x7faec80b1210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:17.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.777+0000 7faec6000640 1 -- 172.21.15.88:0/2473796905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec80c8860 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.778+0000 7faec6000640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 0x7faec814dcc0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7faeb4000c00 tx=0x7faeb4033c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.778+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb403f070 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.778+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeb4031e50 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.778+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec80b1750 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.778+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeb4005290 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.778+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec80c8e20 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.779+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec80d0ef0 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.780+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7faeb4042070 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.781+0000 7faebea00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea80791f0 0x7faea807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.781+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7faeb40bc5b0 con 0x7faec80b9b80 2024-09-21T07:53:17.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.782+0000 7faec6a00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea80791f0 0x7faea807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:17.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.782+0000 7faec6a00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea80791f0 0x7faea807b700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faeb8001720 tx=0x7faeb8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:17.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:17.786+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faeb4083df0 con 0x7faec80b9b80 2024-09-21T07:53:18.010 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.008+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec80d2410 con 0x7faec80b9b80 2024-09-21T07:53:18.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.009+0000 7faebea00640 1 -- 172.21.15.88:0/2473796905 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 83 v83) ==== 94+0+3914 (secure 0 0 0) 0x7faeb4088820 con 0x7faec80b9b80 2024-09-21T07:53:18.011 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:18.011 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":83,"btime":"2024-09-21T07:53:11:241844+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6458,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3979881571","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3979881571},{"type":"v1","addr":"172.21.15.175:6835","nonce":3979881571}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7096,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1148793547","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1148793547},{"type":"v1","addr":"172.21.15.88:6835","nonce":1148793547}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7102,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/260073089","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":260073089},{"type":"v1","addr":"172.21.15.175:6801","nonce":260073089}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":75},{"gid":7830,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1618358921","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1618358921},{"type":"v1","addr":"172.21.15.88:6834","nonce":1618358921}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":83}],"filesystems":[]} 2024-09-21T07:53:18.012 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 83 2024-09-21T07:53:18.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea80791f0 msgr2=0x7faea807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:18.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea80791f0 0x7faea807b700 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7faeb8001720 tx=0x7faeb8009040 comp rx=0 tx=0).stop 2024-09-21T07:53:18.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 msgr2=0x7faec814dcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 0x7faec814dcc0 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7faeb4000c00 tx=0x7faeb4033c90 comp rx=0 tx=0).stop 2024-09-21T07:53:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 shutdown_connections 2024-09-21T07:53:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea80791f0 0x7faea807b700 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec81620e0 0x7faec80c9900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec80b9b80 0x7faec814dcc0 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 --2- 172.21.15.88:0/2473796905 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec80b9610 0x7faec80b1210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 >> 172.21.15.88:0/2473796905 conn(0x7faec809d820 msgr2=0x7faec81611c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 shutdown_connections 2024-09-21T07:53:18.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.011+0000 7faec7e00640 1 -- 172.21.15.88:0/2473796905 wait complete. 2024-09-21T07:53:18.040 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:53:18.040 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:53:18.040 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:53:18.040 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:18.040 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:53:18.040 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:53:18.041 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:53:18.041 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:18.041 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:53:18.041 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:53:18.041 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:53:18.041 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:18.041 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:53:18.041 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:53:18.042 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:53:18.042 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:18.064 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:53:18.065 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:53:18.067 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:53:18.067 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:53:18.070 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:53:18.070 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:53:18.071 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:53:18.071 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:53:18.115 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:53:18.116 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:18.117 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:53:18.118 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:18.119 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:53:18.120 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:18.122 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:53:18.123 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:18.123 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:53:18.311 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:53:18.312 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:53:18.313 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:53:18.314 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:53:18.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.359+0000 7fde74200640 1 -- 172.21.15.88:0/763904274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0b4fe0 msgr2=0x7fde6c0b53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:18.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.359+0000 7fde74200640 1 --2- 172.21.15.88:0/763904274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0b4fe0 0x7fde6c0b53f0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fde6c071590 tx=0x7fde6402cc40 comp rx=0 tx=0).stop 2024-09-21T07:53:18.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.360+0000 7fde74200640 1 -- 172.21.15.88:0/763904274 shutdown_connections 2024-09-21T07:53:18.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.360+0000 7fde74200640 1 --2- 172.21.15.88:0/763904274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0b4fe0 0x7fde6c0b53f0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.360+0000 7fde74200640 1 --2- 172.21.15.88:0/763904274 >> v2:172.21.15.175:3300/0 conn(0x7fde6c0afd10 0x7fde6c0b4180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.360+0000 7fde74200640 1 -- 172.21.15.88:0/763904274 >> 172.21.15.88:0/763904274 conn(0x7fde6c0b2650 msgr2=0x7fde6c0b2a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:18.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.360+0000 7fde74200640 1 -- 172.21.15.88:0/763904274 shutdown_connections 2024-09-21T07:53:18.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.360+0000 7fde74200640 1 -- 172.21.15.88:0/763904274 wait complete. 2024-09-21T07:53:18.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.362+0000 7fde74200640 1 Processor -- start 2024-09-21T07:53:18.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.362+0000 7fde74200640 1 -- start start 2024-09-21T07:53:18.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.363+0000 7fde74200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 0x7fde6c1e69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:18.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.363+0000 7fde74200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c0b4fe0 0x7fde6c1e9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:18.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.363+0000 7fde74200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde6c15b720 0x7fde6c1e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:18.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.363+0000 7fde74200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c0ceb80 con 0x7fde6c0afd10 2024-09-21T07:53:18.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.363+0000 7fde74200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde6c0cea00 con 0x7fde6c0b4fe0 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.363+0000 7fde74200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde6c0ced00 con 0x7fde6c15b720 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 0x7fde6c1e69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 0x7fde6c1e69f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46550/0 (socket says 172.21.15.88:46550) 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 -- 172.21.15.88:0/813158422 learned_addr learned my addr 172.21.15.88:0/813158422 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde70e00640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c0b4fe0 0x7fde6c1e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde72200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde6c15b720 0x7fde6c1e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:18.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 -- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde6c15b720 msgr2=0x7fde6c1e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde6c15b720 0x7fde6c1e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 -- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c0b4fe0 msgr2=0x7fde6c1e9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c0b4fe0 0x7fde6c1e9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 -- 172.21.15.88:0/813158422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde6c1e5480 con 0x7fde6c0afd10 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.364+0000 7fde71800640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 0x7fde6c1e69f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d6d0 tx=0x7fde6000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.365+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60017070 con 0x7fde6c0afd10 2024-09-21T07:53:18.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.365+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde60005330 con 0x7fde6c0afd10 2024-09-21T07:53:18.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.365+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde6c1e5760 con 0x7fde6c0afd10 2024-09-21T07:53:18.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.365+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde60009820 con 0x7fde6c0afd10 2024-09-21T07:53:18.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.365+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde6c1e5d20 con 0x7fde6c0afd10 2024-09-21T07:53:18.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.366+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde6c0cf230 con 0x7fde6c0afd10 2024-09-21T07:53:18.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.367+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fde60009bd0 con 0x7fde6c0afd10 2024-09-21T07:53:18.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.368+0000 7fde69600640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde4c0791f0 0x7fde4c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:18.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.368+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(124..124 src has 1..124) ==== 4971+0+0 (secure 0 0 0) 0x7fde6009b970 con 0x7fde6c0afd10 2024-09-21T07:53:18.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.368+0000 7fde70e00640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde4c0791f0 0x7fde4c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:18.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.369+0000 7fde70e00640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde4c0791f0 0x7fde4c07b700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0cc970 tx=0x7fde58009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:18.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.373+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde60062fb0 con 0x7fde6c0afd10 2024-09-21T07:53:18.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:18.569+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7fde6c0b2d80 con 0x7fde6c0afd10 2024-09-21T07:53:19.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.285+0000 7fde69600640 1 -- 172.21.15.88:0/813158422 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v126) ==== 97+0+0 (secure 0 0 0) 0x7fde600679e0 con 0x7fde6c0afd10 2024-09-21T07:53:19.287 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:53:19.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.288+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde4c0791f0 msgr2=0x7fde4c07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.288+0000 7fde74200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde4c0791f0 0x7fde4c07b700 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fde6c0cc970 tx=0x7fde58009060 comp rx=0 tx=0).stop 2024-09-21T07:53:19.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.288+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 msgr2=0x7fde6c1e69f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.288+0000 7fde74200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 0x7fde6c1e69f0 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d6d0 tx=0x7fde6000db90 comp rx=0 tx=0).stop 2024-09-21T07:53:19.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.288+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 shutdown_connections 2024-09-21T07:53:19.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde4c0791f0 0x7fde4c07b700 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde6c15b720 0x7fde6c1e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde6c0b4fe0 0x7fde6c1e9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 --2- 172.21.15.88:0/813158422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde6c0afd10 0x7fde6c1e69f0 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 >> 172.21.15.88:0/813158422 conn(0x7fde6c0b2650 msgr2=0x7fde6c1e7cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:19.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 shutdown_connections 2024-09-21T07:53:19.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.289+0000 7fde74200640 1 -- 172.21.15.88:0/813158422 wait complete. 2024-09-21T07:53:19.314 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.532+0000 7fdaffe00640 1 -- 172.21.15.88:0/1330931703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00072e60 msgr2=0x7fdb00073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.532+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1330931703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00072e60 0x7fdb00073270 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fdae8009900 tx=0x7fdae802a650 comp rx=0 tx=0).stop 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.533+0000 7fdaffe00640 1 -- 172.21.15.88:0/1330931703 shutdown_connections 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.533+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1330931703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00072e60 0x7fdb00073270 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.533+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1330931703 >> v2:172.21.15.175:3300/0 conn(0x7fdb000af050 0x7fdb000ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.533+0000 7fdaffe00640 1 -- 172.21.15.88:0/1330931703 >> 172.21.15.88:0/1330931703 conn(0x7fdb000b1e40 msgr2=0x7fdb000b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:19.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.533+0000 7fdaffe00640 1 -- 172.21.15.88:0/1330931703 shutdown_connections 2024-09-21T07:53:19.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.533+0000 7fdaffe00640 1 -- 172.21.15.88:0/1330931703 wait complete. 2024-09-21T07:53:19.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.535+0000 7fdaffe00640 1 Processor -- start 2024-09-21T07:53:19.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.535+0000 7fdaffe00640 1 -- start start 2024-09-21T07:53:19.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.536+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 0x7fdb00155a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.536+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb00072e60 0x7fdb001f9600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.536+0000 7fdaffe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb000af050 0x7fdb00146720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.536+0000 7fdaffe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb001658b0 con 0x7fdb00070000 2024-09-21T07:53:19.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.536+0000 7fdaffe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdb00165730 con 0x7fdb00072e60 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.536+0000 7fdaffe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdb00165a30 con 0x7fdb000af050 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 0x7fdb00155a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 0x7fdb00155a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46568/0 (socket says 172.21.15.88:46568) 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 -- 172.21.15.88:0/1474085955 learned_addr learned my addr 172.21.15.88:0/1474085955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdaf6000640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb00072e60 0x7fdb001f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 -- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb000af050 msgr2=0x7fdb00146720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb000af050 0x7fdb00146720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 -- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb00072e60 msgr2=0x7fdb001f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb00072e60 0x7fdb001f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 -- 172.21.15.88:0/1474085955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb001f5780 con 0x7fdb00070000 2024-09-21T07:53:19.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.537+0000 7fdafea00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 0x7fdb00155a60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e7c0 tx=0x7fdaf000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:19.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.538+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf000cc60 con 0x7fdb00070000 2024-09-21T07:53:19.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.538+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdaf0010430 con 0x7fdb00070000 2024-09-21T07:53:19.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.538+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb001f5a20 con 0x7fdb00070000 2024-09-21T07:53:19.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.538+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaf0010860 con 0x7fdb00070000 2024-09-21T07:53:19.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.538+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb001f5fe0 con 0x7fdb00070000 2024-09-21T07:53:19.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.539+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb00073270 con 0x7fdb00070000 2024-09-21T07:53:19.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.540+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fdaf0010a00 con 0x7fdb00070000 2024-09-21T07:53:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.541+0000 7fdaf7400640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdadc0791c0 0x7fdadc07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.541+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4885+0+0 (secure 0 0 0) 0x7fdaf009beb0 con 0x7fdb00070000 2024-09-21T07:53:19.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.541+0000 7fdaf6000640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdadc0791c0 0x7fdadc07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:19.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.542+0000 7fdaf6000640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdadc0791c0 0x7fdadc07b6d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdae8033000 tx=0x7fdae8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:19.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.546+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdaf0063660 con 0x7fdb00070000 2024-09-21T07:53:19.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.799+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fdb00164ba0 con 0x7fdb00070000 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.800+0000 7fdaf7400640 1 -- 172.21.15.88:0/1474085955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) ==== 80+0+5304 (secure 0 0 0) 0x7fdaf0068090 con 0x7fdb00070000 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.801 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.802 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.803 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:53:19.804 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.805 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:53:19.806 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.807 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.808 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:53:19.809 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.810 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.811 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.812 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:53:19.813 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:53:19.814 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.815 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:19.815 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:53:19.815 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:53:19.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdadc0791c0 msgr2=0x7fdadc07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdadc0791c0 0x7fdadc07b6d0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fdae8033000 tx=0x7fdae8004730 comp rx=0 tx=0).stop 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 msgr2=0x7fdb00155a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 0x7fdb00155a60 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000e7c0 tx=0x7fdaf000ec80 comp rx=0 tx=0).stop 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 shutdown_connections 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdadc0791c0 0x7fdadc07b6d0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdb000af050 0x7fdb00146720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdb00072e60 0x7fdb001f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 --2- 172.21.15.88:0/1474085955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb00070000 0x7fdb00155a60 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 >> 172.21.15.88:0/1474085955 conn(0x7fdb000b1e40 msgr2=0x7fdb00148aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:19.816 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 shutdown_connections 2024-09-21T07:53:19.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:19.802+0000 7fdaffe00640 1 -- 172.21.15.88:0/1474085955 wait complete. 2024-09-21T07:53:19.829 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:53:20.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbba400640 1 -- 172.21.15.88:0/4128258512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb98004d40 msgr2=0x7fbb98005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbba400640 1 --2- 172.21.15.88:0/4128258512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb98004d40 0x7fbb98005150 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7fbbb40b5d90 tx=0x7fbb98005830 comp rx=0 tx=0).stop 2024-09-21T07:53:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbb3e00640 1 -- 172.21.15.88:0/4128258512 reap_dead start 2024-09-21T07:53:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbba400640 1 -- 172.21.15.88:0/4128258512 shutdown_connections 2024-09-21T07:53:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbba400640 1 -- 172.21.15.88:0/4128258512 >> 172.21.15.88:0/4128258512 conn(0x7fbbb40d0b50 msgr2=0x7fbbb40d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbba400640 1 -- 172.21.15.88:0/4128258512 shutdown_connections 2024-09-21T07:53:20.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.051+0000 7fbbba400640 1 -- 172.21.15.88:0/4128258512 wait complete. 2024-09-21T07:53:20.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.054+0000 7fbbba400640 1 Processor -- start 2024-09-21T07:53:20.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.054+0000 7fbbba400640 1 -- start start 2024-09-21T07:53:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbba400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb98004d40 0x7fbbb40ce720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbba400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 0x7fbbb41f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbba400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbb41f9d50 0x7fbbb413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbba400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4160ff0 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbba400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4160e70 con 0x7fbb98004d40 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbba400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbbb4161170 con 0x7fbbb41f9d50 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 0x7fbbb41f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbb2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 0x7fbbb41f9810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46578/0 (socket says 172.21.15.88:46578) 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.055+0000 7fbbb2a00640 1 -- 172.21.15.88:0/89608385 learned_addr learned my addr 172.21.15.88:0/89608385 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb3400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb98004d40 0x7fbbb40ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb3e00640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbb41f9d50 0x7fbbb413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb2a00640 1 -- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbb41f9d50 msgr2=0x7fbbb413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb2a00640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbb41f9d50 0x7fbbb413fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb2a00640 1 -- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb98004d40 msgr2=0x7fbbb40ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb2a00640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb98004d40 0x7fbbb40ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb2a00640 1 -- 172.21.15.88:0/89608385 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb40d0280 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbb2a00640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 0x7fbbb41f9810 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbba00109d0 tx=0x7fbba0010e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba000ed40 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbba0004030 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb40d0560 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.056+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba0004480 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.057+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb40cee00 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.058+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbb415f3c0 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.059+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fbba00035f0 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.060+0000 7fbbab400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbb940791f0 0x7fbb9407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.060+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4885+0+0 (secure 0 0 0) 0x7fbba009c2d0 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.060+0000 7fbbb3400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbb940791f0 0x7fbb9407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.061+0000 7fbbb3400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbb940791f0 0x7fbb9407b700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbba4003700 tx=0x7fbba400b260 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.065+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbbb415f3c0 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.251+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fbbb4071b00 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.252+0000 7fbbab400640 1 -- 172.21.15.88:0/89608385 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v128) ==== 84+0+65 (secure 0 0 0) 0x7fbba00639f0 con 0x7fbbb40cf8c0 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:20.253 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:53:20.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbb940791f0 msgr2=0x7fbb9407b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbb940791f0 0x7fbb9407b700 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fbba4003700 tx=0x7fbba400b260 comp rx=0 tx=0).stop 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 msgr2=0x7fbbb41f9810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 0x7fbbb41f9810 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fbba00109d0 tx=0x7fbba0010e90 comp rx=0 tx=0).stop 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 shutdown_connections 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbb940791f0 0x7fbb9407b700 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbb41f9d50 0x7fbbb413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb40cf8c0 0x7fbbb41f9810 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 --2- 172.21.15.88:0/89608385 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbb98004d40 0x7fbbb40ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 >> 172.21.15.88:0/89608385 conn(0x7fbbb40d0b50 msgr2=0x7fbbb415ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 shutdown_connections 2024-09-21T07:53:20.256 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.254+0000 7fbbba400640 1 -- 172.21.15.88:0/89608385 wait complete. 2024-09-21T07:53:20.277 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:53:20.278 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.502+0000 7fedb9800640 1 -- 172.21.15.88:0/2222051612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 msgr2=0x7fedb40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.502+0000 7fedb9800640 1 --2- 172.21.15.88:0/2222051612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb40b4620 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7fedb406d8a0 tx=0x7feda402aa30 comp rx=0 tx=0).stop 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 -- 172.21.15.88:0/2222051612 shutdown_connections 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 --2- 172.21.15.88:0/2222051612 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedb40af100 0x7fedb40af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 --2- 172.21.15.88:0/2222051612 >> v2:172.21.15.175:3300/0 conn(0x7fedb406cda0 0x7fedb406d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 --2- 172.21.15.88:0/2222051612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb40b4620 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 -- 172.21.15.88:0/2222051612 >> 172.21.15.88:0/2222051612 conn(0x7fedb40b2540 msgr2=0x7fedb40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:20.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 -- 172.21.15.88:0/2222051612 shutdown_connections 2024-09-21T07:53:20.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.503+0000 7fedb9800640 1 -- 172.21.15.88:0/2222051612 wait complete. 2024-09-21T07:53:20.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.505+0000 7fedb9800640 1 Processor -- start 2024-09-21T07:53:20.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.505+0000 7fedb9800640 1 -- start start 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.506+0000 7fedb9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedb406cda0 0x7fedb409da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.506+0000 7fedb9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedb40af100 0x7fedb41f67c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb41533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb9800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40b34b0 con 0x7fedb40cc830 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fedb40b3330 con 0x7fedb40af100 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fedb40b3630 con 0x7fedb406cda0 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb41533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb41533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46596/0 (socket says 172.21.15.88:46596) 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 -- 172.21.15.88:0/4144677174 learned_addr learned my addr 172.21.15.88:0/4144677174 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:20.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb2000640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedb40af100 0x7fedb41f67c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb2a00640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedb406cda0 0x7fedb409da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 -- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedb406cda0 msgr2=0x7fedb409da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedb406cda0 0x7fedb409da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 -- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedb40af100 msgr2=0x7fedb41f67c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedb40af100 0x7fedb41f67c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.507+0000 7fedb3400640 1 -- 172.21.15.88:0/4144677174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fedb409e290 con 0x7fedb40cc830 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.508+0000 7fedb3400640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb41533f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fedac00d680 tx=0x7fedac00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.508+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac017070 con 0x7fedb40cc830 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.508+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fedac005310 con 0x7fedb40cc830 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.508+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fedb415ba50 con 0x7fedb40cc830 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.508+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fedac009820 con 0x7fedb40cc830 2024-09-21T07:53:20.510 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.508+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fedb415c010 con 0x7fedb40cc830 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.509+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fedb40af510 con 0x7fedb40cc830 2024-09-21T07:53:20.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.510+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fedac0099c0 con 0x7fedb40cc830 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.511+0000 7fedaaa00640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed940791f0 0x7fed9407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.511+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(128..128 src has 1..128) ==== 4885+0+0 (secure 0 0 0) 0x7fedac09b650 con 0x7fedb40cc830 2024-09-21T07:53:20.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.511+0000 7fedb2a00640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed940791f0 0x7fed9407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:20.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.512+0000 7fedb2a00640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed940791f0 0x7fed9407b700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feda4035000 tx=0x7feda403b040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:20.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.516+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fedac062ce0 con 0x7fedb40cc830 2024-09-21T07:53:20.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:20.701+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fedb409e020 con 0x7fedb40cc830 2024-09-21T07:53:21.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.305+0000 7fedaaa00640 1 -- 172.21.15.88:0/4144677174 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v129) ==== 138+0+0 (secure 0 0 0) 0x7fedac067710 con 0x7fedb40cc830 2024-09-21T07:53:21.307 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:53:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.308+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed940791f0 msgr2=0x7fed9407b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.308+0000 7fedb9800640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed940791f0 0x7fed9407b700 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7feda4035000 tx=0x7feda403b040 comp rx=0 tx=0).stop 2024-09-21T07:53:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.308+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 msgr2=0x7fedb41533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.308+0000 7fedb9800640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb41533f0 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7fedac00d680 tx=0x7fedac00db40 comp rx=0 tx=0).stop 2024-09-21T07:53:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 shutdown_connections 2024-09-21T07:53:21.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed940791f0 0x7fed9407b700 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fedb40cc830 0x7fedb41533f0 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fedb40af100 0x7fedb41f67c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 --2- 172.21.15.88:0/4144677174 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fedb406cda0 0x7fedb409da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 >> 172.21.15.88:0/4144677174 conn(0x7fedb40b2540 msgr2=0x7fedb40ca1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 shutdown_connections 2024-09-21T07:53:21.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.309+0000 7fedb9800640 1 -- 172.21.15.88:0/4144677174 wait complete. 2024-09-21T07:53:21.337 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:53:21.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 -- 172.21.15.88:0/1124903790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 msgr2=0x7f746006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 --2- 172.21.15.88:0/1124903790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746006fb40 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f7450009830 tx=0x7f745002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 -- 172.21.15.88:0/1124903790 shutdown_connections 2024-09-21T07:53:21.564 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 --2- 172.21.15.88:0/1124903790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7460073c50 0x7f74600cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 --2- 172.21.15.88:0/1124903790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746006fb40 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 -- 172.21.15.88:0/1124903790 >> 172.21.15.88:0/1124903790 conn(0x7f746006d1c0 msgr2=0x7f746006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:21.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 -- 172.21.15.88:0/1124903790 shutdown_connections 2024-09-21T07:53:21.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.562+0000 7f7465800640 1 -- 172.21.15.88:0/1124903790 wait complete. 2024-09-21T07:53:21.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.564+0000 7f7465800640 1 Processor -- start 2024-09-21T07:53:21.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.564+0000 7f7465800640 1 -- start start 2024-09-21T07:53:21.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f7465800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746015f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f7465800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7460145a40 0x7f74601f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f7465800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f746015bec0 0x7f746015d860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f7465800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74600ce150 con 0x7f7460145550 2024-09-21T07:53:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f7465800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f74600cdfd0 con 0x7f7460145a40 2024-09-21T07:53:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f7465800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f74600ce2d0 con 0x7f746015bec0 2024-09-21T07:53:21.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746015f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746015f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46612/0 (socket says 172.21.15.88:46612) 2024-09-21T07:53:21.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 -- 172.21.15.88:0/1843603564 learned_addr learned my addr 172.21.15.88:0/1843603564 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:21.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 -- 172.21.15.88:0/1843603564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f746015bec0 msgr2=0x7f746015d860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:53:21.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f746015bec0 0x7f746015d860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 -- 172.21.15.88:0/1843603564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7460145a40 msgr2=0x7f74601f3f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:21.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7460145a40 0x7f74601f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:21.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.566+0000 7f745ea00640 1 -- 172.21.15.88:0/1843603564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74601f46d0 con 0x7f7460145550 2024-09-21T07:53:21.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.567+0000 7f745ea00640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746015f930 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7450009950 tx=0x7f7450033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:21.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.567+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450031d60 con 0x7f7460145550 2024-09-21T07:53:21.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.567+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7450043070 con 0x7f7460145550 2024-09-21T07:53:21.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.567+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74601f49b0 con 0x7f7460145550 2024-09-21T07:53:21.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.567+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7450033310 con 0x7f7460145550 2024-09-21T07:53:21.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.567+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74601f6510 con 0x7f7460145550 2024-09-21T07:53:21.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.568+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f74600ce850 con 0x7f7460145550 2024-09-21T07:53:21.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.570+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f7450031630 con 0x7f7460145550 2024-09-21T07:53:21.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.571+0000 7f7456a00640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f74400791c0 0x7f744007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:21.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.571+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(129..129 src has 1..129) ==== 5255+0+0 (secure 0 0 0) 0x7f74500bcf70 con 0x7f7460145550 2024-09-21T07:53:21.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.571+0000 7f745e000640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f74400791c0 0x7f744007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:21.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.572+0000 7f745e000640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f74400791c0 0x7f744007b6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74600b3100 tx=0x7f744c0079f0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:21.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.575+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7450084cd0 con 0x7f7460145550 2024-09-21T07:53:21.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:21.762+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f74601413c0 con 0x7f7460145550 2024-09-21T07:53:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.312+0000 7f7456a00640 1 -- 172.21.15.88:0/1843603564 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v130) ==== 170+0+0 (secure 0 0 0) 0x7f74500845d0 con 0x7f7460145550 2024-09-21T07:53:22.314 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.315+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f74400791c0 msgr2=0x7f744007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.315+0000 7f7465800640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f74400791c0 0x7f744007b6d0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f74600b3100 tx=0x7f744c0079f0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 msgr2=0x7f746015f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746015f930 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f7450009950 tx=0x7f7450033c20 comp rx=0 tx=0).stop 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 shutdown_connections 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f74400791c0 0x7f744007b6d0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f746015bec0 0x7f746015d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7460145a40 0x7f74601f3f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 --2- 172.21.15.88:0/1843603564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7460145550 0x7f746015f930 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 >> 172.21.15.88:0/1843603564 conn(0x7f746006d1c0 msgr2=0x7f746006f880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 shutdown_connections 2024-09-21T07:53:22.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.316+0000 7f7465800640 1 -- 172.21.15.88:0/1843603564 wait complete. 2024-09-21T07:53:22.357 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:53:22.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 -- 172.21.15.88:0/2185167833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c0cf9d0 msgr2=0x7fb40c1624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 --2- 172.21.15.88:0/2185167833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c0cf9d0 0x7fb40c1624b0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7fb404005fd0 tx=0x7fb40402a820 comp rx=0 tx=0).stop 2024-09-21T07:53:22.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 -- 172.21.15.88:0/2185167833 shutdown_connections 2024-09-21T07:53:22.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 --2- 172.21.15.88:0/2185167833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c0cf9d0 0x7fb40c1624b0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 -- 172.21.15.88:0/2185167833 >> 172.21.15.88:0/2185167833 conn(0x7fb40c06d620 msgr2=0x7fb40c0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:22.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 -- 172.21.15.88:0/2185167833 shutdown_connections 2024-09-21T07:53:22.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.578+0000 7fb410e00640 1 -- 172.21.15.88:0/2185167833 wait complete. 2024-09-21T07:53:22.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.581+0000 7fb410e00640 1 Processor -- start 2024-09-21T07:53:22.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.581+0000 7fb410e00640 1 -- start start 2024-09-21T07:53:22.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb410e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb40c0742b0 0x7fb40c153aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb410e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb40c0b42a0 0x7fb40c153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb410e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 0x7fb40c0afa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb410e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c073280 con 0x7fb40c14f380 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb410e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb40c073100 con 0x7fb40c0742b0 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb410e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb40c073400 con 0x7fb40c0b42a0 2024-09-21T07:53:22.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb40be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 0x7fb40c0afa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb40be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 0x7fb40c0afa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46640/0 (socket says 172.21.15.88:46640) 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.582+0000 7fb40be00640 1 -- 172.21.15.88:0/2018582408 learned_addr learned my addr 172.21.15.88:0/2018582408 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40b400640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb40c0742b0 0x7fb40c153aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40be00640 1 -- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb40c0b42a0 msgr2=0x7fb40c153fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40be00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb40c0b42a0 0x7fb40c153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40be00640 1 -- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb40c0742b0 msgr2=0x7fb40c153aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40be00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb40c0742b0 0x7fb40c153aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40be00640 1 -- 172.21.15.88:0/2018582408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb40c161f70 con 0x7fb40c14f380 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.583+0000 7fb40be00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 0x7fb40c0afa00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb404035040 tx=0x7fb404032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.584+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404037040 con 0x7fb40c14f380 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.584+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb404042070 con 0x7fb40c14f380 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.584+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb40c160be0 con 0x7fb40c14f380 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.584+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb404003e60 con 0x7fb40c14f380 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.584+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb40c0b1580 con 0x7fb40c14f380 2024-09-21T07:53:22.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.585+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb40c1624b0 con 0x7fb40c14f380 2024-09-21T07:53:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.586+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb404004260 con 0x7fb40c14f380 2024-09-21T07:53:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.587+0000 7fb403400640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb3ec0791c0 0x7fb3ec07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.587+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(130..130 src has 1..130) ==== 5637+0+0 (secure 0 0 0) 0x7fb4040bc630 con 0x7fb40c14f380 2024-09-21T07:53:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.587+0000 7fb40b400640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb3ec0791c0 0x7fb3ec07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:22.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.588+0000 7fb40b400640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb3ec0791c0 0x7fb3ec07b6d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007fd0 tx=0x7fb3fc001370 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:22.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.592+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb4040840f0 con 0x7fb40c14f380 2024-09-21T07:53:22.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:22.813+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7fb40c0b2ab0 con 0x7fb40c14f380 2024-09-21T07:53:23.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.314+0000 7fb403400640 1 -- 172.21.15.88:0/2018582408 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v86) ==== 174+0+0 (secure 0 0 0) 0x7fb4040839f0 con 0x7fb40c14f380 2024-09-21T07:53:23.316 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 20 and data pool 21 2024-09-21T07:53:23.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.318+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb3ec0791c0 msgr2=0x7fb3ec07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.318+0000 7fb410e00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb3ec0791c0 0x7fb3ec07b6d0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fb3fc007fd0 tx=0x7fb3fc001370 comp rx=0 tx=0).stop 2024-09-21T07:53:23.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.318+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 msgr2=0x7fb40c0afa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.318+0000 7fb410e00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 0x7fb40c0afa00 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7fb404035040 tx=0x7fb404032d10 comp rx=0 tx=0).stop 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 shutdown_connections 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb3ec0791c0 0x7fb3ec07b6d0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb40c0b42a0 0x7fb40c153fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb40c14f380 0x7fb40c0afa00 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 --2- 172.21.15.88:0/2018582408 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb40c0742b0 0x7fb40c153aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 >> 172.21.15.88:0/2018582408 conn(0x7fb40c06d620 msgr2=0x7fb40c0ad260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:23.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 shutdown_connections 2024-09-21T07:53:23.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.319+0000 7fb410e00640 1 -- 172.21.15.88:0/2018582408 wait complete. 2024-09-21T07:53:23.360 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.580+0000 7fa317c00640 1 -- 172.21.15.88:0/3222290888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 msgr2=0x7fa310073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.580+0000 7fa317c00640 1 --2- 172.21.15.88:0/3222290888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310073500 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7fa3100b2cb0 tx=0x7fa30402a530 comp rx=0 tx=0).stop 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.580+0000 7fa317c00640 1 -- 172.21.15.88:0/3222290888 shutdown_connections 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.580+0000 7fa317c00640 1 --2- 172.21.15.88:0/3222290888 >> v2:172.21.15.175:3300/0 conn(0x7fa310073a40 0x7fa31006f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.580+0000 7fa317c00640 1 --2- 172.21.15.88:0/3222290888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310073500 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.581+0000 7fa317c00640 1 -- 172.21.15.88:0/3222290888 >> 172.21.15.88:0/3222290888 conn(0x7fa31006d730 msgr2=0x7fa3100b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:23.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.581+0000 7fa317c00640 1 -- 172.21.15.88:0/3222290888 shutdown_connections 2024-09-21T07:53:23.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.581+0000 7fa317c00640 1 -- 172.21.15.88:0/3222290888 wait complete. 2024-09-21T07:53:23.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.583+0000 7fa317c00640 1 Processor -- start 2024-09-21T07:53:23.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.583+0000 7fa317c00640 1 -- start start 2024-09-21T07:53:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.584+0000 7fa317c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31006f5e0 0x7fa310156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.584+0000 7fa317c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310073a40 0x7fa3100997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.584+0000 7fa317c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310150940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.584+0000 7fa317c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa310162c70 con 0x7fa3100af630 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.584+0000 7fa317c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa3100b4400 con 0x7fa310073a40 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.584+0000 7fa317c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa310162df0 con 0x7fa31006f5e0 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310150940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310150940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46656/0 (socket says 172.21.15.88:46656) 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 -- 172.21.15.88:0/170562293 learned_addr learned my addr 172.21.15.88:0/170562293 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315200640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31006f5e0 0x7fa310156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa30fe00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310073a40 0x7fa3100997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 -- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31006f5e0 msgr2=0x7fa310156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31006f5e0 0x7fa310156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 -- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310073a40 msgr2=0x7fa3100997e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310073a40 0x7fa3100997e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 -- 172.21.15.88:0/170562293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa310155770 con 0x7fa3100af630 2024-09-21T07:53:23.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.585+0000 7fa315c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310150940 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c760 tx=0x7fa30800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:23.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.586+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa308003da0 con 0x7fa3100af630 2024-09-21T07:53:23.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.586+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa30800ce10 con 0x7fa3100af630 2024-09-21T07:53:23.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.586+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa310154ed0 con 0x7fa3100af630 2024-09-21T07:53:23.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.586+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3080052d0 con 0x7fa3100af630 2024-09-21T07:53:23.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.586+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa310155490 con 0x7fa3100af630 2024-09-21T07:53:23.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.587+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa310163330 con 0x7fa3100af630 2024-09-21T07:53:23.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.588+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fa308005470 con 0x7fa3100af630 2024-09-21T07:53:23.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.589+0000 7fa30cc00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f00791c0 0x7fa2f007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:23.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.589+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5737+0+0 (secure 0 0 0) 0x7fa30809b650 con 0x7fa3100af630 2024-09-21T07:53:23.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.589+0000 7fa315200640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f00791c0 0x7fa2f007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:23.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.590+0000 7fa315200640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f00791c0 0x7fa2f007b6d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa304034000 tx=0x7fa304004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:23.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.594+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa310073e50 con 0x7fa3100af630 2024-09-21T07:53:23.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.780+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa3100afed0 con 0x7fa3100af630 2024-09-21T07:53:23.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.781+0000 7fa30cc00640 1 -- 172.21.15.88:0/170562293 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16808 (secure 0 0 0) 0x7fa3080629d0 con 0x7fa3100af630 2024-09-21T07:53:23.783 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:23.783 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":131,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:22.814841+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:53:20.713784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T07:53:21.774768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:23.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f00791c0 msgr2=0x7fa2f007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f00791c0 0x7fa2f007b6d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fa304034000 tx=0x7fa304004730 comp rx=0 tx=0).stop 2024-09-21T07:53:23.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 msgr2=0x7fa310150940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310150940 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7fa30800c760 tx=0x7fa30800cc20 comp rx=0 tx=0).stop 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 shutdown_connections 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa2f00791c0 0x7fa2f007b6d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3100af630 0x7fa310150940 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa310073a40 0x7fa3100997e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 --2- 172.21.15.88:0/170562293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa31006f5e0 0x7fa310156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 >> 172.21.15.88:0/170562293 conn(0x7fa31006d730 msgr2=0x7fa310162070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 shutdown_connections 2024-09-21T07:53:23.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:23.784+0000 7fa317c00640 1 -- 172.21.15.88:0/170562293 wait complete. 2024-09-21T07:53:23.808 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 -- 172.21.15.88:0/3730131794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f690815d760 msgr2=0x7f69081535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 --2- 172.21.15.88:0/3730131794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f690815d760 0x7f69081535c0 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009830 tx=0x7f68f802a480 comp rx=0 tx=0).stop 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 -- 172.21.15.88:0/3730131794 shutdown_connections 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 --2- 172.21.15.88:0/3730131794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69080ad0e0 0x7f69080ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 --2- 172.21.15.88:0/3730131794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f690815d760 0x7f69081535c0 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 -- 172.21.15.88:0/3730131794 >> 172.21.15.88:0/3730131794 conn(0x7f690806d730 msgr2=0x7f69080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 -- 172.21.15.88:0/3730131794 shutdown_connections 2024-09-21T07:53:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.026+0000 7f690e600640 1 -- 172.21.15.88:0/3730131794 wait complete. 2024-09-21T07:53:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.029+0000 7f690e600640 1 Processor -- start 2024-09-21T07:53:24.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.029+0000 7f690e600640 1 -- start start 2024-09-21T07:53:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f690e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 0x7f690814eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f690e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d760 0x7f69081f8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f690e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6908160310 0x7f6908155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f690e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f690806f220 con 0x7f69080ad0e0 2024-09-21T07:53:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f690e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f690806f0a0 con 0x7f690815d760 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f690e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69080b4c20 con 0x7f6908160310 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.030+0000 7f6907400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 0x7f690814eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 0x7f690814eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46672/0 (socket says 172.21.15.88:46672) 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 -- 172.21.15.88:0/1650322594 learned_addr learned my addr 172.21.15.88:0/1650322594 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907e00640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6908160310 0x7f6908155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6906a00640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d760 0x7f69081f8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 -- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6908160310 msgr2=0x7f6908155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6908160310 0x7f6908155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 -- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d760 msgr2=0x7f69081f8a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d760 0x7f69081f8a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 -- 172.21.15.88:0/1650322594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f690814fe60 con 0x7f69080ad0e0 2024-09-21T07:53:24.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.031+0000 7f6907400640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 0x7f690814eb30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009950 tx=0x7f68f8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:24.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.032+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8005c90 con 0x7f69080ad0e0 2024-09-21T07:53:24.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.032+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68f8031e50 con 0x7f69080ad0e0 2024-09-21T07:53:24.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.032+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68f8005260 con 0x7f69080ad0e0 2024-09-21T07:53:24.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.032+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6908150140 con 0x7f69080ad0e0 2024-09-21T07:53:24.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.032+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6908150700 con 0x7f69080ad0e0 2024-09-21T07:53:24.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.033+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6908157170 con 0x7f69080ad0e0 2024-09-21T07:53:24.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.034+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f68f8005400 con 0x7f69080ad0e0 2024-09-21T07:53:24.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.035+0000 7f68ff400640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f68e80791c0 0x7f68e807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.035+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(131..131 src has 1..131) ==== 5737+0+0 (secure 0 0 0) 0x7f68f80bc870 con 0x7f69080ad0e0 2024-09-21T07:53:24.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.035+0000 7f6906a00640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f68e80791c0 0x7f68e807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.036+0000 7f6906a00640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f68e80791c0 0x7f68e807b6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6908072ff0 tx=0x7f68f400b390 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:24.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.040+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6908157170 con 0x7f69080ad0e0 2024-09-21T07:53:24.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.226+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6908160fa0 con 0x7f69080ad0e0 2024-09-21T07:53:24.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.227+0000 7f68ff400640 1 -- 172.21.15.88:0/1650322594 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v131) ==== 74+0+16808 (secure 0 0 0) 0x7f68f803f030 con 0x7f69080ad0e0 2024-09-21T07:53:24.229 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:24.229 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":131,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:22.814841+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:53:20.713784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T07:53:21.774768+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:24.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f68e80791c0 msgr2=0x7f68e807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f68e80791c0 0x7f68e807b6d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6908072ff0 tx=0x7f68f400b390 comp rx=0 tx=0).stop 2024-09-21T07:53:24.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 msgr2=0x7f690814eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 0x7f690814eb30 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f68f8009950 tx=0x7f68f8033c20 comp rx=0 tx=0).stop 2024-09-21T07:53:24.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 shutdown_connections 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f68e80791c0 0x7f68e807b6d0 secure :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f6908072ff0 tx=0x7f68f400b390 comp rx=0 tx=0).stop 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6908160310 0x7f6908155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f690815d760 0x7f69081f8a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 --2- 172.21.15.88:0/1650322594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69080ad0e0 0x7f690814eb30 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 >> 172.21.15.88:0/1650322594 conn(0x7f690806d730 msgr2=0x7f69080b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 shutdown_connections 2024-09-21T07:53:24.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.230+0000 7f690e600640 1 -- 172.21.15.88:0/1650322594 wait complete. 2024-09-21T07:53:24.254 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:53:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.477+0000 7fc55b000640 1 -- 172.21.15.88:0/1459501033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554073c50 msgr2=0x7fc5540cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.477+0000 7fc55b000640 1 --2- 172.21.15.88:0/1459501033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554073c50 0x7fc5540cd610 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7fc554072bd0 tx=0x7fc54c02a680 comp rx=0 tx=0).stop 2024-09-21T07:53:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.477+0000 7fc55b000640 1 -- 172.21.15.88:0/1459501033 shutdown_connections 2024-09-21T07:53:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.477+0000 7fc55b000640 1 --2- 172.21.15.88:0/1459501033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554073c50 0x7fc5540cd610 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.477+0000 7fc55b000640 1 --2- 172.21.15.88:0/1459501033 >> v2:172.21.15.175:3300/0 conn(0x7fc554145550 0x7fc55406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.479 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.477+0000 7fc55b000640 1 -- 172.21.15.88:0/1459501033 >> 172.21.15.88:0/1459501033 conn(0x7fc55406d1c0 msgr2=0x7fc55406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:24.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.478+0000 7fc55b000640 1 -- 172.21.15.88:0/1459501033 shutdown_connections 2024-09-21T07:53:24.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.478+0000 7fc55b000640 1 -- 172.21.15.88:0/1459501033 wait complete. 2024-09-21T07:53:24.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.480+0000 7fc55b000640 1 Processor -- start 2024-09-21T07:53:24.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.480+0000 7fc55b000640 1 -- start start 2024-09-21T07:53:24.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc55b000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc554145550 0x7fc5541385b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc55b000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 0x7fc5541f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc55b000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc554136050 0x7fc5540b1700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc55b000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5540ce130 con 0x7fc554145a40 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc55b000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5540cdfb0 con 0x7fc554136050 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc55b000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5540ce2b0 con 0x7fc554145550 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.481+0000 7fc553400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 0x7fc5541f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 0x7fc5541f2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46690/0 (socket says 172.21.15.88:46690) 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 -- 172.21.15.88:0/3806926259 learned_addr learned my addr 172.21.15.88:0/3806926259 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc559000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc554136050 0x7fc5540b1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553e00640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc554145550 0x7fc5541385b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 -- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc554145550 msgr2=0x7fc5541385b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.484 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc554145550 0x7fc5541385b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 -- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc554136050 msgr2=0x7fc5540b1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc554136050 0x7fc5540b1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 -- 172.21.15.88:0/3806926259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5541f31f0 con 0x7fc554145a40 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553e00640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc554145550 0x7fc5541385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc553400640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 0x7fc5541f2cb0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c370 tx=0x7fc54000c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540017070 con 0x7fc554145a40 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.482+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5400053c0 con 0x7fc554145a40 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.483+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc540003850 con 0x7fc554145a40 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.483+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc55413c420 con 0x7fc554145a40 2024-09-21T07:53:24.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.483+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc55413c9e0 con 0x7fc554145a40 2024-09-21T07:53:24.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.484+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc55413c190 con 0x7fc554145a40 2024-09-21T07:53:24.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.485+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc54001a070 con 0x7fc554145a40 2024-09-21T07:53:24.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.486+0000 7fc54be00640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5340791c0 0x7fc53407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:24.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.486+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fc54009ac90 con 0x7fc554145a40 2024-09-21T07:53:24.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.486+0000 7fc553e00640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5340791c0 0x7fc53407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:24.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.487+0000 7fc553e00640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5340791c0 0x7fc53407b6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc544005fd0 tx=0x7fc544009480 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:24.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.491+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc540062020 con 0x7fc554145a40 2024-09-21T07:53:24.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:24.715+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fc5540ce7d0 con 0x7fc554145a40 2024-09-21T07:53:25.290 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-09-21T07:53:25.289+0000 7fc5e4800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:53:25.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.343+0000 7fc54be00640 1 -- 172.21.15.88:0/3806926259 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v88) ==== 118+0+0 (secure 0 0 0) 0x7fc540066a50 con 0x7fc554145a40 2024-09-21T07:53:25.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5340791c0 msgr2=0x7fc53407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5340791c0 0x7fc53407b6d0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fc544005fd0 tx=0x7fc544009480 comp rx=0 tx=0).stop 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 msgr2=0x7fc5541f2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 0x7fc5541f2cb0 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7fc54000c370 tx=0x7fc54000c830 comp rx=0 tx=0).stop 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 shutdown_connections 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5340791c0 0x7fc53407b6d0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc554136050 0x7fc5540b1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc554145a40 0x7fc5541f2cb0 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 --2- 172.21.15.88:0/3806926259 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc554145550 0x7fc5541385b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 >> 172.21.15.88:0/3806926259 conn(0x7fc55406d1c0 msgr2=0x7fc5540693b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 shutdown_connections 2024-09-21T07:53:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.346+0000 7fc55b000640 1 -- 172.21.15.88:0/3806926259 wait complete. 2024-09-21T07:53:25.371 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:53:25.371 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:53:25.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 -- 172.21.15.88:0/1735795573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590147b90 msgr2=0x7fb590073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:25.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 --2- 172.21.15.88:0/1735795573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590147b90 0x7fb590073540 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb578009860 tx=0x7fb57802a480 comp rx=0 tx=0).stop 2024-09-21T07:53:25.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 -- 172.21.15.88:0/1735795573 shutdown_connections 2024-09-21T07:53:25.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 --2- 172.21.15.88:0/1735795573 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590147b90 0x7fb590073540 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 -- 172.21.15.88:0/1735795573 >> 172.21.15.88:0/1735795573 conn(0x7fb59006d1c0 msgr2=0x7fb59006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:25.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 -- 172.21.15.88:0/1735795573 shutdown_connections 2024-09-21T07:53:25.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.594+0000 7fb595e00640 1 -- 172.21.15.88:0/1735795573 wait complete. 2024-09-21T07:53:25.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.596+0000 7fb595e00640 1 Processor -- start 2024-09-21T07:53:25.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.596+0000 7fb595e00640 1 -- start start 2024-09-21T07:53:25.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb595e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590073a80 0x7fb5901f8e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:25.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb595e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb590147b90 0x7fb5900c53f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:25.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb595e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 0x7fb5901444a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:25.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb595e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb590148470 con 0x7fb59015f9e0 2024-09-21T07:53:25.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb595e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5901482f0 con 0x7fb590147b90 2024-09-21T07:53:25.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb595e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb59006fee0 con 0x7fb590073a80 2024-09-21T07:53:25.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb58f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 0x7fb5901444a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:25.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb58f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 0x7fb5901444a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46698/0 (socket says 172.21.15.88:46698) 2024-09-21T07:53:25.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.598+0000 7fb58f400640 1 -- 172.21.15.88:0/880710211 learned_addr learned my addr 172.21.15.88:0/880710211 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb58f400640 1 -- 172.21.15.88:0/880710211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590073a80 msgr2=0x7fb5901f8e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb58f400640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590073a80 0x7fb5901f8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb58f400640 1 -- 172.21.15.88:0/880710211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb590147b90 msgr2=0x7fb5900c53f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb58f400640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb590147b90 0x7fb5900c53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb58f400640 1 -- 172.21.15.88:0/880710211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5901449e0 con 0x7fb59015f9e0 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb58f400640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 0x7fb5901444a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c730 tx=0x7fb58800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb588003b70 con 0x7fb59015f9e0 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb588003d10 con 0x7fb59015f9e0 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5901f9450 con 0x7fb59015f9e0 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb58800aab0 con 0x7fb59015f9e0 2024-09-21T07:53:25.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.599+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5901f9a10 con 0x7fb59015f9e0 2024-09-21T07:53:25.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.600+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb590073540 con 0x7fb59015f9e0 2024-09-21T07:53:25.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.601+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb58800ac50 con 0x7fb59015f9e0 2024-09-21T07:53:25.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.602+0000 7fb587400640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb56c0791c0 0x7fb56c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:25.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.602+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fb58809b6e0 con 0x7fb59015f9e0 2024-09-21T07:53:25.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.603+0000 7fb58ea00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb56c0791c0 0x7fb56c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:25.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.603+0000 7fb58ea00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb56c0791c0 0x7fb56c07b6d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb580007fd0 tx=0x7fb5800078a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:25.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.607+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb588063190 con 0x7fb59015f9e0 2024-09-21T07:53:25.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:25.830+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fb5900b46b0 con 0x7fb59015f9e0 2024-09-21T07:53:26.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.350+0000 7fb587400640 1 -- 172.21.15.88:0/880710211 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v89) ==== 122+0+0 (secure 0 0 0) 0x7fb588062a90 con 0x7fb59015f9e0 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb56c0791c0 msgr2=0x7fb56c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb56c0791c0 0x7fb56c07b6d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fb580007fd0 tx=0x7fb5800078a0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 msgr2=0x7fb5901444a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 0x7fb5901444a0 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7fb58800c730 tx=0x7fb58800cbf0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 shutdown_connections 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb56c0791c0 0x7fb56c07b6d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb59015f9e0 0x7fb5901444a0 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb590147b90 0x7fb5900c53f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 --2- 172.21.15.88:0/880710211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb590073a80 0x7fb5901f8e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 >> 172.21.15.88:0/880710211 conn(0x7fb59006d1c0 msgr2=0x7fb5900b3480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.354+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 shutdown_connections 2024-09-21T07:53:26.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.355+0000 7fb595e00640 1 -- 172.21.15.88:0/880710211 wait complete. 2024-09-21T07:53:26.381 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:53:26.381 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:26.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 -- 172.21.15.88:0/1491329521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f899015d670 msgr2=0x7f8990070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 --2- 172.21.15.88:0/1491329521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f899015d670 0x7f8990070530 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f8990051590 tx=0x7f898802a980 comp rx=0 tx=0).stop 2024-09-21T07:53:26.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 -- 172.21.15.88:0/1491329521 shutdown_connections 2024-09-21T07:53:26.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 --2- 172.21.15.88:0/1491329521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f899015d670 0x7f8990070530 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 --2- 172.21.15.88:0/1491329521 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89900b2f60 0x7f899015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 --2- 172.21.15.88:0/1491329521 >> v2:172.21.15.175:3300/0 conn(0x7f89900ad9c0 0x7f89900af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 -- 172.21.15.88:0/1491329521 >> 172.21.15.88:0/1491329521 conn(0x7f89900b1e40 msgr2=0x7f89900b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:26.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 -- 172.21.15.88:0/1491329521 shutdown_connections 2024-09-21T07:53:26.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.600+0000 7f8996600640 1 -- 172.21.15.88:0/1491329521 wait complete. 2024-09-21T07:53:26.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.602+0000 7f8996600640 1 Processor -- start 2024-09-21T07:53:26.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.603+0000 7f8996600640 1 -- start start 2024-09-21T07:53:26.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8996600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 0x7f89900cdb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:26.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8996600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89900b2f60 0x7f8990145a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:26.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8996600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f899015d670 0x7f89900b10a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:26.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8996600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f899006f290 con 0x7f89900ad9c0 2024-09-21T07:53:26.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8996600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8990070c80 con 0x7f899015d670 2024-09-21T07:53:26.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8996600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f899006f410 con 0x7f89900b2f60 2024-09-21T07:53:26.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8995200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 0x7f89900cdb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:26.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8995200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 0x7f89900cdb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35672/0 (socket says 172.21.15.88:35672) 2024-09-21T07:53:26.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.604+0000 7f8995200640 1 -- 172.21.15.88:0/1509592803 learned_addr learned my addr 172.21.15.88:0/1509592803 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:26.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995200640 1 -- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89900b2f60 msgr2=0x7f8990145a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995200640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89900b2f60 0x7f8990145a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995c00640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f899015d670 0x7f89900b10a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:26.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995200640 1 -- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f899015d670 msgr2=0x7f89900b10a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995200640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f899015d670 0x7f89900b10a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995200640 1 -- 172.21.15.88:0/1509592803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f89900ce1d0 con 0x7f89900ad9c0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995c00640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f899015d670 0x7f89900b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8995200640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 0x7f89900cdb10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f898000d770 tx=0x7f898000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8980017070 con 0x7f89900ad9c0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8980009d20 con 0x7f89900ad9c0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f89900ce4b0 con 0x7f89900ad9c0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.605+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f89800106c0 con 0x7f89900ad9c0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.606+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8990157cf0 con 0x7f89900ad9c0 2024-09-21T07:53:26.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.607+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f899006f930 con 0x7f89900ad9c0 2024-09-21T07:53:26.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.608+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f898001a070 con 0x7f89900ad9c0 2024-09-21T07:53:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.609+0000 7f898d600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f896c0791f0 0x7f896c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.609+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7f898009ba30 con 0x7f89900ad9c0 2024-09-21T07:53:26.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.609+0000 7f8987e00640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f896c0791f0 0x7f896c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:26.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.610+0000 7f8987e00640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f896c0791f0 0x7f896c07b700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8990159870 tx=0x7f8978005c90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:26.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.613+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f89800634e0 con 0x7f89900ad9c0 2024-09-21T07:53:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.847+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8990157f50 con 0x7f89900ad9c0 2024-09-21T07:53:26.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.847+0000 7f898d600640 1 -- 172.21.15.88:0/1509592803 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5233 (secure 0 0 0) 0x7f8980062de0 con 0x7f89900ad9c0 2024-09-21T07:53:26.848 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:26.849 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":89,"btime":"2024-09-21T07:53:26:344399+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:53:22.814801+0000","modified":"2024-09-21T07:53:25.831670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2904821684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2904821684},{"type":"v1","addr":"172.21.15.88:6835","nonce":2904821684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7938,"qdb_cluster":[7938]},"id":10}]} 2024-09-21T07:53:26.849 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 89 2024-09-21T07:53:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f896c0791f0 msgr2=0x7f896c07b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f896c0791f0 0x7f896c07b700 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f8990159870 tx=0x7f8978005c90 comp rx=0 tx=0).stop 2024-09-21T07:53:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 msgr2=0x7f89900cdb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 0x7f89900cdb10 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f898000d770 tx=0x7f898000dc30 comp rx=0 tx=0).stop 2024-09-21T07:53:26.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 shutdown_connections 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f896c0791f0 0x7f896c07b700 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f89900b2f60 0x7f8990145a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f899015d670 0x7f89900b10a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 --2- 172.21.15.88:0/1509592803 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f89900ad9c0 0x7f89900cdb10 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 >> 172.21.15.88:0/1509592803 conn(0x7f89900b1e40 msgr2=0x7f89900b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 shutdown_connections 2024-09-21T07:53:26.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:26.850+0000 7f8996600640 1 -- 172.21.15.88:0/1509592803 wait complete. 2024-09-21T07:53:26.874 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 -- 172.21.15.88:0/610259790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 msgr2=0x7f3f64073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 --2- 172.21.15.88:0/610259790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f64073270 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f3f640ad040 tx=0x7f3f4c02a580 comp rx=0 tx=0).stop 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 -- 172.21.15.88:0/610259790 shutdown_connections 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 --2- 172.21.15.88:0/610259790 >> v2:172.21.15.175:3300/0 conn(0x7f3f64070000 0x7f3f64070410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 --2- 172.21.15.88:0/610259790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f64073270 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 --2- 172.21.15.88:0/610259790 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f640af050 0x7f3f640ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 -- 172.21.15.88:0/610259790 >> 172.21.15.88:0/610259790 conn(0x7f3f640b1e40 msgr2=0x7f3f640b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 -- 172.21.15.88:0/610259790 shutdown_connections 2024-09-21T07:53:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.100+0000 7f3f63200640 1 -- 172.21.15.88:0/610259790 wait complete. 2024-09-21T07:53:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.102+0000 7f3f63200640 1 Processor -- start 2024-09-21T07:53:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.103+0000 7f3f63200640 1 -- start start 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f63200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f64070000 0x7f3f6414a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f63200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f641f6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f63200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f640af050 0x7f3f641f6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f63200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6415c090 con 0x7f3f64072e60 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f63200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3f6415bf10 con 0x7f3f64070000 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f63200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3f6415c210 con 0x7f3f640af050 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f641f6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f59400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f641f6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35702/0 (socket says 172.21.15.88:35702) 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.104+0000 7f3f59400640 1 -- 172.21.15.88:0/3676473514 learned_addr learned my addr 172.21.15.88:0/3676473514 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:27.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f61e00640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f64070000 0x7f3f6414a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f59400640 1 -- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f640af050 msgr2=0x7f3f641f6e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f59400640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f640af050 0x7f3f641f6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f59400640 1 -- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f64070000 msgr2=0x7f3f6414a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f59400640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f64070000 0x7f3f6414a260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f59400640 1 -- 172.21.15.88:0/3676473514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3f6414a7a0 con 0x7f3f64072e60 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f61e00640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f64070000 0x7f3f6414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f59400640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f641f6940 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000c00 tx=0x7f3f4c003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c03e030 con 0x7f3f64072e60 2024-09-21T07:53:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3f4c003e00 con 0x7f3f64072e60 2024-09-21T07:53:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3f641e4d40 con 0x7f3f64072e60 2024-09-21T07:53:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c035040 con 0x7f3f64072e60 2024-09-21T07:53:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.105+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3f641f77e0 con 0x7f3f64072e60 2024-09-21T07:53:27.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.106+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3f641f7550 con 0x7f3f64072e60 2024-09-21T07:53:27.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.108+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f3f4c041030 con 0x7f3f64072e60 2024-09-21T07:53:27.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.109+0000 7f3f5aa00640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3f400791c0 0x7f3f4007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.109+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7f3f4c0bc090 con 0x7f3f64072e60 2024-09-21T07:53:27.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.109+0000 7f3f61e00640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3f400791c0 0x7f3f4007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.110+0000 7f3f61e00640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3f400791c0 0x7f3f4007b6d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f54005fd0 tx=0x7f3f54009340 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:27.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.114+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3f4c083bd0 con 0x7f3f64072e60 2024-09-21T07:53:27.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.310+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3f6415c730 con 0x7f3f64072e60 2024-09-21T07:53:27.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.311+0000 7f3f5aa00640 1 -- 172.21.15.88:0/3676473514 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v132) ==== 74+0+16795 (secure 0 0 0) 0x7f3f4c0834d0 con 0x7f3f64072e60 2024-09-21T07:53:27.313 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:27.313 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":132,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:24.326990+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:53:20.713784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T07:53:21.774768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3f400791c0 msgr2=0x7f3f4007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3f400791c0 0x7f3f4007b6d0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f3f54005fd0 tx=0x7f3f54009340 comp rx=0 tx=0).stop 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 msgr2=0x7f3f641f6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f641f6940 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f3f4c000c00 tx=0x7f3f4c003910 comp rx=0 tx=0).stop 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 shutdown_connections 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3f400791c0 0x7f3f4007b6d0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3f640af050 0x7f3f641f6e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3f64072e60 0x7f3f641f6940 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 --2- 172.21.15.88:0/3676473514 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3f64070000 0x7f3f6414a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 >> 172.21.15.88:0/3676473514 conn(0x7f3f640b1e40 msgr2=0x7f3f640afbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 shutdown_connections 2024-09-21T07:53:27.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.314+0000 7f3f63200640 1 -- 172.21.15.88:0/3676473514 wait complete. 2024-09-21T07:53:27.316 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:27.320 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:27.339 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:53:27.340 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:53:27.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 -- 172.21.15.88:0/3027041395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad80b2f60 msgr2=0x7faad815a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 --2- 172.21.15.88:0/3027041395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad80b2f60 0x7faad815a0f0 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7faad8051590 tx=0x7faac402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 -- 172.21.15.88:0/3027041395 shutdown_connections 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 --2- 172.21.15.88:0/3027041395 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faad815d670 0x7faad8070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 --2- 172.21.15.88:0/3027041395 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad80b2f60 0x7faad815a0f0 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 --2- 172.21.15.88:0/3027041395 >> v2:172.21.15.175:3300/0 conn(0x7faad80ad9c0 0x7faad80af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 -- 172.21.15.88:0/3027041395 >> 172.21.15.88:0/3027041395 conn(0x7faad80b1e40 msgr2=0x7faad80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 -- 172.21.15.88:0/3027041395 shutdown_connections 2024-09-21T07:53:27.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.606+0000 7faad7200640 1 -- 172.21.15.88:0/3027041395 wait complete. 2024-09-21T07:53:27.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.608+0000 7faad7200640 1 Processor -- start 2024-09-21T07:53:27.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.609+0000 7faad7200640 1 -- start start 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faad80ad9c0 0x7faad8151950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faad80b2f60 0x7faad8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 0x7faad8156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faad8070c80 con 0x7faad815d670 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faad806f290 con 0x7faad80ad9c0 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faad809ca80 con 0x7faad80b2f60 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 0x7faad8156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 0x7faad8156b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35720/0 (socket says 172.21.15.88:35720) 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad6800640 1 -- 172.21.15.88:0/3620634450 learned_addr learned my addr 172.21.15.88:0/3620634450 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad5e00640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faad80ad9c0 0x7faad8151950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.610+0000 7faad5400640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faad80b2f60 0x7faad8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad6800640 1 -- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faad80b2f60 msgr2=0x7faad8156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad6800640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faad80b2f60 0x7faad8156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad6800640 1 -- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faad80ad9c0 msgr2=0x7faad8151950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad6800640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faad80ad9c0 0x7faad8151950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad6800640 1 -- 172.21.15.88:0/3620634450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faad81571c0 con 0x7faad815d670 2024-09-21T07:53:27.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad6800640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 0x7faad8156b60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7faad000caa0 tx=0x7faad000cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:27.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad000adf0 con 0x7faad815d670 2024-09-21T07:53:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faad000f540 con 0x7faad815d670 2024-09-21T07:53:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faad81574a0 con 0x7faad815d670 2024-09-21T07:53:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.611+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faad000f990 con 0x7faad815d670 2024-09-21T07:53:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.612+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faad8157a60 con 0x7faad815d670 2024-09-21T07:53:27.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.613+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faad815a0f0 con 0x7faad815d670 2024-09-21T07:53:27.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.614+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7faad000fb30 con 0x7faad815d670 2024-09-21T07:53:27.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.615+0000 7faace000640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faab80791c0 0x7faab807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:27.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.615+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7faad009c330 con 0x7faad815d670 2024-09-21T07:53:27.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.615+0000 7faad5e00640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faab80791c0 0x7faab807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:27.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.616+0000 7faad5e00640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faab80791c0 0x7faab807b6d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faac8005fd0 tx=0x7faac800a250 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:27.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.619+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faad0014030 con 0x7faad815d670 2024-09-21T07:53:27.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.808+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7faad8070530 con 0x7faab80791c0 2024-09-21T07:53:27.814 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.813+0000 7faace000640 1 -- 172.21.15.88:0/3620634450 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7faad8070530 con 0x7faab80791c0 2024-09-21T07:53:27.817 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:27.818 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:53:27.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faab80791c0 msgr2=0x7faab807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faab80791c0 0x7faab807b6d0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7faac8005fd0 tx=0x7faac800a250 comp rx=0 tx=0).stop 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 msgr2=0x7faad8156b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 0x7faad8156b60 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7faad000caa0 tx=0x7faad000cf60 comp rx=0 tx=0).stop 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 shutdown_connections 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faab80791c0 0x7faab807b6d0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faad80b2f60 0x7faad8156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faad815d670 0x7faad8156b60 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 --2- 172.21.15.88:0/3620634450 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faad80ad9c0 0x7faad8151950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 >> 172.21.15.88:0/3620634450 conn(0x7faad80b1e40 msgr2=0x7faad80b1020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 shutdown_connections 2024-09-21T07:53:27.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:27.817+0000 7faad7200640 1 -- 172.21.15.88:0/3620634450 wait complete. 2024-09-21T07:53:27.844 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-21T07:53:27.732525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263568,"kb_used_data":14056,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711152,"statfs":{"total":773094113280,"available":772824219648,"internally_reserved":0,"allocated":14393344,"data_stored":6983108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001272"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342737+0000","last_change":"2024-09-21T07:53:23.346330+0000","last_active":"2024-09-21T07:53:24.342737+0000","last_peered":"2024-09-21T07:53:24.342737+0000","last_clean":"2024-09-21T07:53:24.342737+0000","last_became_active":"2024-09-21T07:53:23.345700+0000","last_became_peered":"2024-09-21T07:53:23.345700+0000","last_unstale":"2024-09-21T07:53:24.342737+0000","last_undegraded":"2024-09-21T07:53:24.342737+0000","last_fullsized":"2024-09-21T07:53:24.342737+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:44.177076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161593+0000","last_change":"2024-09-21T07:53:23.351779+0000","last_active":"2024-09-21T07:53:25.161593+0000","last_peered":"2024-09-21T07:53:25.161593+0000","last_clean":"2024-09-21T07:53:25.161593+0000","last_became_active":"2024-09-21T07:53:23.351478+0000","last_became_peered":"2024-09-21T07:53:23.351478+0000","last_unstale":"2024-09-21T07:53:25.161593+0000","last_undegraded":"2024-09-21T07:53:25.161593+0000","last_fullsized":"2024-09-21T07:53:25.161593+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:58.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339529+0000","last_change":"2024-09-21T07:53:23.352174+0000","last_active":"2024-09-21T07:53:24.339529+0000","last_peered":"2024-09-21T07:53:24.339529+0000","last_clean":"2024-09-21T07:53:24.339529+0000","last_became_active":"2024-09-21T07:53:23.351860+0000","last_became_peered":"2024-09-21T07:53:23.351860+0000","last_unstale":"2024-09-21T07:53:24.339529+0000","last_undegraded":"2024-09-21T07:53:24.339529+0000","last_fullsized":"2024-09-21T07:53:24.339529+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:33.997257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659532+0000","last_change":"2024-09-21T07:53:23.361522+0000","last_active":"2024-09-21T07:53:24.659532+0000","last_peered":"2024-09-21T07:53:24.659532+0000","last_clean":"2024-09-21T07:53:24.659532+0000","last_became_active":"2024-09-21T07:53:23.361198+0000","last_became_peered":"2024-09-21T07:53:23.361198+0000","last_unstale":"2024-09-21T07:53:24.659532+0000","last_undegraded":"2024-09-21T07:53:24.659532+0000","last_fullsized":"2024-09-21T07:53:24.659532+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:09.289682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159653+0000","last_change":"2024-09-21T07:53:23.348464+0000","last_active":"2024-09-21T07:53:25.159653+0000","last_peered":"2024-09-21T07:53:25.159653+0000","last_clean":"2024-09-21T07:53:25.159653+0000","last_became_active":"2024-09-21T07:53:23.344682+0000","last_became_peered":"2024-09-21T07:53:23.344682+0000","last_unstale":"2024-09-21T07:53:25.159653+0000","last_undegraded":"2024-09-21T07:53:25.159653+0000","last_fullsized":"2024-09-21T07:53:25.159653+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:19.478639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160582+0000","last_change":"2024-09-21T07:53:23.352697+0000","last_active":"2024-09-21T07:53:25.160582+0000","last_peered":"2024-09-21T07:53:25.160582+0000","last_clean":"2024-09-21T07:53:25.160582+0000","last_became_active":"2024-09-21T07:53:23.352366+0000","last_became_peered":"2024-09-21T07:53:23.352366+0000","last_unstale":"2024-09-21T07:53:25.160582+0000","last_undegraded":"2024-09-21T07:53:25.160582+0000","last_fullsized":"2024-09-21T07:53:25.160582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:58.875225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343138+0000","last_change":"2024-09-21T07:53:23.337428+0000","last_active":"2024-09-21T07:53:24.343138+0000","last_peered":"2024-09-21T07:53:24.343138+0000","last_clean":"2024-09-21T07:53:24.343138+0000","last_became_active":"2024-09-21T07:53:23.336933+0000","last_became_peered":"2024-09-21T07:53:23.336933+0000","last_unstale":"2024-09-21T07:53:24.343138+0000","last_undegraded":"2024-09-21T07:53:24.343138+0000","last_fullsized":"2024-09-21T07:53:24.343138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341738+0000","last_change":"2024-09-21T07:53:23.341543+0000","last_active":"2024-09-21T07:53:24.341738+0000","last_peered":"2024-09-21T07:53:24.341738+0000","last_clean":"2024-09-21T07:53:24.341738+0000","last_became_active":"2024-09-21T07:53:23.341175+0000","last_became_peered":"2024-09-21T07:53:23.341175+0000","last_unstale":"2024-09-21T07:53:24.341738+0000","last_undegraded":"2024-09-21T07:53:24.341738+0000","last_fullsized":"2024-09-21T07:53:24.341738+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:20.362969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160885+0000","last_change":"2024-09-21T07:53:23.336289+0000","last_active":"2024-09-21T07:53:25.160885+0000","last_peered":"2024-09-21T07:53:25.160885+0000","last_clean":"2024-09-21T07:53:25.160885+0000","last_became_active":"2024-09-21T07:53:23.335516+0000","last_became_peered":"2024-09-21T07:53:23.335516+0000","last_unstale":"2024-09-21T07:53:25.160885+0000","last_undegraded":"2024-09-21T07:53:25.160885+0000","last_fullsized":"2024-09-21T07:53:25.160885+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:59.097440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339850+0000","last_change":"2024-09-21T07:53:23.355115+0000","last_active":"2024-09-21T07:53:24.339850+0000","last_peered":"2024-09-21T07:53:24.339850+0000","last_clean":"2024-09-21T07:53:24.339850+0000","last_became_active":"2024-09-21T07:53:23.354759+0000","last_became_peered":"2024-09-21T07:53:23.354759+0000","last_unstale":"2024-09-21T07:53:24.339850+0000","last_undegraded":"2024-09-21T07:53:24.339850+0000","last_fullsized":"2024-09-21T07:53:24.339850+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:05.879117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160856+0000","last_change":"2024-09-21T07:53:23.344727+0000","last_active":"2024-09-21T07:53:25.160856+0000","last_peered":"2024-09-21T07:53:25.160856+0000","last_clean":"2024-09-21T07:53:25.160856+0000","last_became_active":"2024-09-21T07:53:23.344378+0000","last_became_peered":"2024-09-21T07:53:23.344378+0000","last_unstale":"2024-09-21T07:53:25.160856+0000","last_undegraded":"2024-09-21T07:53:25.160856+0000","last_fullsized":"2024-09-21T07:53:25.160856+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:12.332318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633932+0000","last_change":"2024-09-21T07:53:23.348908+0000","last_active":"2024-09-21T07:53:24.633932+0000","last_peered":"2024-09-21T07:53:24.633932+0000","last_clean":"2024-09-21T07:53:24.633932+0000","last_became_active":"2024-09-21T07:53:23.348553+0000","last_became_peered":"2024-09-21T07:53:23.348553+0000","last_unstale":"2024-09-21T07:53:24.633932+0000","last_undegraded":"2024-09-21T07:53:24.633932+0000","last_fullsized":"2024-09-21T07:53:24.633932+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:37.807129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159635+0000","last_change":"2024-09-21T07:53:23.348131+0000","last_active":"2024-09-21T07:53:25.159635+0000","last_peered":"2024-09-21T07:53:25.159635+0000","last_clean":"2024-09-21T07:53:25.159635+0000","last_became_active":"2024-09-21T07:53:23.344317+0000","last_became_peered":"2024-09-21T07:53:23.344317+0000","last_unstale":"2024-09-21T07:53:25.159635+0000","last_undegraded":"2024-09-21T07:53:25.159635+0000","last_fullsized":"2024-09-21T07:53:25.159635+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:22.681966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634041+0000","last_change":"2024-09-21T07:53:23.359245+0000","last_active":"2024-09-21T07:53:24.634041+0000","last_peered":"2024-09-21T07:53:24.634041+0000","last_clean":"2024-09-21T07:53:24.634041+0000","last_became_active":"2024-09-21T07:53:23.358932+0000","last_became_peered":"2024-09-21T07:53:23.358932+0000","last_unstale":"2024-09-21T07:53:24.634041+0000","last_undegraded":"2024-09-21T07:53:24.634041+0000","last_fullsized":"2024-09-21T07:53:24.634041+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:47.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159673+0000","last_change":"2024-09-21T07:53:23.328227+0000","last_active":"2024-09-21T07:53:25.159673+0000","last_peered":"2024-09-21T07:53:25.159673+0000","last_clean":"2024-09-21T07:53:25.159673+0000","last_became_active":"2024-09-21T07:53:23.325718+0000","last_became_peered":"2024-09-21T07:53:23.325718+0000","last_unstale":"2024-09-21T07:53:25.159673+0000","last_undegraded":"2024-09-21T07:53:25.159673+0000","last_fullsized":"2024-09-21T07:53:25.159673+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:46.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341889+0000","last_change":"2024-09-21T07:53:23.334074+0000","last_active":"2024-09-21T07:53:24.341889+0000","last_peered":"2024-09-21T07:53:24.341889+0000","last_clean":"2024-09-21T07:53:24.341889+0000","last_became_active":"2024-09-21T07:53:23.333730+0000","last_became_peered":"2024-09-21T07:53:23.333730+0000","last_unstale":"2024-09-21T07:53:24.341889+0000","last_undegraded":"2024-09-21T07:53:24.341889+0000","last_fullsized":"2024-09-21T07:53:24.341889+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:46.745190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339678+0000","last_change":"2024-09-21T07:53:23.349261+0000","last_active":"2024-09-21T07:53:24.339678+0000","last_peered":"2024-09-21T07:53:24.339678+0000","last_clean":"2024-09-21T07:53:24.339678+0000","last_became_active":"2024-09-21T07:53:23.348964+0000","last_became_peered":"2024-09-21T07:53:23.348964+0000","last_unstale":"2024-09-21T07:53:24.339678+0000","last_undegraded":"2024-09-21T07:53:24.339678+0000","last_fullsized":"2024-09-21T07:53:24.339678+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:56.672740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159648+0000","last_change":"2024-09-21T07:53:23.338050+0000","last_active":"2024-09-21T07:53:25.159648+0000","last_peered":"2024-09-21T07:53:25.159648+0000","last_clean":"2024-09-21T07:53:25.159648+0000","last_became_active":"2024-09-21T07:53:23.337355+0000","last_became_peered":"2024-09-21T07:53:23.337355+0000","last_unstale":"2024-09-21T07:53:25.159648+0000","last_undegraded":"2024-09-21T07:53:25.159648+0000","last_fullsized":"2024-09-21T07:53:25.159648+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:20.228459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342238+0000","last_change":"2024-09-21T07:53:23.334489+0000","last_active":"2024-09-21T07:53:24.342238+0000","last_peered":"2024-09-21T07:53:24.342238+0000","last_clean":"2024-09-21T07:53:24.342238+0000","last_became_active":"2024-09-21T07:53:23.331209+0000","last_became_peered":"2024-09-21T07:53:23.331209+0000","last_unstale":"2024-09-21T07:53:24.342238+0000","last_undegraded":"2024-09-21T07:53:24.342238+0000","last_fullsized":"2024-09-21T07:53:24.342238+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:52.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342078+0000","last_change":"2024-09-21T07:53:23.345801+0000","last_active":"2024-09-21T07:53:24.342078+0000","last_peered":"2024-09-21T07:53:24.342078+0000","last_clean":"2024-09-21T07:53:24.342078+0000","last_became_active":"2024-09-21T07:53:23.345447+0000","last_became_peered":"2024-09-21T07:53:23.345447+0000","last_unstale":"2024-09-21T07:53:24.342078+0000","last_undegraded":"2024-09-21T07:53:24.342078+0000","last_fullsized":"2024-09-21T07:53:24.342078+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:28.814323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339576+0000","last_change":"2024-09-21T07:53:23.350589+0000","last_active":"2024-09-21T07:53:24.339576+0000","last_peered":"2024-09-21T07:53:24.339576+0000","last_clean":"2024-09-21T07:53:24.339576+0000","last_became_active":"2024-09-21T07:53:23.350231+0000","last_became_peered":"2024-09-21T07:53:23.350231+0000","last_unstale":"2024-09-21T07:53:24.339576+0000","last_undegraded":"2024-09-21T07:53:24.339576+0000","last_fullsized":"2024-09-21T07:53:24.339576+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:13.370509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633971+0000","last_change":"2024-09-21T07:53:23.351644+0000","last_active":"2024-09-21T07:53:24.633971+0000","last_peered":"2024-09-21T07:53:24.633971+0000","last_clean":"2024-09-21T07:53:24.633971+0000","last_became_active":"2024-09-21T07:53:23.351336+0000","last_became_peered":"2024-09-21T07:53:23.351336+0000","last_unstale":"2024-09-21T07:53:24.633971+0000","last_undegraded":"2024-09-21T07:53:24.633971+0000","last_fullsized":"2024-09-21T07:53:24.633971+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:05.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659077+0000","last_change":"2024-09-21T07:53:23.340055+0000","last_active":"2024-09-21T07:53:24.659077+0000","last_peered":"2024-09-21T07:53:24.659077+0000","last_clean":"2024-09-21T07:53:24.659077+0000","last_became_active":"2024-09-21T07:53:23.339796+0000","last_became_peered":"2024-09-21T07:53:23.339796+0000","last_unstale":"2024-09-21T07:53:24.659077+0000","last_undegraded":"2024-09-21T07:53:24.659077+0000","last_fullsized":"2024-09-21T07:53:24.659077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:42.532733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159630+0000","last_change":"2024-09-21T07:53:23.345888+0000","last_active":"2024-09-21T07:53:25.159630+0000","last_peered":"2024-09-21T07:53:25.159630+0000","last_clean":"2024-09-21T07:53:25.159630+0000","last_became_active":"2024-09-21T07:53:23.345623+0000","last_became_peered":"2024-09-21T07:53:23.345623+0000","last_unstale":"2024-09-21T07:53:25.159630+0000","last_undegraded":"2024-09-21T07:53:25.159630+0000","last_fullsized":"2024-09-21T07:53:25.159630+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.936282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161760+0000","last_change":"2024-09-21T07:53:23.353048+0000","last_active":"2024-09-21T07:53:25.161760+0000","last_peered":"2024-09-21T07:53:25.161760+0000","last_clean":"2024-09-21T07:53:25.161760+0000","last_became_active":"2024-09-21T07:53:23.352717+0000","last_became_peered":"2024-09-21T07:53:23.352717+0000","last_unstale":"2024-09-21T07:53:25.161760+0000","last_undegraded":"2024-09-21T07:53:25.161760+0000","last_fullsized":"2024-09-21T07:53:25.161760+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:08.365643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343120+0000","last_change":"2024-09-21T07:53:23.344361+0000","last_active":"2024-09-21T07:53:24.343120+0000","last_peered":"2024-09-21T07:53:24.343120+0000","last_clean":"2024-09-21T07:53:24.343120+0000","last_became_active":"2024-09-21T07:53:23.342635+0000","last_became_peered":"2024-09-21T07:53:23.342635+0000","last_unstale":"2024-09-21T07:53:24.343120+0000","last_undegraded":"2024-09-21T07:53:24.343120+0000","last_fullsized":"2024-09-21T07:53:24.343120+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:31.577680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161649+0000","last_change":"2024-09-21T07:53:23.357160+0000","last_active":"2024-09-21T07:53:25.161649+0000","last_peered":"2024-09-21T07:53:25.161649+0000","last_clean":"2024-09-21T07:53:25.161649+0000","last_became_active":"2024-09-21T07:53:23.354546+0000","last_became_peered":"2024-09-21T07:53:23.354546+0000","last_unstale":"2024-09-21T07:53:25.161649+0000","last_undegraded":"2024-09-21T07:53:25.161649+0000","last_fullsized":"2024-09-21T07:53:25.161649+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:43.152366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161182+0000","last_change":"2024-09-21T07:53:23.353557+0000","last_active":"2024-09-21T07:53:25.161182+0000","last_peered":"2024-09-21T07:53:25.161182+0000","last_clean":"2024-09-21T07:53:25.161182+0000","last_became_active":"2024-09-21T07:53:23.353227+0000","last_became_peered":"2024-09-21T07:53:23.353227+0000","last_unstale":"2024-09-21T07:53:25.161182+0000","last_undegraded":"2024-09-21T07:53:25.161182+0000","last_fullsized":"2024-09-21T07:53:25.161182+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:56.037928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342841+0000","last_change":"2024-09-21T07:53:23.347421+0000","last_active":"2024-09-21T07:53:24.342841+0000","last_peered":"2024-09-21T07:53:24.342841+0000","last_clean":"2024-09-21T07:53:24.342841+0000","last_became_active":"2024-09-21T07:53:23.345414+0000","last_became_peered":"2024-09-21T07:53:23.345414+0000","last_unstale":"2024-09-21T07:53:24.342841+0000","last_undegraded":"2024-09-21T07:53:24.342841+0000","last_fullsized":"2024-09-21T07:53:24.342841+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:49.234044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634240+0000","last_change":"2024-09-21T07:53:23.347689+0000","last_active":"2024-09-21T07:53:24.634240+0000","last_peered":"2024-09-21T07:53:24.634240+0000","last_clean":"2024-09-21T07:53:24.634240+0000","last_became_active":"2024-09-21T07:53:23.347378+0000","last_became_peered":"2024-09-21T07:53:23.347378+0000","last_unstale":"2024-09-21T07:53:24.634240+0000","last_undegraded":"2024-09-21T07:53:24.634240+0000","last_fullsized":"2024-09-21T07:53:24.634240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:14.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160921+0000","last_change":"2024-09-21T07:53:23.356723+0000","last_active":"2024-09-21T07:53:25.160921+0000","last_peered":"2024-09-21T07:53:25.160921+0000","last_clean":"2024-09-21T07:53:25.160921+0000","last_became_active":"2024-09-21T07:53:23.355978+0000","last_became_peered":"2024-09-21T07:53:23.355978+0000","last_unstale":"2024-09-21T07:53:25.160921+0000","last_undegraded":"2024-09-21T07:53:25.160921+0000","last_fullsized":"2024-09-21T07:53:25.160921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:55.217591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160582+0000","last_change":"2024-09-21T07:53:23.349494+0000","last_active":"2024-09-21T07:53:25.160582+0000","last_peered":"2024-09-21T07:53:25.160582+0000","last_clean":"2024-09-21T07:53:25.160582+0000","last_became_active":"2024-09-21T07:53:23.348873+0000","last_became_peered":"2024-09-21T07:53:23.348873+0000","last_unstale":"2024-09-21T07:53:25.160582+0000","last_undegraded":"2024-09-21T07:53:25.160582+0000","last_fullsized":"2024-09-21T07:53:25.160582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:10.236171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159604+0000","last_change":"2024-09-21T07:53:23.338363+0000","last_active":"2024-09-21T07:53:25.159604+0000","last_peered":"2024-09-21T07:53:25.159604+0000","last_clean":"2024-09-21T07:53:25.159604+0000","last_became_active":"2024-09-21T07:53:23.337764+0000","last_became_peered":"2024-09-21T07:53:23.337764+0000","last_unstale":"2024-09-21T07:53:25.159604+0000","last_undegraded":"2024-09-21T07:53:25.159604+0000","last_fullsized":"2024-09-21T07:53:25.159604+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:14.026339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"131'4","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339735+0000","last_change":"2024-09-21T07:53:22.336923+0000","last_active":"2024-09-21T07:53:24.339735+0000","last_peered":"2024-09-21T07:53:24.339735+0000","last_clean":"2024-09-21T07:53:24.339735+0000","last_became_active":"2024-09-21T07:53:22.335849+0000","last_became_peered":"2024-09-21T07:53:22.335849+0000","last_unstale":"2024-09-21T07:53:24.339735+0000","last_undegraded":"2024-09-21T07:53:24.339735+0000","last_fullsized":"2024-09-21T07:53:24.339735+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:41.158853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343293+0000","last_change":"2024-09-21T07:53:23.330283+0000","last_active":"2024-09-21T07:53:24.343293+0000","last_peered":"2024-09-21T07:53:24.343293+0000","last_clean":"2024-09-21T07:53:24.343293+0000","last_became_active":"2024-09-21T07:53:23.329971+0000","last_became_peered":"2024-09-21T07:53:23.329971+0000","last_unstale":"2024-09-21T07:53:24.343293+0000","last_undegraded":"2024-09-21T07:53:24.343293+0000","last_fullsized":"2024-09-21T07:53:24.343293+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:14.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161118+0000","last_change":"2024-09-21T07:53:22.335291+0000","last_active":"2024-09-21T07:53:25.161118+0000","last_peered":"2024-09-21T07:53:25.161118+0000","last_clean":"2024-09-21T07:53:25.161118+0000","last_became_active":"2024-09-21T07:53:22.334974+0000","last_became_peered":"2024-09-21T07:53:22.334974+0000","last_unstale":"2024-09-21T07:53:25.161118+0000","last_undegraded":"2024-09-21T07:53:25.161118+0000","last_fullsized":"2024-09-21T07:53:25.161118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:42.709866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160733+0000","last_change":"2024-09-21T07:53:23.344768+0000","last_active":"2024-09-21T07:53:25.160733+0000","last_peered":"2024-09-21T07:53:25.160733+0000","last_clean":"2024-09-21T07:53:25.160733+0000","last_became_active":"2024-09-21T07:53:23.343531+0000","last_became_peered":"2024-09-21T07:53:23.343531+0000","last_unstale":"2024-09-21T07:53:25.160733+0000","last_undegraded":"2024-09-21T07:53:25.160733+0000","last_fullsized":"2024-09-21T07:53:25.160733+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:24.703754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339572+0000","last_change":"2024-09-21T07:53:22.330500+0000","last_active":"2024-09-21T07:53:24.339572+0000","last_peered":"2024-09-21T07:53:24.339572+0000","last_clean":"2024-09-21T07:53:24.339572+0000","last_became_active":"2024-09-21T07:53:22.330179+0000","last_became_peered":"2024-09-21T07:53:22.330179+0000","last_unstale":"2024-09-21T07:53:24.339572+0000","last_undegraded":"2024-09-21T07:53:24.339572+0000","last_fullsized":"2024-09-21T07:53:24.339572+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:26.996930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659165+0000","last_change":"2024-09-21T07:53:23.360077+0000","last_active":"2024-09-21T07:53:24.659165+0000","last_peered":"2024-09-21T07:53:24.659165+0000","last_clean":"2024-09-21T07:53:24.659165+0000","last_became_active":"2024-09-21T07:53:23.359622+0000","last_became_peered":"2024-09-21T07:53:23.359622+0000","last_unstale":"2024-09-21T07:53:24.659165+0000","last_undegraded":"2024-09-21T07:53:24.659165+0000","last_fullsized":"2024-09-21T07:53:24.659165+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:23.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339513+0000","last_change":"2024-09-21T07:53:22.337829+0000","last_active":"2024-09-21T07:53:24.339513+0000","last_peered":"2024-09-21T07:53:24.339513+0000","last_clean":"2024-09-21T07:53:24.339513+0000","last_became_active":"2024-09-21T07:53:22.337298+0000","last_became_peered":"2024-09-21T07:53:22.337298+0000","last_unstale":"2024-09-21T07:53:24.339513+0000","last_undegraded":"2024-09-21T07:53:24.339513+0000","last_fullsized":"2024-09-21T07:53:24.339513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:44.599956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160895+0000","last_change":"2024-09-21T07:53:23.344990+0000","last_active":"2024-09-21T07:53:25.160895+0000","last_peered":"2024-09-21T07:53:25.160895+0000","last_clean":"2024-09-21T07:53:25.160895+0000","last_became_active":"2024-09-21T07:53:23.344686+0000","last_became_peered":"2024-09-21T07:53:23.344686+0000","last_unstale":"2024-09-21T07:53:25.160895+0000","last_undegraded":"2024-09-21T07:53:25.160895+0000","last_fullsized":"2024-09-21T07:53:25.160895+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:12.356811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342921+0000","last_change":"2024-09-21T07:53:22.338312+0000","last_active":"2024-09-21T07:53:24.342921+0000","last_peered":"2024-09-21T07:53:24.342921+0000","last_clean":"2024-09-21T07:53:24.342921+0000","last_became_active":"2024-09-21T07:53:22.337906+0000","last_became_peered":"2024-09-21T07:53:22.337906+0000","last_unstale":"2024-09-21T07:53:24.342921+0000","last_undegraded":"2024-09-21T07:53:24.342921+0000","last_fullsized":"2024-09-21T07:53:24.342921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:51.297854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659763+0000","last_change":"2024-09-21T07:53:23.356604+0000","last_active":"2024-09-21T07:53:24.659763+0000","last_peered":"2024-09-21T07:53:24.659763+0000","last_clean":"2024-09-21T07:53:24.659763+0000","last_became_active":"2024-09-21T07:53:23.355855+0000","last_became_peered":"2024-09-21T07:53:23.355855+0000","last_unstale":"2024-09-21T07:53:24.659763+0000","last_undegraded":"2024-09-21T07:53:24.659763+0000","last_fullsized":"2024-09-21T07:53:24.659763+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:05.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159229+0000","last_change":"2024-09-21T07:53:22.334217+0000","last_active":"2024-09-21T07:53:25.159229+0000","last_peered":"2024-09-21T07:53:25.159229+0000","last_clean":"2024-09-21T07:53:25.159229+0000","last_became_active":"2024-09-21T07:53:22.333851+0000","last_became_peered":"2024-09-21T07:53:22.333851+0000","last_unstale":"2024-09-21T07:53:25.159229+0000","last_undegraded":"2024-09-21T07:53:25.159229+0000","last_fullsized":"2024-09-21T07:53:25.159229+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:42.592475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161205+0000","last_change":"2024-09-21T07:53:23.355989+0000","last_active":"2024-09-21T07:53:25.161205+0000","last_peered":"2024-09-21T07:53:25.161205+0000","last_clean":"2024-09-21T07:53:25.161205+0000","last_became_active":"2024-09-21T07:53:23.355584+0000","last_became_peered":"2024-09-21T07:53:23.355584+0000","last_unstale":"2024-09-21T07:53:25.161205+0000","last_undegraded":"2024-09-21T07:53:25.161205+0000","last_fullsized":"2024-09-21T07:53:25.161205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:26.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160457+0000","last_change":"2024-09-21T07:53:22.337478+0000","last_active":"2024-09-21T07:53:25.160457+0000","last_peered":"2024-09-21T07:53:25.160457+0000","last_clean":"2024-09-21T07:53:25.160457+0000","last_became_active":"2024-09-21T07:53:22.337105+0000","last_became_peered":"2024-09-21T07:53:22.337105+0000","last_unstale":"2024-09-21T07:53:25.160457+0000","last_undegraded":"2024-09-21T07:53:25.160457+0000","last_fullsized":"2024-09-21T07:53:25.160457+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:24.195429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659209+0000","last_change":"2024-09-21T07:53:23.343654+0000","last_active":"2024-09-21T07:53:24.659209+0000","last_peered":"2024-09-21T07:53:24.659209+0000","last_clean":"2024-09-21T07:53:24.659209+0000","last_became_active":"2024-09-21T07:53:23.343302+0000","last_became_peered":"2024-09-21T07:53:23.343302+0000","last_unstale":"2024-09-21T07:53:24.659209+0000","last_undegraded":"2024-09-21T07:53:24.659209+0000","last_fullsized":"2024-09-21T07:53:24.659209+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:33.271211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339169+0000","last_change":"2024-09-21T07:53:22.343143+0000","last_active":"2024-09-21T07:53:24.339169+0000","last_peered":"2024-09-21T07:53:24.339169+0000","last_clean":"2024-09-21T07:53:24.339169+0000","last_became_active":"2024-09-21T07:53:22.342842+0000","last_became_peered":"2024-09-21T07:53:22.342842+0000","last_unstale":"2024-09-21T07:53:24.339169+0000","last_undegraded":"2024-09-21T07:53:24.339169+0000","last_fullsized":"2024-09-21T07:53:24.339169+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:56.140214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659512+0000","last_change":"2024-09-21T07:53:23.342441+0000","last_active":"2024-09-21T07:53:24.659512+0000","last_peered":"2024-09-21T07:53:24.659512+0000","last_clean":"2024-09-21T07:53:24.659512+0000","last_became_active":"2024-09-21T07:53:23.341785+0000","last_became_peered":"2024-09-21T07:53:23.341785+0000","last_unstale":"2024-09-21T07:53:24.659512+0000","last_undegraded":"2024-09-21T07:53:24.659512+0000","last_fullsized":"2024-09-21T07:53:24.659512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:27.814113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160965+0000","last_change":"2024-09-21T07:53:22.337188+0000","last_active":"2024-09-21T07:53:25.160965+0000","last_peered":"2024-09-21T07:53:25.160965+0000","last_clean":"2024-09-21T07:53:25.160965+0000","last_became_active":"2024-09-21T07:53:22.336636+0000","last_became_peered":"2024-09-21T07:53:22.336636+0000","last_unstale":"2024-09-21T07:53:25.160965+0000","last_undegraded":"2024-09-21T07:53:25.160965+0000","last_fullsized":"2024-09-21T07:53:25.160965+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:41.781693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634038+0000","last_change":"2024-09-21T07:53:23.353349+0000","last_active":"2024-09-21T07:53:24.634038+0000","last_peered":"2024-09-21T07:53:24.634038+0000","last_clean":"2024-09-21T07:53:24.634038+0000","last_became_active":"2024-09-21T07:53:23.352301+0000","last_became_peered":"2024-09-21T07:53:23.352301+0000","last_unstale":"2024-09-21T07:53:24.634038+0000","last_undegraded":"2024-09-21T07:53:24.634038+0000","last_fullsized":"2024-09-21T07:53:24.634038+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.782721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159735+0000","last_change":"2024-09-21T07:53:22.328129+0000","last_active":"2024-09-21T07:53:25.159735+0000","last_peered":"2024-09-21T07:53:25.159735+0000","last_clean":"2024-09-21T07:53:25.159735+0000","last_became_active":"2024-09-21T07:53:22.327767+0000","last_became_peered":"2024-09-21T07:53:22.327767+0000","last_unstale":"2024-09-21T07:53:25.159735+0000","last_undegraded":"2024-09-21T07:53:25.159735+0000","last_fullsized":"2024-09-21T07:53:25.159735+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.924703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342549+0000","last_change":"2024-09-21T07:53:23.334234+0000","last_active":"2024-09-21T07:53:24.342549+0000","last_peered":"2024-09-21T07:53:24.342549+0000","last_clean":"2024-09-21T07:53:24.342549+0000","last_became_active":"2024-09-21T07:53:23.333192+0000","last_became_peered":"2024-09-21T07:53:23.333192+0000","last_unstale":"2024-09-21T07:53:24.342549+0000","last_undegraded":"2024-09-21T07:53:24.342549+0000","last_fullsized":"2024-09-21T07:53:24.342549+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.782721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"131'6","reported_seq":21,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342486+0000","last_change":"2024-09-21T07:53:22.336657+0000","last_active":"2024-09-21T07:53:24.342486+0000","last_peered":"2024-09-21T07:53:24.342486+0000","last_clean":"2024-09-21T07:53:24.342486+0000","last_became_active":"2024-09-21T07:53:22.335160+0000","last_became_peered":"2024-09-21T07:53:22.335160+0000","last_unstale":"2024-09-21T07:53:24.342486+0000","last_undegraded":"2024-09-21T07:53:24.342486+0000","last_fullsized":"2024-09-21T07:53:24.342486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:28.497583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659427+0000","last_change":"2024-09-21T07:53:23.344658+0000","last_active":"2024-09-21T07:53:24.659427+0000","last_peered":"2024-09-21T07:53:24.659427+0000","last_clean":"2024-09-21T07:53:24.659427+0000","last_became_active":"2024-09-21T07:53:23.343353+0000","last_became_peered":"2024-09-21T07:53:23.343353+0000","last_unstale":"2024-09-21T07:53:24.659427+0000","last_undegraded":"2024-09-21T07:53:24.659427+0000","last_fullsized":"2024-09-21T07:53:24.659427+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.875543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339630+0000","last_change":"2024-09-21T07:53:22.333123+0000","last_active":"2024-09-21T07:53:24.339630+0000","last_peered":"2024-09-21T07:53:24.339630+0000","last_clean":"2024-09-21T07:53:24.339630+0000","last_became_active":"2024-09-21T07:53:22.332816+0000","last_became_peered":"2024-09-21T07:53:22.332816+0000","last_unstale":"2024-09-21T07:53:24.339630+0000","last_undegraded":"2024-09-21T07:53:24.339630+0000","last_fullsized":"2024-09-21T07:53:24.339630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:28.798447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343169+0000","last_change":"2024-09-21T07:53:23.344841+0000","last_active":"2024-09-21T07:53:24.343169+0000","last_peered":"2024-09-21T07:53:24.343169+0000","last_clean":"2024-09-21T07:53:24.343169+0000","last_became_active":"2024-09-21T07:53:23.342810+0000","last_became_peered":"2024-09-21T07:53:23.342810+0000","last_unstale":"2024-09-21T07:53:24.343169+0000","last_undegraded":"2024-09-21T07:53:24.343169+0000","last_fullsized":"2024-09-21T07:53:24.343169+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:20.299724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634232+0000","last_change":"2024-09-21T07:53:22.336709+0000","last_active":"2024-09-21T07:53:24.634232+0000","last_peered":"2024-09-21T07:53:24.634232+0000","last_clean":"2024-09-21T07:53:24.634232+0000","last_became_active":"2024-09-21T07:53:22.336363+0000","last_became_peered":"2024-09-21T07:53:22.336363+0000","last_unstale":"2024-09-21T07:53:24.634232+0000","last_undegraded":"2024-09-21T07:53:24.634232+0000","last_fullsized":"2024-09-21T07:53:24.634232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:37.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.658963+0000","last_change":"2024-09-21T07:53:23.356694+0000","last_active":"2024-09-21T07:53:24.658963+0000","last_peered":"2024-09-21T07:53:24.658963+0000","last_clean":"2024-09-21T07:53:24.658963+0000","last_became_active":"2024-09-21T07:53:23.356424+0000","last_became_peered":"2024-09-21T07:53:23.356424+0000","last_unstale":"2024-09-21T07:53:24.658963+0000","last_undegraded":"2024-09-21T07:53:24.658963+0000","last_fullsized":"2024-09-21T07:53:24.658963+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:32.082961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160994+0000","last_change":"2024-09-21T07:53:22.340448+0000","last_active":"2024-09-21T07:53:25.160994+0000","last_peered":"2024-09-21T07:53:25.160994+0000","last_clean":"2024-09-21T07:53:25.160994+0000","last_became_active":"2024-09-21T07:53:22.339515+0000","last_became_peered":"2024-09-21T07:53:22.339515+0000","last_unstale":"2024-09-21T07:53:25.160994+0000","last_undegraded":"2024-09-21T07:53:25.160994+0000","last_fullsized":"2024-09-21T07:53:25.160994+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:06.963231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659355+0000","last_change":"2024-09-21T07:53:23.342714+0000","last_active":"2024-09-21T07:53:24.659355+0000","last_peered":"2024-09-21T07:53:24.659355+0000","last_clean":"2024-09-21T07:53:24.659355+0000","last_became_active":"2024-09-21T07:53:23.342112+0000","last_became_peered":"2024-09-21T07:53:23.342112+0000","last_unstale":"2024-09-21T07:53:24.659355+0000","last_undegraded":"2024-09-21T07:53:24.659355+0000","last_fullsized":"2024-09-21T07:53:24.659355+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:49.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"131'8","reported_seq":23,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341623+0000","last_change":"2024-09-21T07:53:22.343094+0000","last_active":"2024-09-21T07:53:24.341623+0000","last_peered":"2024-09-21T07:53:24.341623+0000","last_clean":"2024-09-21T07:53:24.341623+0000","last_became_active":"2024-09-21T07:53:22.342480+0000","last_became_peered":"2024-09-21T07:53:22.342480+0000","last_unstale":"2024-09-21T07:53:24.341623+0000","last_undegraded":"2024-09-21T07:53:24.341623+0000","last_fullsized":"2024-09-21T07:53:24.341623+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:35.559106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159789+0000","last_change":"2024-09-21T07:53:23.345811+0000","last_active":"2024-09-21T07:53:25.159789+0000","last_peered":"2024-09-21T07:53:25.159789+0000","last_clean":"2024-09-21T07:53:25.159789+0000","last_became_active":"2024-09-21T07:53:23.345473+0000","last_became_peered":"2024-09-21T07:53:23.345473+0000","last_unstale":"2024-09-21T07:53:25.159789+0000","last_undegraded":"2024-09-21T07:53:25.159789+0000","last_fullsized":"2024-09-21T07:53:25.159789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:20.242396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659588+0000","last_change":"2024-09-21T07:53:22.340228+0000","last_active":"2024-09-21T07:53:24.659588+0000","last_peered":"2024-09-21T07:53:24.659588+0000","last_clean":"2024-09-21T07:53:24.659588+0000","last_became_active":"2024-09-21T07:53:22.339985+0000","last_became_peered":"2024-09-21T07:53:22.339985+0000","last_unstale":"2024-09-21T07:53:24.659588+0000","last_undegraded":"2024-09-21T07:53:24.659588+0000","last_fullsized":"2024-09-21T07:53:24.659588+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:35.291827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161687+0000","last_change":"2024-09-21T07:53:23.356600+0000","last_active":"2024-09-21T07:53:25.161687+0000","last_peered":"2024-09-21T07:53:25.161687+0000","last_clean":"2024-09-21T07:53:25.161687+0000","last_became_active":"2024-09-21T07:53:23.356276+0000","last_became_peered":"2024-09-21T07:53:23.356276+0000","last_unstale":"2024-09-21T07:53:25.161687+0000","last_undegraded":"2024-09-21T07:53:25.161687+0000","last_fullsized":"2024-09-21T07:53:25.161687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:17.767413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343206+0000","last_change":"2024-09-21T07:53:22.335995+0000","last_active":"2024-09-21T07:53:24.343206+0000","last_peered":"2024-09-21T07:53:24.343206+0000","last_clean":"2024-09-21T07:53:24.343206+0000","last_became_active":"2024-09-21T07:53:22.335692+0000","last_became_peered":"2024-09-21T07:53:22.335692+0000","last_unstale":"2024-09-21T07:53:24.343206+0000","last_undegraded":"2024-09-21T07:53:24.343206+0000","last_fullsized":"2024-09-21T07:53:24.343206+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:48.857131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159695+0000","last_change":"2024-09-21T07:53:23.347873+0000","last_active":"2024-09-21T07:53:25.159695+0000","last_peered":"2024-09-21T07:53:25.159695+0000","last_clean":"2024-09-21T07:53:25.159695+0000","last_became_active":"2024-09-21T07:53:23.347517+0000","last_became_peered":"2024-09-21T07:53:23.347517+0000","last_unstale":"2024-09-21T07:53:25.159695+0000","last_undegraded":"2024-09-21T07:53:25.159695+0000","last_fullsized":"2024-09-21T07:53:25.159695+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:10.201321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342648+0000","last_change":"2024-09-21T07:53:22.328072+0000","last_active":"2024-09-21T07:53:24.342648+0000","last_peered":"2024-09-21T07:53:24.342648+0000","last_clean":"2024-09-21T07:53:24.342648+0000","last_became_active":"2024-09-21T07:53:22.327744+0000","last_became_peered":"2024-09-21T07:53:22.327744+0000","last_unstale":"2024-09-21T07:53:24.342648+0000","last_undegraded":"2024-09-21T07:53:24.342648+0000","last_fullsized":"2024-09-21T07:53:24.342648+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:00.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160953+0000","last_change":"2024-09-21T07:53:23.339644+0000","last_active":"2024-09-21T07:53:25.160953+0000","last_peered":"2024-09-21T07:53:25.160953+0000","last_clean":"2024-09-21T07:53:25.160953+0000","last_became_active":"2024-09-21T07:53:23.339294+0000","last_became_peered":"2024-09-21T07:53:23.339294+0000","last_unstale":"2024-09-21T07:53:25.160953+0000","last_undegraded":"2024-09-21T07:53:25.160953+0000","last_fullsized":"2024-09-21T07:53:25.160953+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:04.713568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161332+0000","last_change":"2024-09-21T07:53:22.337373+0000","last_active":"2024-09-21T07:53:25.161332+0000","last_peered":"2024-09-21T07:53:25.161332+0000","last_clean":"2024-09-21T07:53:25.161332+0000","last_became_active":"2024-09-21T07:53:22.337058+0000","last_became_peered":"2024-09-21T07:53:22.337058+0000","last_unstale":"2024-09-21T07:53:25.161332+0000","last_undegraded":"2024-09-21T07:53:25.161332+0000","last_fullsized":"2024-09-21T07:53:25.161332+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:06.528971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634193+0000","last_change":"2024-09-21T07:53:23.355007+0000","last_active":"2024-09-21T07:53:24.634193+0000","last_peered":"2024-09-21T07:53:24.634193+0000","last_clean":"2024-09-21T07:53:24.634193+0000","last_became_active":"2024-09-21T07:53:23.353826+0000","last_became_peered":"2024-09-21T07:53:23.353826+0000","last_unstale":"2024-09-21T07:53:24.634193+0000","last_undegraded":"2024-09-21T07:53:24.634193+0000","last_fullsized":"2024-09-21T07:53:24.634193+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160401+0000","last_change":"2024-09-21T07:53:22.334709+0000","last_active":"2024-09-21T07:53:25.160401+0000","last_peered":"2024-09-21T07:53:25.160401+0000","last_clean":"2024-09-21T07:53:25.160401+0000","last_became_active":"2024-09-21T07:53:22.334412+0000","last_became_peered":"2024-09-21T07:53:22.334412+0000","last_unstale":"2024-09-21T07:53:25.160401+0000","last_undegraded":"2024-09-21T07:53:25.160401+0000","last_fullsized":"2024-09-21T07:53:25.160401+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:58.758955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161158+0000","last_change":"2024-09-21T07:53:23.358223+0000","last_active":"2024-09-21T07:53:25.161158+0000","last_peered":"2024-09-21T07:53:25.161158+0000","last_clean":"2024-09-21T07:53:25.161158+0000","last_became_active":"2024-09-21T07:53:23.357283+0000","last_became_peered":"2024-09-21T07:53:23.357283+0000","last_unstale":"2024-09-21T07:53:25.161158+0000","last_undegraded":"2024-09-21T07:53:25.161158+0000","last_fullsized":"2024-09-21T07:53:25.161158+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:40.443115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659099+0000","last_change":"2024-09-21T07:53:22.340320+0000","last_active":"2024-09-21T07:53:24.659099+0000","last_peered":"2024-09-21T07:53:24.659099+0000","last_clean":"2024-09-21T07:53:24.659099+0000","last_became_active":"2024-09-21T07:53:22.339934+0000","last_became_peered":"2024-09-21T07:53:22.339934+0000","last_unstale":"2024-09-21T07:53:24.659099+0000","last_undegraded":"2024-09-21T07:53:24.659099+0000","last_fullsized":"2024-09-21T07:53:24.659099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:57.604526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633941+0000","last_change":"2024-09-21T07:53:23.349773+0000","last_active":"2024-09-21T07:53:24.633941+0000","last_peered":"2024-09-21T07:53:24.633941+0000","last_clean":"2024-09-21T07:53:24.633941+0000","last_became_active":"2024-09-21T07:53:23.348237+0000","last_became_peered":"2024-09-21T07:53:23.348237+0000","last_unstale":"2024-09-21T07:53:24.633941+0000","last_undegraded":"2024-09-21T07:53:24.633941+0000","last_fullsized":"2024-09-21T07:53:24.633941+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:44.580689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659272+0000","last_change":"2024-09-21T07:53:22.339444+0000","last_active":"2024-09-21T07:53:24.659272+0000","last_peered":"2024-09-21T07:53:24.659272+0000","last_clean":"2024-09-21T07:53:24.659272+0000","last_became_active":"2024-09-21T07:53:22.339132+0000","last_became_peered":"2024-09-21T07:53:22.339132+0000","last_unstale":"2024-09-21T07:53:24.659272+0000","last_undegraded":"2024-09-21T07:53:24.659272+0000","last_fullsized":"2024-09-21T07:53:24.659272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:41.060845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161438+0000","last_change":"2024-09-21T07:53:23.356717+0000","last_active":"2024-09-21T07:53:25.161438+0000","last_peered":"2024-09-21T07:53:25.161438+0000","last_clean":"2024-09-21T07:53:25.161438+0000","last_became_active":"2024-09-21T07:53:23.356251+0000","last_became_peered":"2024-09-21T07:53:23.356251+0000","last_unstale":"2024-09-21T07:53:25.161438+0000","last_undegraded":"2024-09-21T07:53:25.161438+0000","last_fullsized":"2024-09-21T07:53:25.161438+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:58.016595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659036+0000","last_change":"2024-09-21T07:53:22.341823+0000","last_active":"2024-09-21T07:53:24.659036+0000","last_peered":"2024-09-21T07:53:24.659036+0000","last_clean":"2024-09-21T07:53:24.659036+0000","last_became_active":"2024-09-21T07:53:22.341439+0000","last_became_peered":"2024-09-21T07:53:22.341439+0000","last_unstale":"2024-09-21T07:53:24.659036+0000","last_undegraded":"2024-09-21T07:53:24.659036+0000","last_fullsized":"2024-09-21T07:53:24.659036+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:15.205417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659477+0000","last_change":"2024-09-21T07:53:23.362153+0000","last_active":"2024-09-21T07:53:24.659477+0000","last_peered":"2024-09-21T07:53:24.659477+0000","last_clean":"2024-09-21T07:53:24.659477+0000","last_became_active":"2024-09-21T07:53:23.361417+0000","last_became_peered":"2024-09-21T07:53:23.361417+0000","last_unstale":"2024-09-21T07:53:24.659477+0000","last_undegraded":"2024-09-21T07:53:24.659477+0000","last_fullsized":"2024-09-21T07:53:24.659477+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:18.908556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659783+0000","last_change":"2024-09-21T07:53:22.341810+0000","last_active":"2024-09-21T07:53:24.659783+0000","last_peered":"2024-09-21T07:53:24.659783+0000","last_clean":"2024-09-21T07:53:24.659783+0000","last_became_active":"2024-09-21T07:53:22.341487+0000","last_became_peered":"2024-09-21T07:53:22.341487+0000","last_unstale":"2024-09-21T07:53:24.659783+0000","last_undegraded":"2024-09-21T07:53:24.659783+0000","last_fullsized":"2024-09-21T07:53:24.659783+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:50.458298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659525+0000","last_change":"2024-09-21T07:53:23.361340+0000","last_active":"2024-09-21T07:53:24.659525+0000","last_peered":"2024-09-21T07:53:24.659525+0000","last_clean":"2024-09-21T07:53:24.659525+0000","last_became_active":"2024-09-21T07:53:23.360988+0000","last_became_peered":"2024-09-21T07:53:23.360988+0000","last_unstale":"2024-09-21T07:53:24.659525+0000","last_undegraded":"2024-09-21T07:53:24.659525+0000","last_fullsized":"2024-09-21T07:53:24.659525+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:42.301815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161524+0000","last_change":"2024-09-21T07:53:22.342377+0000","last_active":"2024-09-21T07:53:25.161524+0000","last_peered":"2024-09-21T07:53:25.161524+0000","last_clean":"2024-09-21T07:53:25.161524+0000","last_became_active":"2024-09-21T07:53:22.341847+0000","last_became_peered":"2024-09-21T07:53:22.341847+0000","last_unstale":"2024-09-21T07:53:25.161524+0000","last_undegraded":"2024-09-21T07:53:25.161524+0000","last_fullsized":"2024-09-21T07:53:25.161524+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:33.610911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342967+0000","last_change":"2024-09-21T07:53:23.337287+0000","last_active":"2024-09-21T07:53:24.342967+0000","last_peered":"2024-09-21T07:53:24.342967+0000","last_clean":"2024-09-21T07:53:24.342967+0000","last_became_active":"2024-09-21T07:53:23.336412+0000","last_became_peered":"2024-09-21T07:53:23.336412+0000","last_unstale":"2024-09-21T07:53:24.342967+0000","last_undegraded":"2024-09-21T07:53:24.342967+0000","last_fullsized":"2024-09-21T07:53:24.342967+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:43.879086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342769+0000","last_change":"2024-09-21T07:53:22.339874+0000","last_active":"2024-09-21T07:53:24.342769+0000","last_peered":"2024-09-21T07:53:24.342769+0000","last_clean":"2024-09-21T07:53:24.342769+0000","last_became_active":"2024-09-21T07:53:22.337912+0000","last_became_peered":"2024-09-21T07:53:22.337912+0000","last_unstale":"2024-09-21T07:53:24.342769+0000","last_undegraded":"2024-09-21T07:53:24.342769+0000","last_fullsized":"2024-09-21T07:53:24.342769+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:22.879760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341930+0000","last_change":"2024-09-21T07:53:23.338732+0000","last_active":"2024-09-21T07:53:24.341930+0000","last_peered":"2024-09-21T07:53:24.341930+0000","last_clean":"2024-09-21T07:53:24.341930+0000","last_became_active":"2024-09-21T07:53:23.338408+0000","last_became_peered":"2024-09-21T07:53:23.338408+0000","last_unstale":"2024-09-21T07:53:24.341930+0000","last_undegraded":"2024-09-21T07:53:24.341930+0000","last_fullsized":"2024-09-21T07:53:24.341930+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:58.846181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342977+0000","last_change":"2024-09-21T07:53:22.331768+0000","last_active":"2024-09-21T07:53:24.342977+0000","last_peered":"2024-09-21T07:53:24.342977+0000","last_clean":"2024-09-21T07:53:24.342977+0000","last_became_active":"2024-09-21T07:53:22.330854+0000","last_became_peered":"2024-09-21T07:53:22.330854+0000","last_unstale":"2024-09-21T07:53:24.342977+0000","last_undegraded":"2024-09-21T07:53:24.342977+0000","last_fullsized":"2024-09-21T07:53:24.342977+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:41.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342685+0000","last_change":"2024-09-21T07:53:23.344317+0000","last_active":"2024-09-21T07:53:24.342685+0000","last_peered":"2024-09-21T07:53:24.342685+0000","last_clean":"2024-09-21T07:53:24.342685+0000","last_became_active":"2024-09-21T07:53:23.343868+0000","last_became_peered":"2024-09-21T07:53:23.343868+0000","last_unstale":"2024-09-21T07:53:24.342685+0000","last_undegraded":"2024-09-21T07:53:24.342685+0000","last_fullsized":"2024-09-21T07:53:24.342685+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:40.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161154+0000","last_change":"2024-09-21T07:53:22.335389+0000","last_active":"2024-09-21T07:53:25.161154+0000","last_peered":"2024-09-21T07:53:25.161154+0000","last_clean":"2024-09-21T07:53:25.161154+0000","last_became_active":"2024-09-21T07:53:22.335123+0000","last_became_peered":"2024-09-21T07:53:22.335123+0000","last_unstale":"2024-09-21T07:53:25.161154+0000","last_undegraded":"2024-09-21T07:53:25.161154+0000","last_fullsized":"2024-09-21T07:53:25.161154+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:53.707505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659747+0000","last_change":"2024-09-21T07:53:23.338094+0000","last_active":"2024-09-21T07:53:24.659747+0000","last_peered":"2024-09-21T07:53:24.659747+0000","last_clean":"2024-09-21T07:53:24.659747+0000","last_became_active":"2024-09-21T07:53:23.337353+0000","last_became_peered":"2024-09-21T07:53:23.337353+0000","last_unstale":"2024-09-21T07:53:24.659747+0000","last_undegraded":"2024-09-21T07:53:24.659747+0000","last_fullsized":"2024-09-21T07:53:24.659747+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:46.745190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339293+0000","last_change":"2024-09-21T07:53:22.333468+0000","last_active":"2024-09-21T07:53:24.339293+0000","last_peered":"2024-09-21T07:53:24.339293+0000","last_clean":"2024-09-21T07:53:24.339293+0000","last_became_active":"2024-09-21T07:53:22.332967+0000","last_became_peered":"2024-09-21T07:53:22.332967+0000","last_unstale":"2024-09-21T07:53:24.339293+0000","last_undegraded":"2024-09-21T07:53:24.339293+0000","last_fullsized":"2024-09-21T07:53:24.339293+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:20.670538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659020+0000","last_change":"2024-09-21T07:53:23.345771+0000","last_active":"2024-09-21T07:53:24.659020+0000","last_peered":"2024-09-21T07:53:24.659020+0000","last_clean":"2024-09-21T07:53:24.659020+0000","last_became_active":"2024-09-21T07:53:23.345548+0000","last_became_peered":"2024-09-21T07:53:23.345548+0000","last_unstale":"2024-09-21T07:53:24.659020+0000","last_undegraded":"2024-09-21T07:53:24.659020+0000","last_fullsized":"2024-09-21T07:53:24.659020+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:29.500564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633470+0000","last_change":"2024-09-21T07:53:22.338716+0000","last_active":"2024-09-21T07:53:24.633470+0000","last_peered":"2024-09-21T07:53:24.633470+0000","last_clean":"2024-09-21T07:53:24.633470+0000","last_became_active":"2024-09-21T07:53:22.338345+0000","last_became_peered":"2024-09-21T07:53:22.338345+0000","last_unstale":"2024-09-21T07:53:24.633470+0000","last_undegraded":"2024-09-21T07:53:24.633470+0000","last_fullsized":"2024-09-21T07:53:24.633470+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:46.884181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339325+0000","last_change":"2024-09-21T07:53:23.330293+0000","last_active":"2024-09-21T07:53:24.339325+0000","last_peered":"2024-09-21T07:53:24.339325+0000","last_clean":"2024-09-21T07:53:24.339325+0000","last_became_active":"2024-09-21T07:53:23.329992+0000","last_became_peered":"2024-09-21T07:53:23.329992+0000","last_unstale":"2024-09-21T07:53:24.339325+0000","last_undegraded":"2024-09-21T07:53:24.339325+0000","last_fullsized":"2024-09-21T07:53:24.339325+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:56.536413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160169+0000","last_change":"2024-09-21T07:53:22.333056+0000","last_active":"2024-09-21T07:53:25.160169+0000","last_peered":"2024-09-21T07:53:25.160169+0000","last_clean":"2024-09-21T07:53:25.160169+0000","last_became_active":"2024-09-21T07:53:22.332736+0000","last_became_peered":"2024-09-21T07:53:22.332736+0000","last_unstale":"2024-09-21T07:53:25.160169+0000","last_undegraded":"2024-09-21T07:53:25.160169+0000","last_fullsized":"2024-09-21T07:53:25.160169+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:58.891674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342395+0000","last_change":"2024-09-21T07:53:23.336076+0000","last_active":"2024-09-21T07:53:24.342395+0000","last_peered":"2024-09-21T07:53:24.342395+0000","last_clean":"2024-09-21T07:53:24.342395+0000","last_became_active":"2024-09-21T07:53:23.335079+0000","last_became_peered":"2024-09-21T07:53:23.335079+0000","last_unstale":"2024-09-21T07:53:24.342395+0000","last_undegraded":"2024-09-21T07:53:24.342395+0000","last_fullsized":"2024-09-21T07:53:24.342395+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:05.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"131'3","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633820+0000","last_change":"2024-09-21T07:53:22.339067+0000","last_active":"2024-09-21T07:53:24.633820+0000","last_peered":"2024-09-21T07:53:24.633820+0000","last_clean":"2024-09-21T07:53:24.633820+0000","last_became_active":"2024-09-21T07:53:22.338465+0000","last_became_peered":"2024-09-21T07:53:22.338465+0000","last_unstale":"2024-09-21T07:53:24.633820+0000","last_undegraded":"2024-09-21T07:53:24.633820+0000","last_fullsized":"2024-09-21T07:53:24.633820+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:59.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":347,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159133+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:53:25.159133+0000","last_peered":"2024-09-21T07:53:25.159133+0000","last_clean":"2024-09-21T07:53:25.159133+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:53:25.159133+0000","last_undegraded":"2024-09-21T07:53:25.159133+0000","last_fullsized":"2024-09-21T07:53:25.159133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542238,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":724694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":724694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509534,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2260992,"data_stored":1316868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1662976,"data_stored":726910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476832,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1650688,"data_stored":725310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476832,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1646592,"data_stored":724716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476829,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33380,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338460,"statfs":{"total":96636764160,"available":96602583040,"internally_reserved":0,"allocated":2240512,"data_stored":1315132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":724784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:53:27.849 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 -- 172.21.15.88:0/3719988220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0d1580 msgr2=0x7f944c0d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 --2- 172.21.15.88:0/3719988220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0d1580 0x7f944c0d1990 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f9444005fd0 tx=0x7f944402a840 comp rx=0 tx=0).stop 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 -- 172.21.15.88:0/3719988220 shutdown_connections 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 --2- 172.21.15.88:0/3719988220 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0d1580 0x7f944c0d1990 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 -- 172.21.15.88:0/3719988220 >> 172.21.15.88:0/3719988220 conn(0x7f944c08bfe0 msgr2=0x7f944c09d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 -- 172.21.15.88:0/3719988220 shutdown_connections 2024-09-21T07:53:28.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.109+0000 7f944b200640 1 -- 172.21.15.88:0/3719988220 wait complete. 2024-09-21T07:53:28.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.112+0000 7f944b200640 1 Processor -- start 2024-09-21T07:53:28.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.112+0000 7f944b200640 1 -- start start 2024-09-21T07:53:28.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f944b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0b3160 0x7f944c152c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f944b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 0x7f944c0a13d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f944b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f944c0d1580 0x7f944c0a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f944b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0d4700 con 0x7f944c0b9e00 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f944b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f944c0d4580 con 0x7f944c0b3160 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f944b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f944c0d4880 con 0x7f944c0d1580 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f9443e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 0x7f944c0a13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.113+0000 7f9443e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 0x7f944c0a13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35732/0 (socket says 172.21.15.88:35732) 2024-09-21T07:53:28.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 -- 172.21.15.88:0/1004086653 learned_addr learned my addr 172.21.15.88:0/1004086653 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:28.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9449e00640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0b3160 0x7f944c152c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:28.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f944a800640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f944c0d1580 0x7f944c0a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:28.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 -- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f944c0d1580 msgr2=0x7f944c0a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f944c0d1580 0x7f944c0a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 -- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0b3160 msgr2=0x7f944c152c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0b3160 0x7f944c152c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 -- 172.21.15.88:0/1004086653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c09fe20 con 0x7f944c0b9e00 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f944a800640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f944c0d1580 0x7f944c0a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9443e00640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 0x7f944c0a13d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f943400c4d0 tx=0x7f943400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434016020 con 0x7f944c0b9e00 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9434012920 con 0x7f944c0b9e00 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9434012d70 con 0x7f944c0b9e00 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.114+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c15efc0 con 0x7f944c0b9e00 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.115+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c09e2e0 con 0x7f944c0b9e00 2024-09-21T07:53:28.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.116+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f944c0d1990 con 0x7f944c0b9e00 2024-09-21T07:53:28.118 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.117+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f9434019020 con 0x7f944c0b9e00 2024-09-21T07:53:28.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.118+0000 7f9442000640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f94280791c0 0x7f942807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.118+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7f943409b7d0 con 0x7f944c0b9e00 2024-09-21T07:53:28.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.118+0000 7f9449e00640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f94280791c0 0x7f942807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:28.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.119+0000 7f9449e00640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f94280791c0 0x7f942807b6d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f943c001120 tx=0x7f943c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:28.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.123+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9434020030 con 0x7f944c0b9e00 2024-09-21T07:53:28.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.311+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f944c15f4e0 con 0x7f94280791c0 2024-09-21T07:53:28.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.315+0000 7f9442000640 1 -- 172.21.15.88:0/1004086653 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252476 (secure 0 0 0) 0x7f943c0050d0 con 0x7f94280791c0 2024-09-21T07:53:28.319 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:28.320 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f94280791c0 msgr2=0x7f942807b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f94280791c0 0x7f942807b6d0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f943c001120 tx=0x7f943c009040 comp rx=0 tx=0).stop 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 msgr2=0x7f944c0a13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 0x7f944c0a13d0 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f943400c4d0 tx=0x7f943400c990 comp rx=0 tx=0).stop 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 shutdown_connections 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f94280791c0 0x7f942807b6d0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f944c0d1580 0x7f944c0a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c0b9e00 0x7f944c0a13d0 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 --2- 172.21.15.88:0/1004086653 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f944c0b3160 0x7f944c152c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 >> 172.21.15.88:0/1004086653 conn(0x7f944c08bfe0 msgr2=0x7f944c162a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 shutdown_connections 2024-09-21T07:53:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.319+0000 7f944b200640 1 -- 172.21.15.88:0/1004086653 wait complete. 2024-09-21T07:53:28.342 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-09-21T07:53:27.732525+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":187,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":263568,"kb_used_data":14056,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754711152,"statfs":{"total":773094113280,"available":772824219648,"internally_reserved":0,"allocated":14393344,"data_stored":6983108,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001272"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342737+0000","last_change":"2024-09-21T07:53:23.346330+0000","last_active":"2024-09-21T07:53:24.342737+0000","last_peered":"2024-09-21T07:53:24.342737+0000","last_clean":"2024-09-21T07:53:24.342737+0000","last_became_active":"2024-09-21T07:53:23.345700+0000","last_became_peered":"2024-09-21T07:53:23.345700+0000","last_unstale":"2024-09-21T07:53:24.342737+0000","last_undegraded":"2024-09-21T07:53:24.342737+0000","last_fullsized":"2024-09-21T07:53:24.342737+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:21:44.177076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161593+0000","last_change":"2024-09-21T07:53:23.351779+0000","last_active":"2024-09-21T07:53:25.161593+0000","last_peered":"2024-09-21T07:53:25.161593+0000","last_clean":"2024-09-21T07:53:25.161593+0000","last_became_active":"2024-09-21T07:53:23.351478+0000","last_became_peered":"2024-09-21T07:53:23.351478+0000","last_unstale":"2024-09-21T07:53:25.161593+0000","last_undegraded":"2024-09-21T07:53:25.161593+0000","last_fullsized":"2024-09-21T07:53:25.161593+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:58:58.373634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339529+0000","last_change":"2024-09-21T07:53:23.352174+0000","last_active":"2024-09-21T07:53:24.339529+0000","last_peered":"2024-09-21T07:53:24.339529+0000","last_clean":"2024-09-21T07:53:24.339529+0000","last_became_active":"2024-09-21T07:53:23.351860+0000","last_became_peered":"2024-09-21T07:53:23.351860+0000","last_unstale":"2024-09-21T07:53:24.339529+0000","last_undegraded":"2024-09-21T07:53:24.339529+0000","last_fullsized":"2024-09-21T07:53:24.339529+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:20:33.997257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659532+0000","last_change":"2024-09-21T07:53:23.361522+0000","last_active":"2024-09-21T07:53:24.659532+0000","last_peered":"2024-09-21T07:53:24.659532+0000","last_clean":"2024-09-21T07:53:24.659532+0000","last_became_active":"2024-09-21T07:53:23.361198+0000","last_became_peered":"2024-09-21T07:53:23.361198+0000","last_unstale":"2024-09-21T07:53:24.659532+0000","last_undegraded":"2024-09-21T07:53:24.659532+0000","last_fullsized":"2024-09-21T07:53:24.659532+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:09.289682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159653+0000","last_change":"2024-09-21T07:53:23.348464+0000","last_active":"2024-09-21T07:53:25.159653+0000","last_peered":"2024-09-21T07:53:25.159653+0000","last_clean":"2024-09-21T07:53:25.159653+0000","last_became_active":"2024-09-21T07:53:23.344682+0000","last_became_peered":"2024-09-21T07:53:23.344682+0000","last_unstale":"2024-09-21T07:53:25.159653+0000","last_undegraded":"2024-09-21T07:53:25.159653+0000","last_fullsized":"2024-09-21T07:53:25.159653+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:01:19.478639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160582+0000","last_change":"2024-09-21T07:53:23.352697+0000","last_active":"2024-09-21T07:53:25.160582+0000","last_peered":"2024-09-21T07:53:25.160582+0000","last_clean":"2024-09-21T07:53:25.160582+0000","last_became_active":"2024-09-21T07:53:23.352366+0000","last_became_peered":"2024-09-21T07:53:23.352366+0000","last_unstale":"2024-09-21T07:53:25.160582+0000","last_undegraded":"2024-09-21T07:53:25.160582+0000","last_fullsized":"2024-09-21T07:53:25.160582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:43:58.875225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343138+0000","last_change":"2024-09-21T07:53:23.337428+0000","last_active":"2024-09-21T07:53:24.343138+0000","last_peered":"2024-09-21T07:53:24.343138+0000","last_clean":"2024-09-21T07:53:24.343138+0000","last_became_active":"2024-09-21T07:53:23.336933+0000","last_became_peered":"2024-09-21T07:53:23.336933+0000","last_unstale":"2024-09-21T07:53:24.343138+0000","last_undegraded":"2024-09-21T07:53:24.343138+0000","last_fullsized":"2024-09-21T07:53:24.343138+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341738+0000","last_change":"2024-09-21T07:53:23.341543+0000","last_active":"2024-09-21T07:53:24.341738+0000","last_peered":"2024-09-21T07:53:24.341738+0000","last_clean":"2024-09-21T07:53:24.341738+0000","last_became_active":"2024-09-21T07:53:23.341175+0000","last_became_peered":"2024-09-21T07:53:23.341175+0000","last_unstale":"2024-09-21T07:53:24.341738+0000","last_undegraded":"2024-09-21T07:53:24.341738+0000","last_fullsized":"2024-09-21T07:53:24.341738+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:49:20.362969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160885+0000","last_change":"2024-09-21T07:53:23.336289+0000","last_active":"2024-09-21T07:53:25.160885+0000","last_peered":"2024-09-21T07:53:25.160885+0000","last_clean":"2024-09-21T07:53:25.160885+0000","last_became_active":"2024-09-21T07:53:23.335516+0000","last_became_peered":"2024-09-21T07:53:23.335516+0000","last_unstale":"2024-09-21T07:53:25.160885+0000","last_undegraded":"2024-09-21T07:53:25.160885+0000","last_fullsized":"2024-09-21T07:53:25.160885+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:59.097440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339850+0000","last_change":"2024-09-21T07:53:23.355115+0000","last_active":"2024-09-21T07:53:24.339850+0000","last_peered":"2024-09-21T07:53:24.339850+0000","last_clean":"2024-09-21T07:53:24.339850+0000","last_became_active":"2024-09-21T07:53:23.354759+0000","last_became_peered":"2024-09-21T07:53:23.354759+0000","last_unstale":"2024-09-21T07:53:24.339850+0000","last_undegraded":"2024-09-21T07:53:24.339850+0000","last_fullsized":"2024-09-21T07:53:24.339850+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:10:05.879117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160856+0000","last_change":"2024-09-21T07:53:23.344727+0000","last_active":"2024-09-21T07:53:25.160856+0000","last_peered":"2024-09-21T07:53:25.160856+0000","last_clean":"2024-09-21T07:53:25.160856+0000","last_became_active":"2024-09-21T07:53:23.344378+0000","last_became_peered":"2024-09-21T07:53:23.344378+0000","last_unstale":"2024-09-21T07:53:25.160856+0000","last_undegraded":"2024-09-21T07:53:25.160856+0000","last_fullsized":"2024-09-21T07:53:25.160856+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:12.332318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633932+0000","last_change":"2024-09-21T07:53:23.348908+0000","last_active":"2024-09-21T07:53:24.633932+0000","last_peered":"2024-09-21T07:53:24.633932+0000","last_clean":"2024-09-21T07:53:24.633932+0000","last_became_active":"2024-09-21T07:53:23.348553+0000","last_became_peered":"2024-09-21T07:53:23.348553+0000","last_unstale":"2024-09-21T07:53:24.633932+0000","last_undegraded":"2024-09-21T07:53:24.633932+0000","last_fullsized":"2024-09-21T07:53:24.633932+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:42:37.807129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159635+0000","last_change":"2024-09-21T07:53:23.348131+0000","last_active":"2024-09-21T07:53:25.159635+0000","last_peered":"2024-09-21T07:53:25.159635+0000","last_clean":"2024-09-21T07:53:25.159635+0000","last_became_active":"2024-09-21T07:53:23.344317+0000","last_became_peered":"2024-09-21T07:53:23.344317+0000","last_unstale":"2024-09-21T07:53:25.159635+0000","last_undegraded":"2024-09-21T07:53:25.159635+0000","last_fullsized":"2024-09-21T07:53:25.159635+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:28:22.681966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634041+0000","last_change":"2024-09-21T07:53:23.359245+0000","last_active":"2024-09-21T07:53:24.634041+0000","last_peered":"2024-09-21T07:53:24.634041+0000","last_clean":"2024-09-21T07:53:24.634041+0000","last_became_active":"2024-09-21T07:53:23.358932+0000","last_became_peered":"2024-09-21T07:53:23.358932+0000","last_unstale":"2024-09-21T07:53:24.634041+0000","last_undegraded":"2024-09-21T07:53:24.634041+0000","last_fullsized":"2024-09-21T07:53:24.634041+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:48:47.150927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159673+0000","last_change":"2024-09-21T07:53:23.328227+0000","last_active":"2024-09-21T07:53:25.159673+0000","last_peered":"2024-09-21T07:53:25.159673+0000","last_clean":"2024-09-21T07:53:25.159673+0000","last_became_active":"2024-09-21T07:53:23.325718+0000","last_became_peered":"2024-09-21T07:53:23.325718+0000","last_unstale":"2024-09-21T07:53:25.159673+0000","last_undegraded":"2024-09-21T07:53:25.159673+0000","last_fullsized":"2024-09-21T07:53:25.159673+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:08:46.965926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341889+0000","last_change":"2024-09-21T07:53:23.334074+0000","last_active":"2024-09-21T07:53:24.341889+0000","last_peered":"2024-09-21T07:53:24.341889+0000","last_clean":"2024-09-21T07:53:24.341889+0000","last_became_active":"2024-09-21T07:53:23.333730+0000","last_became_peered":"2024-09-21T07:53:23.333730+0000","last_unstale":"2024-09-21T07:53:24.341889+0000","last_undegraded":"2024-09-21T07:53:24.341889+0000","last_fullsized":"2024-09-21T07:53:24.341889+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:46.745190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339678+0000","last_change":"2024-09-21T07:53:23.349261+0000","last_active":"2024-09-21T07:53:24.339678+0000","last_peered":"2024-09-21T07:53:24.339678+0000","last_clean":"2024-09-21T07:53:24.339678+0000","last_became_active":"2024-09-21T07:53:23.348964+0000","last_became_peered":"2024-09-21T07:53:23.348964+0000","last_unstale":"2024-09-21T07:53:24.339678+0000","last_undegraded":"2024-09-21T07:53:24.339678+0000","last_fullsized":"2024-09-21T07:53:24.339678+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:56.672740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159648+0000","last_change":"2024-09-21T07:53:23.338050+0000","last_active":"2024-09-21T07:53:25.159648+0000","last_peered":"2024-09-21T07:53:25.159648+0000","last_clean":"2024-09-21T07:53:25.159648+0000","last_became_active":"2024-09-21T07:53:23.337355+0000","last_became_peered":"2024-09-21T07:53:23.337355+0000","last_unstale":"2024-09-21T07:53:25.159648+0000","last_undegraded":"2024-09-21T07:53:25.159648+0000","last_fullsized":"2024-09-21T07:53:25.159648+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:20.228459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342238+0000","last_change":"2024-09-21T07:53:23.334489+0000","last_active":"2024-09-21T07:53:24.342238+0000","last_peered":"2024-09-21T07:53:24.342238+0000","last_clean":"2024-09-21T07:53:24.342238+0000","last_became_active":"2024-09-21T07:53:23.331209+0000","last_became_peered":"2024-09-21T07:53:23.331209+0000","last_unstale":"2024-09-21T07:53:24.342238+0000","last_undegraded":"2024-09-21T07:53:24.342238+0000","last_fullsized":"2024-09-21T07:53:24.342238+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:52.094982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342078+0000","last_change":"2024-09-21T07:53:23.345801+0000","last_active":"2024-09-21T07:53:24.342078+0000","last_peered":"2024-09-21T07:53:24.342078+0000","last_clean":"2024-09-21T07:53:24.342078+0000","last_became_active":"2024-09-21T07:53:23.345447+0000","last_became_peered":"2024-09-21T07:53:23.345447+0000","last_unstale":"2024-09-21T07:53:24.342078+0000","last_undegraded":"2024-09-21T07:53:24.342078+0000","last_fullsized":"2024-09-21T07:53:24.342078+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:28.814323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339576+0000","last_change":"2024-09-21T07:53:23.350589+0000","last_active":"2024-09-21T07:53:24.339576+0000","last_peered":"2024-09-21T07:53:24.339576+0000","last_clean":"2024-09-21T07:53:24.339576+0000","last_became_active":"2024-09-21T07:53:23.350231+0000","last_became_peered":"2024-09-21T07:53:23.350231+0000","last_unstale":"2024-09-21T07:53:24.339576+0000","last_undegraded":"2024-09-21T07:53:24.339576+0000","last_fullsized":"2024-09-21T07:53:24.339576+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:04:13.370509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633971+0000","last_change":"2024-09-21T07:53:23.351644+0000","last_active":"2024-09-21T07:53:24.633971+0000","last_peered":"2024-09-21T07:53:24.633971+0000","last_clean":"2024-09-21T07:53:24.633971+0000","last_became_active":"2024-09-21T07:53:23.351336+0000","last_became_peered":"2024-09-21T07:53:23.351336+0000","last_unstale":"2024-09-21T07:53:24.633971+0000","last_undegraded":"2024-09-21T07:53:24.633971+0000","last_fullsized":"2024-09-21T07:53:24.633971+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:05.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659077+0000","last_change":"2024-09-21T07:53:23.340055+0000","last_active":"2024-09-21T07:53:24.659077+0000","last_peered":"2024-09-21T07:53:24.659077+0000","last_clean":"2024-09-21T07:53:24.659077+0000","last_became_active":"2024-09-21T07:53:23.339796+0000","last_became_peered":"2024-09-21T07:53:23.339796+0000","last_unstale":"2024-09-21T07:53:24.659077+0000","last_undegraded":"2024-09-21T07:53:24.659077+0000","last_fullsized":"2024-09-21T07:53:24.659077+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:42.532733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159630+0000","last_change":"2024-09-21T07:53:23.345888+0000","last_active":"2024-09-21T07:53:25.159630+0000","last_peered":"2024-09-21T07:53:25.159630+0000","last_clean":"2024-09-21T07:53:25.159630+0000","last_became_active":"2024-09-21T07:53:23.345623+0000","last_became_peered":"2024-09-21T07:53:23.345623+0000","last_unstale":"2024-09-21T07:53:25.159630+0000","last_undegraded":"2024-09-21T07:53:25.159630+0000","last_fullsized":"2024-09-21T07:53:25.159630+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:22.936282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161760+0000","last_change":"2024-09-21T07:53:23.353048+0000","last_active":"2024-09-21T07:53:25.161760+0000","last_peered":"2024-09-21T07:53:25.161760+0000","last_clean":"2024-09-21T07:53:25.161760+0000","last_became_active":"2024-09-21T07:53:23.352717+0000","last_became_peered":"2024-09-21T07:53:23.352717+0000","last_unstale":"2024-09-21T07:53:25.161760+0000","last_undegraded":"2024-09-21T07:53:25.161760+0000","last_fullsized":"2024-09-21T07:53:25.161760+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:08.365643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343120+0000","last_change":"2024-09-21T07:53:23.344361+0000","last_active":"2024-09-21T07:53:24.343120+0000","last_peered":"2024-09-21T07:53:24.343120+0000","last_clean":"2024-09-21T07:53:24.343120+0000","last_became_active":"2024-09-21T07:53:23.342635+0000","last_became_peered":"2024-09-21T07:53:23.342635+0000","last_unstale":"2024-09-21T07:53:24.343120+0000","last_undegraded":"2024-09-21T07:53:24.343120+0000","last_fullsized":"2024-09-21T07:53:24.343120+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:31.577680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161649+0000","last_change":"2024-09-21T07:53:23.357160+0000","last_active":"2024-09-21T07:53:25.161649+0000","last_peered":"2024-09-21T07:53:25.161649+0000","last_clean":"2024-09-21T07:53:25.161649+0000","last_became_active":"2024-09-21T07:53:23.354546+0000","last_became_peered":"2024-09-21T07:53:23.354546+0000","last_unstale":"2024-09-21T07:53:25.161649+0000","last_undegraded":"2024-09-21T07:53:25.161649+0000","last_fullsized":"2024-09-21T07:53:25.161649+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:06:43.152366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161182+0000","last_change":"2024-09-21T07:53:23.353557+0000","last_active":"2024-09-21T07:53:25.161182+0000","last_peered":"2024-09-21T07:53:25.161182+0000","last_clean":"2024-09-21T07:53:25.161182+0000","last_became_active":"2024-09-21T07:53:23.353227+0000","last_became_peered":"2024-09-21T07:53:23.353227+0000","last_unstale":"2024-09-21T07:53:25.161182+0000","last_undegraded":"2024-09-21T07:53:25.161182+0000","last_fullsized":"2024-09-21T07:53:25.161182+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:56.037928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342841+0000","last_change":"2024-09-21T07:53:23.347421+0000","last_active":"2024-09-21T07:53:24.342841+0000","last_peered":"2024-09-21T07:53:24.342841+0000","last_clean":"2024-09-21T07:53:24.342841+0000","last_became_active":"2024-09-21T07:53:23.345414+0000","last_became_peered":"2024-09-21T07:53:23.345414+0000","last_unstale":"2024-09-21T07:53:24.342841+0000","last_undegraded":"2024-09-21T07:53:24.342841+0000","last_fullsized":"2024-09-21T07:53:24.342841+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:47:49.234044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634240+0000","last_change":"2024-09-21T07:53:23.347689+0000","last_active":"2024-09-21T07:53:24.634240+0000","last_peered":"2024-09-21T07:53:24.634240+0000","last_clean":"2024-09-21T07:53:24.634240+0000","last_became_active":"2024-09-21T07:53:23.347378+0000","last_became_peered":"2024-09-21T07:53:23.347378+0000","last_unstale":"2024-09-21T07:53:24.634240+0000","last_undegraded":"2024-09-21T07:53:24.634240+0000","last_fullsized":"2024-09-21T07:53:24.634240+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:14.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160921+0000","last_change":"2024-09-21T07:53:23.356723+0000","last_active":"2024-09-21T07:53:25.160921+0000","last_peered":"2024-09-21T07:53:25.160921+0000","last_clean":"2024-09-21T07:53:25.160921+0000","last_became_active":"2024-09-21T07:53:23.355978+0000","last_became_peered":"2024-09-21T07:53:23.355978+0000","last_unstale":"2024-09-21T07:53:25.160921+0000","last_undegraded":"2024-09-21T07:53:25.160921+0000","last_fullsized":"2024-09-21T07:53:25.160921+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:32:55.217591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160582+0000","last_change":"2024-09-21T07:53:23.349494+0000","last_active":"2024-09-21T07:53:25.160582+0000","last_peered":"2024-09-21T07:53:25.160582+0000","last_clean":"2024-09-21T07:53:25.160582+0000","last_became_active":"2024-09-21T07:53:23.348873+0000","last_became_peered":"2024-09-21T07:53:23.348873+0000","last_unstale":"2024-09-21T07:53:25.160582+0000","last_undegraded":"2024-09-21T07:53:25.160582+0000","last_fullsized":"2024-09-21T07:53:25.160582+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:10.236171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159604+0000","last_change":"2024-09-21T07:53:23.338363+0000","last_active":"2024-09-21T07:53:25.159604+0000","last_peered":"2024-09-21T07:53:25.159604+0000","last_clean":"2024-09-21T07:53:25.159604+0000","last_became_active":"2024-09-21T07:53:23.337764+0000","last_became_peered":"2024-09-21T07:53:23.337764+0000","last_unstale":"2024-09-21T07:53:25.159604+0000","last_undegraded":"2024-09-21T07:53:25.159604+0000","last_fullsized":"2024-09-21T07:53:25.159604+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:14:14.026339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1f","version":"131'4","reported_seq":19,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339735+0000","last_change":"2024-09-21T07:53:22.336923+0000","last_active":"2024-09-21T07:53:24.339735+0000","last_peered":"2024-09-21T07:53:24.339735+0000","last_clean":"2024-09-21T07:53:24.339735+0000","last_became_active":"2024-09-21T07:53:22.335849+0000","last_became_peered":"2024-09-21T07:53:22.335849+0000","last_unstale":"2024-09-21T07:53:24.339735+0000","last_undegraded":"2024-09-21T07:53:24.339735+0000","last_fullsized":"2024-09-21T07:53:24.339735+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:41.158853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343293+0000","last_change":"2024-09-21T07:53:23.330283+0000","last_active":"2024-09-21T07:53:24.343293+0000","last_peered":"2024-09-21T07:53:24.343293+0000","last_clean":"2024-09-21T07:53:24.343293+0000","last_became_active":"2024-09-21T07:53:23.329971+0000","last_became_peered":"2024-09-21T07:53:23.329971+0000","last_unstale":"2024-09-21T07:53:24.343293+0000","last_undegraded":"2024-09-21T07:53:24.343293+0000","last_fullsized":"2024-09-21T07:53:24.343293+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:14.601392+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.1e","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161118+0000","last_change":"2024-09-21T07:53:22.335291+0000","last_active":"2024-09-21T07:53:25.161118+0000","last_peered":"2024-09-21T07:53:25.161118+0000","last_clean":"2024-09-21T07:53:25.161118+0000","last_became_active":"2024-09-21T07:53:22.334974+0000","last_became_peered":"2024-09-21T07:53:22.334974+0000","last_unstale":"2024-09-21T07:53:25.161118+0000","last_undegraded":"2024-09-21T07:53:25.161118+0000","last_fullsized":"2024-09-21T07:53:25.161118+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:42.709866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160733+0000","last_change":"2024-09-21T07:53:23.344768+0000","last_active":"2024-09-21T07:53:25.160733+0000","last_peered":"2024-09-21T07:53:25.160733+0000","last_clean":"2024-09-21T07:53:25.160733+0000","last_became_active":"2024-09-21T07:53:23.343531+0000","last_became_peered":"2024-09-21T07:53:23.343531+0000","last_unstale":"2024-09-21T07:53:25.160733+0000","last_undegraded":"2024-09-21T07:53:25.160733+0000","last_fullsized":"2024-09-21T07:53:25.160733+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:24.703754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.1d","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339572+0000","last_change":"2024-09-21T07:53:22.330500+0000","last_active":"2024-09-21T07:53:24.339572+0000","last_peered":"2024-09-21T07:53:24.339572+0000","last_clean":"2024-09-21T07:53:24.339572+0000","last_became_active":"2024-09-21T07:53:22.330179+0000","last_became_peered":"2024-09-21T07:53:22.330179+0000","last_unstale":"2024-09-21T07:53:24.339572+0000","last_undegraded":"2024-09-21T07:53:24.339572+0000","last_fullsized":"2024-09-21T07:53:24.339572+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:26.996930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659165+0000","last_change":"2024-09-21T07:53:23.360077+0000","last_active":"2024-09-21T07:53:24.659165+0000","last_peered":"2024-09-21T07:53:24.659165+0000","last_clean":"2024-09-21T07:53:24.659165+0000","last_became_active":"2024-09-21T07:53:23.359622+0000","last_became_peered":"2024-09-21T07:53:23.359622+0000","last_unstale":"2024-09-21T07:53:24.659165+0000","last_undegraded":"2024-09-21T07:53:24.659165+0000","last_fullsized":"2024-09-21T07:53:24.659165+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:23.882742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1c","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339513+0000","last_change":"2024-09-21T07:53:22.337829+0000","last_active":"2024-09-21T07:53:24.339513+0000","last_peered":"2024-09-21T07:53:24.339513+0000","last_clean":"2024-09-21T07:53:24.339513+0000","last_became_active":"2024-09-21T07:53:22.337298+0000","last_became_peered":"2024-09-21T07:53:22.337298+0000","last_unstale":"2024-09-21T07:53:24.339513+0000","last_undegraded":"2024-09-21T07:53:24.339513+0000","last_fullsized":"2024-09-21T07:53:24.339513+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:07:44.599956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160895+0000","last_change":"2024-09-21T07:53:23.344990+0000","last_active":"2024-09-21T07:53:25.160895+0000","last_peered":"2024-09-21T07:53:25.160895+0000","last_clean":"2024-09-21T07:53:25.160895+0000","last_became_active":"2024-09-21T07:53:23.344686+0000","last_became_peered":"2024-09-21T07:53:23.344686+0000","last_unstale":"2024-09-21T07:53:25.160895+0000","last_undegraded":"2024-09-21T07:53:25.160895+0000","last_fullsized":"2024-09-21T07:53:25.160895+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:17:12.356811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1b","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342921+0000","last_change":"2024-09-21T07:53:22.338312+0000","last_active":"2024-09-21T07:53:24.342921+0000","last_peered":"2024-09-21T07:53:24.342921+0000","last_clean":"2024-09-21T07:53:24.342921+0000","last_became_active":"2024-09-21T07:53:22.337906+0000","last_became_peered":"2024-09-21T07:53:22.337906+0000","last_unstale":"2024-09-21T07:53:24.342921+0000","last_undegraded":"2024-09-21T07:53:24.342921+0000","last_fullsized":"2024-09-21T07:53:24.342921+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:43:51.297854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659763+0000","last_change":"2024-09-21T07:53:23.356604+0000","last_active":"2024-09-21T07:53:24.659763+0000","last_peered":"2024-09-21T07:53:24.659763+0000","last_clean":"2024-09-21T07:53:24.659763+0000","last_became_active":"2024-09-21T07:53:23.355855+0000","last_became_peered":"2024-09-21T07:53:23.355855+0000","last_unstale":"2024-09-21T07:53:24.659763+0000","last_undegraded":"2024-09-21T07:53:24.659763+0000","last_fullsized":"2024-09-21T07:53:24.659763+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:05.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1a","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159229+0000","last_change":"2024-09-21T07:53:22.334217+0000","last_active":"2024-09-21T07:53:25.159229+0000","last_peered":"2024-09-21T07:53:25.159229+0000","last_clean":"2024-09-21T07:53:25.159229+0000","last_became_active":"2024-09-21T07:53:22.333851+0000","last_became_peered":"2024-09-21T07:53:22.333851+0000","last_unstale":"2024-09-21T07:53:25.159229+0000","last_undegraded":"2024-09-21T07:53:25.159229+0000","last_fullsized":"2024-09-21T07:53:25.159229+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:42.592475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161205+0000","last_change":"2024-09-21T07:53:23.355989+0000","last_active":"2024-09-21T07:53:25.161205+0000","last_peered":"2024-09-21T07:53:25.161205+0000","last_clean":"2024-09-21T07:53:25.161205+0000","last_became_active":"2024-09-21T07:53:23.355584+0000","last_became_peered":"2024-09-21T07:53:23.355584+0000","last_unstale":"2024-09-21T07:53:25.161205+0000","last_undegraded":"2024-09-21T07:53:25.161205+0000","last_fullsized":"2024-09-21T07:53:25.161205+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:39:26.807553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160457+0000","last_change":"2024-09-21T07:53:22.337478+0000","last_active":"2024-09-21T07:53:25.160457+0000","last_peered":"2024-09-21T07:53:25.160457+0000","last_clean":"2024-09-21T07:53:25.160457+0000","last_became_active":"2024-09-21T07:53:22.337105+0000","last_became_peered":"2024-09-21T07:53:22.337105+0000","last_unstale":"2024-09-21T07:53:25.160457+0000","last_undegraded":"2024-09-21T07:53:25.160457+0000","last_fullsized":"2024-09-21T07:53:25.160457+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:13:24.195429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659209+0000","last_change":"2024-09-21T07:53:23.343654+0000","last_active":"2024-09-21T07:53:24.659209+0000","last_peered":"2024-09-21T07:53:24.659209+0000","last_clean":"2024-09-21T07:53:24.659209+0000","last_became_active":"2024-09-21T07:53:23.343302+0000","last_became_peered":"2024-09-21T07:53:23.343302+0000","last_unstale":"2024-09-21T07:53:24.659209+0000","last_undegraded":"2024-09-21T07:53:24.659209+0000","last_fullsized":"2024-09-21T07:53:24.659209+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:45:33.271211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339169+0000","last_change":"2024-09-21T07:53:22.343143+0000","last_active":"2024-09-21T07:53:24.339169+0000","last_peered":"2024-09-21T07:53:24.339169+0000","last_clean":"2024-09-21T07:53:24.339169+0000","last_became_active":"2024-09-21T07:53:22.342842+0000","last_became_peered":"2024-09-21T07:53:22.342842+0000","last_unstale":"2024-09-21T07:53:24.339169+0000","last_undegraded":"2024-09-21T07:53:24.339169+0000","last_fullsized":"2024-09-21T07:53:24.339169+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:56.140214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659512+0000","last_change":"2024-09-21T07:53:23.342441+0000","last_active":"2024-09-21T07:53:24.659512+0000","last_peered":"2024-09-21T07:53:24.659512+0000","last_clean":"2024-09-21T07:53:24.659512+0000","last_became_active":"2024-09-21T07:53:23.341785+0000","last_became_peered":"2024-09-21T07:53:23.341785+0000","last_unstale":"2024-09-21T07:53:24.659512+0000","last_undegraded":"2024-09-21T07:53:24.659512+0000","last_fullsized":"2024-09-21T07:53:24.659512+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:28:27.814113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160965+0000","last_change":"2024-09-21T07:53:22.337188+0000","last_active":"2024-09-21T07:53:25.160965+0000","last_peered":"2024-09-21T07:53:25.160965+0000","last_clean":"2024-09-21T07:53:25.160965+0000","last_became_active":"2024-09-21T07:53:22.336636+0000","last_became_peered":"2024-09-21T07:53:22.336636+0000","last_unstale":"2024-09-21T07:53:25.160965+0000","last_undegraded":"2024-09-21T07:53:25.160965+0000","last_fullsized":"2024-09-21T07:53:25.160965+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:41.781693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634038+0000","last_change":"2024-09-21T07:53:23.353349+0000","last_active":"2024-09-21T07:53:24.634038+0000","last_peered":"2024-09-21T07:53:24.634038+0000","last_clean":"2024-09-21T07:53:24.634038+0000","last_became_active":"2024-09-21T07:53:23.352301+0000","last_became_peered":"2024-09-21T07:53:23.352301+0000","last_unstale":"2024-09-21T07:53:24.634038+0000","last_undegraded":"2024-09-21T07:53:24.634038+0000","last_fullsized":"2024-09-21T07:53:24.634038+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.782721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159735+0000","last_change":"2024-09-21T07:53:22.328129+0000","last_active":"2024-09-21T07:53:25.159735+0000","last_peered":"2024-09-21T07:53:25.159735+0000","last_clean":"2024-09-21T07:53:25.159735+0000","last_became_active":"2024-09-21T07:53:22.327767+0000","last_became_peered":"2024-09-21T07:53:22.327767+0000","last_unstale":"2024-09-21T07:53:25.159735+0000","last_undegraded":"2024-09-21T07:53:25.159735+0000","last_fullsized":"2024-09-21T07:53:25.159735+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:39:12.924703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342549+0000","last_change":"2024-09-21T07:53:23.334234+0000","last_active":"2024-09-21T07:53:24.342549+0000","last_peered":"2024-09-21T07:53:24.342549+0000","last_clean":"2024-09-21T07:53:24.342549+0000","last_became_active":"2024-09-21T07:53:23.333192+0000","last_became_peered":"2024-09-21T07:53:23.333192+0000","last_unstale":"2024-09-21T07:53:24.342549+0000","last_undegraded":"2024-09-21T07:53:24.342549+0000","last_fullsized":"2024-09-21T07:53:24.342549+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:11:39.782721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.14","version":"131'6","reported_seq":21,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342486+0000","last_change":"2024-09-21T07:53:22.336657+0000","last_active":"2024-09-21T07:53:24.342486+0000","last_peered":"2024-09-21T07:53:24.342486+0000","last_clean":"2024-09-21T07:53:24.342486+0000","last_became_active":"2024-09-21T07:53:22.335160+0000","last_became_peered":"2024-09-21T07:53:22.335160+0000","last_unstale":"2024-09-21T07:53:24.342486+0000","last_undegraded":"2024-09-21T07:53:24.342486+0000","last_fullsized":"2024-09-21T07:53:24.342486+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:28.497583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659427+0000","last_change":"2024-09-21T07:53:23.344658+0000","last_active":"2024-09-21T07:53:24.659427+0000","last_peered":"2024-09-21T07:53:24.659427+0000","last_clean":"2024-09-21T07:53:24.659427+0000","last_became_active":"2024-09-21T07:53:23.343353+0000","last_became_peered":"2024-09-21T07:53:23.343353+0000","last_unstale":"2024-09-21T07:53:24.659427+0000","last_undegraded":"2024-09-21T07:53:24.659427+0000","last_fullsized":"2024-09-21T07:53:24.659427+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:01.875543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.13","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339630+0000","last_change":"2024-09-21T07:53:22.333123+0000","last_active":"2024-09-21T07:53:24.339630+0000","last_peered":"2024-09-21T07:53:24.339630+0000","last_clean":"2024-09-21T07:53:24.339630+0000","last_became_active":"2024-09-21T07:53:22.332816+0000","last_became_peered":"2024-09-21T07:53:22.332816+0000","last_unstale":"2024-09-21T07:53:24.339630+0000","last_undegraded":"2024-09-21T07:53:24.339630+0000","last_fullsized":"2024-09-21T07:53:24.339630+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:13:28.798447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343169+0000","last_change":"2024-09-21T07:53:23.344841+0000","last_active":"2024-09-21T07:53:24.343169+0000","last_peered":"2024-09-21T07:53:24.343169+0000","last_clean":"2024-09-21T07:53:24.343169+0000","last_became_active":"2024-09-21T07:53:23.342810+0000","last_became_peered":"2024-09-21T07:53:23.342810+0000","last_unstale":"2024-09-21T07:53:24.343169+0000","last_undegraded":"2024-09-21T07:53:24.343169+0000","last_fullsized":"2024-09-21T07:53:24.343169+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:14:20.299724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634232+0000","last_change":"2024-09-21T07:53:22.336709+0000","last_active":"2024-09-21T07:53:24.634232+0000","last_peered":"2024-09-21T07:53:24.634232+0000","last_clean":"2024-09-21T07:53:24.634232+0000","last_became_active":"2024-09-21T07:53:22.336363+0000","last_became_peered":"2024-09-21T07:53:22.336363+0000","last_unstale":"2024-09-21T07:53:24.634232+0000","last_undegraded":"2024-09-21T07:53:24.634232+0000","last_fullsized":"2024-09-21T07:53:24.634232+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:40:37.336862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.658963+0000","last_change":"2024-09-21T07:53:23.356694+0000","last_active":"2024-09-21T07:53:24.658963+0000","last_peered":"2024-09-21T07:53:24.658963+0000","last_clean":"2024-09-21T07:53:24.658963+0000","last_became_active":"2024-09-21T07:53:23.356424+0000","last_became_peered":"2024-09-21T07:53:23.356424+0000","last_unstale":"2024-09-21T07:53:24.658963+0000","last_undegraded":"2024-09-21T07:53:24.658963+0000","last_fullsized":"2024-09-21T07:53:24.658963+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:32.082961+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160994+0000","last_change":"2024-09-21T07:53:22.340448+0000","last_active":"2024-09-21T07:53:25.160994+0000","last_peered":"2024-09-21T07:53:25.160994+0000","last_clean":"2024-09-21T07:53:25.160994+0000","last_became_active":"2024-09-21T07:53:22.339515+0000","last_became_peered":"2024-09-21T07:53:22.339515+0000","last_unstale":"2024-09-21T07:53:25.160994+0000","last_undegraded":"2024-09-21T07:53:25.160994+0000","last_fullsized":"2024-09-21T07:53:25.160994+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:06.963231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659355+0000","last_change":"2024-09-21T07:53:23.342714+0000","last_active":"2024-09-21T07:53:24.659355+0000","last_peered":"2024-09-21T07:53:24.659355+0000","last_clean":"2024-09-21T07:53:24.659355+0000","last_became_active":"2024-09-21T07:53:23.342112+0000","last_became_peered":"2024-09-21T07:53:23.342112+0000","last_unstale":"2024-09-21T07:53:24.659355+0000","last_undegraded":"2024-09-21T07:53:24.659355+0000","last_fullsized":"2024-09-21T07:53:24.659355+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:49.401487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.10","version":"131'8","reported_seq":23,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341623+0000","last_change":"2024-09-21T07:53:22.343094+0000","last_active":"2024-09-21T07:53:24.341623+0000","last_peered":"2024-09-21T07:53:24.341623+0000","last_clean":"2024-09-21T07:53:24.341623+0000","last_became_active":"2024-09-21T07:53:22.342480+0000","last_became_peered":"2024-09-21T07:53:22.342480+0000","last_unstale":"2024-09-21T07:53:24.341623+0000","last_undegraded":"2024-09-21T07:53:24.341623+0000","last_fullsized":"2024-09-21T07:53:24.341623+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:35.559106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159789+0000","last_change":"2024-09-21T07:53:23.345811+0000","last_active":"2024-09-21T07:53:25.159789+0000","last_peered":"2024-09-21T07:53:25.159789+0000","last_clean":"2024-09-21T07:53:25.159789+0000","last_became_active":"2024-09-21T07:53:23.345473+0000","last_became_peered":"2024-09-21T07:53:23.345473+0000","last_unstale":"2024-09-21T07:53:25.159789+0000","last_undegraded":"2024-09-21T07:53:25.159789+0000","last_fullsized":"2024-09-21T07:53:25.159789+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:03:20.242396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.f","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659588+0000","last_change":"2024-09-21T07:53:22.340228+0000","last_active":"2024-09-21T07:53:24.659588+0000","last_peered":"2024-09-21T07:53:24.659588+0000","last_clean":"2024-09-21T07:53:24.659588+0000","last_became_active":"2024-09-21T07:53:22.339985+0000","last_became_peered":"2024-09-21T07:53:22.339985+0000","last_unstale":"2024-09-21T07:53:24.659588+0000","last_undegraded":"2024-09-21T07:53:24.659588+0000","last_fullsized":"2024-09-21T07:53:24.659588+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:37:35.291827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161687+0000","last_change":"2024-09-21T07:53:23.356600+0000","last_active":"2024-09-21T07:53:25.161687+0000","last_peered":"2024-09-21T07:53:25.161687+0000","last_clean":"2024-09-21T07:53:25.161687+0000","last_became_active":"2024-09-21T07:53:23.356276+0000","last_became_peered":"2024-09-21T07:53:23.356276+0000","last_unstale":"2024-09-21T07:53:25.161687+0000","last_undegraded":"2024-09-21T07:53:25.161687+0000","last_fullsized":"2024-09-21T07:53:25.161687+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:17.767413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.e","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.343206+0000","last_change":"2024-09-21T07:53:22.335995+0000","last_active":"2024-09-21T07:53:24.343206+0000","last_peered":"2024-09-21T07:53:24.343206+0000","last_clean":"2024-09-21T07:53:24.343206+0000","last_became_active":"2024-09-21T07:53:22.335692+0000","last_became_peered":"2024-09-21T07:53:22.335692+0000","last_unstale":"2024-09-21T07:53:24.343206+0000","last_undegraded":"2024-09-21T07:53:24.343206+0000","last_fullsized":"2024-09-21T07:53:24.343206+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:03:48.857131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159695+0000","last_change":"2024-09-21T07:53:23.347873+0000","last_active":"2024-09-21T07:53:25.159695+0000","last_peered":"2024-09-21T07:53:25.159695+0000","last_clean":"2024-09-21T07:53:25.159695+0000","last_became_active":"2024-09-21T07:53:23.347517+0000","last_became_peered":"2024-09-21T07:53:23.347517+0000","last_unstale":"2024-09-21T07:53:25.159695+0000","last_undegraded":"2024-09-21T07:53:25.159695+0000","last_fullsized":"2024-09-21T07:53:25.159695+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:10.201321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.d","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342648+0000","last_change":"2024-09-21T07:53:22.328072+0000","last_active":"2024-09-21T07:53:24.342648+0000","last_peered":"2024-09-21T07:53:24.342648+0000","last_clean":"2024-09-21T07:53:24.342648+0000","last_became_active":"2024-09-21T07:53:22.327744+0000","last_became_peered":"2024-09-21T07:53:22.327744+0000","last_unstale":"2024-09-21T07:53:24.342648+0000","last_undegraded":"2024-09-21T07:53:24.342648+0000","last_fullsized":"2024-09-21T07:53:24.342648+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:00.872561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160953+0000","last_change":"2024-09-21T07:53:23.339644+0000","last_active":"2024-09-21T07:53:25.160953+0000","last_peered":"2024-09-21T07:53:25.160953+0000","last_clean":"2024-09-21T07:53:25.160953+0000","last_became_active":"2024-09-21T07:53:23.339294+0000","last_became_peered":"2024-09-21T07:53:23.339294+0000","last_unstale":"2024-09-21T07:53:25.160953+0000","last_undegraded":"2024-09-21T07:53:25.160953+0000","last_fullsized":"2024-09-21T07:53:25.160953+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:46:04.713568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161332+0000","last_change":"2024-09-21T07:53:22.337373+0000","last_active":"2024-09-21T07:53:25.161332+0000","last_peered":"2024-09-21T07:53:25.161332+0000","last_clean":"2024-09-21T07:53:25.161332+0000","last_became_active":"2024-09-21T07:53:22.337058+0000","last_became_peered":"2024-09-21T07:53:22.337058+0000","last_unstale":"2024-09-21T07:53:25.161332+0000","last_undegraded":"2024-09-21T07:53:25.161332+0000","last_fullsized":"2024-09-21T07:53:25.161332+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:47:06.528971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.634193+0000","last_change":"2024-09-21T07:53:23.355007+0000","last_active":"2024-09-21T07:53:24.634193+0000","last_peered":"2024-09-21T07:53:24.634193+0000","last_clean":"2024-09-21T07:53:24.634193+0000","last_became_active":"2024-09-21T07:53:23.353826+0000","last_became_peered":"2024-09-21T07:53:23.353826+0000","last_unstale":"2024-09-21T07:53:24.634193+0000","last_undegraded":"2024-09-21T07:53:24.634193+0000","last_fullsized":"2024-09-21T07:53:24.634193+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:54:12.731701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.b","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160401+0000","last_change":"2024-09-21T07:53:22.334709+0000","last_active":"2024-09-21T07:53:25.160401+0000","last_peered":"2024-09-21T07:53:25.160401+0000","last_clean":"2024-09-21T07:53:25.160401+0000","last_became_active":"2024-09-21T07:53:22.334412+0000","last_became_peered":"2024-09-21T07:53:22.334412+0000","last_unstale":"2024-09-21T07:53:25.160401+0000","last_undegraded":"2024-09-21T07:53:25.160401+0000","last_fullsized":"2024-09-21T07:53:25.160401+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:45:58.758955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161158+0000","last_change":"2024-09-21T07:53:23.358223+0000","last_active":"2024-09-21T07:53:25.161158+0000","last_peered":"2024-09-21T07:53:25.161158+0000","last_clean":"2024-09-21T07:53:25.161158+0000","last_became_active":"2024-09-21T07:53:23.357283+0000","last_became_peered":"2024-09-21T07:53:23.357283+0000","last_unstale":"2024-09-21T07:53:25.161158+0000","last_undegraded":"2024-09-21T07:53:25.161158+0000","last_fullsized":"2024-09-21T07:53:25.161158+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:40.443115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659099+0000","last_change":"2024-09-21T07:53:22.340320+0000","last_active":"2024-09-21T07:53:24.659099+0000","last_peered":"2024-09-21T07:53:24.659099+0000","last_clean":"2024-09-21T07:53:24.659099+0000","last_became_active":"2024-09-21T07:53:22.339934+0000","last_became_peered":"2024-09-21T07:53:22.339934+0000","last_unstale":"2024-09-21T07:53:24.659099+0000","last_undegraded":"2024-09-21T07:53:24.659099+0000","last_fullsized":"2024-09-21T07:53:24.659099+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:57.604526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633941+0000","last_change":"2024-09-21T07:53:23.349773+0000","last_active":"2024-09-21T07:53:24.633941+0000","last_peered":"2024-09-21T07:53:24.633941+0000","last_clean":"2024-09-21T07:53:24.633941+0000","last_became_active":"2024-09-21T07:53:23.348237+0000","last_became_peered":"2024-09-21T07:53:23.348237+0000","last_unstale":"2024-09-21T07:53:24.633941+0000","last_undegraded":"2024-09-21T07:53:24.633941+0000","last_fullsized":"2024-09-21T07:53:24.633941+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:33:44.580689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659272+0000","last_change":"2024-09-21T07:53:22.339444+0000","last_active":"2024-09-21T07:53:24.659272+0000","last_peered":"2024-09-21T07:53:24.659272+0000","last_clean":"2024-09-21T07:53:24.659272+0000","last_became_active":"2024-09-21T07:53:22.339132+0000","last_became_peered":"2024-09-21T07:53:22.339132+0000","last_unstale":"2024-09-21T07:53:24.659272+0000","last_undegraded":"2024-09-21T07:53:24.659272+0000","last_fullsized":"2024-09-21T07:53:24.659272+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:41:41.060845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161438+0000","last_change":"2024-09-21T07:53:23.356717+0000","last_active":"2024-09-21T07:53:25.161438+0000","last_peered":"2024-09-21T07:53:25.161438+0000","last_clean":"2024-09-21T07:53:25.161438+0000","last_became_active":"2024-09-21T07:53:23.356251+0000","last_became_peered":"2024-09-21T07:53:23.356251+0000","last_unstale":"2024-09-21T07:53:25.161438+0000","last_undegraded":"2024-09-21T07:53:25.161438+0000","last_fullsized":"2024-09-21T07:53:25.161438+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:58.016595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659036+0000","last_change":"2024-09-21T07:53:22.341823+0000","last_active":"2024-09-21T07:53:24.659036+0000","last_peered":"2024-09-21T07:53:24.659036+0000","last_clean":"2024-09-21T07:53:24.659036+0000","last_became_active":"2024-09-21T07:53:22.341439+0000","last_became_peered":"2024-09-21T07:53:22.341439+0000","last_unstale":"2024-09-21T07:53:24.659036+0000","last_undegraded":"2024-09-21T07:53:24.659036+0000","last_fullsized":"2024-09-21T07:53:24.659036+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:15.205417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659477+0000","last_change":"2024-09-21T07:53:23.362153+0000","last_active":"2024-09-21T07:53:24.659477+0000","last_peered":"2024-09-21T07:53:24.659477+0000","last_clean":"2024-09-21T07:53:24.659477+0000","last_became_active":"2024-09-21T07:53:23.361417+0000","last_became_peered":"2024-09-21T07:53:23.361417+0000","last_unstale":"2024-09-21T07:53:24.659477+0000","last_undegraded":"2024-09-21T07:53:24.659477+0000","last_fullsized":"2024-09-21T07:53:24.659477+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:18.908556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.7","version":"131'2","reported_seq":17,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659783+0000","last_change":"2024-09-21T07:53:22.341810+0000","last_active":"2024-09-21T07:53:24.659783+0000","last_peered":"2024-09-21T07:53:24.659783+0000","last_clean":"2024-09-21T07:53:24.659783+0000","last_became_active":"2024-09-21T07:53:22.341487+0000","last_became_peered":"2024-09-21T07:53:22.341487+0000","last_unstale":"2024-09-21T07:53:24.659783+0000","last_undegraded":"2024-09-21T07:53:24.659783+0000","last_fullsized":"2024-09-21T07:53:24.659783+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:34:50.458298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659525+0000","last_change":"2024-09-21T07:53:23.361340+0000","last_active":"2024-09-21T07:53:24.659525+0000","last_peered":"2024-09-21T07:53:24.659525+0000","last_clean":"2024-09-21T07:53:24.659525+0000","last_became_active":"2024-09-21T07:53:23.360988+0000","last_became_peered":"2024-09-21T07:53:23.360988+0000","last_unstale":"2024-09-21T07:53:24.659525+0000","last_undegraded":"2024-09-21T07:53:24.659525+0000","last_fullsized":"2024-09-21T07:53:24.659525+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:05:42.301815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.6","version":"131'1","reported_seq":16,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161524+0000","last_change":"2024-09-21T07:53:22.342377+0000","last_active":"2024-09-21T07:53:25.161524+0000","last_peered":"2024-09-21T07:53:25.161524+0000","last_clean":"2024-09-21T07:53:25.161524+0000","last_became_active":"2024-09-21T07:53:22.341847+0000","last_became_peered":"2024-09-21T07:53:22.341847+0000","last_unstale":"2024-09-21T07:53:25.161524+0000","last_undegraded":"2024-09-21T07:53:25.161524+0000","last_fullsized":"2024-09-21T07:53:25.161524+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:56:33.610911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342967+0000","last_change":"2024-09-21T07:53:23.337287+0000","last_active":"2024-09-21T07:53:24.342967+0000","last_peered":"2024-09-21T07:53:24.342967+0000","last_clean":"2024-09-21T07:53:24.342967+0000","last_became_active":"2024-09-21T07:53:23.336412+0000","last_became_peered":"2024-09-21T07:53:23.336412+0000","last_unstale":"2024-09-21T07:53:24.342967+0000","last_undegraded":"2024-09-21T07:53:24.342967+0000","last_fullsized":"2024-09-21T07:53:24.342967+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:30:43.879086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342769+0000","last_change":"2024-09-21T07:53:22.339874+0000","last_active":"2024-09-21T07:53:24.342769+0000","last_peered":"2024-09-21T07:53:24.342769+0000","last_clean":"2024-09-21T07:53:24.342769+0000","last_became_active":"2024-09-21T07:53:22.337912+0000","last_became_peered":"2024-09-21T07:53:22.337912+0000","last_unstale":"2024-09-21T07:53:24.342769+0000","last_undegraded":"2024-09-21T07:53:24.342769+0000","last_fullsized":"2024-09-21T07:53:24.342769+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:38:22.879760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.341930+0000","last_change":"2024-09-21T07:53:23.338732+0000","last_active":"2024-09-21T07:53:24.341930+0000","last_peered":"2024-09-21T07:53:24.341930+0000","last_clean":"2024-09-21T07:53:24.341930+0000","last_became_active":"2024-09-21T07:53:23.338408+0000","last_became_peered":"2024-09-21T07:53:23.338408+0000","last_unstale":"2024-09-21T07:53:24.341930+0000","last_undegraded":"2024-09-21T07:53:24.341930+0000","last_fullsized":"2024-09-21T07:53:24.341930+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:58.846181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342977+0000","last_change":"2024-09-21T07:53:22.331768+0000","last_active":"2024-09-21T07:53:24.342977+0000","last_peered":"2024-09-21T07:53:24.342977+0000","last_clean":"2024-09-21T07:53:24.342977+0000","last_became_active":"2024-09-21T07:53:22.330854+0000","last_became_peered":"2024-09-21T07:53:22.330854+0000","last_unstale":"2024-09-21T07:53:24.342977+0000","last_undegraded":"2024-09-21T07:53:24.342977+0000","last_fullsized":"2024-09-21T07:53:24.342977+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:13:41.529751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342685+0000","last_change":"2024-09-21T07:53:23.344317+0000","last_active":"2024-09-21T07:53:24.342685+0000","last_peered":"2024-09-21T07:53:24.342685+0000","last_clean":"2024-09-21T07:53:24.342685+0000","last_became_active":"2024-09-21T07:53:23.343868+0000","last_became_peered":"2024-09-21T07:53:23.343868+0000","last_unstale":"2024-09-21T07:53:24.342685+0000","last_undegraded":"2024-09-21T07:53:24.342685+0000","last_fullsized":"2024-09-21T07:53:24.342685+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:40.083579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.161154+0000","last_change":"2024-09-21T07:53:22.335389+0000","last_active":"2024-09-21T07:53:25.161154+0000","last_peered":"2024-09-21T07:53:25.161154+0000","last_clean":"2024-09-21T07:53:25.161154+0000","last_became_active":"2024-09-21T07:53:22.335123+0000","last_became_peered":"2024-09-21T07:53:22.335123+0000","last_unstale":"2024-09-21T07:53:25.161154+0000","last_undegraded":"2024-09-21T07:53:25.161154+0000","last_fullsized":"2024-09-21T07:53:25.161154+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:53.707505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659747+0000","last_change":"2024-09-21T07:53:23.338094+0000","last_active":"2024-09-21T07:53:24.659747+0000","last_peered":"2024-09-21T07:53:24.659747+0000","last_clean":"2024-09-21T07:53:24.659747+0000","last_became_active":"2024-09-21T07:53:23.337353+0000","last_became_peered":"2024-09-21T07:53:23.337353+0000","last_unstale":"2024-09-21T07:53:24.659747+0000","last_undegraded":"2024-09-21T07:53:24.659747+0000","last_fullsized":"2024-09-21T07:53:24.659747+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:46.745190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339293+0000","last_change":"2024-09-21T07:53:22.333468+0000","last_active":"2024-09-21T07:53:24.339293+0000","last_peered":"2024-09-21T07:53:24.339293+0000","last_clean":"2024-09-21T07:53:24.339293+0000","last_became_active":"2024-09-21T07:53:22.332967+0000","last_became_peered":"2024-09-21T07:53:22.332967+0000","last_unstale":"2024-09-21T07:53:24.339293+0000","last_undegraded":"2024-09-21T07:53:24.339293+0000","last_fullsized":"2024-09-21T07:53:24.339293+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:58:20.670538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.659020+0000","last_change":"2024-09-21T07:53:23.345771+0000","last_active":"2024-09-21T07:53:24.659020+0000","last_peered":"2024-09-21T07:53:24.659020+0000","last_clean":"2024-09-21T07:53:24.659020+0000","last_became_active":"2024-09-21T07:53:23.345548+0000","last_became_peered":"2024-09-21T07:53:23.345548+0000","last_unstale":"2024-09-21T07:53:24.659020+0000","last_undegraded":"2024-09-21T07:53:24.659020+0000","last_fullsized":"2024-09-21T07:53:24.659020+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:29.500564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633470+0000","last_change":"2024-09-21T07:53:22.338716+0000","last_active":"2024-09-21T07:53:24.633470+0000","last_peered":"2024-09-21T07:53:24.633470+0000","last_clean":"2024-09-21T07:53:24.633470+0000","last_became_active":"2024-09-21T07:53:22.338345+0000","last_became_peered":"2024-09-21T07:53:22.338345+0000","last_unstale":"2024-09-21T07:53:24.633470+0000","last_undegraded":"2024-09-21T07:53:24.633470+0000","last_fullsized":"2024-09-21T07:53:24.633470+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:46.884181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.339325+0000","last_change":"2024-09-21T07:53:23.330293+0000","last_active":"2024-09-21T07:53:24.339325+0000","last_peered":"2024-09-21T07:53:24.339325+0000","last_clean":"2024-09-21T07:53:24.339325+0000","last_became_active":"2024-09-21T07:53:23.329992+0000","last_became_peered":"2024-09-21T07:53:23.329992+0000","last_unstale":"2024-09-21T07:53:24.339325+0000","last_undegraded":"2024-09-21T07:53:24.339325+0000","last_fullsized":"2024-09-21T07:53:24.339325+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:19:56.536413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":15,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.160169+0000","last_change":"2024-09-21T07:53:22.333056+0000","last_active":"2024-09-21T07:53:25.160169+0000","last_peered":"2024-09-21T07:53:25.160169+0000","last_clean":"2024-09-21T07:53:25.160169+0000","last_became_active":"2024-09-21T07:53:22.332736+0000","last_became_peered":"2024-09-21T07:53:22.332736+0000","last_unstale":"2024-09-21T07:53:25.160169+0000","last_undegraded":"2024-09-21T07:53:25.160169+0000","last_fullsized":"2024-09-21T07:53:25.160169+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:58.891674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":13,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.342395+0000","last_change":"2024-09-21T07:53:23.336076+0000","last_active":"2024-09-21T07:53:24.342395+0000","last_peered":"2024-09-21T07:53:24.342395+0000","last_clean":"2024-09-21T07:53:24.342395+0000","last_became_active":"2024-09-21T07:53:23.335079+0000","last_became_peered":"2024-09-21T07:53:23.335079+0000","last_unstale":"2024-09-21T07:53:24.342395+0000","last_undegraded":"2024-09-21T07:53:24.342395+0000","last_fullsized":"2024-09-21T07:53:24.342395+0000","mapping_epoch":130,"log_start":"0'0","ondisk_log_start":"0'0","created":130,"last_epoch_clean":131,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:22.306408+0000","last_clean_scrub_stamp":"2024-09-21T07:53:22.306408+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:47:05.987109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.15","version":"131'3","reported_seq":18,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:24.633820+0000","last_change":"2024-09-21T07:53:22.339067+0000","last_active":"2024-09-21T07:53:24.633820+0000","last_peered":"2024-09-21T07:53:24.633820+0000","last_clean":"2024-09-21T07:53:24.633820+0000","last_became_active":"2024-09-21T07:53:22.338465+0000","last_became_peered":"2024-09-21T07:53:22.338465+0000","last_unstale":"2024-09-21T07:53:24.633820+0000","last_undegraded":"2024-09-21T07:53:24.633820+0000","last_fullsized":"2024-09-21T07:53:24.633820+0000","mapping_epoch":129,"log_start":"0'0","ondisk_log_start":"0'0","created":129,"last_epoch_clean":130,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:53:21.303427+0000","last_clean_scrub_stamp":"2024-09-21T07:53:21.303427+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:59.342731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":347,"reported_epoch":132,"state":"active+clean","last_fresh":"2024-09-21T07:53:25.159133+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:53:25.159133+0000","last_peered":"2024-09-21T07:53:25.159133+0000","last_clean":"2024-09-21T07:53:25.159133+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:53:25.159133+0000","last_undegraded":"2024-09-21T07:53:25.159133+0000","last_fullsized":"2024-09-21T07:53:25.159133+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542238,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":724694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509533,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32792,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339048,"statfs":{"total":96636764160,"available":96603185152,"internally_reserved":0,"allocated":1642496,"data_stored":724694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509534,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33396,"kb_used_data":2208,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338444,"statfs":{"total":96636764160,"available":96602566656,"internally_reserved":0,"allocated":2260992,"data_stored":1316868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509534,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32812,"kb_used_data":1624,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339028,"statfs":{"total":96636764160,"available":96603164672,"internally_reserved":0,"allocated":1662976,"data_stored":726910,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476832,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1612,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1650688,"data_stored":725310,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476832,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32796,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339044,"statfs":{"total":96636764160,"available":96603181056,"internally_reserved":0,"allocated":1646592,"data_stored":724716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476829,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33380,"kb_used_data":2188,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338460,"statfs":{"total":96636764160,"available":96602583040,"internally_reserved":0,"allocated":2240512,"data_stored":1315132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476829,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32800,"kb_used_data":1608,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94339040,"statfs":{"total":96636764160,"available":96603176960,"internally_reserved":0,"allocated":1646592,"data_stored":724784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1806,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:53:28.346 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:53:28.346 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:53:28.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 -- 172.21.15.88:0/1955970946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 msgr2=0x7fd68c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 --2- 172.21.15.88:0/1955970946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd68c00b9c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fd69c001d90 tx=0x7fd68c005830 comp rx=0 tx=0).stop 2024-09-21T07:53:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 -- 172.21.15.88:0/1955970946 shutdown_connections 2024-09-21T07:53:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 --2- 172.21.15.88:0/1955970946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd68c00b9c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 -- 172.21.15.88:0/1955970946 >> 172.21.15.88:0/1955970946 conn(0x7fd6a4073920 msgr2=0x7fd6a4073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 -- 172.21.15.88:0/1955970946 shutdown_connections 2024-09-21T07:53:28.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.609+0000 7fd6ac600640 1 -- 172.21.15.88:0/1955970946 wait complete. 2024-09-21T07:53:28.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.611+0000 7fd6ac600640 1 Processor -- start 2024-09-21T07:53:28.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.611+0000 7fd6ac600640 1 -- start start 2024-09-21T07:53:28.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6ac600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd6a4150ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6ac600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6a415b550 0x7fd6a415b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6ac600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6a40b53c0 0x7fd6a40b57d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6ac600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40ce2b0 con 0x7fd68c005350 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6ac600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd6a40ce130 con 0x7fd6a415b550 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6ac600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd6a40ce430 con 0x7fd6a40b53c0 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd6a4150ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd6a4150ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35746/0 (socket says 172.21.15.88:35746) 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 -- 172.21.15.88:0/2840738630 learned_addr learned my addr 172.21.15.88:0/2840738630 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 -- 172.21.15.88:0/2840738630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6a40b53c0 msgr2=0x7fd6a40b57d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6a40b53c0 0x7fd6a40b57d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 -- 172.21.15.88:0/2840738630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6a415b550 msgr2=0x7fd6a415b960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6a415b550 0x7fd6a415b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.613+0000 7fd6a9c00640 1 -- 172.21.15.88:0/2840738630 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6a41f90a0 con 0x7fd68c005350 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.614+0000 7fd6a9c00640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd6a4150ef0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd698011ae0 tx=0x7fd698009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.614+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69801e070 con 0x7fd68c005350 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.614+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd698004480 con 0x7fd68c005350 2024-09-21T07:53:28.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.614+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6a41f9380 con 0x7fd68c005350 2024-09-21T07:53:28.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.614+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd69800fc60 con 0x7fd68c005350 2024-09-21T07:53:28.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.614+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6a41f9940 con 0x7fd68c005350 2024-09-21T07:53:28.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.615+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd6a406e380 con 0x7fd68c005350 2024-09-21T07:53:28.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.616+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd698021070 con 0x7fd68c005350 2024-09-21T07:53:28.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.617+0000 7fd6a1600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd6840791c0 0x7fd68407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:28.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.617+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fd69809c410 con 0x7fd68c005350 2024-09-21T07:53:28.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.618+0000 7fd6a9200640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd6840791c0 0x7fd68407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:28.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.618+0000 7fd6a9200640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd6840791c0 0x7fd68407b6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd690000fb0 tx=0x7fd690000bc0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:28.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.622+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd6980637e0 con 0x7fd68c005350 2024-09-21T07:53:28.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.878+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fd6a406f640 con 0x7fd68c005350 2024-09-21T07:53:28.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.881+0000 7fd6a1600640 1 -- 172.21.15.88:0/2840738630 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) ==== 190+0+147 (secure 0 0 0) 0x7fd698068210 con 0x7fd68c005350 2024-09-21T07:53:28.883 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:53:28.883 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:53:28.883 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:53:28.883 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:53:28.883 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:53:28.883 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd6840791c0 msgr2=0x7fd68407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd6840791c0 0x7fd68407b6d0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fd690000fb0 tx=0x7fd690000bc0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 msgr2=0x7fd6a4150ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd6a4150ef0 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fd698011ae0 tx=0x7fd698009e40 comp rx=0 tx=0).stop 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 shutdown_connections 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd6840791c0 0x7fd68407b6d0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd6a40b53c0 0x7fd6a40b57d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd6a415b550 0x7fd6a415b960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 --2- 172.21.15.88:0/2840738630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd68c005350 0x7fd6a4150ef0 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:28.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 >> 172.21.15.88:0/2840738630 conn(0x7fd6a4073920 msgr2=0x7fd6a40b10b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:28.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 shutdown_connections 2024-09-21T07:53:28.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:28.884+0000 7fd6ac600640 1 -- 172.21.15.88:0/2840738630 wait complete. 2024-09-21T07:53:28.910 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:29.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.130+0000 7f50ad800640 1 -- 172.21.15.88:0/3287237950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a80ad0e0 msgr2=0x7f50a80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:29.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.130+0000 7f50ad800640 1 --2- 172.21.15.88:0/3287237950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a80ad0e0 0x7f50a80ad4f0 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7f5090009830 tx=0x7f509002a530 comp rx=0 tx=0).stop 2024-09-21T07:53:29.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.131+0000 7f50ad800640 1 -- 172.21.15.88:0/3287237950 shutdown_connections 2024-09-21T07:53:29.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.131+0000 7f50ad800640 1 --2- 172.21.15.88:0/3287237950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a80ad0e0 0x7f50a80ad4f0 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.131+0000 7f50ad800640 1 -- 172.21.15.88:0/3287237950 >> 172.21.15.88:0/3287237950 conn(0x7f50a806d730 msgr2=0x7f50a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:29.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.131+0000 7f50ad800640 1 -- 172.21.15.88:0/3287237950 shutdown_connections 2024-09-21T07:53:29.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.131+0000 7f50ad800640 1 -- 172.21.15.88:0/3287237950 wait complete. 2024-09-21T07:53:29.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.133+0000 7f50ad800640 1 Processor -- start 2024-09-21T07:53:29.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.133+0000 7f50ad800640 1 -- start start 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.134+0000 7f50ad800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80ad0e0 0x7f50a815a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.134+0000 7f50ad800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a80b3550 0x7f50a8155560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.134+0000 7f50ad800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 0x7f50a81568a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.134+0000 7f50ad800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80b4da0 con 0x7f50a815d760 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.134+0000 7f50ad800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f50a80b4c20 con 0x7f50a80b3550 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.134+0000 7f50ad800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f50a80b4070 con 0x7f50a80ad0e0 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 0x7f50a81568a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 0x7f50a81568a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35770/0 (socket says 172.21.15.88:35770) 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 -- 172.21.15.88:0/2736380917 learned_addr learned my addr 172.21.15.88:0/2736380917 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:29.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f509e000640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a80b3550 0x7f50a8155560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a6a00640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80ad0e0 0x7f50a815a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 -- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80ad0e0 msgr2=0x7f50a815a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80ad0e0 0x7f50a815a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 -- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a80b3550 msgr2=0x7f50a8155560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a80b3550 0x7f50a8155560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 -- 172.21.15.88:0/2736380917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50a81f9160 con 0x7f50a815d760 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.135+0000 7f50a7400640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 0x7f50a81568a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c730 tx=0x7f50a000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.136+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a0003b70 con 0x7f50a815d760 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.136+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f50a0003d30 con 0x7f50a815d760 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.136+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50a00052d0 con 0x7f50a815d760 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.136+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50a8157580 con 0x7f50a815d760 2024-09-21T07:53:29.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.136+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50a81f9400 con 0x7f50a815d760 2024-09-21T07:53:29.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.137+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50a80ad4f0 con 0x7f50a815d760 2024-09-21T07:53:29.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.138+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f50a0005470 con 0x7f50a815d760 2024-09-21T07:53:29.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.139+0000 7f509f400640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f50840791c0 0x7f508407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:29.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.139+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7f50a009b8d0 con 0x7f50a815d760 2024-09-21T07:53:29.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.139+0000 7f50a6a00640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f50840791c0 0x7f508407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:29.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.140+0000 7f50a6a00640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f50840791c0 0x7f508407b6d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5098000f60 tx=0x7f5098009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:29.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.144+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f50a80ada30 con 0x7f50a815d760 2024-09-21T07:53:29.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.368+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f50a815cda0 con 0x7f50a815d760 2024-09-21T07:53:29.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.369+0000 7f509f400640 1 -- 172.21.15.88:0/2736380917 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) ==== 94+0+5233 (secure 0 0 0) 0x7f50a00630e0 con 0x7f50a815d760 2024-09-21T07:53:29.370 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:29.370 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":89,"btime":"2024-09-21T07:53:26:344399+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:53:22.814801+0000","modified":"2024-09-21T07:53:25.831670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2904821684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2904821684},{"type":"v1","addr":"172.21.15.88:6835","nonce":2904821684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7938,"qdb_cluster":[7938]},"id":10}]} 2024-09-21T07:53:29.370 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 89 2024-09-21T07:53:29.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f50840791c0 msgr2=0x7f508407b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:29.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f50840791c0 0x7f508407b6d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5098000f60 tx=0x7f5098009040 comp rx=0 tx=0).stop 2024-09-21T07:53:29.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 msgr2=0x7f50a81568a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:29.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 0x7f50a81568a0 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7f50a000c730 tx=0x7f50a000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 shutdown_connections 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f50840791c0 0x7f508407b6d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50a815d760 0x7f50a81568a0 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f50a80b3550 0x7f50a8155560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 --2- 172.21.15.88:0/2736380917 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f50a80ad0e0 0x7f50a815a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 >> 172.21.15.88:0/2736380917 conn(0x7f50a806d730 msgr2=0x7f50a815f8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 shutdown_connections 2024-09-21T07:53:29.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:29.371+0000 7f50ad800640 1 -- 172.21.15.88:0/2736380917 wait complete. 2024-09-21T07:53:29.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 89, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:53:22.814801+0000', 'modified': '2024-09-21T07:53:25.831670+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7938}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7938': {'gid': 7938, 'name': 'a', 'rank': 0, 'incarnation': 86, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/2904821684', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6833', 'nonce': 2904821684}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 2904821684}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 7938, 'qdb_cluster': [7938]} 2024-09-21T07:53:29.797 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:53:29.797 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 -- 172.21.15.88:0/2887810684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff24070cc0 msgr2=0x7eff24156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 --2- 172.21.15.88:0/2887810684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff24070cc0 0x7eff24156e40 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009830 tx=0x7eff0c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 -- 172.21.15.88:0/2887810684 shutdown_connections 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 --2- 172.21.15.88:0/2887810684 >> v2:172.21.15.175:3300/0 conn(0x7eff2415ce30 0x7eff2415d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 --2- 172.21.15.88:0/2887810684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff24070cc0 0x7eff24156e40 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 -- 172.21.15.88:0/2887810684 >> 172.21.15.88:0/2887810684 conn(0x7eff2406d3d0 msgr2=0x7eff240b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 -- 172.21.15.88:0/2887810684 shutdown_connections 2024-09-21T07:53:30.022 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.019+0000 7eff2a400640 1 -- 172.21.15.88:0/2887810684 wait complete. 2024-09-21T07:53:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.021+0000 7eff2a400640 1 Processor -- start 2024-09-21T07:53:30.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.022+0000 7eff2a400640 1 -- start start 2024-09-21T07:53:30.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff2a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 0x7eff241f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff2a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff2415ce30 0x7eff241f8f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff2a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff24154390 0x7eff241f9440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff2a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff240cd050 con 0x7eff240cfc80 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff2a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff240cced0 con 0x7eff24154390 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff2a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7eff240cd1d0 con 0x7eff2415ce30 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 0x7eff241f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 0x7eff241f51f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35800/0 (socket says 172.21.15.88:35800) 2024-09-21T07:53:30.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.023+0000 7eff23400640 1 -- 172.21.15.88:0/2790994297 learned_addr learned my addr 172.21.15.88:0/2790994297 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:30.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff1be00640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff2415ce30 0x7eff241f8f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23400640 1 -- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff2415ce30 msgr2=0x7eff241f8f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff2415ce30 0x7eff241f8f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23400640 1 -- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff24154390 msgr2=0x7eff241f9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23e00640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff24154390 0x7eff241f9440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff24154390 0x7eff241f9440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23400640 1 -- 172.21.15.88:0/2790994297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff240af3e0 con 0x7eff240cfc80 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff1be00640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff2415ce30 0x7eff241f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23e00640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff24154390 0x7eff241f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff23400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 0x7eff241f51f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c430 tx=0x7eff1400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff1b400640 1 -- 172.21.15.88:0/2790994297 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14011790 con 0x7eff240cfc80 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff1b400640 1 -- 172.21.15.88:0/2790994297 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff14017070 con 0x7eff240cfc80 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff1b400640 1 -- 172.21.15.88:0/2790994297 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14011cc0 con 0x7eff240cfc80 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.024+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff241f9bd0 con 0x7eff240cfc80 2024-09-21T07:53:30.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.025+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff240ad1c0 con 0x7eff240cfc80 2024-09-21T07:53:30.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.028+0000 7eff1b400640 1 -- 172.21.15.88:0/2790994297 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7eff1401c460 con 0x7eff240cfc80 2024-09-21T07:53:30.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.029+0000 7eff1b400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff000791c0 0x7eff0007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.030 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.029+0000 7eff1b400640 1 -- 172.21.15.88:0/2790994297 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7eff1409b4b0 con 0x7eff240cfc80 2024-09-21T07:53:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.029+0000 7eff314a3740 1 Processor -- start 2024-09-21T07:53:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.029+0000 7eff1be00640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff000791c0 0x7eff0007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.029+0000 7eff314a3740 1 -- start start 2024-09-21T07:53:30.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.030+0000 7eff1be00640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff000791c0 0x7eff0007b6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7eff0c034000 tx=0x7eff0c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff314a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 0x55d054373970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054373eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d054343f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff314a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d0542cd430 con 0x55d053fa3270 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0542bd5d0 con 0x55d053f82590 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55d0542aca00 con 0x55d053f7a3b0 2024-09-21T07:53:30.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 0x55d054373970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 0x55d054373970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35810/0 (socket says 172.21.15.88:35810) 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 -- 172.21.15.88:0/511475896 learned_addr learned my addr 172.21.15.88:0/511475896 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff1be00640 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054373eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23e00640 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d054343f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 -- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 msgr2=0x55d054373eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054373eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 -- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 msgr2=0x55d054343f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d054343f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.031+0000 7eff23400640 1 -- 172.21.15.88:0/511475896 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d054275520 con 0x55d053fa3270 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.032+0000 7eff23400640 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 0x55d054373970 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7eff14062760 tx=0x7eff14062040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a8fec2eb37d085 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.032+0000 7eff13e00640 1 -- 172.21.15.88:0/511475896 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14067070 con 0x55d053fa3270 2024-09-21T07:53:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.032+0000 7eff13e00640 1 -- 172.21.15.88:0/511475896 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff140627d0 con 0x55d053fa3270 2024-09-21T07:53:30.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.032+0000 7eff13e00640 1 -- 172.21.15.88:0/511475896 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14061c70 con 0x55d053fa3270 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 -- 172.21.15.88:0/511475896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 msgr2=0x55d054373970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 0x55d054373970 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7eff14062760 tx=0x7eff14062040 comp rx=0 tx=0).stop 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 -- 172.21.15.88:0/511475896 shutdown_connections 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054373eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d054343f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 --2- 172.21.15.88:0/511475896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053fa3270 0x55d054373970 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 -- 172.21.15.88:0/511475896 >> 172.21.15.88:0/511475896 conn(0x55d053f973c0 msgr2=0x55d054349970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 -- 172.21.15.88:0/511475896 shutdown_connections 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.033+0000 7eff314a3740 1 -- 172.21.15.88:0/511475896 wait complete. 2024-09-21T07:53:30.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.034+0000 7eff314a3740 1 -- start start 2024-09-21T07:53:30.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.034+0000 7eff314a3740 1 Processor -- start 2024-09-21T07:53:30.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054388d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d0543885f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053fa3270 0x55d0543a1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d054253330 con 0x55d053f7a3b0 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0542b0810 con 0x55d053f82590 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55d05401d620 con 0x55d053fa3270 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff314a3740 10 client.0 _ll_register_callbacks cb 0x55d054258420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054388d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054388d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35816/0 (socket says 172.21.15.88:35816) 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 -- 172.21.15.88:0/1462141446 learned_addr learned my addr 172.21.15.88:0/1462141446 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff1be00640 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d0543885f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 -- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053fa3270 msgr2=0x55d0543a1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053fa3270 0x55d0543a1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 -- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 msgr2=0x55d0543885f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d0543885f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 -- 172.21.15.88:0/1462141446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d05400e850 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.036+0000 7eff23400640 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054388d40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7eff14062760 tx=0x7eff14061ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80af2184d0c8cd57 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff10c00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff10c00640 1 -- 172.21.15.88:0/1462141446 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140672b0 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff10c00640 1 -- 172.21.15.88:0/1462141446 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7eff14063d10 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff140627d0 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff10c00640 1 -- 172.21.15.88:0/1462141446 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7eff1406c160 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 10 client.8067 fetch_fsmap learned FSMap version 89 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55d0540856b0 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff10c00640 1 -- 172.21.15.88:0/1462141446 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff140627d0 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff10c00640 1 -- 172.21.15.88:0/1462141446 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 89) ==== 3502+0+0 (secure 0 0 0) 0x7eff1405e120 con 0x55d053f7a3b0 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 10 client.8067 fetch_fsmap finished waiting for FSMap version 89 2024-09-21T07:53:30.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 10 client.8067 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 20 client.8067 populate_metadata read hostname 'smithi088' 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d053f61e40 0x55d05436d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 4 client.8067 mds_command: new command op to 7938 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.037+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 --> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0542bf520 con 0x55d053f61e40 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.038+0000 7eff1be00640 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d053f61e40 0x55d05436d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.038+0000 7eff10c00640 10 client.8067 ms_handle_connect on v2:172.21.15.88:6833/2904821684 2024-09-21T07:53:30.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.038+0000 7eff1be00640 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d053f61e40 0x55d05436d950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1ef77296932ac611 server_cookie=eb3b6cbb67d4a0e in_seq=0 out_seq=0 2024-09-21T07:53:30.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff10c00640 1 -- 172.21.15.88:0/1462141446 <== mds.0 v2:172.21.15.88:6833/2904821684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d0542bf520 con 0x55d053f61e40 2024-09-21T07:53:30.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff10c00640 10 client.8067 handle_command_reply: tid=0 2024-09-21T07:53:30.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 client.8067 shutdown 2024-09-21T07:53:30.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 msgr2=0x55d054388d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054388d40 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7eff14062760 tx=0x7eff14061ce0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 shutdown_connections 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d053f61e40 0x55d05436d950 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053fa3270 0x55d0543a1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053f82590 0x55d0543885f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 --2- 172.21.15.88:0/1462141446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054388d40 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff10c00640 0 client.8067 ms_handle_reset on v2:172.21.15.88:6833/2904821684 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.039+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 >> 172.21.15.88:0/1462141446 conn(0x55d053f973c0 msgr2=0x55d0543908c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.040+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 shutdown_connections 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.040+0000 7eff314a3740 1 -- 172.21.15.88:0/1462141446 wait complete. 2024-09-21T07:53:30.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.040+0000 7eff314a3740 20 client.8067 trim_cache size 0 max 16384 2024-09-21T07:53:30.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.059+0000 7eff314a3740 1 Processor -- start 2024-09-21T07:53:30.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.059+0000 7eff314a3740 1 -- start start 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff314a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054390750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f82590 0x55d054390c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d05436b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff314a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff242012a0 con 0x55d053f7a3b0 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7eff24159800 con 0x55d053fa3270 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7eff24159e50 con 0x55d053f82590 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054390750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff23400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054390750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35830/0 (socket says 172.21.15.88:35830) 2024-09-21T07:53:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff23400640 1 -- 172.21.15.88:0/2072923703 learned_addr learned my addr 172.21.15.88:0/2072923703 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.060+0000 7eff23e00640 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d05436b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23400640 1 -- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f82590 msgr2=0x55d054390c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23400640 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f82590 0x55d054390c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23400640 1 -- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 msgr2=0x55d05436b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23400640 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d05436b6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23400640 1 -- 172.21.15.88:0/2072923703 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0543697a0 con 0x55d053f7a3b0 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23e00640 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d05436b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff23400640 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054390750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c400 tx=0x7eff14098f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70bd35bb69323696 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff13400640 1 -- 172.21.15.88:0/2072923703 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff14067040 con 0x55d053f7a3b0 2024-09-21T07:53:30.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff13400640 1 -- 172.21.15.88:0/2072923703 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff14061d50 con 0x55d053f7a3b0 2024-09-21T07:53:30.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.061+0000 7eff13400640 1 -- 172.21.15.88:0/2072923703 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff140627d0 con 0x55d053f7a3b0 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 -- 172.21.15.88:0/2072923703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 msgr2=0x55d054390750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054390750 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c400 tx=0x7eff14098f70 comp rx=0 tx=0).stop 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 -- 172.21.15.88:0/2072923703 shutdown_connections 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f82590 0x55d054390c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d05436b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 --2- 172.21.15.88:0/2072923703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f7a3b0 0x55d054390750 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 -- 172.21.15.88:0/2072923703 >> 172.21.15.88:0/2072923703 conn(0x55d053f973c0 msgr2=0x55d054392560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 -- 172.21.15.88:0/2072923703 shutdown_connections 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.062+0000 7eff314a3740 1 -- 172.21.15.88:0/2072923703 wait complete. 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.063+0000 7eff314a3740 1 -- start start 2024-09-21T07:53:30.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.063+0000 7eff314a3740 1 Processor -- start 2024-09-21T07:53:30.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054405250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 0x55d05436d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d054390750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d054253330 con 0x55d053f82590 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d0542b0810 con 0x55d053fa3270 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55d05436cc40 con 0x55d053f7a3b0 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff314a3740 10 client.0 _ll_register_callbacks cb 0x7eff14061f40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 0x55d05436d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 0x55d05436d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35846/0 (socket says 172.21.15.88:35846) 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 -- 172.21.15.88:0/2007861026 learned_addr learned my addr 172.21.15.88:0/2007861026 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff23e00640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d054390750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff23400640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054405250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 -- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 msgr2=0x55d054405250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054405250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 -- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 msgr2=0x55d054390750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.065+0000 7eff1be00640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d054390750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff1be00640 1 -- 172.21.15.88:0/2007861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d054412200 con 0x55d053f82590 2024-09-21T07:53:30.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff1be00640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 0x55d05436d7e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7eff0c02aac0 tx=0x7eff0c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8162fe570f0090a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff10c00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff10c00640 1 -- 172.21.15.88:0/2007861026 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c058060 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff10c00640 1 -- 172.21.15.88:0/2007861026 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7eff0c003d50 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7eff14061d50 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff10c00640 1 -- 172.21.15.88:0/2007861026 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7eff0c049420 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff314a3740 10 client.8073 fetch_fsmap learned FSMap version 89 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.066+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=89,monmap=2+}) -- 0x55d054411c00 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff10c00640 1 -- 172.21.15.88:0/2007861026 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7eff14061d50 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff10c00640 1 -- 172.21.15.88:0/2007861026 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 89) ==== 3502+0+0 (secure 0 0 0) 0x7eff0c0495c0 con 0x55d053f82590 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff314a3740 10 client.8073 fetch_fsmap finished waiting for FSMap version 89 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff314a3740 10 client.8073 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff314a3740 20 client.8073 populate_metadata read hostname 'smithi088' 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff314a3740 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d0543b1b00 0x55d0543a0170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff314a3740 4 client.8073 mds_command: new command op to 7938 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:53:30.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 --> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] -- command(tid 0: {"prefix": "status"}) -- 0x55d0542bafd0 con 0x55d0543b1b00 2024-09-21T07:53:30.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff23400640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d0543b1b00 0x55d0543a0170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff23400640 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d0543b1b00 0x55d0543a0170 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b07fe517f736432c server_cookie=ac16e4c975b3fc75 in_seq=0 out_seq=0 2024-09-21T07:53:30.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.067+0000 7eff10c00640 10 client.8073 ms_handle_connect on v2:172.21.15.88:6833/2904821684 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff10c00640 1 -- 172.21.15.88:0/2007861026 <== mds.0 v2:172.21.15.88:6833/2904821684 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d0542bafd0 con 0x55d0543b1b00 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff10c00640 10 client.8073 handle_command_reply: tid=0 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 client.8073 shutdown 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 msgr2=0x55d05436d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 0x55d05436d7e0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7eff0c02aac0 tx=0x7eff0c0046c0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 shutdown_connections 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d053fa3270 0x55d054390750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d053f82590 0x55d05436d7e0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.88:6833/2904821684,v1:172.21.15.88:6835/2904821684] conn(0x55d0543b1b00 0x55d0543a0170 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 --2- 172.21.15.88:0/2007861026 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d053f7a3b0 0x55d054405250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 >> 172.21.15.88:0/2007861026 conn(0x55d053f973c0 msgr2=0x55d0543b2130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff10c00640 0 client.8073 ms_handle_reset on v2:172.21.15.88:6833/2904821684 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 shutdown_connections 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 1 -- 172.21.15.88:0/2007861026 wait complete. 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.068+0000 7eff314a3740 20 client.8073 trim_cache size 0 max 16384 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:53:30.073 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 7938, 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.7520917139999996, 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 89, 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 132, 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 132, 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.752987587 2024-09-21T07:53:30.074 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff000791c0 msgr2=0x7eff0007b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff000791c0 0x7eff0007b6d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7eff0c034000 tx=0x7eff0c004730 comp rx=0 tx=0).stop 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 msgr2=0x7eff241f51f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 0x7eff241f51f0 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7eff1400c430 tx=0x7eff1400c8f0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 shutdown_connections 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7eff000791c0 0x7eff0007b6d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7eff2415ce30 0x7eff241f8f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7eff24154390 0x7eff241f9440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 --2- 172.21.15.88:0/2790994297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff240cfc80 0x7eff241f51f0 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 >> 172.21.15.88:0/2790994297 conn(0x7eff2406d3d0 msgr2=0x7eff240b4030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 shutdown_connections 2024-09-21T07:53:30.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.069+0000 7eff2a400640 1 -- 172.21.15.88:0/2790994297 wait complete. 2024-09-21T07:53:30.091 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 -- 172.21.15.88:0/1706442863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 msgr2=0x7fda081535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 --2- 172.21.15.88:0/1706442863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081535c0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7fda08072d60 tx=0x7fd9f802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 -- 172.21.15.88:0/1706442863 shutdown_connections 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 --2- 172.21.15.88:0/1706442863 >> v2:172.21.15.175:3300/0 conn(0x7fda080ad0e0 0x7fda080ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 --2- 172.21.15.88:0/1706442863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081535c0 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 -- 172.21.15.88:0/1706442863 >> 172.21.15.88:0/1706442863 conn(0x7fda0806d730 msgr2=0x7fda080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 -- 172.21.15.88:0/1706442863 shutdown_connections 2024-09-21T07:53:30.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.310+0000 7fda0f200640 1 -- 172.21.15.88:0/1706442863 wait complete. 2024-09-21T07:53:30.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.312+0000 7fda0f200640 1 Processor -- start 2024-09-21T07:53:30.314 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.313+0000 7fda0f200640 1 -- start start 2024-09-21T07:53:30.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda080ad0e0 0x7fda080701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081f8830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda08160310 0x7fda0815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fda080b4c20 con 0x7fda0815d760 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fda08153c70 con 0x7fda08160310 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fda080b4da0 con 0x7fda080ad0e0 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081f8830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda07400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081f8830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35876/0 (socket says 172.21.15.88:35876) 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda07400640 1 -- 172.21.15.88:0/4146177163 learned_addr learned my addr 172.21.15.88:0/4146177163 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.314+0000 7fda0d200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda08160310 0x7fda0815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07e00640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda080ad0e0 0x7fda080701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07400640 1 -- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda080ad0e0 msgr2=0x7fda080701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07400640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda080ad0e0 0x7fda080701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07400640 1 -- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda08160310 msgr2=0x7fda0815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07400640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda08160310 0x7fda0815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07400640 1 -- 172.21.15.88:0/4146177163 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fda08070710 con 0x7fda0815d760 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda07400640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081f8830 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c9a0 tx=0x7fd9f400ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f4007b50 con 0x7fda0815d760 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd9f4007de0 con 0x7fda0815d760 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fda081f8d70 con 0x7fda0815d760 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9f400f6b0 con 0x7fda0815d760 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.315+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fda080736a0 con 0x7fda0815d760 2024-09-21T07:53:30.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.317+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fda08160ff0 con 0x7fda0815d760 2024-09-21T07:53:30.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.318+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd9f400f850 con 0x7fda0815d760 2024-09-21T07:53:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.319+0000 7fd9ffe00640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd9e80791f0 0x7fd9e807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.319+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fd9f409c100 con 0x7fda0815d760 2024-09-21T07:53:30.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.319+0000 7fda07e00640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd9e80791f0 0x7fd9e807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:30.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.320+0000 7fda07e00640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd9e80791f0 0x7fd9e807b700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8034000 tx=0x7fd9f8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:30.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.323+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fda08070710 con 0x7fda0815d760 2024-09-21T07:53:30.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.552+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fda081537f0 con 0x7fda0815d760 2024-09-21T07:53:30.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.552+0000 7fd9ffe00640 1 -- 172.21.15.88:0/4146177163 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7fd9f4014030 con 0x7fda0815d760 2024-09-21T07:53:30.553 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:53:30.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd9e80791f0 msgr2=0x7fd9e807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd9e80791f0 0x7fd9e807b700 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8034000 tx=0x7fd9f8004730 comp rx=0 tx=0).stop 2024-09-21T07:53:30.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 msgr2=0x7fda081f8830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081f8830 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7fd9f400c9a0 tx=0x7fd9f400ce60 comp rx=0 tx=0).stop 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 shutdown_connections 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd9e80791f0 0x7fd9e807b700 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fda08160310 0x7fda0815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fda0815d760 0x7fda081f8830 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 --2- 172.21.15.88:0/4146177163 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fda080ad0e0 0x7fda080701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 >> 172.21.15.88:0/4146177163 conn(0x7fda0806d730 msgr2=0x7fda080cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 shutdown_connections 2024-09-21T07:53:30.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30.555+0000 7fda0f200640 1 -- 172.21.15.88:0/4146177163 wait complete. 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:53:30.578 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:30.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:30.579 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:30.614 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42284sec preferred_lft 42284sec 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.615 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:53:30.616 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:30.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:30.617 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:53:30.710 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:53:30.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:30.713 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:53:30.749 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:53:30.749 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:53:30.749 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:53:30.749 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:53:30.750 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:53:30.750 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:53:30.750 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:53:30.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:53:30.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:53:30.750 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:53:30.750 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:53:30.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:53:30.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:30.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:30.946 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:53:30.981 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:53:30.981 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:53:30.981 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:30.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:30.982 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:53:31.078 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:53:31.078 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:31.113 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:53:31.113 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:53:31.113 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:31.187 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:53:31.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:31.187 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:53:31.386 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:53:31.386 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:53:31.386 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:53:31.407 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:53:31.407 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:53:31.407 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:53:31.407 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:53:31.407 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:53:31.437 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:53:31.437 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:53:31.437 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:53:31.437 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:53:31.437 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:53:31.437 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:53:31.438 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:53:31.438 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:53:31.439 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:53:31.439 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:53:31.439 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:53:31.439 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:53:31.439 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:53:31.442 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:53:31.442 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:31.442 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:31.498 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:53:31.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:53:31.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:31.593 INFO:teuthology.orchestra.run.smithi088.stdout:client6785 2024-09-21T07:53:31.594 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:31.594 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:31.630 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:53:31.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 -- 172.21.15.88:0/2277998293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc48815f910 msgr2=0x7fc4880b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:31.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 --2- 172.21.15.88:0/2277998293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc48815f910 0x7fc4880b3750 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fc480005fd0 tx=0x7fc48002aa10 comp rx=0 tx=0).stop 2024-09-21T07:53:31.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 -- 172.21.15.88:0/2277998293 shutdown_connections 2024-09-21T07:53:31.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 --2- 172.21.15.88:0/2277998293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc48815f910 0x7fc4880b3750 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:31.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 --2- 172.21.15.88:0/2277998293 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc488152920 0x7fc48815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:31.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 -- 172.21.15.88:0/2277998293 >> 172.21.15.88:0/2277998293 conn(0x7fc48806d730 msgr2=0x7fc4880b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:31.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 -- 172.21.15.88:0/2277998293 shutdown_connections 2024-09-21T07:53:31.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.891+0000 7fc48f800640 1 -- 172.21.15.88:0/2277998293 wait complete. 2024-09-21T07:53:31.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.893+0000 7fc48f800640 1 Processor -- start 2024-09-21T07:53:31.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.893+0000 7fc48f800640 1 -- start start 2024-09-21T07:53:31.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 0x7fc48814b740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:31.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48f800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc488152920 0x7fc4881f8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:31.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48f800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc48815f910 0x7fc4881f9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:31.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880b41e0 con 0x7fc4880aebc0 2024-09-21T07:53:31.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48f800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4880b4060 con 0x7fc488152920 2024-09-21T07:53:31.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48f800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4880b4360 con 0x7fc48815f910 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 0x7fc48814b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 0x7fc48814b740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35894/0 (socket says 172.21.15.88:35894) 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 -- 172.21.15.88:0/1306461989 learned_addr learned my addr 172.21.15.88:0/1306461989 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48d800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc48815f910 0x7fc4881f9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc487e00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc488152920 0x7fc4881f8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 -- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc48815f910 msgr2=0x7fc4881f9270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc48815f910 0x7fc4881f9270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 -- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc488152920 msgr2=0x7fc4881f8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc488152920 0x7fc4881f8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48ce00640 1 -- 172.21.15.88:0/1306461989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc488073e20 con 0x7fc4880aebc0 2024-09-21T07:53:31.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.895+0000 7fc48d800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc48815f910 0x7fc4881f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:31.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.896+0000 7fc48ce00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 0x7fc48814b740 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c420 tx=0x7fc47800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:31.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.896+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478017070 con 0x7fc4880aebc0 2024-09-21T07:53:31.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.896+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4780035f0 con 0x7fc4880aebc0 2024-09-21T07:53:31.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.896+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc478003a40 con 0x7fc4880aebc0 2024-09-21T07:53:31.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.896+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc488074100 con 0x7fc4880aebc0 2024-09-21T07:53:31.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.896+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc48814bc80 con 0x7fc4880aebc0 2024-09-21T07:53:31.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.897+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4880b48f0 con 0x7fc4880aebc0 2024-09-21T07:53:31.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.898+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc47801a070 con 0x7fc4880aebc0 2024-09-21T07:53:31.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.900+0000 7fc484c00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc4680791f0 0x7fc46807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:31.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.900+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fc47809ace0 con 0x7fc4880aebc0 2024-09-21T07:53:31.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.900+0000 7fc487e00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc4680791f0 0x7fc46807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:31.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.901+0000 7fc487e00640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc4680791f0 0x7fc46807b700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc474005fd0 tx=0x7fc474005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:31.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:31.904+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc478062700 con 0x7fc4880aebc0 2024-09-21T07:53:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.126+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fc4880695d0 con 0x7fc4880aebc0 2024-09-21T07:53:32.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.127+0000 7fc484c00640 1 -- 172.21.15.88:0/1306461989 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v89) ==== 53+0+73 (secure 0 0 0) 0x7fc478062000 con 0x7fc4880aebc0 2024-09-21T07:53:32.128 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.129+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc4680791f0 msgr2=0x7fc46807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.129+0000 7fc48f800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc4680791f0 0x7fc46807b700 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fc474005fd0 tx=0x7fc474005850 comp rx=0 tx=0).stop 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.129+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 msgr2=0x7fc48814b740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.129+0000 7fc48f800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 0x7fc48814b740 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fc47800c420 tx=0x7fc47800c8e0 comp rx=0 tx=0).stop 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.129+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 shutdown_connections 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.129+0000 7fc48f800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc4680791f0 0x7fc46807b700 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.130+0000 7fc48f800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc48815f910 0x7fc4881f9270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.130+0000 7fc48f800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc488152920 0x7fc4881f8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.130+0000 7fc48f800640 1 --2- 172.21.15.88:0/1306461989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4880aebc0 0x7fc48814b740 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:32.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.130+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 >> 172.21.15.88:0/1306461989 conn(0x7fc48806d730 msgr2=0x7fc48806f930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:32.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.130+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 shutdown_connections 2024-09-21T07:53:32.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:32.130+0000 7fc48f800640 1 -- 172.21.15.88:0/1306461989 wait complete. 2024-09-21T07:53:32.156 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:53:32.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:53:32.156 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:53:32.156 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:53:32.156 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:53:32.156 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:53:32.157 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:53:32.157 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:53:32.157 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:53:32.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:32.157 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:53:32.197 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:53:32.197 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:53:32.197 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:53:32.197 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.197 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:53:32.197 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42233sec preferred_lft 42233sec 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:53:32.198 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.199 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:53:32.200 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:53:32.200 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:53:32.200 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:53:32.201 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:32.201 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:53:32.299 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:53:32.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:32.302 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:53:32.339 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:53:32.339 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:53:32.339 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:53:32.340 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:53:32.340 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:53:32.340 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:53:32.340 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:53:32.340 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:53:32.340 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:53:32.340 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:53:32.340 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:53:32.458 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:53:32 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:53:32.529 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:53:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:32.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:32.537 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:53:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:53:32.572 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:53:32.572 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:53:32.573 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:32.573 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:53:32.669 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:53:32.669 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:53:32.704 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:53:32.704 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:53:32.704 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:53:32.780 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:53:32.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:32.780 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:53:32.973 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:53:32.974 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:53:32.974 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:53:32.993 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:53:32.993 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:53:32.993 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:53:32.994 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:53:32.994 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:53:33.025 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:53:33.027 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:53:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:53:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:53:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:53:33.027 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:53:33.030 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:53:33.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:33.030 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:53:33.086 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:53:33.164 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:53:33 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:53:33.167 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:53:33 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:53:33.175 INFO:teuthology.orchestra.run.smithi175.stdout:client8091 2024-09-21T07:53:33.175 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:33.175 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:33.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:53:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 -- 172.21.15.88:0/4113464466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 msgr2=0x7fd2680b36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 --2- 172.21.15.88:0/4113464466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd2680b36f0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fd260005fd0 tx=0x7fd26002aa10 comp rx=0 tx=0).stop 2024-09-21T07:53:33.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 -- 172.21.15.88:0/4113464466 shutdown_connections 2024-09-21T07:53:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 --2- 172.21.15.88:0/4113464466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd2680b36f0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 -- 172.21.15.88:0/4113464466 >> 172.21.15.88:0/4113464466 conn(0x7fd2680b2540 msgr2=0x7fd2680d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 -- 172.21.15.88:0/4113464466 shutdown_connections 2024-09-21T07:53:33.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.440+0000 7fd26e200640 1 -- 172.21.15.88:0/4113464466 wait complete. 2024-09-21T07:53:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.442+0000 7fd26e200640 1 Processor -- start 2024-09-21T07:53:33.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.442+0000 7fd26e200640 1 -- start start 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.443+0000 7fd26e200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd26809dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.443+0000 7fd26e200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680cc7d0 0x7fd2681f3240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.443+0000 7fd26e200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815ba50 0x7fd26809cd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.443+0000 7fd26e200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd26806fab0 con 0x7fd2680b3120 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.443+0000 7fd26e200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd26806f930 con 0x7fd2680cc7d0 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.443+0000 7fd26e200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd26806fc30 con 0x7fd26815ba50 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd26809dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.446 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd26809dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35906/0 (socket says 172.21.15.88:35906) 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 -- 172.21.15.88:0/171938607 learned_addr learned my addr 172.21.15.88:0/171938607 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267e00640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815ba50 0x7fd26809cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd25ea00640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680cc7d0 0x7fd2681f3240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 -- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815ba50 msgr2=0x7fd26809cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815ba50 0x7fd26809cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 -- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680cc7d0 msgr2=0x7fd2681f3240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680cc7d0 0x7fd2681f3240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 -- 172.21.15.88:0/171938607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd26815e640 con 0x7fd2680b3120 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.444+0000 7fd267400640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd26809dd90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c400 tx=0x7fd25800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.445+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258017070 con 0x7fd2680b3120 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.445+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd2580053c0 con 0x7fd2680b3120 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.445+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd26815e920 con 0x7fd2680b3120 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.445+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd258003850 con 0x7fd2680b3120 2024-09-21T07:53:33.447 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.445+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd26815d920 con 0x7fd2680b3120 2024-09-21T07:53:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.446+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2680732a0 con 0x7fd2680b3120 2024-09-21T07:53:33.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.447+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fd25801a070 con 0x7fd2680b3120 2024-09-21T07:53:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.448+0000 7fd25fe00640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2440791f0 0x7fd24407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.448+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fd25809b050 con 0x7fd2680b3120 2024-09-21T07:53:33.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.449+0000 7fd25ea00640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2440791f0 0x7fd24407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.449+0000 7fd25ea00640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2440791f0 0x7fd24407b700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd26809e610 tx=0x7fd250009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:33.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.451+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd258021020 con 0x7fd2680b3120 2024-09-21T07:53:33.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.675+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fd2680b0e00 con 0x7fd2680b3120 2024-09-21T07:53:33.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.727+0000 7fd25fe00640 1 -- 172.21.15.88:0/171938607 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v90) ==== 136+0+0 (secure 0 0 0) 0x7fd2580624a0 con 0x7fd2680b3120 2024-09-21T07:53:33.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2440791f0 msgr2=0x7fd24407b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2440791f0 0x7fd24407b700 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd26809e610 tx=0x7fd250009040 comp rx=0 tx=0).stop 2024-09-21T07:53:33.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 msgr2=0x7fd26809dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd26809dd90 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fd25800c400 tx=0x7fd25800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 shutdown_connections 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd2440791f0 0x7fd24407b700 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd26815ba50 0x7fd26809cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd2680cc7d0 0x7fd2681f3240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 --2- 172.21.15.88:0/171938607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2680b3120 0x7fd26809dd90 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 >> 172.21.15.88:0/171938607 conn(0x7fd2680b2540 msgr2=0x7fd268073750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 shutdown_connections 2024-09-21T07:53:33.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.730+0000 7fd26e200640 1 -- 172.21.15.88:0/171938607 wait complete. 2024-09-21T07:53:33.759 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:53:33.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 -- 172.21.15.88:0/4183870466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf20004800 msgr2=0x7faf20004cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 --2- 172.21.15.88:0/4183870466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf20004800 0x7faf20004cb0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7faf28009b20 tx=0x7faf20005830 comp rx=0 tx=0).stop 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 -- 172.21.15.88:0/4183870466 shutdown_connections 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 --2- 172.21.15.88:0/4183870466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf20004800 0x7faf20004cb0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 -- 172.21.15.88:0/4183870466 >> 172.21.15.88:0/4183870466 conn(0x7faf3c06d3d0 msgr2=0x7faf3c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 -- 172.21.15.88:0/4183870466 shutdown_connections 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.978+0000 7faf41800640 1 -- 172.21.15.88:0/4183870466 wait complete. 2024-09-21T07:53:33.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.981+0000 7faf41800640 1 Processor -- start 2024-09-21T07:53:33.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.981+0000 7faf41800640 1 -- start start 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf41800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf20004800 0x7faf3c0c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf41800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 0x7faf3c1566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf41800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf3c157350 0x7faf3c157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf41800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c0ad340 con 0x7faf3c1f7250 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf41800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf3c0ad1c0 con 0x7faf3c157350 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf41800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf3c0ad4c0 con 0x7faf20004800 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf3a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 0x7faf3c1566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf3a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 0x7faf3c1566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35928/0 (socket says 172.21.15.88:35928) 2024-09-21T07:53:33.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.982+0000 7faf3a000640 1 -- 172.21.15.88:0/2614341966 learned_addr learned my addr 172.21.15.88:0/2614341966 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3b400640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf3c157350 0x7faf3c157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3aa00640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf20004800 0x7faf3c0c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3a000640 1 -- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf20004800 msgr2=0x7faf3c0c6a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3a000640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf20004800 0x7faf3c0c6a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3a000640 1 -- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf3c157350 msgr2=0x7faf3c157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3a000640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf3c157350 0x7faf3c157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3a000640 1 -- 172.21.15.88:0/2614341966 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf3c0c3fa0 con 0x7faf3c1f7250 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3aa00640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf20004800 0x7faf3c0c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3b400640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf3c157350 0x7faf3c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:33.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf3a000640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 0x7faf3c1566d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7faf28009a70 tx=0x7faf28033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28038040 con 0x7faf3c1f7250 2024-09-21T07:53:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf28033950 con 0x7faf3c1f7250 2024-09-21T07:53:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf3c0c4280 con 0x7faf3c1f7250 2024-09-21T07:53:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf28033da0 con 0x7faf3c1f7250 2024-09-21T07:53:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.983+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf3c1f6d30 con 0x7faf3c1f7250 2024-09-21T07:53:33.986 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.985+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf3c1f6840 con 0x7faf3c1f7250 2024-09-21T07:53:33.987 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.986+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7faf28043020 con 0x7faf3c1f7250 2024-09-21T07:53:33.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.987+0000 7faf32a00640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf1c0791c0 0x7faf1c07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:33.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.988+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7faf280bbba0 con 0x7faf3c1f7250 2024-09-21T07:53:33.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.988+0000 7faf3aa00640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf1c0791c0 0x7faf1c07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:33.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.988+0000 7faf3aa00640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf1c0791c0 0x7faf1c07b6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faf2c001860 tx=0x7faf2c000ed0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:33.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:33.992+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf280bde20 con 0x7faf3c1f7250 2024-09-21T07:53:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.199+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7faf3c074290 con 0x7faf1c0791c0 2024-09-21T07:53:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.200+0000 7faf32a00640 1 -- 172.21.15.88:0/2614341966 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7faf3c074290 con 0x7faf1c0791c0 2024-09-21T07:53:34.201 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:53:34.202 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:53:34.202 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:53:34.202 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:53:34.202 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:53:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf1c0791c0 msgr2=0x7faf1c07b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf1c0791c0 0x7faf1c07b6d0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7faf2c001860 tx=0x7faf2c000ed0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 msgr2=0x7faf3c1566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 0x7faf3c1566d0 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7faf28009a70 tx=0x7faf28033670 comp rx=0 tx=0).stop 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 shutdown_connections 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf1c0791c0 0x7faf1c07b6d0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf3c157350 0x7faf3c157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf3c1f7250 0x7faf3c1566d0 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 --2- 172.21.15.88:0/2614341966 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf20004800 0x7faf3c0c6a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 >> 172.21.15.88:0/2614341966 conn(0x7faf3c06d3d0 msgr2=0x7faf3c0730b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 shutdown_connections 2024-09-21T07:53:34.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.203+0000 7faf41800640 1 -- 172.21.15.88:0/2614341966 wait complete. 2024-09-21T07:53:34.226 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:53:34.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 -- 172.21.15.88:0/3255877827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb558145550 msgr2=0x7fb55806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 --2- 172.21.15.88:0/3255877827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb558145550 0x7fb55806fb40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb548005fd0 tx=0x7fb54802a670 comp rx=0 tx=0).stop 2024-09-21T07:53:34.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 -- 172.21.15.88:0/3255877827 shutdown_connections 2024-09-21T07:53:34.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 --2- 172.21.15.88:0/3255877827 >> v2:172.21.15.175:3300/0 conn(0x7fb558073c50 0x7fb5580cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 --2- 172.21.15.88:0/3255877827 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb558145550 0x7fb55806fb40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 -- 172.21.15.88:0/3255877827 >> 172.21.15.88:0/3255877827 conn(0x7fb55806d1c0 msgr2=0x7fb55806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:34.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 -- 172.21.15.88:0/3255877827 shutdown_connections 2024-09-21T07:53:34.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.447+0000 7fb55da00640 1 -- 172.21.15.88:0/3255877827 wait complete. 2024-09-21T07:53:34.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.449+0000 7fb55da00640 1 Processor -- start 2024-09-21T07:53:34.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.449+0000 7fb55da00640 1 -- start start 2024-09-21T07:53:34.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb55da00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 0x7fb5581f96e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb55da00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb558145a40 0x7fb55815bfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb55da00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb55813e1b0 0x7fb55813e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb55da00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5580ce1e0 con 0x7fb558145550 2024-09-21T07:53:34.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb55da00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb5580ce060 con 0x7fb558145a40 2024-09-21T07:53:34.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb55da00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb5580ce360 con 0x7fb55813e1b0 2024-09-21T07:53:34.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb556a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 0x7fb5581f96e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb556a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 0x7fb5581f96e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35936/0 (socket says 172.21.15.88:35936) 2024-09-21T07:53:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb556a00640 1 -- 172.21.15.88:0/2070963912 learned_addr learned my addr 172.21.15.88:0/2070963912 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb556000640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb558145a40 0x7fb55815bfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.451+0000 7fb557400640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb55813e1b0 0x7fb55813e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb556a00640 1 -- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb55813e1b0 msgr2=0x7fb55813e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb556a00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb55813e1b0 0x7fb55813e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb556a00640 1 -- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb558145a40 msgr2=0x7fb55815bfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb556a00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb558145a40 0x7fb55815bfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb556a00640 1 -- 172.21.15.88:0/2070963912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb55815c500 con 0x7fb558145550 2024-09-21T07:53:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb556a00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 0x7fb5581f96e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb548006010 tx=0x7fb5480038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:34.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54803e030 con 0x7fb558145550 2024-09-21T07:53:34.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb548003d90 con 0x7fb558145550 2024-09-21T07:53:34.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548035040 con 0x7fb558145550 2024-09-21T07:53:34.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb55813eb00 con 0x7fb558145550 2024-09-21T07:53:34.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.452+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5580b54e0 con 0x7fb558145550 2024-09-21T07:53:34.459 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.453+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb55806a7f0 con 0x7fb558145550 2024-09-21T07:53:34.460 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.455+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb548041030 con 0x7fb558145550 2024-09-21T07:53:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.455+0000 7fb54ea00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5380791f0 0x7fb53807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.456+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fb5480bc060 con 0x7fb558145550 2024-09-21T07:53:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.456+0000 7fb556000640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5380791f0 0x7fb53807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.456+0000 7fb556000640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5380791f0 0x7fb53807b700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb55813edb0 tx=0x7fb544009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:34.462 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.460+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb55813eb00 con 0x7fb558145550 2024-09-21T07:53:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.644+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fb5580ce8c0 con 0x7fb558145550 2024-09-21T07:53:34.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.645+0000 7fb54ea00640 1 -- 172.21.15.88:0/2070963912 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) ==== 122+0+0 (secure 0 0 0) 0x7fb55815c500 con 0x7fb558145550 2024-09-21T07:53:34.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5380791f0 msgr2=0x7fb53807b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5380791f0 0x7fb53807b700 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb55813edb0 tx=0x7fb544009040 comp rx=0 tx=0).stop 2024-09-21T07:53:34.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 msgr2=0x7fb5581f96e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 0x7fb5581f96e0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fb548006010 tx=0x7fb5480038a0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 shutdown_connections 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb5380791f0 0x7fb53807b700 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb55813e1b0 0x7fb55813e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb558145a40 0x7fb55815bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 --2- 172.21.15.88:0/2070963912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558145550 0x7fb5581f96e0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 >> 172.21.15.88:0/2070963912 conn(0x7fb55806d1c0 msgr2=0x7fb5580b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 shutdown_connections 2024-09-21T07:53:34.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.648+0000 7fb55da00640 1 -- 172.21.15.88:0/2070963912 wait complete. 2024-09-21T07:53:34.676 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create non_existent_subvolume subvolume_1 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 -- 172.21.15.88:0/3671322433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 msgr2=0x7fc8ec156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 --2- 172.21.15.88:0/3671322433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 0x7fc8ec156ca0 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc009830 tx=0x7fc8dc02a510 comp rx=0 tx=0).stop 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 -- 172.21.15.88:0/3671322433 shutdown_connections 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 --2- 172.21.15.88:0/3671322433 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8ec0cd1d0 0x7fc8ec0af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 --2- 172.21.15.88:0/3671322433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 0x7fc8ec156ca0 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 -- 172.21.15.88:0/3671322433 >> 172.21.15.88:0/3671322433 conn(0x7fc8ec06d620 msgr2=0x7fc8ec0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 -- 172.21.15.88:0/3671322433 shutdown_connections 2024-09-21T07:53:34.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.897+0000 7fc8f1200640 1 -- 172.21.15.88:0/3671322433 wait complete. 2024-09-21T07:53:34.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.900+0000 7fc8f1200640 1 Processor -- start 2024-09-21T07:53:34.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.900+0000 7fc8f1200640 1 -- start start 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 0x7fc8ec153d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ec0cd1d0 0x7fc8ec144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8f1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8ec15d6e0 0x7fc8ec145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8f1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0af9a0 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8f1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc8ec0af820 con 0x7fc8ec0cd1d0 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8f1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc8ec0afb20 con 0x7fc8ec15d6e0 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.901+0000 7fc8eb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 0x7fc8ec153d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 0x7fc8ec153d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35952/0 (socket says 172.21.15.88:35952) 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 -- 172.21.15.88:0/1292790150 learned_addr learned my addr 172.21.15.88:0/1292790150 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:34.905 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8ebe00640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8ec15d6e0 0x7fc8ec145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eaa00640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ec0cd1d0 0x7fc8ec144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 -- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8ec15d6e0 msgr2=0x7fc8ec145120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc8ec15d6e0 0x7fc8ec145120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 -- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ec0cd1d0 msgr2=0x7fc8ec144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ec0cd1d0 0x7fc8ec144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 -- 172.21.15.88:0/1292790150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8ec1f4560 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eaa00640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc8ec0cd1d0 0x7fc8ec144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8eb400640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc8ec0b3f30 0x7fc8ec153d60 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7fc8dc02ab00 tx=0x7fc8dc033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.902+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc031d60 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.903+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc8dc033e10 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.903+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8dc005260 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.903+0000 7fc8f1200640 1 -- 172.21.15.88:0/1292790150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8ec1f4b40 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.903+0000 7fc8f1200640 1 -- 172.21.15.88:0/1292790150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8ec1f66a0 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.904+0000 7fc8f1200640 1 -- 172.21.15.88:0/1292790150 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8ec1f6900 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.905+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc8dc005400 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.906+0000 7fc8e3400640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc8cc0791c0 0x7fc8cc07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:34.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.906+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fc8dc0bc9d0 con 0x7fc8ec0b3f30 2024-09-21T07:53:34.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.906+0000 7fc8eaa00640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc8cc0791c0 0x7fc8cc07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:34.909 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.907+0000 7fc8eaa00640 1 --2- 172.21.15.88:0/1292790150 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc8cc0791c0 0x7fc8cc07b6d0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec15ecc0 tx=0x7fc8d8007d50 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:34.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:34.910+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc8dc03f030 con 0x7fc8ec0b3f30 2024-09-21T07:53:35.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.123+0000 7fc8f1200640 1 -- 172.21.15.88:0/1292790150 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fc8ec1f4930 con 0x7fc8cc0791c0 2024-09-21T07:53:35.126 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:35.124+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:35.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.125+0000 7fc8e3400640 1 -- 172.21.15.88:0/1292790150 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc8ec1f4930 con 0x7fc8cc0791c0 2024-09-21T07:53:35.127 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:35.144 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:35.145 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm non_existent_subvolume subvolume_1 2024-09-21T07:53:35.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.367+0000 7f0ab3e00640 1 -- 172.21.15.88:0/4175071785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a94003a30 msgr2=0x7f0a94003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.367+0000 7f0ab3e00640 1 --2- 172.21.15.88:0/4175071785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a94003a30 0x7f0a94003ea0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c310 tx=0x7f0a94005830 comp rx=0 tx=0).stop 2024-09-21T07:53:35.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.367+0000 7f0ab1e00640 1 -- 172.21.15.88:0/4175071785 reap_dead start 2024-09-21T07:53:35.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.368+0000 7f0ab3e00640 1 -- 172.21.15.88:0/4175071785 shutdown_connections 2024-09-21T07:53:35.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.368+0000 7f0ab3e00640 1 -- 172.21.15.88:0/4175071785 >> 172.21.15.88:0/4175071785 conn(0x7f0aac06d1c0 msgr2=0x7f0aac06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:35.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.368+0000 7f0ab3e00640 1 -- 172.21.15.88:0/4175071785 shutdown_connections 2024-09-21T07:53:35.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.368+0000 7f0ab3e00640 1 -- 172.21.15.88:0/4175071785 wait complete. 2024-09-21T07:53:35.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.370+0000 7f0ab3e00640 1 Processor -- start 2024-09-21T07:53:35.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.370+0000 7f0ab3e00640 1 -- start start 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.371+0000 7f0ab3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a94003a30 0x7f0aac1f9a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.371+0000 7f0ab3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b3480 0x7f0aac0b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0b0030 0x7f0aac150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac1648d0 con 0x7f0a94003a30 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac164750 con 0x7f0aac0b3480 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac164a50 con 0x7f0aac0b0030 2024-09-21T07:53:35.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a94003a30 0x7f0aac1f9a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a94003a30 0x7f0aac1f9a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35964/0 (socket says 172.21.15.88:35964) 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 -- 172.21.15.88:0/789561739 learned_addr learned my addr 172.21.15.88:0/789561739 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab0a00640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b3480 0x7f0aac0b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1e00640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0b0030 0x7f0aac150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 -- 172.21.15.88:0/789561739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0b0030 msgr2=0x7f0aac150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0b0030 0x7f0aac150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 -- 172.21.15.88:0/789561739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b3480 msgr2=0x7f0aac0b4290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b3480 0x7f0aac0b4290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 -- 172.21.15.88:0/789561739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac0af020 con 0x7f0a94003a30 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.372+0000 7f0ab1400640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a94003a30 0x7f0aac1f9a60 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f0aa0001d10 tx=0x7f0aa0001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.373+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0031040 con 0x7f0a94003a30 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.373+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0aa00065e0 con 0x7f0a94003a30 2024-09-21T07:53:35.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.373+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0003a40 con 0x7f0a94003a30 2024-09-21T07:53:35.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.373+0000 7f0ab3e00640 1 -- 172.21.15.88:0/789561739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aac0af300 con 0x7f0a94003a30 2024-09-21T07:53:35.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.373+0000 7f0ab3e00640 1 -- 172.21.15.88:0/789561739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aac1508c0 con 0x7f0a94003a30 2024-09-21T07:53:35.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.374+0000 7f0ab3e00640 1 -- 172.21.15.88:0/789561739 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aac0cf570 con 0x7f0a94003a30 2024-09-21T07:53:35.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.375+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0aa003e030 con 0x7f0a94003a30 2024-09-21T07:53:35.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.377+0000 7f0aa9600640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a8c0791f0 0x7f0a8c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.377+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7f0aa00bb650 con 0x7f0a94003a30 2024-09-21T07:53:35.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.377+0000 7f0ab0a00640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a8c0791f0 0x7f0a8c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.378+0000 7f0ab0a00640 1 --2- 172.21.15.88:0/789561739 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a8c0791f0 0x7f0a8c07b700 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f0aac1596c0 tx=0x7f0a98006b20 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:35.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.381+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0aa00831c0 con 0x7f0a94003a30 2024-09-21T07:53:35.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.590+0000 7f0ab3e00640 1 -- 172.21.15.88:0/789561739 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f0aac0b0810 con 0x7f0a8c0791f0 2024-09-21T07:53:35.594 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:35.592+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.592+0000 7f0aa9600640 1 -- 172.21.15.88:0/789561739 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0a98012500 con 0x7f0a8c0791f0 2024-09-21T07:53:35.594 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:35.615 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:35.615 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath non_existent_subvolume subvolume_1 2024-09-21T07:53:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 -- 172.21.15.88:0/4223343353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc073a80 msgr2=0x7f0dcc0cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 --2- 172.21.15.88:0/4223343353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc073a80 0x7f0dcc0cd610 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f0dcc160280 tx=0x7f0dc402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 -- 172.21.15.88:0/4223343353 shutdown_connections 2024-09-21T07:53:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 --2- 172.21.15.88:0/4223343353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc073a80 0x7f0dcc0cd610 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 --2- 172.21.15.88:0/4223343353 >> v2:172.21.15.175:3300/0 conn(0x7f0dcc145a40 0x7f0dcc073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 -- 172.21.15.88:0/4223343353 >> 172.21.15.88:0/4223343353 conn(0x7f0dcc06d1c0 msgr2=0x7f0dcc06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 -- 172.21.15.88:0/4223343353 shutdown_connections 2024-09-21T07:53:35.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.840+0000 7f0dd2200640 1 -- 172.21.15.88:0/4223343353 wait complete. 2024-09-21T07:53:35.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.842+0000 7f0dd2200640 1 Processor -- start 2024-09-21T07:53:35.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.842+0000 7f0dd2200640 1 -- start start 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dd2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc073a80 0x7f0dcc15eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dd2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc145550 0x7f0dcc1f3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dd2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dcc145a40 0x7f0dcc13dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dd2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0ce0e0 con 0x7f0dcc145550 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dd2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0dcc0cdf60 con 0x7f0dcc073a80 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dd2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0dcc0ce260 con 0x7f0dcc145a40 2024-09-21T07:53:35.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc145550 0x7f0dcc1f3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc145550 0x7f0dcc1f3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49042/0 (socket says 172.21.15.88:49042) 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 -- 172.21.15.88:0/1395299840 learned_addr learned my addr 172.21.15.88:0/1395299840 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dcbe00640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dcc145a40 0x7f0dcc13dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dcb400640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc073a80 0x7f0dcc15eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 -- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dcc145a40 msgr2=0x7f0dcc13dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dcc145a40 0x7f0dcc13dab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 -- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc073a80 msgr2=0x7f0dcc15eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc073a80 0x7f0dcc15eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dc2a00640 1 -- 172.21.15.88:0/1395299840 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0dcc15f700 con 0x7f0dcc145550 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dcbe00640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0dcc145a40 0x7f0dcc13dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.844+0000 7f0dcb400640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0dcc073a80 0x7f0dcc15eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.845+0000 7f0dc2a00640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0dcc145550 0x7f0dcc1f3c80 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7f0db400e9e0 tx=0x7f0db400eea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.845+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db400cd50 con 0x7f0dcc145550 2024-09-21T07:53:35.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.845+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0db4011470 con 0x7f0dcc145550 2024-09-21T07:53:35.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.845+0000 7f0dd2200640 1 -- 172.21.15.88:0/1395299840 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0dcc15f9e0 con 0x7f0dcc145550 2024-09-21T07:53:35.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.845+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0db4011860 con 0x7f0dcc145550 2024-09-21T07:53:35.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.845+0000 7f0dd2200640 1 -- 172.21.15.88:0/1395299840 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0dcc15ffa0 con 0x7f0dcc145550 2024-09-21T07:53:35.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.846+0000 7f0dd2200640 1 -- 172.21.15.88:0/1395299840 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0dcc15f450 con 0x7f0dcc145550 2024-09-21T07:53:35.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.847+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f0db4011a00 con 0x7f0dcc145550 2024-09-21T07:53:35.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.848+0000 7f0dc3e00640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0da80791f0 0x7f0da807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:35.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.848+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7f0db409c6c0 con 0x7f0dcc145550 2024-09-21T07:53:35.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.848+0000 7f0dcb400640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0da80791f0 0x7f0da807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:35.851 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.849+0000 7f0dcb400640 1 --2- 172.21.15.88:0/1395299840 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0da80791f0 0x7f0da807b700 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f0dbc000f60 tx=0x7f0dbc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:35.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:35.853+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0db4016030 con 0x7f0dcc145550 2024-09-21T07:53:36.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.065+0000 7f0dd2200640 1 -- 172.21.15.88:0/1395299840 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f0dcc0ce780 con 0x7f0da80791f0 2024-09-21T07:53:36.068 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:36.066+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:36.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.066+0000 7f0dc3e00640 1 -- 172.21.15.88:0/1395299840 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f0dcc0ce780 con 0x7f0da80791f0 2024-09-21T07:53:36.069 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:36.084 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:36.084 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume info non_existent_subvolume subvolume_1 2024-09-21T07:53:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe342000640 1 -- 172.21.15.88:0/3085510526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe320004800 msgr2=0x7fe320004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe342000640 1 --2- 172.21.15.88:0/3085510526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe320004800 0x7fe320004c10 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fe33c1613e0 tx=0x7fe320005830 comp rx=0 tx=0).stop 2024-09-21T07:53:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe33be00640 1 -- 172.21.15.88:0/3085510526 reap_dead start 2024-09-21T07:53:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe342000640 1 -- 172.21.15.88:0/3085510526 shutdown_connections 2024-09-21T07:53:36.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe342000640 1 -- 172.21.15.88:0/3085510526 >> 172.21.15.88:0/3085510526 conn(0x7fe33c06d1c0 msgr2=0x7fe33c06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:36.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe342000640 1 -- 172.21.15.88:0/3085510526 shutdown_connections 2024-09-21T07:53:36.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.309+0000 7fe342000640 1 -- 172.21.15.88:0/3085510526 wait complete. 2024-09-21T07:53:36.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.311+0000 7fe342000640 1 Processor -- start 2024-09-21T07:53:36.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.311+0000 7fe342000640 1 -- start start 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe342000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe320004800 0x7fe33c0cbb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe342000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe33c0cc070 0x7fe33c1f3610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe342000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33c1f3b50 0x7fe33c1f6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe342000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c0cdcd0 con 0x7fe33c0cc070 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe342000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe33c0cdb50 con 0x7fe33c1f3b50 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe342000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe33c0cde50 con 0x7fe320004800 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe33c0cc070 0x7fe33c1f3610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe33c0cc070 0x7fe33c1f3610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49052/0 (socket says 172.21.15.88:49052) 2024-09-21T07:53:36.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 -- 172.21.15.88:0/2387948475 learned_addr learned my addr 172.21.15.88:0/2387948475 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:36.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33b400640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe320004800 0x7fe33c0cbb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33be00640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33c1f3b50 0x7fe33c1f6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 -- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe320004800 msgr2=0x7fe33c0cbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe320004800 0x7fe33c0cbb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 -- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33c1f3b50 msgr2=0x7fe33c1f6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33c1f3b50 0x7fe33c1f6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33aa00640 1 -- 172.21.15.88:0/2387948475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe33c0a9790 con 0x7fe33c0cc070 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33b400640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe320004800 0x7fe33c0cbb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.313+0000 7fe33be00640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe33c1f3b50 0x7fe33c1f6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.314+0000 7fe33aa00640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe33c0cc070 0x7fe33c1f3610 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fe33c072330 tx=0x7fe32802b9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.314+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe32802bd60 con 0x7fe33c0cc070 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.314+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe328044070 con 0x7fe33c0cc070 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.314+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe328004a00 con 0x7fe33c0cc070 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.314+0000 7fe342000640 1 -- 172.21.15.88:0/2387948475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe33c0a9a70 con 0x7fe33c0cc070 2024-09-21T07:53:36.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.314+0000 7fe342000640 1 -- 172.21.15.88:0/2387948475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe33c0aa030 con 0x7fe33c0cc070 2024-09-21T07:53:36.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.315+0000 7fe342000640 1 -- 172.21.15.88:0/2387948475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe33c1f4110 con 0x7fe33c0cc070 2024-09-21T07:53:36.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.317+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fe328004ba0 con 0x7fe33c0cc070 2024-09-21T07:53:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.317+0000 7fe333400640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe31c0791f0 0x7fe31c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.319 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.318+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(132..132 src has 1..132) ==== 5737+0+0 (secure 0 0 0) 0x7fe3280bccd0 con 0x7fe33c0cc070 2024-09-21T07:53:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.318+0000 7fe33b400640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe31c0791f0 0x7fe31c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.318+0000 7fe33b400640 1 --2- 172.21.15.88:0/2387948475 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe31c0791f0 0x7fe31c07b700 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fe32c000fb0 tx=0x7fe32c001290 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:36.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.322+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe3280846a0 con 0x7fe33c0cc070 2024-09-21T07:53:36.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.535+0000 7fe342000640 1 -- 172.21.15.88:0/2387948475 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7fe33c0a9540 con 0x7fe31c0791f0 2024-09-21T07:53:36.538 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:36.536+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:36.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.536+0000 7fe333400640 1 -- 172.21.15.88:0/2387948475 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fe33c0a9540 con 0x7fe31c0791f0 2024-09-21T07:53:36.539 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:36.554 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:36.555 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume resize non_existent_subvolume subvolname_1 inf 2024-09-21T07:53:36.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 -- 172.21.15.88:0/4089851047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600705b0 msgr2=0x7fed600b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:36.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 --2- 172.21.15.88:0/4089851047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600705b0 0x7fed600b47c0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fed4c009860 tx=0x7fed4c02a510 comp rx=0 tx=0).stop 2024-09-21T07:53:36.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 -- 172.21.15.88:0/4089851047 shutdown_connections 2024-09-21T07:53:36.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 --2- 172.21.15.88:0/4089851047 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600705b0 0x7fed600b47c0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:36.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 -- 172.21.15.88:0/4089851047 >> 172.21.15.88:0/4089851047 conn(0x7fed600b1dd0 msgr2=0x7fed600b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:36.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 -- 172.21.15.88:0/4089851047 shutdown_connections 2024-09-21T07:53:36.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.783+0000 7fed65800640 1 -- 172.21.15.88:0/4089851047 wait complete. 2024-09-21T07:53:36.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.785+0000 7fed65800640 1 Processor -- start 2024-09-21T07:53:36.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.785+0000 7fed65800640 1 -- start start 2024-09-21T07:53:36.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed65800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed600705b0 0x7fed60152a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed65800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600af3c0 0x7fed60152fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed65800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed600b3ec0 0x7fed6013d280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed65800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed60167e20 con 0x7fed600705b0 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed65800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fed60160bd0 con 0x7fed600b3ec0 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed65800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fed60167fa0 con 0x7fed600af3c0 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed600705b0 0x7fed60152a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed600705b0 0x7fed60152a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49068/0 (socket says 172.21.15.88:49068) 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 -- 172.21.15.88:0/3788266708 learned_addr learned my addr 172.21.15.88:0/3788266708 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5f400640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600af3c0 0x7fed60152fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 -- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600af3c0 msgr2=0x7fed60152fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600af3c0 0x7fed60152fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed64e00640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed600b3ec0 0x7fed6013d280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 -- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed600b3ec0 msgr2=0x7fed6013d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:36.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed600b3ec0 0x7fed6013d280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:36.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5fe00640 1 -- 172.21.15.88:0/3788266708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed6014eee0 con 0x7fed600705b0 2024-09-21T07:53:36.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed5f400640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fed600af3c0 0x7fed60152fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:36.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.787+0000 7fed64e00640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fed600b3ec0 0x7fed6013d280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:36.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.788+0000 7fed5fe00640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed600705b0 0x7fed60152a70 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fed5000c730 tx=0x7fed5000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.788+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed50016020 con 0x7fed600705b0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.788+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fed5000a8f0 con 0x7fed600705b0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.788+0000 7fed65800640 1 -- 172.21.15.88:0/3788266708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed6014f1c0 con 0x7fed600705b0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.788+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed5000ad40 con 0x7fed600705b0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.788+0000 7fed65800640 1 -- 172.21.15.88:0/3788266708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed6014f780 con 0x7fed600705b0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.789+0000 7fed65800640 1 -- 172.21.15.88:0/3788266708 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed600b47c0 con 0x7fed600705b0 2024-09-21T07:53:36.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.790+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fed50019020 con 0x7fed600705b0 2024-09-21T07:53:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.791+0000 7fed57e00640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed400791c0 0x7fed4007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.791+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5795+0+0 (secure 0 0 0) 0x7fed5009b6d0 con 0x7fed600705b0 2024-09-21T07:53:36.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.791+0000 7fed5f400640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed400791c0 0x7fed4007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:36.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.792+0000 7fed5f400640 1 --2- 172.21.15.88:0/3788266708 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fed400791c0 0x7fed4007b6d0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fed4c034000 tx=0x7fed4c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:36.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:36.795+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fed50020030 con 0x7fed600705b0 2024-09-21T07:53:37.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.017+0000 7fed65800640 1 -- 172.21.15.88:0/3788266708 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume resize", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "new_size": "inf", "target": ["mon-mgr", ""]}) -- 0x7fed600b3780 con 0x7fed400791c0 2024-09-21T07:53:37.020 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:37.018+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:37.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.018+0000 7fed57e00640 1 -- 172.21.15.88:0/3788266708 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fed600b3780 con 0x7fed400791c0 2024-09-21T07:53:37.020 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:37.040 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:37.041 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume pin non_existent_subvolume subvolname_1 export 1 2024-09-21T07:53:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 -- 172.21.15.88:0/3984781576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff28c005250 msgr2=0x7ff28c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 --2- 172.21.15.88:0/3984781576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff28c005250 0x7ff28c00b9c0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0001d90 tx=0x7ff28c005830 comp rx=0 tx=0).stop 2024-09-21T07:53:37.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 -- 172.21.15.88:0/3984781576 shutdown_connections 2024-09-21T07:53:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 --2- 172.21.15.88:0/3984781576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff28c005250 0x7ff28c00b9c0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 -- 172.21.15.88:0/3984781576 >> 172.21.15.88:0/3984781576 conn(0x7ff2a80b2540 msgr2=0x7ff2a80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 -- 172.21.15.88:0/3984781576 shutdown_connections 2024-09-21T07:53:37.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.263+0000 7ff2af600640 1 -- 172.21.15.88:0/3984781576 wait complete. 2024-09-21T07:53:37.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.265+0000 7ff2af600640 1 Processor -- start 2024-09-21T07:53:37.267 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.265+0000 7ff2af600640 1 -- start start 2024-09-21T07:53:37.269 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2af600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28c005250 0x7ff2a81f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2af600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a814f5c0 0x7ff2a81f6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2af600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a81f7290 0x7ff2a81f78c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2af600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a806d540 con 0x7ff2a81f7290 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2af600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff2a806d3c0 con 0x7ff2a814f5c0 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2af600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff2a806d6c0 con 0x7ff28c005250 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a81f7290 0x7ff2a81f78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a81f7290 0x7ff2a81f78c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49076/0 (socket says 172.21.15.88:49076) 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 -- 172.21.15.88:0/3361820593 learned_addr learned my addr 172.21.15.88:0/3361820593 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2a7e00640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a814f5c0 0x7ff2a81f6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2acc00640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28c005250 0x7ff2a81f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 -- 172.21.15.88:0/3361820593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28c005250 msgr2=0x7ff2a81f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff28c005250 0x7ff2a81f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.270 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 -- 172.21.15.88:0/3361820593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a814f5c0 msgr2=0x7ff2a81f6d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff2a814f5c0 0x7ff2a81f6d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.267+0000 7ff2ad600640 1 -- 172.21.15.88:0/3361820593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2a81f5770 con 0x7ff2a81f7290 2024-09-21T07:53:37.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.268+0000 7ff2ad600640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff2a81f7290 0x7ff2a81f78c0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7ff2a0001eb0 tx=0x7ff2a0006d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.268+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0003d80 con 0x7ff2a81f7290 2024-09-21T07:53:37.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.268+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff2a0005a90 con 0x7ff2a81f7290 2024-09-21T07:53:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.268+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2a0005ec0 con 0x7ff2a81f7290 2024-09-21T07:53:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.268+0000 7ff2af600640 1 -- 172.21.15.88:0/3361820593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2a81f5a50 con 0x7ff2a81f7290 2024-09-21T07:53:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.268+0000 7ff2af600640 1 -- 172.21.15.88:0/3361820593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2a81f6010 con 0x7ff2a81f7290 2024-09-21T07:53:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.269+0000 7ff2af600640 1 -- 172.21.15.88:0/3361820593 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2a81f54e0 con 0x7ff2a81f7290 2024-09-21T07:53:37.272 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.270+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7ff2a0006060 con 0x7ff2a81f7290 2024-09-21T07:53:37.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.271+0000 7ff2a4c00640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2880791f0 0x7ff28807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.273 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.271+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5795+0+0 (secure 0 0 0) 0x7ff2a00bb610 con 0x7ff2a81f7290 2024-09-21T07:53:37.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.272+0000 7ff2acc00640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2880791f0 0x7ff28807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.274 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.272+0000 7ff2acc00640 1 --2- 172.21.15.88:0/3361820593 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2880791f0 0x7ff28807b700 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7ff298009b10 tx=0x7ff298005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.278 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.276+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff2a0083050 con 0x7ff2a81f7290 2024-09-21T07:53:37.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.494+0000 7ff2af600640 1 -- 172.21.15.88:0/3361820593 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume pin", "vol_name": "non_existent_subvolume", "sub_name": "subvolname_1", "pin_type": "export", "pin_setting": "1", "target": ["mon-mgr", ""]}) -- 0x7ff2a80ae9e0 con 0x7ff2880791f0 2024-09-21T07:53:37.497 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:37.496+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:37.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.496+0000 7ff2a4c00640 1 -- 172.21.15.88:0/3361820593 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7ff2a80ae9e0 con 0x7ff2880791f0 2024-09-21T07:53:37.498 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:37.515 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:37.515 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume ls non_existent_subvolume 2024-09-21T07:53:37.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 -- 172.21.15.88:0/738248200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa4073c50 msgr2=0x7feaa406f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 --2- 172.21.15.88:0/738248200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa4073c50 0x7feaa406f7d0 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7fea9c005fd0 tx=0x7fea9c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 -- 172.21.15.88:0/738248200 shutdown_connections 2024-09-21T07:53:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 --2- 172.21.15.88:0/738248200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa4073c50 0x7feaa406f7d0 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 -- 172.21.15.88:0/738248200 >> 172.21.15.88:0/738248200 conn(0x7feaa406d1c0 msgr2=0x7feaa406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 -- 172.21.15.88:0/738248200 shutdown_connections 2024-09-21T07:53:37.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.740+0000 7feaa8e00640 1 -- 172.21.15.88:0/738248200 wait complete. 2024-09-21T07:53:37.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.742+0000 7feaa8e00640 1 Processor -- start 2024-09-21T07:53:37.744 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.742+0000 7feaa8e00640 1 -- start start 2024-09-21T07:53:37.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaa4147b90 0x7feaa413fa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa415f9e0 0x7feaa40b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaa413ff50 0x7feaa40b23e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa8e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feaa406fe90 con 0x7feaa415f9e0 2024-09-21T07:53:37.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feaa406fd10 con 0x7feaa413ff50 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feaa4070010 con 0x7feaa4147b90 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa415f9e0 0x7feaa40b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa415f9e0 0x7feaa40b1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49092/0 (socket says 172.21.15.88:49092) 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 -- 172.21.15.88:0/1349597243 learned_addr learned my addr 172.21.15.88:0/1349597243 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa2800640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaa413ff50 0x7feaa40b23e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1e00640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaa4147b90 0x7feaa413fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 -- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaa4147b90 msgr2=0x7feaa413fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaa4147b90 0x7feaa413fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 -- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaa413ff50 msgr2=0x7feaa40b23e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feaa413ff50 0x7feaa40b23e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:37.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1400640 1 -- 172.21.15.88:0/1349597243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaa4140780 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.744+0000 7feaa1e00640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feaa4147b90 0x7feaa413fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.745+0000 7feaa1400640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feaa415f9e0 0x7feaa40b1ea0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7fea9000c3a0 tx=0x7fea9000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.745+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90017070 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.745+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fea900053c0 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.745+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea90003850 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.745+0000 7feaa8e00640 1 -- 172.21.15.88:0/1349597243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaa40b10e0 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.745+0000 7feaa8e00640 1 -- 172.21.15.88:0/1349597243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaa40b16a0 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.746+0000 7feaa8e00640 1 -- 172.21.15.88:0/1349597243 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaa41404f0 con 0x7feaa415f9e0 2024-09-21T07:53:37.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.748+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fea9001a070 con 0x7feaa415f9e0 2024-09-21T07:53:37.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.748+0000 7fea9a000640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fea840791f0 0x7fea8407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:37.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.749+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(133..133 src has 1..133) ==== 5795+0+0 (secure 0 0 0) 0x7fea9009acb0 con 0x7feaa415f9e0 2024-09-21T07:53:37.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.749+0000 7feaa1e00640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fea840791f0 0x7fea8407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:37.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.750+0000 7feaa1e00640 1 --2- 172.21.15.88:0/1349597243 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fea840791f0 0x7fea8407b700 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feaa40b3240 tx=0x7fea94009550 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:37.754 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.752+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fea90062180 con 0x7feaa415f9e0 2024-09-21T07:53:37.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.969+0000 7feaa8e00640 1 -- 172.21.15.88:0/1349597243 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7feaa406f7d0 con 0x7fea840791f0 2024-09-21T07:53:37.972 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:37.971+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:37.971+0000 7fea9a000640 1 -- 172.21.15.88:0/1349597243 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7feaa406f7d0 con 0x7fea840791f0 2024-09-21T07:53:37.973 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:37.988 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:37.989 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot create non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 -- 172.21.15.88:0/2487215994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0b7510 msgr2=0x7fc32c0b7920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 --2- 172.21.15.88:0/2487215994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0b7510 0x7fc32c0b7920 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7fc32c08ee00 tx=0x7fc32402a8f0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 -- 172.21.15.88:0/2487215994 shutdown_connections 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 --2- 172.21.15.88:0/2487215994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0b7510 0x7fc32c0b7920 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 --2- 172.21.15.88:0/2487215994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc32c0b6320 0x7fc32c0b6730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 --2- 172.21.15.88:0/2487215994 >> v2:172.21.15.175:3300/0 conn(0x7fc32c0d5300 0x7fc32c0d2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 -- 172.21.15.88:0/2487215994 >> 172.21.15.88:0/2487215994 conn(0x7fc32c0d46c0 msgr2=0x7fc32c0d4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.215+0000 7fc332a00640 1 -- 172.21.15.88:0/2487215994 shutdown_connections 2024-09-21T07:53:38.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.216+0000 7fc332a00640 1 -- 172.21.15.88:0/2487215994 wait complete. 2024-09-21T07:53:38.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.218+0000 7fc332a00640 1 Processor -- start 2024-09-21T07:53:38.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.218+0000 7fc332a00640 1 -- start start 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.219+0000 7fc332a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc32c0b6320 0x7fc32c09e280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.219+0000 7fc332a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc32c0b7510 0x7fc32c14c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.219+0000 7fc332a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0d5300 0x7fc32c14c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.219+0000 7fc332a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0d3830 con 0x7fc32c0d5300 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.219+0000 7fc332a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc32c0d36b0 con 0x7fc32c0b7510 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.219+0000 7fc332a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc32c0d39b0 con 0x7fc32c0b6320 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0d5300 0x7fc32c14c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0d5300 0x7fc32c14c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49108/0 (socket says 172.21.15.88:49108) 2024-09-21T07:53:38.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 -- 172.21.15.88:0/958779860 learned_addr learned my addr 172.21.15.88:0/958779860 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc323e00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc32c0b7510 0x7fc32c14c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc32be00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc32c0b6320 0x7fc32c09e280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 -- 172.21.15.88:0/958779860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc32c0b6320 msgr2=0x7fc32c09e280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc32c0b6320 0x7fc32c09e280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 -- 172.21.15.88:0/958779860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc32c0b7510 msgr2=0x7fc32c14c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc32c0b7510 0x7fc32c14c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.220+0000 7fc330a00640 1 -- 172.21.15.88:0/958779860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc32c15a9a0 con 0x7fc32c0d5300 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.221+0000 7fc330a00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc32c0d5300 0x7fc32c14c910 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7fc324033040 tx=0x7fc324003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.221+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324004040 con 0x7fc32c0d5300 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.221+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc32402d070 con 0x7fc32c0d5300 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.221+0000 7fc332a00640 1 -- 172.21.15.88:0/958779860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc32c15ac10 con 0x7fc32c0d5300 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.221+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc324004460 con 0x7fc32c0d5300 2024-09-21T07:53:38.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.221+0000 7fc332a00640 1 -- 172.21.15.88:0/958779860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc32c1f6d30 con 0x7fc32c0d5300 2024-09-21T07:53:38.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.222+0000 7fc332a00640 1 -- 172.21.15.88:0/958779860 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc32c08cd50 con 0x7fc32c0d5300 2024-09-21T07:53:38.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.224+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc324004600 con 0x7fc32c0d5300 2024-09-21T07:53:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.225+0000 7fc328c00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc3080791f0 0x7fc30807b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.225+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7fc3240bceb0 con 0x7fc32c0d5300 2024-09-21T07:53:38.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.225+0000 7fc32be00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc3080791f0 0x7fc30807b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.226+0000 7fc32be00640 1 --2- 172.21.15.88:0/958779860 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc3080791f0 0x7fc30807b700 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fc31c009b30 tx=0x7fc31c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:38.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.230+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc32c08cd50 con 0x7fc32c0d5300 2024-09-21T07:53:38.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.453+0000 7fc332a00640 1 -- 172.21.15.88:0/958779860 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot create", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc32c160130 con 0x7fc3080791f0 2024-09-21T07:53:38.457 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:38.455+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:38.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.455+0000 7fc328c00640 1 -- 172.21.15.88:0/958779860 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc31c011cf0 con 0x7fc3080791f0 2024-09-21T07:53:38.457 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:38.476 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:38.476 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot rm non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 -- 172.21.15.88:0/3998524176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300b3ec0 msgr2=0x7efe300b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 --2- 172.21.15.88:0/3998524176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300b3ec0 0x7efe300b47c0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7efe300d0050 tx=0x7efe2002a560 comp rx=0 tx=0).stop 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 -- 172.21.15.88:0/3998524176 shutdown_connections 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 --2- 172.21.15.88:0/3998524176 >> v2:172.21.15.175:3300/0 conn(0x7efe300af3c0 0x7efe301602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 --2- 172.21.15.88:0/3998524176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300b3ec0 0x7efe300b47c0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 -- 172.21.15.88:0/3998524176 >> 172.21.15.88:0/3998524176 conn(0x7efe300b1dd0 msgr2=0x7efe300b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 -- 172.21.15.88:0/3998524176 shutdown_connections 2024-09-21T07:53:38.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.698+0000 7efe35e00640 1 -- 172.21.15.88:0/3998524176 wait complete. 2024-09-21T07:53:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.701+0000 7efe35e00640 1 Processor -- start 2024-09-21T07:53:38.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.701+0000 7efe35e00640 1 -- start start 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.702+0000 7efe35e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe300705b0 0x7efe30157380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.702+0000 7efe35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300af3c0 0x7efe30145a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.702+0000 7efe35e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe300b3ec0 0x7efe3015b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.702+0000 7efe35e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe30160b80 con 0x7efe300af3c0 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.702+0000 7efe35e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe30160a00 con 0x7efe300705b0 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.702+0000 7efe35e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe3015cc60 con 0x7efe300b3ec0 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300af3c0 0x7efe30145a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe34a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe300705b0 0x7efe30157380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300af3c0 0x7efe30145a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49134/0 (socket says 172.21.15.88:49134) 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 -- 172.21.15.88:0/571804391 learned_addr learned my addr 172.21.15.88:0/571804391 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe35400640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe300b3ec0 0x7efe3015b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 -- 172.21.15.88:0/571804391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe300b3ec0 msgr2=0x7efe3015b930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe300b3ec0 0x7efe3015b930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 -- 172.21.15.88:0/571804391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe300705b0 msgr2=0x7efe30157380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe300705b0 0x7efe30157380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 -- 172.21.15.88:0/571804391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301578c0 con 0x7efe300af3c0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.703+0000 7efe27e00640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe300af3c0 0x7efe30145a60 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7efe1800c3a0 tx=0x7efe1800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.704+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18017070 con 0x7efe300af3c0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.704+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe180053c0 con 0x7efe300af3c0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.704+0000 7efe35e00640 1 -- 172.21.15.88:0/571804391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe30157dd0 con 0x7efe300af3c0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.704+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe18003850 con 0x7efe300af3c0 2024-09-21T07:53:38.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.704+0000 7efe35e00640 1 -- 172.21.15.88:0/571804391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe30158390 con 0x7efe300af3c0 2024-09-21T07:53:38.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.705+0000 7efe35e00640 1 -- 172.21.15.88:0/571804391 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30157b40 con 0x7efe300af3c0 2024-09-21T07:53:38.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.706+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7efe1801a070 con 0x7efe300af3c0 2024-09-21T07:53:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.707+0000 7efe2d600640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe0c0791f0 0x7efe0c07b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.708+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7efe1809acc0 con 0x7efe300af3c0 2024-09-21T07:53:38.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.708+0000 7efe34a00640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe0c0791f0 0x7efe0c07b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:38.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.708+0000 7efe34a00640 1 --2- 172.21.15.88:0/571804391 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe0c0791f0 0x7efe0c07b700 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7efe20033000 tx=0x7efe20004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:38.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.712+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efe300b4ec0 con 0x7efe300af3c0 2024-09-21T07:53:38.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.933+0000 7efe35e00640 1 -- 172.21.15.88:0/571804391 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot rm", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7efe301602b0 con 0x7efe0c0791f0 2024-09-21T07:53:38.937 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:38.935+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:38.935+0000 7efe2d600640 1 -- 172.21.15.88:0/571804391 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7efe301602b0 con 0x7efe0c0791f0 2024-09-21T07:53:38.937 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:38.956 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:38.956 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot info non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T07:53:39.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.179+0000 7fc5cb200640 1 -- 172.21.15.88:0/2932479733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0af3c0 msgr2=0x7fc5cc1602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.179+0000 7fc5cb200640 1 --2- 172.21.15.88:0/2932479733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0af3c0 0x7fc5cc1602b0 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7fc5c4005fd0 tx=0x7fc5c402aa10 comp rx=0 tx=0).stop 2024-09-21T07:53:39.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.180+0000 7fc5cb200640 1 -- 172.21.15.88:0/2932479733 shutdown_connections 2024-09-21T07:53:39.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.180+0000 7fc5cb200640 1 --2- 172.21.15.88:0/2932479733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0af3c0 0x7fc5cc1602b0 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.180+0000 7fc5cb200640 1 --2- 172.21.15.88:0/2932479733 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5cc0b3ec0 0x7fc5cc0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.180+0000 7fc5cb200640 1 -- 172.21.15.88:0/2932479733 >> 172.21.15.88:0/2932479733 conn(0x7fc5cc0b1dd0 msgr2=0x7fc5cc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:39.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.180+0000 7fc5cb200640 1 -- 172.21.15.88:0/2932479733 shutdown_connections 2024-09-21T07:53:39.183 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.180+0000 7fc5cb200640 1 -- 172.21.15.88:0/2932479733 wait complete. 2024-09-21T07:53:39.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.182+0000 7fc5cb200640 1 Processor -- start 2024-09-21T07:53:39.184 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.182+0000 7fc5cb200640 1 -- start start 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.183+0000 7fc5cb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0705b0 0x7fc5cc156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5cb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc0af3c0 0x7fc5cc156940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5cb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5cc0b3ec0 0x7fc5cc1e7000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5cb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc160b80 con 0x7fc5cc0705b0 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5cb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc5cc160a00 con 0x7fc5cc0af3c0 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5cb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc5cc15cc60 con 0x7fc5cc0b3ec0 2024-09-21T07:53:39.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0705b0 0x7fc5cc156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0705b0 0x7fc5cc156400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49156/0 (socket says 172.21.15.88:49156) 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 -- 172.21.15.88:0/2040620750 learned_addr learned my addr 172.21.15.88:0/2040620750 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9400640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc0af3c0 0x7fc5cc156940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5ca800640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5cc0b3ec0 0x7fc5cc1e7000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 -- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5cc0b3ec0 msgr2=0x7fc5cc1e7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5cc0b3ec0 0x7fc5cc1e7000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 -- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc0af3c0 msgr2=0x7fc5cc156940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc0af3c0 0x7fc5cc156940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 -- 172.21.15.88:0/2040620750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5cc153f80 con 0x7fc5cc0705b0 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5ca800640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc5cc0b3ec0 0x7fc5cc1e7000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9400640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc5cc0af3c0 0x7fc5cc156940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.184+0000 7fc5c9e00640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5cc0705b0 0x7fc5cc156400 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7fc5bc00d6d0 tx=0x7fc5bc00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:39.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.185+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc017070 con 0x7fc5cc0705b0 2024-09-21T07:53:39.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.185+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5bc005330 con 0x7fc5cc0705b0 2024-09-21T07:53:39.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.185+0000 7fc5cb200640 1 -- 172.21.15.88:0/2040620750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5cc155920 con 0x7fc5cc0705b0 2024-09-21T07:53:39.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.185+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5bc009820 con 0x7fc5cc0705b0 2024-09-21T07:53:39.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.185+0000 7fc5cb200640 1 -- 172.21.15.88:0/2040620750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5cc155ee0 con 0x7fc5cc0705b0 2024-09-21T07:53:39.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.186+0000 7fc5cb200640 1 -- 172.21.15.88:0/2040620750 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5cc1602b0 con 0x7fc5cc0705b0 2024-09-21T07:53:39.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.187+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc5bc009bd0 con 0x7fc5cc0705b0 2024-09-21T07:53:39.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.188+0000 7fc5c2000640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5ac0791c0 0x7fc5ac07b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.188+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7fc5bc09bd70 con 0x7fc5cc0705b0 2024-09-21T07:53:39.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.188+0000 7fc5c9400640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5ac0791c0 0x7fc5ac07b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.189+0000 7fc5c9400640 1 --2- 172.21.15.88:0/2040620750 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5ac0791c0 0x7fc5ac07b6d0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc5cc051010 tx=0x7fc5b8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:39.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.193+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5bc0636c0 con 0x7fc5cc0705b0 2024-09-21T07:53:39.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.416+0000 7fc5cb200640 1 -- 172.21.15.88:0/2040620750 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot info", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc5cc154260 con 0x7fc5ac0791c0 2024-09-21T07:53:39.420 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:39.418+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:39.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.418+0000 7fc5c2000640 1 -- 172.21.15.88:0/2040620750 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc5b8001bf0 con 0x7fc5ac0791c0 2024-09-21T07:53:39.420 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:39.439 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:39.439 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot protect non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.662+0000 7fc60b800640 1 -- 172.21.15.88:0/4196457770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc60406cda0 msgr2=0x7fc60406d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.662+0000 7fc60b800640 1 --2- 172.21.15.88:0/4196457770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc60406cda0 0x7fc60406d1b0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec005fd0 tx=0x7fc5ec02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.663+0000 7fc60b800640 1 -- 172.21.15.88:0/4196457770 shutdown_connections 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.663+0000 7fc60b800640 1 --2- 172.21.15.88:0/4196457770 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc60406cda0 0x7fc60406d1b0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.663+0000 7fc60b800640 1 --2- 172.21.15.88:0/4196457770 >> v2:172.21.15.175:3300/0 conn(0x7fc6040cc830 0x7fc6040b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.663+0000 7fc60b800640 1 -- 172.21.15.88:0/4196457770 >> 172.21.15.88:0/4196457770 conn(0x7fc6040b2540 msgr2=0x7fc6040d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.663+0000 7fc60b800640 1 -- 172.21.15.88:0/4196457770 shutdown_connections 2024-09-21T07:53:39.665 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.663+0000 7fc60b800640 1 -- 172.21.15.88:0/4196457770 wait complete. 2024-09-21T07:53:39.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.665+0000 7fc60b800640 1 Processor -- start 2024-09-21T07:53:39.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.665+0000 7fc60b800640 1 -- start start 2024-09-21T07:53:39.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.666+0000 7fc60b800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc60406cda0 0x7fc60409a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc60b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6040af100 0x7fc604156340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc60b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6040cc830 0x7fc60415f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc60b800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6040b3670 con 0x7fc60406cda0 2024-09-21T07:53:39.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc60b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6040b34f0 con 0x7fc6040af100 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc60b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6040b37f0 con 0x7fc6040cc830 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc60406cda0 0x7fc60409a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc60406cda0 0x7fc60409a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49158/0 (socket says 172.21.15.88:49158) 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 -- 172.21.15.88:0/95822463 learned_addr learned my addr 172.21.15.88:0/95822463 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc609800640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6040cc830 0x7fc60415f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc5fbe00640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6040af100 0x7fc604156340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 -- 172.21.15.88:0/95822463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6040cc830 msgr2=0x7fc60415f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6040cc830 0x7fc60415f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 -- 172.21.15.88:0/95822463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6040af100 msgr2=0x7fc604156340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc6040af100 0x7fc604156340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 -- 172.21.15.88:0/95822463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6040ca5b0 con 0x7fc60406cda0 2024-09-21T07:53:39.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.667+0000 7fc608e00640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc60406cda0 0x7fc60409a420 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7fc5f400e7c0 tx=0x7fc5f400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:39.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.668+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f400ccc0 con 0x7fc60406cda0 2024-09-21T07:53:39.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.668+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5f400ce80 con 0x7fc60406cda0 2024-09-21T07:53:39.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.668+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f4005280 con 0x7fc60406cda0 2024-09-21T07:53:39.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.668+0000 7fc60b800640 1 -- 172.21.15.88:0/95822463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6041f3de0 con 0x7fc60406cda0 2024-09-21T07:53:39.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.668+0000 7fc60b800640 1 -- 172.21.15.88:0/95822463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6041f3af0 con 0x7fc60406cda0 2024-09-21T07:53:39.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.669+0000 7fc60b800640 1 -- 172.21.15.88:0/95822463 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6040b3d10 con 0x7fc60406cda0 2024-09-21T07:53:39.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.670+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fc5f4005420 con 0x7fc60406cda0 2024-09-21T07:53:39.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.671+0000 7fc601600640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e00791f0 0x7fc5e007b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:39.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.671+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7fc5f409c2f0 con 0x7fc60406cda0 2024-09-21T07:53:39.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.671+0000 7fc5fbe00640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e00791f0 0x7fc5e007b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:39.674 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.672+0000 7fc5fbe00640 1 --2- 172.21.15.88:0/95822463 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e00791f0 0x7fc5e007b700 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec033000 tx=0x7fc5ec004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:39.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.676+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5f4063da0 con 0x7fc60406cda0 2024-09-21T07:53:39.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.898+0000 7fc60b800640 1 -- 172.21.15.88:0/95822463 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot protect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fc6040d1030 con 0x7fc5e00791f0 2024-09-21T07:53:39.901 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:39.899+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:39.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:39.900+0000 7fc601600640 1 -- 172.21.15.88:0/95822463 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fc6040d1030 con 0x7fc5e00791f0 2024-09-21T07:53:39.902 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:39.921 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:39.921 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot unprotect non_existent_subvolume subvolume_1 snapshot_1 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 -- 172.21.15.88:0/2588160935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84806cda0 msgr2=0x7fb84806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 --2- 172.21.15.88:0/2588160935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84806cda0 0x7fb84806d1b0 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7fb84806d8a0 tx=0x7fb83002a580 comp rx=0 tx=0).stop 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 -- 172.21.15.88:0/2588160935 shutdown_connections 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 --2- 172.21.15.88:0/2588160935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8480af100 0x7fb8480af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 --2- 172.21.15.88:0/2588160935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb84806cda0 0x7fb84806d1b0 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 --2- 172.21.15.88:0/2588160935 >> v2:172.21.15.175:3300/0 conn(0x7fb8480cc830 0x7fb8480b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 -- 172.21.15.88:0/2588160935 >> 172.21.15.88:0/2588160935 conn(0x7fb8480b2540 msgr2=0x7fb8480d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 -- 172.21.15.88:0/2588160935 shutdown_connections 2024-09-21T07:53:40.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.144+0000 7fb84d200640 1 -- 172.21.15.88:0/2588160935 wait complete. 2024-09-21T07:53:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.146+0000 7fb84d200640 1 Processor -- start 2024-09-21T07:53:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.146+0000 7fb84d200640 1 -- start start 2024-09-21T07:53:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.147+0000 7fb84d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb84806cda0 0x7fb8481f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.147+0000 7fb84d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8480af100 0x7fb8481f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb84d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8480cc830 0x7fb8480ae150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb84d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b34b0 con 0x7fb8480cc830 2024-09-21T07:53:40.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb84d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb8480b3330 con 0x7fb84806cda0 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb84d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb8480b3630 con 0x7fb8480af100 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8480cc830 0x7fb8480ae150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8480cc830 0x7fb8480ae150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49176/0 (socket says 172.21.15.88:49176) 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 -- 172.21.15.88:0/1871778763 learned_addr learned my addr 172.21.15.88:0/1871778763 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb845e00640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb84806cda0 0x7fb8481f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 -- 172.21.15.88:0/1871778763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8480af100 msgr2=0x7fb8481f5a10 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8480af100 0x7fb8481f5a10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 -- 172.21.15.88:0/1871778763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb84806cda0 msgr2=0x7fb8481f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb84806cda0 0x7fb8481f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.148+0000 7fb846800640 1 -- 172.21.15.88:0/1871778763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8481f6700 con 0x7fb8480cc830 2024-09-21T07:53:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.149+0000 7fb846800640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8480cc830 0x7fb8480ae150 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7fb84000cab0 tx=0x7fb84000cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.149+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb840007df0 con 0x7fb8480cc830 2024-09-21T07:53:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.149+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb84000f560 con 0x7fb8480cc830 2024-09-21T07:53:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.149+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb84000f9b0 con 0x7fb8480cc830 2024-09-21T07:53:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.149+0000 7fb84d200640 1 -- 172.21.15.88:0/1871778763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb848155390 con 0x7fb8480cc830 2024-09-21T07:53:40.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.149+0000 7fb84d200640 1 -- 172.21.15.88:0/1871778763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8481f9e60 con 0x7fb8480cc830 2024-09-21T07:53:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.150+0000 7fb84d200640 1 -- 172.21.15.88:0/1871778763 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb8481f6470 con 0x7fb8480cc830 2024-09-21T07:53:40.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.151+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7fb84000fb50 con 0x7fb8480cc830 2024-09-21T07:53:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.152+0000 7fb83ea00640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8240791f0 0x7fb82407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.152+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7fb84009c450 con 0x7fb8480cc830 2024-09-21T07:53:40.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.153+0000 7fb845e00640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8240791f0 0x7fb82407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.153+0000 7fb845e00640 1 --2- 172.21.15.88:0/1871778763 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8240791f0 0x7fb82407b700 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fb838005d70 tx=0x7fb838005d00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:40.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.157+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb840014070 con 0x7fb8480cc830 2024-09-21T07:53:40.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.378+0000 7fb84d200640 1 -- 172.21.15.88:0/1871778763 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot unprotect", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7fb8480af510 con 0x7fb8240791f0 2024-09-21T07:53:40.381 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:40.379+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:40.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.379+0000 7fb83ea00640 1 -- 172.21.15.88:0/1871778763 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7fb8480af510 con 0x7fb8240791f0 2024-09-21T07:53:40.381 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:40.401 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:40.401 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot ls non_existent_subvolume subvolume_1 2024-09-21T07:53:40.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.623+0000 7f7e5be00640 1 -- 172.21.15.88:0/2568145743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0af3c0 msgr2=0x7f7e5c1602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.623+0000 7f7e5be00640 1 --2- 172.21.15.88:0/2568145743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0af3c0 0x7f7e5c1602b0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c06ddf0 tx=0x7f7e5402aa30 comp rx=0 tx=0).stop 2024-09-21T07:53:40.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.624+0000 7f7e5be00640 1 -- 172.21.15.88:0/2568145743 shutdown_connections 2024-09-21T07:53:40.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.624+0000 7f7e5be00640 1 --2- 172.21.15.88:0/2568145743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0af3c0 0x7f7e5c1602b0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.624+0000 7f7e5be00640 1 --2- 172.21.15.88:0/2568145743 >> v2:172.21.15.175:3300/0 conn(0x7f7e5c0b3ec0 0x7f7e5c0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.624+0000 7f7e5be00640 1 -- 172.21.15.88:0/2568145743 >> 172.21.15.88:0/2568145743 conn(0x7f7e5c0b1dd0 msgr2=0x7f7e5c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:40.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.624+0000 7f7e5be00640 1 -- 172.21.15.88:0/2568145743 shutdown_connections 2024-09-21T07:53:40.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.624+0000 7f7e5be00640 1 -- 172.21.15.88:0/2568145743 wait complete. 2024-09-21T07:53:40.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.626+0000 7f7e5be00640 1 Processor -- start 2024-09-21T07:53:40.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.626+0000 7f7e5be00640 1 -- start start 2024-09-21T07:53:40.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.627+0000 7f7e5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0705b0 0x7f7e5c1f4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.627+0000 7f7e5be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e5c0af3c0 0x7f7e5c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.627+0000 7f7e5be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5c0b3ec0 0x7f7e5c09af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c160b80 con 0x7f7e5c0705b0 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7e5c160a00 con 0x7f7e5c0b3ec0 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7e5c09b7e0 con 0x7f7e5c0af3c0 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0705b0 0x7f7e5c1f4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0705b0 0x7f7e5c1f4230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49198/0 (socket says 172.21.15.88:49198) 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 -- 172.21.15.88:0/1548695360 learned_addr learned my addr 172.21.15.88:0/1548695360 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5b400640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5c0b3ec0 0x7f7e5c09af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e52000640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e5c0af3c0 0x7f7e5c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 -- 172.21.15.88:0/1548695360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e5c0af3c0 msgr2=0x7f7e5c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7e5c0af3c0 0x7f7e5c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 -- 172.21.15.88:0/1548695360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5c0b3ec0 msgr2=0x7f7e5c09af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:40.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7e5c0b3ec0 0x7f7e5c09af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.628+0000 7f7e5aa00640 1 -- 172.21.15.88:0/1548695360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7e5c09d090 con 0x7f7e5c0705b0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.629+0000 7f7e5aa00640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7e5c0705b0 0x7f7e5c1f4230 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7f7e4c00d770 tx=0x7f7e4c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.629+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c017070 con 0x7f7e5c0705b0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.629+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7e4c009d20 con 0x7f7e5c0705b0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.629+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e4c0106c0 con 0x7f7e5c0705b0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.629+0000 7f7e5be00640 1 -- 172.21.15.88:0/1548695360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7e5c09d370 con 0x7f7e5c0705b0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.629+0000 7f7e5be00640 1 -- 172.21.15.88:0/1548695360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7e5c1f3d10 con 0x7f7e5c0705b0 2024-09-21T07:53:40.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.630+0000 7f7e5be00640 1 -- 172.21.15.88:0/1548695360 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7e5c0717c0 con 0x7f7e5c0705b0 2024-09-21T07:53:40.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.631+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f7e4c01a070 con 0x7f7e5c0705b0 2024-09-21T07:53:40.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.632+0000 7f7e53400640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7e380791c0 0x7f7e3807b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:40.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.632+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f7e4c09bbb0 con 0x7f7e5c0705b0 2024-09-21T07:53:40.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.633+0000 7f7e52000640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7e380791c0 0x7f7e3807b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:40.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.633+0000 7f7e52000640 1 --2- 172.21.15.88:0/1548695360 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7e380791c0 0x7f7e3807b6d0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f7e5c0b33a0 tx=0x7f7e44009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:40.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.637+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7e5c0717c0 con 0x7f7e5c0705b0 2024-09-21T07:53:40.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.855+0000 7f7e5be00640 1 -- 172.21.15.88:0/1548695360 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot ls", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "target": ["mon-mgr", ""]}) -- 0x7f7e5c153c20 con 0x7f7e380791c0 2024-09-21T07:53:40.859 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:40.857+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:40.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:40.857+0000 7f7e53400640 1 -- 172.21.15.88:0/1548695360 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f7e44001cd0 con 0x7f7e380791c0 2024-09-21T07:53:40.859 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:40.875 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:40.876 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume snapshot clone non_existent_subvolume subvolume_1 snapshot_1 clone_1 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 -- 172.21.15.88:0/3909050104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240d16f0 msgr2=0x7f85240b9d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 --2- 172.21.15.88:0/3909050104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240d16f0 0x7f85240b9d40 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8514009be0 tx=0x7f851402a8b0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 -- 172.21.15.88:0/3909050104 shutdown_connections 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 --2- 172.21.15.88:0/3909050104 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240d16f0 0x7f85240b9d40 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 -- 172.21.15.88:0/3909050104 >> 172.21.15.88:0/3909050104 conn(0x7f852409d820 msgr2=0x7f852408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 -- 172.21.15.88:0/3909050104 shutdown_connections 2024-09-21T07:53:41.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.102+0000 7f8528a00640 1 -- 172.21.15.88:0/3909050104 wait complete. 2024-09-21T07:53:41.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.105+0000 7f8528a00640 1 Processor -- start 2024-09-21T07:53:41.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.105+0000 7f8528a00640 1 -- start start 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8528a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240b3c40 0x7f8524159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8528a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85240d16f0 0x7f85240b1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8528a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f852415ac30 0x7f85240b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8528a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85240cfd90 con 0x7f852415ac30 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8528a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f85240cfc10 con 0x7f85240d16f0 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8528a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f85240cff10 con 0x7f85240b3c40 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8523200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f852415ac30 0x7f85240b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8523200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f852415ac30 0x7f85240b2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49214/0 (socket says 172.21.15.88:49214) 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.106+0000 7f8523200640 1 -- 172.21.15.88:0/621818903 learned_addr learned my addr 172.21.15.88:0/621818903 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8521e00640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85240d16f0 0x7f85240b1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8522800640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240b3c40 0x7f8524159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8523200640 1 -- 172.21.15.88:0/621818903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240b3c40 msgr2=0x7f8524159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8523200640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f85240b3c40 0x7f8524159ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8523200640 1 -- 172.21.15.88:0/621818903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85240d16f0 msgr2=0x7f85240b1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8523200640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f85240d16f0 0x7f85240b1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8523200640 1 -- 172.21.15.88:0/621818903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85241f9910 con 0x7f852415ac30 2024-09-21T07:53:41.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8523200640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f852415ac30 0x7f85240b2270 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7f851c00c3c0 tx=0x7f851c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c017070 con 0x7f852415ac30 2024-09-21T07:53:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f851c0053c0 con 0x7f852415ac30 2024-09-21T07:53:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f851c003850 con 0x7f852415ac30 2024-09-21T07:53:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.107+0000 7f8528a00640 1 -- 172.21.15.88:0/621818903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85241f9e70 con 0x7f852415ac30 2024-09-21T07:53:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.108+0000 7f8528a00640 1 -- 172.21.15.88:0/621818903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85241f9b90 con 0x7f852415ac30 2024-09-21T07:53:41.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.109+0000 7f8528a00640 1 -- 172.21.15.88:0/621818903 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f852415a8e0 con 0x7f852415ac30 2024-09-21T07:53:41.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.110+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f851c01a070 con 0x7f852415ac30 2024-09-21T07:53:41.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.111+0000 7f851aa00640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f85040791c0 0x7f850407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.111+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f851c09adb0 con 0x7f852415ac30 2024-09-21T07:53:41.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.111+0000 7f8522800640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f85040791c0 0x7f850407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.114 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.112+0000 7f8522800640 1 --2- 172.21.15.88:0/621818903 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f85040791c0 0x7f850407b6d0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f8514034000 tx=0x7f8514004ab0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:41.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.115+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f85241f9910 con 0x7f852415ac30 2024-09-21T07:53:41.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.333+0000 7f8528a00640 1 -- 172.21.15.88:0/621818903 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume snapshot clone", "vol_name": "non_existent_subvolume", "sub_name": "subvolume_1", "snap_name": "snapshot_1", "target_sub_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f85240d03c0 con 0x7f85040791c0 2024-09-21T07:53:41.342 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.340+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:41.343 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.340+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:41.343 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.340+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:41.343 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.340+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:41.343 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.340+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:53:41.349 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.347+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:41.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.347+0000 7f851aa00640 1 -- 172.21.15.88:0/621818903 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f85240d03c0 con 0x7f85040791c0 2024-09-21T07:53:41.350 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:41.364 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:41.365 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs clone status non_existent_subvolume clone_1 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.584+0000 7f5d34c00640 1 -- 172.21.15.88:0/3936684836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 msgr2=0x7f5d300b42a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.584+0000 7f5d34c00640 1 --2- 172.21.15.88:0/3936684836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 0x7f5d300b42a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f5d20005fd0 tx=0x7f5d2002a540 comp rx=0 tx=0).stop 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.585+0000 7f5d34c00640 1 -- 172.21.15.88:0/3936684836 shutdown_connections 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.585+0000 7f5d34c00640 1 --2- 172.21.15.88:0/3936684836 >> v2:172.21.15.175:3300/0 conn(0x7f5d300cf2c0 0x7f5d300cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.585+0000 7f5d34c00640 1 --2- 172.21.15.88:0/3936684836 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 0x7f5d300b42a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.585+0000 7f5d34c00640 1 -- 172.21.15.88:0/3936684836 >> 172.21.15.88:0/3936684836 conn(0x7f5d3006d620 msgr2=0x7f5d300b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.585+0000 7f5d34c00640 1 -- 172.21.15.88:0/3936684836 shutdown_connections 2024-09-21T07:53:41.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.585+0000 7f5d34c00640 1 -- 172.21.15.88:0/3936684836 wait complete. 2024-09-21T07:53:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.587+0000 7f5d34c00640 1 Processor -- start 2024-09-21T07:53:41.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.587+0000 7f5d34c00640 1 -- start start 2024-09-21T07:53:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d34c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 0x7f5d30152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d34c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d300cf2c0 0x7f5d30145db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d34c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d301624b0 0x7f5d300ae080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d34c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5d300730c0 con 0x7f5d300cf2c0 2024-09-21T07:53:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d34c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5d30162a00 con 0x7f5d301624b0 2024-09-21T07:53:41.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d34c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5d30073240 con 0x7f5d300742b0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d300cf2c0 0x7f5d30145db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d300cf2c0 0x7f5d30145db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49232/0 (socket says 172.21.15.88:49232) 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 -- 172.21.15.88:0/3332898296 learned_addr learned my addr 172.21.15.88:0/3332898296 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2f400640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 0x7f5d30152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2fe00640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d301624b0 0x7f5d300ae080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 -- 172.21.15.88:0/3332898296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 msgr2=0x7f5d30152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5d300742b0 0x7f5d30152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 -- 172.21.15.88:0/3332898296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d301624b0 msgr2=0x7f5d300ae080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5d301624b0 0x7f5d300ae080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.589+0000 7f5d2ea00640 1 -- 172.21.15.88:0/3332898296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d300b1360 con 0x7f5d300cf2c0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.590+0000 7f5d2ea00640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d300cf2c0 0x7f5d30145db0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f5d1c00c3a0 tx=0x7f5d1c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.590+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c017070 con 0x7f5d300cf2c0 2024-09-21T07:53:41.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.590+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5d1c0053c0 con 0x7f5d300cf2c0 2024-09-21T07:53:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.590+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d1c003850 con 0x7f5d300cf2c0 2024-09-21T07:53:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.590+0000 7f5d34c00640 1 -- 172.21.15.88:0/3332898296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d30140490 con 0x7f5d300cf2c0 2024-09-21T07:53:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.590+0000 7f5d34c00640 1 -- 172.21.15.88:0/3332898296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d300b1600 con 0x7f5d300cf2c0 2024-09-21T07:53:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.591+0000 7f5d34c00640 1 -- 172.21.15.88:0/3332898296 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5d30073760 con 0x7f5d300cf2c0 2024-09-21T07:53:41.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.592+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f5d1c01a070 con 0x7f5d300cf2c0 2024-09-21T07:53:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.593+0000 7f5d27400640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5d100791c0 0x7f5d1007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:41.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.593+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f5d1c09adf0 con 0x7f5d300cf2c0 2024-09-21T07:53:41.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.594+0000 7f5d2f400640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5d100791c0 0x7f5d1007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:41.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.594+0000 7f5d2f400640 1 --2- 172.21.15.88:0/3332898296 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5d100791c0 0x7f5d1007b6d0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f5d20033000 tx=0x7f5d20004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:41.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.598+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5d300b1360 con 0x7f5d300cf2c0 2024-09-21T07:53:41.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.821+0000 7f5d34c00640 1 -- 172.21.15.88:0/3332898296 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs clone status", "vol_name": "non_existent_subvolume", "clone_name": "clone_1", "target": ["mon-mgr", ""]}) -- 0x7f5d30156320 con 0x7f5d100791c0 2024-09-21T07:53:41.825 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:41.823+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:41.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:41.823+0000 7f5d27400640 1 -- 172.21.15.88:0/3332898296 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f5d30156320 con 0x7f5d100791c0 2024-09-21T07:53:41.825 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:41.841 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:41.841 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create non_existent_subvolume group_1 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 -- 172.21.15.88:0/1081039213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145550 msgr2=0x7f927006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 --2- 172.21.15.88:0/1081039213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145550 0x7f927006fb40 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7f92701464e0 tx=0x7f926002a6a0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 -- 172.21.15.88:0/1081039213 shutdown_connections 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 --2- 172.21.15.88:0/1081039213 >> v2:172.21.15.175:3300/0 conn(0x7f9270073c50 0x7f92700cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 --2- 172.21.15.88:0/1081039213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145550 0x7f927006fb40 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 -- 172.21.15.88:0/1081039213 >> 172.21.15.88:0/1081039213 conn(0x7f927006d1c0 msgr2=0x7f927006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 -- 172.21.15.88:0/1081039213 shutdown_connections 2024-09-21T07:53:42.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.062+0000 7f9276400640 1 -- 172.21.15.88:0/1081039213 wait complete. 2024-09-21T07:53:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.065+0000 7f9276400640 1 Processor -- start 2024-09-21T07:53:42.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.065+0000 7f9276400640 1 -- start start 2024-09-21T07:53:42.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f9276400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9270145550 0x7f927015be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f9276400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145a40 0x7f92701f94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f9276400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92701f9a00 0x7f92701f81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f9276400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f92700ce150 con 0x7f9270145a40 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f9276400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92700cdfd0 con 0x7f9270145550 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f9276400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92700ce2d0 con 0x7f92701f9a00 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.066+0000 7f926ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145a40 0x7f92701f94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145a40 0x7f92701f94c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49246/0 (socket says 172.21.15.88:49246) 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 -- 172.21.15.88:0/2862852349 learned_addr learned my addr 172.21.15.88:0/2862852349 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926f400640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9270145550 0x7f927015be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926fe00640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92701f9a00 0x7f92701f81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 -- 172.21.15.88:0/2862852349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92701f9a00 msgr2=0x7f92701f81d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92701f9a00 0x7f92701f81d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 -- 172.21.15.88:0/2862852349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9270145550 msgr2=0x7f927015be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9270145550 0x7f927015be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 -- 172.21.15.88:0/2862852349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92701f9e10 con 0x7f9270145a40 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f926ea00640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9270145a40 0x7f92701f94c0 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7f925c00c3c0 tx=0x7f925c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.067+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c017070 con 0x7f9270145a40 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.068+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f925c0035f0 con 0x7f9270145a40 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.068+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f925c003a20 con 0x7f9270145a40 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.068+0000 7f9276400640 1 -- 172.21.15.88:0/2862852349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92701f9250 con 0x7f9270145a40 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.068+0000 7f9276400640 1 -- 172.21.15.88:0/2862852349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92701f7410 con 0x7f9270145a40 2024-09-21T07:53:42.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.069+0000 7f9276400640 1 -- 172.21.15.88:0/2862852349 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92700ce870 con 0x7f9270145a40 2024-09-21T07:53:42.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.070+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f925c01a070 con 0x7f9270145a40 2024-09-21T07:53:42.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.071+0000 7f9267400640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f92500791c0 0x7f925007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.072+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f925c09ad00 con 0x7f9270145a40 2024-09-21T07:53:42.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.072+0000 7f926f400640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f92500791c0 0x7f925007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.072+0000 7f926f400640 1 --2- 172.21.15.88:0/2862852349 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f92500791c0 0x7f925007b6d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9260033000 tx=0x7f9260004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:42.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.076+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f92701f9e10 con 0x7f9270145a40 2024-09-21T07:53:42.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.287+0000 7f9276400640 1 -- 172.21.15.88:0/2862852349 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f92701437f0 con 0x7f92500791c0 2024-09-21T07:53:42.290 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:42.288+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:42.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.288+0000 7f9267400640 1 -- 172.21.15.88:0/2862852349 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f92701437f0 con 0x7f92500791c0 2024-09-21T07:53:42.291 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:42.306 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:42.307 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm non_existent_subvolume group_1 2024-09-21T07:53:42.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 -- 172.21.15.88:0/4135924888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440ccc30 msgr2=0x7f65440b4390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 --2- 172.21.15.88:0/4135924888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440ccc30 0x7f65440b4390 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f6538009830 tx=0x7f653802a510 comp rx=0 tx=0).stop 2024-09-21T07:53:42.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 -- 172.21.15.88:0/4135924888 shutdown_connections 2024-09-21T07:53:42.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 --2- 172.21.15.88:0/4135924888 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6544070290 0x7f65440706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 --2- 172.21.15.88:0/4135924888 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440ccc30 0x7f65440b4390 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 -- 172.21.15.88:0/4135924888 >> 172.21.15.88:0/4135924888 conn(0x7f654406d1c0 msgr2=0x7f654406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:42.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 -- 172.21.15.88:0/4135924888 shutdown_connections 2024-09-21T07:53:42.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.528+0000 7f654c200640 1 -- 172.21.15.88:0/4135924888 wait complete. 2024-09-21T07:53:42.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.530+0000 7f654c200640 1 Processor -- start 2024-09-21T07:53:42.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.530+0000 7f654c200640 1 -- start start 2024-09-21T07:53:42.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654c200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6544070290 0x7f654415a6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654c200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65440ccc30 0x7f65441f9cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654c200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440cf7c0 0x7f654414b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654c200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f654415c410 con 0x7f65440ccc30 2024-09-21T07:53:42.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654c200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f654415c290 con 0x7f65440cf7c0 2024-09-21T07:53:42.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654c200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f654415c590 con 0x7f6544070290 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65440ccc30 0x7f65441f9cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65440ccc30 0x7f65441f9cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49252/0 (socket says 172.21.15.88:49252) 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 -- 172.21.15.88:0/510278372 learned_addr learned my addr 172.21.15.88:0/510278372 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654a200640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440cf7c0 0x7f654414b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6549800640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6544070290 0x7f654415a6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 -- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6544070290 msgr2=0x7f654415a6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6544070290 0x7f654415a6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 -- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440cf7c0 msgr2=0x7f654414b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440cf7c0 0x7f654414b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f6548e00640 1 -- 172.21.15.88:0/510278372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65441fa230 con 0x7f65440ccc30 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.532+0000 7f654a200640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f65440cf7c0 0x7f654414b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.533+0000 7f6548e00640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f65440ccc30 0x7f65441f9cf0 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7f653000c3a0 tx=0x7f653000c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.533+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530011790 con 0x7f65440ccc30 2024-09-21T07:53:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.533+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6530017070 con 0x7f65440ccc30 2024-09-21T07:53:42.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.533+0000 7f654c200640 1 -- 172.21.15.88:0/510278372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f654415af10 con 0x7f65440ccc30 2024-09-21T07:53:42.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.533+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6530011cc0 con 0x7f65440ccc30 2024-09-21T07:53:42.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.533+0000 7f654c200640 1 -- 172.21.15.88:0/510278372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65441f94f0 con 0x7f65440ccc30 2024-09-21T07:53:42.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.534+0000 7f654c200640 1 -- 172.21.15.88:0/510278372 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f654415ac80 con 0x7f65440ccc30 2024-09-21T07:53:42.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.535+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f653001c460 con 0x7f65440ccc30 2024-09-21T07:53:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.536+0000 7f6541600640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f65240791c0 0x7f652407b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.536+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f653009b720 con 0x7f65440ccc30 2024-09-21T07:53:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.537+0000 7f6549800640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f65240791c0 0x7f652407b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.537+0000 7f6549800640 1 --2- 172.21.15.88:0/510278372 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f65240791c0 0x7f652407b6d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f6538034000 tx=0x7f6538004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:42.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.541+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6544070be0 con 0x7f65440ccc30 2024-09-21T07:53:42.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.749+0000 7f654c200640 1 -- 172.21.15.88:0/510278372 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f65441f9310 con 0x7f65240791c0 2024-09-21T07:53:42.753 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:42.751+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:42.753 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.751+0000 7f6541600640 1 -- 172.21.15.88:0/510278372 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f65441f9310 con 0x7f65240791c0 2024-09-21T07:53:42.753 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:42.769 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:42.770 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup getpath non_existent_subvolume group_1 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 -- 172.21.15.88:0/1093843764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0073c50 msgr2=0x7f1de01674a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 --2- 172.21.15.88:0/1093843764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0073c50 0x7f1de01674a0 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8005fd0 tx=0x7f1dd802ab20 comp rx=0 tx=0).stop 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 -- 172.21.15.88:0/1093843764 shutdown_connections 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 --2- 172.21.15.88:0/1093843764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0073c50 0x7f1de01674a0 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 -- 172.21.15.88:0/1093843764 >> 172.21.15.88:0/1093843764 conn(0x7f1de006d1c0 msgr2=0x7f1de006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 -- 172.21.15.88:0/1093843764 shutdown_connections 2024-09-21T07:53:42.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.989+0000 7f1de5600640 1 -- 172.21.15.88:0/1093843764 wait complete. 2024-09-21T07:53:42.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.992+0000 7f1de5600640 1 Processor -- start 2024-09-21T07:53:42.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.992+0000 7f1de5600640 1 -- start start 2024-09-21T07:53:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1de5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0145550 0x7f1de00b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1de5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de0145a40 0x7f1de00b1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1de5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de01322a0 0x7f1de01f70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1de5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0167db0 con 0x7f1de0145550 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1de5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1de0167c30 con 0x7f1de01322a0 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1de5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1de0167f30 con 0x7f1de0145a40 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1ddea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0145550 0x7f1de00b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1ddea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0145550 0x7f1de00b2090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49268/0 (socket says 172.21.15.88:49268) 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.993+0000 7f1ddea00640 1 -- 172.21.15.88:0/2517268877 learned_addr learned my addr 172.21.15.88:0/2517268877 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddf400640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de01322a0 0x7f1de01f70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1dde000640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de0145a40 0x7f1de00b1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddea00640 1 -- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de0145a40 msgr2=0x7f1de00b1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddea00640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1de0145a40 0x7f1de00b1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddea00640 1 -- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de01322a0 msgr2=0x7f1de01f70f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:42.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddea00640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de01322a0 0x7f1de01f70f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddea00640 1 -- 172.21.15.88:0/2517268877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de00b25d0 con 0x7f1de0145550 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddf400640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1de01322a0 0x7f1de01f70f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1ddea00640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1de0145550 0x7f1de00b2090 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x7f1dd000b3f0 tx=0x7f1dd000b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0015020 con 0x7f1de0145550 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1dd0011920 con 0x7f1de0145550 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd0011d70 con 0x7f1de0145550 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1de5600640 1 -- 172.21.15.88:0/2517268877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de0132b40 con 0x7f1de0145550 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.994+0000 7f1de5600640 1 -- 172.21.15.88:0/2517268877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de0133100 con 0x7f1de0145550 2024-09-21T07:53:42.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.995+0000 7f1de5600640 1 -- 172.21.15.88:0/2517268877 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de006fb40 con 0x7f1de0145550 2024-09-21T07:53:42.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.997+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f1dd0018020 con 0x7f1de0145550 2024-09-21T07:53:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.998+0000 7f1dd6a00640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1dc00791c0 0x7f1dc007b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.998+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f1dd009b660 con 0x7f1de0145550 2024-09-21T07:53:43.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.998+0000 7f1dde000640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1dc00791c0 0x7f1dc007b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:42.999+0000 7f1dde000640 1 --2- 172.21.15.88:0/2517268877 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1dc00791c0 0x7f1dc007b6d0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f1de0137710 tx=0x7f1dcc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:43.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.003+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1dd001f030 con 0x7f1de0145550 2024-09-21T07:53:43.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.213+0000 7f1de5600640 1 -- 172.21.15.88:0/2517268877 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup getpath", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f1de0138060 con 0x7f1dc00791c0 2024-09-21T07:53:43.217 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:43.215+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:43.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.215+0000 7f1dd6a00640 1 -- 172.21.15.88:0/2517268877 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1de0138060 con 0x7f1dc00791c0 2024-09-21T07:53:43.217 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:43.235 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:43.236 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup pin non_existent_subvolume group_1 export 0 2024-09-21T07:53:43.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 -- 172.21.15.88:0/3026862140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180af3c0 msgr2=0x7f3e181602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:43.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 --2- 172.21.15.88:0/3026862140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180af3c0 0x7f3e181602b0 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7f3e10005fd0 tx=0x7f3e1002aa10 comp rx=0 tx=0).stop 2024-09-21T07:53:43.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 -- 172.21.15.88:0/3026862140 shutdown_connections 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 --2- 172.21.15.88:0/3026862140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180af3c0 0x7f3e181602b0 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 --2- 172.21.15.88:0/3026862140 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e180b3ec0 0x7f3e180b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 -- 172.21.15.88:0/3026862140 >> 172.21.15.88:0/3026862140 conn(0x7f3e180b1dd0 msgr2=0x7f3e180b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 -- 172.21.15.88:0/3026862140 shutdown_connections 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.461+0000 7f3e1ca00640 1 -- 172.21.15.88:0/3026862140 wait complete. 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.464+0000 7f3e1ca00640 1 Processor -- start 2024-09-21T07:53:43.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.464+0000 7f3e1ca00640 1 -- start start 2024-09-21T07:53:43.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180705b0 0x7f3e181f5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e180af3c0 0x7f3e18155370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e1ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e180b3ec0 0x7f3e181558b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e1ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18160b80 con 0x7f3e180705b0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e1ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e18160a00 con 0x7f3e180af3c0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e1ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3e1815cc60 con 0x7f3e180b3ec0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.465+0000 7f3e17400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180705b0 0x7f3e181f5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180705b0 0x7f3e181f5040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49276/0 (socket says 172.21.15.88:49276) 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 -- 172.21.15.88:0/478865721 learned_addr learned my addr 172.21.15.88:0/478865721 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17e00640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e180b3ec0 0x7f3e181558b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e0ea00640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e180af3c0 0x7f3e18155370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 -- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e180b3ec0 msgr2=0x7f3e181558b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e180b3ec0 0x7f3e181558b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 -- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e180af3c0 msgr2=0x7f3e18155370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e180af3c0 0x7f3e18155370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 -- 172.21.15.88:0/478865721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e1809d990 con 0x7f3e180705b0 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17e00640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e180b3ec0 0x7f3e181558b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:43.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.466+0000 7f3e17400640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e180705b0 0x7f3e181f5040 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7f3e0800d740 tx=0x7f3e0800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.467+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e08017070 con 0x7f3e180705b0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.467+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3e08009d20 con 0x7f3e180705b0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.467+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3e080106c0 con 0x7f3e180705b0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.467+0000 7f3e1ca00640 1 -- 172.21.15.88:0/478865721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e1809d3d0 con 0x7f3e180705b0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.467+0000 7f3e1ca00640 1 -- 172.21.15.88:0/478865721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e18154b70 con 0x7f3e180705b0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.468+0000 7f3e1ca00640 1 -- 172.21.15.88:0/478865721 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e181602b0 con 0x7f3e180705b0 2024-09-21T07:53:43.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.469+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 102002+0+0 (secure 0 0 0) 0x7f3e0801a070 con 0x7f3e180705b0 2024-09-21T07:53:43.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.470+0000 7f3e0fe00640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3df40791f0 0x7f3df407b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.470+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f3e0809ba90 con 0x7f3e180705b0 2024-09-21T07:53:43.472 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.470+0000 7f3e0ea00640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3df40791f0 0x7f3df407b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.471+0000 7f3e0ea00640 1 --2- 172.21.15.88:0/478865721 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3df40791f0 0x7f3df407b700 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f3e1815c550 tx=0x7f3e00009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:43.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.475+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3e08063500 con 0x7f3e180705b0 2024-09-21T07:53:43.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.689+0000 7f3e1ca00640 1 -- 172.21.15.88:0/478865721 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "non_existent_subvolume", "group_name": "group_1", "pin_type": "export", "pin_setting": "0", "target": ["mon-mgr", ""]}) -- 0x7f3e1815c2a0 con 0x7f3df40791f0 2024-09-21T07:53:43.693 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:43.691+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:43.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.691+0000 7f3e0fe00640 1 -- 172.21.15.88:0/478865721 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f3e1815c2a0 con 0x7f3df40791f0 2024-09-21T07:53:43.693 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:43.712 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:43.713 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup ls non_existent_subvolume 2024-09-21T07:53:43.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.936+0000 7f223e800640 1 -- 172.21.15.88:0/3416019744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 msgr2=0x7f22380af1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:43.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.936+0000 7f223e800640 1 --2- 172.21.15.88:0/3416019744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 0x7f22380af1f0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f2238074db0 tx=0x7f222802a420 comp rx=0 tx=0).stop 2024-09-21T07:53:43.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 -- 172.21.15.88:0/3416019744 shutdown_connections 2024-09-21T07:53:43.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 --2- 172.21.15.88:0/3416019744 >> v2:172.21.15.175:3300/0 conn(0x7f22380af730 0x7f22380d0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 --2- 172.21.15.88:0/3416019744 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22380b7d70 0x7f22380d0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 --2- 172.21.15.88:0/3416019744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 0x7f22380af1f0 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 -- 172.21.15.88:0/3416019744 >> 172.21.15.88:0/3416019744 conn(0x7f22380cef20 msgr2=0x7f22380cf330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:43.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 -- 172.21.15.88:0/3416019744 shutdown_connections 2024-09-21T07:53:43.940 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.937+0000 7f223e800640 1 -- 172.21.15.88:0/3416019744 wait complete. 2024-09-21T07:53:43.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.939+0000 7f223e800640 1 Processor -- start 2024-09-21T07:53:43.941 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.939+0000 7f223e800640 1 -- start start 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.940+0000 7f223e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22380af730 0x7f2238153ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.940+0000 7f223e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 0x7f2238153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f223e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22380b7d70 0x7f223815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f223e800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f22380ceb90 con 0x7f22380b7960 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f223e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f22380b8310 con 0x7f22380af730 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f223e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f22380ced10 con 0x7f22380b7d70 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 0x7f2238153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 0x7f2238153290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49294/0 (socket says 172.21.15.88:49294) 2024-09-21T07:53:43.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 -- 172.21.15.88:0/205870492 learned_addr learned my addr 172.21.15.88:0/205870492 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f2237400640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22380af730 0x7f2238153ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f2237e00640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22380b7d70 0x7f223815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 -- 172.21.15.88:0/205870492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22380b7d70 msgr2=0x7f223815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f22380b7d70 0x7f223815fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 -- 172.21.15.88:0/205870492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22380af730 msgr2=0x7f2238153ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f22380af730 0x7f2238153ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 -- 172.21.15.88:0/205870492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22381547c0 con 0x7f22380b7960 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.941+0000 7f222fe00640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f22380b7960 0x7f2238153290 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f222000d650 tx=0x7f222000db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.942+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220017070 con 0x7f22380b7960 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.942+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2220005310 con 0x7f22380b7960 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.942+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2220009820 con 0x7f22380b7960 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.942+0000 7f223e800640 1 -- 172.21.15.88:0/205870492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2238155e20 con 0x7f22380b7960 2024-09-21T07:53:43.945 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.942+0000 7f223e800640 1 -- 172.21.15.88:0/205870492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22381563e0 con 0x7f22380b7960 2024-09-21T07:53:43.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.943+0000 7f223e800640 1 -- 172.21.15.88:0/205870492 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22380b31b0 con 0x7f22380b7960 2024-09-21T07:53:43.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.944+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f22200099c0 con 0x7f22380b7960 2024-09-21T07:53:43.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.945+0000 7f222f400640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2214079250 0x7f221407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:43.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.945+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f222009bae0 con 0x7f22380b7960 2024-09-21T07:53:43.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.945+0000 7f2237400640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2214079250 0x7f221407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:43.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.946+0000 7f2237400640 1 --2- 172.21.15.88:0/205870492 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2214079250 0x7f221407b760 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2228034000 tx=0x7f2228004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:43.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:43.950+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2220062d90 con 0x7f22380b7960 2024-09-21T07:53:44.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.162+0000 7f223e800640 1 -- 172.21.15.88:0/205870492 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup ls", "vol_name": "non_existent_subvolume", "target": ["mon-mgr", ""]}) -- 0x7f22380b2ed0 con 0x7f2214079250 2024-09-21T07:53:44.165 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:44.164+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:44.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.164+0000 7f222f400640 1 -- 172.21.15.88:0/205870492 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f22380b2ed0 con 0x7f2214079250 2024-09-21T07:53:44.166 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:44.183 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:44.183 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot create non_existent_subvolume group_1 snapshot_1 2024-09-21T07:53:44.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2814200640 1 -- 172.21.15.88:0/1324072546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4003a00 msgr2=0x7f27f4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:44.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2814200640 1 --2- 172.21.15.88:0/1324072546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f4003a00 0x7f27f4003e70 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f2800009b50 tx=0x7f27f4005830 comp rx=0 tx=0).stop 2024-09-21T07:53:44.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2812200640 1 -- 172.21.15.88:0/1324072546 reap_dead start 2024-09-21T07:53:44.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2814200640 1 -- 172.21.15.88:0/1324072546 shutdown_connections 2024-09-21T07:53:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2814200640 1 -- 172.21.15.88:0/1324072546 >> 172.21.15.88:0/1324072546 conn(0x7f280c0b2540 msgr2=0x7f280c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2814200640 1 -- 172.21.15.88:0/1324072546 shutdown_connections 2024-09-21T07:53:44.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.408+0000 7f2814200640 1 -- 172.21.15.88:0/1324072546 wait complete. 2024-09-21T07:53:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.410+0000 7f2814200640 1 Processor -- start 2024-09-21T07:53:44.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.410+0000 7f2814200640 1 -- start start 2024-09-21T07:53:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2814200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4003a00 0x7f280c09b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2814200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c157330 0x7f280c15cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2814200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c157740 0x7f280c15d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2814200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f280c06fb30 con 0x7f280c157740 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2814200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0b34f0 con 0x7f27f4003a00 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2814200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f280c06fcb0 con 0x7f280c157330 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c157740 0x7f280c15d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c157740 0x7f280c15d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49318/0 (socket says 172.21.15.88:49318) 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 -- 172.21.15.88:0/1996043218 learned_addr learned my addr 172.21.15.88:0/1996043218 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 -- 172.21.15.88:0/1996043218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c157330 msgr2=0x7f280c15cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 --2- 172.21.15.88:0/1996043218 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c157330 0x7f280c15cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:44.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 -- 172.21.15.88:0/1996043218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4003a00 msgr2=0x7f280c09b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 --2- 172.21.15.88:0/1996043218 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27f4003a00 0x7f280c09b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.412+0000 7f2812200640 1 -- 172.21.15.88:0/1996043218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c0c9c70 con 0x7f280c157740 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.413+0000 7f2812200640 1 --2- 172.21.15.88:0/1996043218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f280c157740 0x7f280c15d480 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f280c0d13a0 tx=0x7f280400eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.413+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400c8f0 con 0x7f280c157740 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.413+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2804017070 con 0x7f280c157740 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.413+0000 7f2814200640 1 -- 172.21.15.88:0/1996043218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c15aa40 con 0x7f280c157740 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.413+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280400ce20 con 0x7f280c157740 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.413+0000 7f2814200640 1 -- 172.21.15.88:0/1996043218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c09aef0 con 0x7f280c157740 2024-09-21T07:53:44.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.414+0000 7f2814200640 1 -- 172.21.15.88:0/1996043218 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c09ace0 con 0x7f280c157740 2024-09-21T07:53:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.415+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f280401c460 con 0x7f280c157740 2024-09-21T07:53:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.416+0000 7f2809600640 1 --2- 172.21.15.88:0/1996043218 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079250 0x7f27ec07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.416+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f280409bd70 con 0x7f280c157740 2024-09-21T07:53:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.416+0000 7f2811800640 1 --2- 172.21.15.88:0/1996043218 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079250 0x7f27ec07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:44.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.417+0000 7f2811800640 1 --2- 172.21.15.88:0/1996043218 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079250 0x7f27ec07b760 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2800035000 tx=0x7f2800004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:44.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.420+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2804063770 con 0x7f280c157740 2024-09-21T07:53:44.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.629+0000 7f2814200640 1 -- 172.21.15.88:0/1996043218 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot create", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f280c15a7d0 con 0x7f27ec079250 2024-09-21T07:53:44.633 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:44.631+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:44.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.631+0000 7f2809600640 1 -- 172.21.15.88:0/1996043218 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f280c15a7d0 con 0x7f27ec079250 2024-09-21T07:53:44.634 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:44.651 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:44.652 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot rm non_existent_subvolume group_1 snapshot_1 2024-09-21T07:53:44.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1abb200640 1 -- 172.21.15.88:0/3630735627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aa0003a00 msgr2=0x7f1aa0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:44.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1abb200640 1 --2- 172.21.15.88:0/3630735627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aa0003a00 0x7f1aa0003e70 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00c4a0 tx=0x7f1aa0005830 comp rx=0 tx=0).stop 2024-09-21T07:53:44.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1aba800640 1 -- 172.21.15.88:0/3630735627 reap_dead start 2024-09-21T07:53:44.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1abb200640 1 -- 172.21.15.88:0/3630735627 shutdown_connections 2024-09-21T07:53:44.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1abb200640 1 -- 172.21.15.88:0/3630735627 >> 172.21.15.88:0/3630735627 conn(0x7f1abc0b1dd0 msgr2=0x7f1abc0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:44.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1abb200640 1 -- 172.21.15.88:0/3630735627 shutdown_connections 2024-09-21T07:53:44.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.878+0000 7f1abb200640 1 -- 172.21.15.88:0/3630735627 wait complete. 2024-09-21T07:53:44.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.880+0000 7f1abb200640 1 Processor -- start 2024-09-21T07:53:44.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.881+0000 7f1abb200640 1 -- start start 2024-09-21T07:53:44.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1abb200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aa0003a00 0x7f1abc152cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1abb200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc15bae0 0x7f1abc074500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1abb200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc074a40 0x7f1abc159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1abb200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc160b80 con 0x7f1aa0003a00 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1abb200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1abc160a00 con 0x7f1abc074a40 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1abb200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1abc0b47c0 con 0x7f1abc15bae0 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1ab9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aa0003a00 0x7f1abc152cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1ab9e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aa0003a00 0x7f1abc152cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49332/0 (socket says 172.21.15.88:49332) 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.882+0000 7f1ab9e00640 1 -- 172.21.15.88:0/1673365460 learned_addr learned my addr 172.21.15.88:0/1673365460 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab1400640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc15bae0 0x7f1abc074500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:44.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1aba800640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc074a40 0x7f1abc159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab9e00640 1 -- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc15bae0 msgr2=0x7f1abc074500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab9e00640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc15bae0 0x7f1abc074500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab9e00640 1 -- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc074a40 msgr2=0x7f1abc159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab9e00640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1abc074a40 0x7f1abc159af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab9e00640 1 -- 172.21.15.88:0/1673365460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1abc1f9630 con 0x7f1aa0003a00 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab1400640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1abc15bae0 0x7f1abc074500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab9e00640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1aa0003a00 0x7f1abc152cc0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f1aac02b300 tx=0x7f1aac0060e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac042070 con 0x7f1aa0003a00 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1aac02b650 con 0x7f1aa0003a00 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1aac039070 con 0x7f1aa0003a00 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1abb200640 1 -- 172.21.15.88:0/1673365460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1abc1f8a20 con 0x7f1aa0003a00 2024-09-21T07:53:44.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.883+0000 7f1abb200640 1 -- 172.21.15.88:0/1673365460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1abc1f8fe0 con 0x7f1aa0003a00 2024-09-21T07:53:44.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.884+0000 7f1abb200640 1 -- 172.21.15.88:0/1673365460 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1abc1602b0 con 0x7f1aa0003a00 2024-09-21T07:53:44.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.886+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f1aac045070 con 0x7f1aa0003a00 2024-09-21T07:53:44.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.887+0000 7f1ab2a00640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1a98079250 0x7f1a9807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:44.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.887+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f1aac0bb6b0 con 0x7f1aa0003a00 2024-09-21T07:53:44.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.887+0000 7f1ab1400640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1a98079250 0x7f1a9807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:44.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.888+0000 7f1ab1400640 1 --2- 172.21.15.88:0/1673365460 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1a98079250 0x7f1a9807b760 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f1abc09af00 tx=0x7f1aa4007320 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:44.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:44.892+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1abc0b4ec0 con 0x7f1aa0003a00 2024-09-21T07:53:45.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.104+0000 7f1abb200640 1 -- 172.21.15.88:0/1673365460 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot rm", "vol_name": "non_existent_subvolume", "group_name": "group_1", "snap_name": "snapshot_1", "target": ["mon-mgr", ""]}) -- 0x7f1abc15bef0 con 0x7f1a98079250 2024-09-21T07:53:45.110 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:45.105+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:45.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.105+0000 7f1ab2a00640 1 -- 172.21.15.88:0/1673365460 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f1abc15bef0 con 0x7f1a98079250 2024-09-21T07:53:45.110 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:45.124 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:45.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup snapshot ls non_existent_subvolume group_1 2024-09-21T07:53:45.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 -- 172.21.15.88:0/2335477290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 msgr2=0x7f81500b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 --2- 172.21.15.88:0/2335477290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 0x7f81500b47c0 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f81500b38b0 tx=0x7f814002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 -- 172.21.15.88:0/2335477290 shutdown_connections 2024-09-21T07:53:45.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 --2- 172.21.15.88:0/2335477290 >> v2:172.21.15.175:3300/0 conn(0x7f81501532e0 0x7f81501536f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 --2- 172.21.15.88:0/2335477290 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81500705b0 0x7f81500af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 --2- 172.21.15.88:0/2335477290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 0x7f81500b47c0 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 -- 172.21.15.88:0/2335477290 >> 172.21.15.88:0/2335477290 conn(0x7f81500b1dd0 msgr2=0x7f81500b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:45.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 -- 172.21.15.88:0/2335477290 shutdown_connections 2024-09-21T07:53:45.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.348+0000 7f814f200640 1 -- 172.21.15.88:0/2335477290 wait complete. 2024-09-21T07:53:45.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.351+0000 7f814f200640 1 Processor -- start 2024-09-21T07:53:45.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.351+0000 7f814f200640 1 -- start start 2024-09-21T07:53:45.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81500705b0 0x7f815013f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 0x7f8150157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81501532e0 0x7f8150141710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8150160880 con 0x7f81500b3ec0 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8150160700 con 0x7f81500705b0 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8150160a00 con 0x7f81501532e0 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 0x7f8150157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 0x7f8150157790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49360/0 (socket says 172.21.15.88:49360) 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.352+0000 7f814d400640 1 -- 172.21.15.88:0/1364846475 learned_addr learned my addr 172.21.15.88:0/1364846475 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:45.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814d400640 1 -- 172.21.15.88:0/1364846475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81501532e0 msgr2=0x7f8150141710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814d400640 1 --2- 172.21.15.88:0/1364846475 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81501532e0 0x7f8150141710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814d400640 1 -- 172.21.15.88:0/1364846475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81500705b0 msgr2=0x7f815013f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814d400640 1 --2- 172.21.15.88:0/1364846475 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81500705b0 0x7f815013f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814d400640 1 -- 172.21.15.88:0/1364846475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f815013fbc0 con 0x7f81500b3ec0 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814d400640 1 --2- 172.21.15.88:0/1364846475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81500b3ec0 0x7f8150157790 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f813c00b410 tx=0x7f813c00b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c015020 con 0x7f81500b3ec0 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f814f200640 1 -- 172.21.15.88:0/1364846475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8150142280 con 0x7f81500b3ec0 2024-09-21T07:53:45.357 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f813c011940 con 0x7f81500b3ec0 2024-09-21T07:53:45.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.353+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f813c011d90 con 0x7f81500b3ec0 2024-09-21T07:53:45.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.354+0000 7f814f200640 1 -- 172.21.15.88:0/1364846475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81501592a0 con 0x7f81500b3ec0 2024-09-21T07:53:45.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.355+0000 7f814f200640 1 -- 172.21.15.88:0/1364846475 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8150141ff0 con 0x7f81500b3ec0 2024-09-21T07:53:45.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.356+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f813c018020 con 0x7f81500b3ec0 2024-09-21T07:53:45.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.357+0000 7f8146000640 1 --2- 172.21.15.88:0/1364846475 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8130079280 0x7f813007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:45.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.357+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f813c09baa0 con 0x7f81500b3ec0 2024-09-21T07:53:45.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.357+0000 7f814de00640 1 --2- 172.21.15.88:0/1364846475 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8130079280 0x7f813007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:45.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.358+0000 7f814de00640 1 --2- 172.21.15.88:0/1364846475 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8130079280 0x7f813007b790 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f8140033000 tx=0x7f81400046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:45.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.361+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f813c01f020 con 0x7f81500b3ec0 2024-09-21T07:53:45.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.570+0000 7f814f200640 1 -- 172.21.15.88:0/1364846475 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup snapshot ls", "vol_name": "non_existent_subvolume", "group_name": "group_1", "target": ["mon-mgr", ""]}) -- 0x7f8150153800 con 0x7f8130079280 2024-09-21T07:53:45.573 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:53:45.571+0000 7faca9000640 -1 mgr.server reply reply (2) No such file or directory FS 'non_existent_subvolume' not found 2024-09-21T07:53:45.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:45.572+0000 7f8146000640 1 -- 172.21.15.88:0/1364846475 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: -2 FS 'non_existent_subvolume' not found) ==== 45+0+0 (secure 0 0 0) 0x7f8150153800 con 0x7f8130079280 2024-09-21T07:53:45.574 INFO:teuthology.orchestra.run.smithi088.stderr:Error ENOENT: FS 'non_existent_subvolume' not found 2024-09-21T07:53:45.592 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-09-21T07:53:45.593 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:53:45.593 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:45.675 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:45.773 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:53:45.773 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:45.773 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:45.773 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:45.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:45.811 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:45.888 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:53:46.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 -- 172.21.15.88:0/1408249096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 msgr2=0x7efd38156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 --2- 172.21.15.88:0/1408249096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38156e40 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7efd2c009900 tx=0x7efd2c02a580 comp rx=0 tx=0).stop 2024-09-21T07:53:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 -- 172.21.15.88:0/1408249096 shutdown_connections 2024-09-21T07:53:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 --2- 172.21.15.88:0/1408249096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38156e40 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 -- 172.21.15.88:0/1408249096 >> 172.21.15.88:0/1408249096 conn(0x7efd3806d3d0 msgr2=0x7efd380b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 -- 172.21.15.88:0/1408249096 shutdown_connections 2024-09-21T07:53:46.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.150+0000 7efd40000640 1 -- 172.21.15.88:0/1408249096 wait complete. 2024-09-21T07:53:46.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.152+0000 7efd40000640 1 Processor -- start 2024-09-21T07:53:46.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.152+0000 7efd40000640 1 -- start start 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd40000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd380705b0 0x7efd381f9d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd40000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd40000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd3815ce30 0x7efd3815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd40000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd380cd100 con 0x7efd380cfc80 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd40000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd380ccf80 con 0x7efd380705b0 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd40000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd380cd280 con 0x7efd3815ce30 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd3cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd3cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38155e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55342/0 (socket says 172.21.15.88:55342) 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd3cc00640 1 -- 172.21.15.88:0/720415846 learned_addr learned my addr 172.21.15.88:0/720415846 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd3d600640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd380705b0 0x7efd381f9d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.154+0000 7efd3e000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd3815ce30 0x7efd3815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3cc00640 1 -- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd3815ce30 msgr2=0x7efd3815b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3cc00640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd3815ce30 0x7efd3815b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3cc00640 1 -- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd380705b0 msgr2=0x7efd381f9d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3cc00640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd380705b0 0x7efd381f9d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3cc00640 1 -- 172.21.15.88:0/720415846 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd381fa240 con 0x7efd380cfc80 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3e000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd3815ce30 0x7efd3815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:46.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd3cc00640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38155e30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efd2400b410 tx=0x7efd2400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24015020 con 0x7efd380cfc80 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd24011920 con 0x7efd380cfc80 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24011d70 con 0x7efd380cfc80 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd38156370 con 0x7efd380cfc80 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.155+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd3809a970 con 0x7efd380cfc80 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.156+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd380b47b0 con 0x7efd380cfc80 2024-09-21T07:53:46.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.157+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7efd24018020 con 0x7efd380cfc80 2024-09-21T07:53:46.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.159+0000 7efd35600640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd18079280 0x7efd1807b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.159+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7efd2409bad0 con 0x7efd380cfc80 2024-09-21T07:53:46.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.159+0000 7efd3d600640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd18079280 0x7efd1807b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.160+0000 7efd3d600640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd18079280 0x7efd1807b790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd2c033000 tx=0x7efd2c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:46.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.164+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd380b47b0 con 0x7efd380cfc80 2024-09-21T07:53:46.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.362+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7efd380cd7b0 con 0x7efd380cfc80 2024-09-21T07:53:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.363+0000 7efd35600640 1 -- 172.21.15.88:0/720415846 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7efd2401f020 con 0x7efd380cfc80 2024-09-21T07:53:46.365 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:53:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd18079280 msgr2=0x7efd1807b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd18079280 0x7efd1807b790 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7efd2c033000 tx=0x7efd2c004730 comp rx=0 tx=0).stop 2024-09-21T07:53:46.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 msgr2=0x7efd38155e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38155e30 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7efd2400b410 tx=0x7efd2400b8d0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 shutdown_connections 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd18079280 0x7efd1807b790 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd3815ce30 0x7efd3815b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd380cfc80 0x7efd38155e30 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 --2- 172.21.15.88:0/720415846 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd380705b0 0x7efd381f9d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 >> 172.21.15.88:0/720415846 conn(0x7efd3806d3d0 msgr2=0x7efd3815e970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 shutdown_connections 2024-09-21T07:53:46.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.365+0000 7efd40000640 1 -- 172.21.15.88:0/720415846 wait complete. 2024-09-21T07:53:46.392 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:53:46.392 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:46.393 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:53:46.456 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:53:46.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:46.457 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:53:46.493 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:46.493 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:46.493 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:46.533 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:46.534 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:46.612 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:53:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 -- 172.21.15.88:0/2534386105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477815ce30 msgr2=0x7f477815d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 --2- 172.21.15.88:0/2534386105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477815ce30 0x7f477815d240 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f4770005fd0 tx=0x7f477002a5a0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 -- 172.21.15.88:0/2534386105 shutdown_connections 2024-09-21T07:53:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 --2- 172.21.15.88:0/2534386105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f477815ce30 0x7f477815d240 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 --2- 172.21.15.88:0/2534386105 >> v2:172.21.15.175:3300/0 conn(0x7f4778070cc0 0x7f4778156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 -- 172.21.15.88:0/2534386105 >> 172.21.15.88:0/2534386105 conn(0x7f477806d3d0 msgr2=0x7f47780b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:46.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 -- 172.21.15.88:0/2534386105 shutdown_connections 2024-09-21T07:53:46.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.833+0000 7f477de00640 1 -- 172.21.15.88:0/2534386105 wait complete. 2024-09-21T07:53:46.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.835+0000 7f477de00640 1 Processor -- start 2024-09-21T07:53:46.838 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.835+0000 7f477de00640 1 -- start start 2024-09-21T07:53:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f477de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 0x7f47781f70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f477de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477815ce30 0x7f47781562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f477de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47780b04c0 0x7f47780c55b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f477de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47780cd050 con 0x7f47780cfc80 2024-09-21T07:53:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f477de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f47780cced0 con 0x7f47780b04c0 2024-09-21T07:53:46.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f477de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f47780cd1d0 con 0x7f477815ce30 2024-09-21T07:53:46.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f4776a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 0x7f47781f70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f4776a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 0x7f47781f70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55352/0 (socket says 172.21.15.88:55352) 2024-09-21T07:53:46.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f4776a00640 1 -- 172.21.15.88:0/510878859 learned_addr learned my addr 172.21.15.88:0/510878859 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:46.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f4777400640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47780b04c0 0x7f47780c55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.837+0000 7f4776000640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477815ce30 0x7f47781562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776a00640 1 -- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477815ce30 msgr2=0x7f47781562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776a00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477815ce30 0x7f47781562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776a00640 1 -- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47780b04c0 msgr2=0x7f47780c55b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776a00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47780b04c0 0x7f47780c55b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776a00640 1 -- 172.21.15.88:0/510878859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47781f3620 con 0x7f47780cfc80 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776000640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477815ce30 0x7f47781562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:46.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f4776a00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 0x7f47781f70e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f476800c6c0 tx=0x7f476800cb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:46.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768016020 con 0x7f47780cfc80 2024-09-21T07:53:46.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4768006e40 con 0x7f47780cfc80 2024-09-21T07:53:46.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4768003bf0 con 0x7f47780cfc80 2024-09-21T07:53:46.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47781f3900 con 0x7f47780cfc80 2024-09-21T07:53:46.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.838+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47781f3ec0 con 0x7f47780cfc80 2024-09-21T07:53:46.845 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.839+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47780cd6f0 con 0x7f47780cfc80 2024-09-21T07:53:46.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.840+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f47680050d0 con 0x7f47780cfc80 2024-09-21T07:53:46.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.841+0000 7f476f400640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4758079250 0x7f475807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:46.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.841+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f476809b6f0 con 0x7f47780cfc80 2024-09-21T07:53:46.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.842+0000 7f4776000640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4758079250 0x7f475807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:46.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.842+0000 7f4776000640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4758079250 0x7f475807b760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f47780b20c0 tx=0x7f4764005f10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:46.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:46.846+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4768020020 con 0x7f47780cfc80 2024-09-21T07:53:47.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.047+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f47780afa70 con 0x7f47780cfc80 2024-09-21T07:53:47.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.047+0000 7f476f400640 1 -- 172.21.15.88:0/510878859 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v134) ==== 80+0+0 (secure 0 0 0) 0x7f4768062ff0 con 0x7f47780cfc80 2024-09-21T07:53:47.049 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4758079250 msgr2=0x7f475807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4758079250 0x7f475807b760 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f47780b20c0 tx=0x7f4764005f10 comp rx=0 tx=0).stop 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 msgr2=0x7f47781f70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 0x7f47781f70e0 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f476800c6c0 tx=0x7f476800cb80 comp rx=0 tx=0).stop 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 shutdown_connections 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4758079250 0x7f475807b760 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f477815ce30 0x7f47781562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f47780b04c0 0x7f47780c55b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 --2- 172.21.15.88:0/510878859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47780cfc80 0x7f47781f70e0 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 >> 172.21.15.88:0/510878859 conn(0x7f477806d3d0 msgr2=0x7f47781603b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:47.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 shutdown_connections 2024-09-21T07:53:47.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.050+0000 7f477de00640 1 -- 172.21.15.88:0/510878859 wait complete. 2024-09-21T07:53:47.074 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:53:47.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:47.075 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:53:47.143 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:53:47.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:53:47.143 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:53:47.180 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:47.180 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:47.181 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:47.216 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:47.216 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:53:47.216 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:47.216 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:47.256 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:53:47.257 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:47.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f5245e00640 1 -- 172.21.15.88:0/2722416202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 msgr2=0x7f5224003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f5245e00640 1 --2- 172.21.15.88:0/2722416202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f5224003e70 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f5230009b50 tx=0x7f5224005830 comp rx=0 tx=0).stop 2024-09-21T07:53:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f523f400640 1 -- 172.21.15.88:0/2722416202 reap_dead start 2024-09-21T07:53:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f5245e00640 1 -- 172.21.15.88:0/2722416202 shutdown_connections 2024-09-21T07:53:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f5245e00640 1 -- 172.21.15.88:0/2722416202 >> 172.21.15.88:0/2722416202 conn(0x7f524006d730 msgr2=0x7f52400b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f5245e00640 1 -- 172.21.15.88:0/2722416202 shutdown_connections 2024-09-21T07:53:47.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.478+0000 7f5245e00640 1 -- 172.21.15.88:0/2722416202 wait complete. 2024-09-21T07:53:47.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.481+0000 7f5245e00640 1 Processor -- start 2024-09-21T07:53:47.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.481+0000 7f5245e00640 1 -- start start 2024-09-21T07:53:47.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f5245e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f52401f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f5245e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524015aa90 0x7f524015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f5245e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5240099500 0x7f5240099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f5245e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52400b3750 con 0x7f5224003a00 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f5245e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52400b38d0 con 0x7f5240099500 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f5245e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52400b3a50 con 0x7f524015aa90 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f523ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f52401f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f523ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f52401f5470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55360/0 (socket says 172.21.15.88:55360) 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.482+0000 7f523ea00640 1 -- 172.21.15.88:0/2517003606 learned_addr learned my addr 172.21.15.88:0/2517003606 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f5237e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524015aa90 0x7f524015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523ea00640 1 -- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524015aa90 msgr2=0x7f524015aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523ea00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524015aa90 0x7f524015aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523ea00640 1 -- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5240099500 msgr2=0x7f5240099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523f400640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5240099500 0x7f5240099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523ea00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5240099500 0x7f5240099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523ea00640 1 -- 172.21.15.88:0/2517003606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52401543a0 con 0x7f5224003a00 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f5237e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524015aa90 0x7f524015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523f400640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5240099500 0x7f5240099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f523ea00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f52401f5470 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5230009a70 tx=0x7f5230033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230038040 con 0x7f5224003a00 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5230004330 con 0x7f5224003a00 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52401f8fe0 con 0x7f5224003a00 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.483+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5230004520 con 0x7f5224003a00 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.484+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52401f95a0 con 0x7f5224003a00 2024-09-21T07:53:47.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.485+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5240152920 con 0x7f5224003a00 2024-09-21T07:53:47.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.486+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f5230045060 con 0x7f5224003a00 2024-09-21T07:53:47.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.487+0000 7f5237400640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f521c079250 0x7f521c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.487+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f52300bbdf0 con 0x7f5224003a00 2024-09-21T07:53:47.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.487+0000 7f5237e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f521c079250 0x7f521c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.488+0000 7f5237e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f521c079250 0x7f521c07b760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f52400cfdd0 tx=0x7f5228009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:47.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.492+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5230083910 con 0x7f5224003a00 2024-09-21T07:53:47.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.714+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f524015b940 con 0x7f5224003a00 2024-09-21T07:53:47.717 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.715+0000 7f5237400640 1 -- 172.21.15.88:0/2517003606 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5233 (secure 0 0 0) 0x7f5230083210 con 0x7f5224003a00 2024-09-21T07:53:47.717 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:47.717 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":90,"btime":"2024-09-21T07:53:33:725994+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:53:22.814801+0000","modified":"2024-09-21T07:53:25.831670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2904821684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2904821684},{"type":"v1","addr":"172.21.15.88:6835","nonce":2904821684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7938,"qdb_cluster":[7938]},"id":10}]} 2024-09-21T07:53:47.717 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 90 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.717+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f521c079250 msgr2=0x7f521c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.717+0000 7f5245e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f521c079250 0x7f521c07b760 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f52400cfdd0 tx=0x7f5228009040 comp rx=0 tx=0).stop 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.717+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 msgr2=0x7f52401f5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.717+0000 7f5245e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f52401f5470 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5230009a70 tx=0x7f5230033670 comp rx=0 tx=0).stop 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 shutdown_connections 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f521c079250 0x7f521c07b760 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524015aa90 0x7f524015aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5240099500 0x7f5240099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 --2- 172.21.15.88:0/2517003606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5224003a00 0x7f52401f5470 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 >> 172.21.15.88:0/2517003606 conn(0x7f524006d730 msgr2=0x7f524015d950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:47.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 shutdown_connections 2024-09-21T07:53:47.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.718+0000 7f5245e00640 1 -- 172.21.15.88:0/2517003606 wait complete. 2024-09-21T07:53:47.745 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:47.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1ee800640 1 -- 172.21.15.88:0/489331751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1cc003fd0 msgr2=0x7ff1cc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1ee800640 1 --2- 172.21.15.88:0/489331751 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1cc003fd0 0x7ff1cc004800 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7ff1e815cd60 tx=0x7ff1cc005830 comp rx=0 tx=0).stop 2024-09-21T07:53:47.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1e7e00640 1 -- 172.21.15.88:0/489331751 reap_dead start 2024-09-21T07:53:47.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1ee800640 1 -- 172.21.15.88:0/489331751 shutdown_connections 2024-09-21T07:53:47.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1ee800640 1 -- 172.21.15.88:0/489331751 >> 172.21.15.88:0/489331751 conn(0x7ff1e806d730 msgr2=0x7ff1e80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1ee800640 1 -- 172.21.15.88:0/489331751 shutdown_connections 2024-09-21T07:53:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.967+0000 7ff1ee800640 1 -- 172.21.15.88:0/489331751 wait complete. 2024-09-21T07:53:47.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.969+0000 7ff1ee800640 1 Processor -- start 2024-09-21T07:53:47.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.969+0000 7ff1ee800640 1 -- start start 2024-09-21T07:53:47.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1ee800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1cc003fd0 0x7ff1e8151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1ee800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 0x7ff1e814ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1ee800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1e8074090 0x7ff1e80744a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1ee800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e80b4da0 con 0x7ff1e81f9790 2024-09-21T07:53:47.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1ee800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff1e80b4c20 con 0x7ff1e8074090 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1ee800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff1e80b3960 con 0x7ff1cc003fd0 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 0x7ff1e814ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 0x7ff1e814ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55372/0 (socket says 172.21.15.88:55372) 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 -- 172.21.15.88:0/2459061610 learned_addr learned my addr 172.21.15.88:0/2459061610 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e7400640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1cc003fd0 0x7ff1e8151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e7e00640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1e8074090 0x7ff1e80744a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 -- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1cc003fd0 msgr2=0x7ff1e8151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1cc003fd0 0x7ff1e8151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 -- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1e8074090 msgr2=0x7ff1e80744a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1e8074090 0x7ff1e80744a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e6a00640 1 -- 172.21.15.88:0/2459061610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1e80749e0 con 0x7ff1e81f9790 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e7400640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1cc003fd0 0x7ff1e8151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.971+0000 7ff1e7e00640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1e8074090 0x7ff1e80744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.972+0000 7ff1e6a00640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 0x7ff1e814ed10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009a70 tx=0x7ff1d4033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:47.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.972+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4038040 con 0x7ff1e81f9790 2024-09-21T07:53:47.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.972+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff1d4033970 con 0x7ff1e81f9790 2024-09-21T07:53:47.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.972+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1e8152290 con 0x7ff1e81f9790 2024-09-21T07:53:47.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.972+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff1d4033dc0 con 0x7ff1e81f9790 2024-09-21T07:53:47.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.972+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1e81f8f90 con 0x7ff1e81f9790 2024-09-21T07:53:47.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.973+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1e8152000 con 0x7ff1e81f9790 2024-09-21T07:53:47.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.975+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff1d4043020 con 0x7ff1e81f9790 2024-09-21T07:53:47.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.975+0000 7ff1df400640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff1c8079250 0x7ff1c807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:47.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.976+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7ff1d40bbd40 con 0x7ff1e81f9790 2024-09-21T07:53:47.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.976+0000 7ff1e7400640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff1c8079250 0x7ff1c807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:47.978 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.977+0000 7ff1e7400640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff1c8079250 0x7ff1c807b760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8001cc0 tx=0x7ff1d8005040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:47.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:47.980+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff1d404a020 con 0x7ff1e81f9790 2024-09-21T07:53:48.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.203+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff1e80ad4f0 con 0x7ff1e81f9790 2024-09-21T07:53:48.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.203+0000 7ff1df400640 1 -- 172.21.15.88:0/2459061610 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5233 (secure 0 0 0) 0x7ff1e80749e0 con 0x7ff1e81f9790 2024-09-21T07:53:48.206 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:48.206 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":90,"btime":"2024-09-21T07:53:33:725994+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:53:22.814801+0000","modified":"2024-09-21T07:53:25.831670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2904821684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2904821684},{"type":"v1","addr":"172.21.15.88:6835","nonce":2904821684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7938,"qdb_cluster":[7938]},"id":10}]} 2024-09-21T07:53:48.206 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 90 2024-09-21T07:53:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff1c8079250 msgr2=0x7ff1c807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff1c8079250 0x7ff1c807b760 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff1d8001cc0 tx=0x7ff1d8005040 comp rx=0 tx=0).stop 2024-09-21T07:53:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 msgr2=0x7ff1e814ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 0x7ff1e814ed10 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7ff1d4009a70 tx=0x7ff1d4033670 comp rx=0 tx=0).stop 2024-09-21T07:53:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 shutdown_connections 2024-09-21T07:53:48.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff1c8079250 0x7ff1c807b760 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff1e8074090 0x7ff1e80744a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff1e81f9790 0x7ff1e814ed10 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 --2- 172.21.15.88:0/2459061610 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff1cc003fd0 0x7ff1e8151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 >> 172.21.15.88:0/2459061610 conn(0x7ff1e806d730 msgr2=0x7ff1e815efd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:48.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 shutdown_connections 2024-09-21T07:53:48.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.206+0000 7ff1ee800640 1 -- 172.21.15.88:0/2459061610 wait complete. 2024-09-21T07:53:48.229 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 -- 172.21.15.88:0/226653597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 msgr2=0x7fe47c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 --2- 172.21.15.88:0/226653597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1535c0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7fe47c072d60 tx=0x7fe47002a980 comp rx=0 tx=0).stop 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 -- 172.21.15.88:0/226653597 shutdown_connections 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 --2- 172.21.15.88:0/226653597 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c160310 0x7fe47c160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 --2- 172.21.15.88:0/226653597 >> v2:172.21.15.175:3300/0 conn(0x7fe47c0ad0e0 0x7fe47c0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 --2- 172.21.15.88:0/226653597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1535c0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 -- 172.21.15.88:0/226653597 >> 172.21.15.88:0/226653597 conn(0x7fe47c06d730 msgr2=0x7fe47c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 -- 172.21.15.88:0/226653597 shutdown_connections 2024-09-21T07:53:48.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.446+0000 7fe484400640 1 -- 172.21.15.88:0/226653597 wait complete. 2024-09-21T07:53:48.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.449+0000 7fe484400640 1 Processor -- start 2024-09-21T07:53:48.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.449+0000 7fe484400640 1 -- start start 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe484400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c0ad0e0 0x7fe47c155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe484400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1f5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe484400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c160310 0x7fe47c14f650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe484400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c0b4da0 con 0x7fe47c15d760 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe484400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe47c0b4c20 con 0x7fe47c0ad0e0 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe484400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe47c160c60 con 0x7fe47c160310 2024-09-21T07:53:48.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe481000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1f5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe481000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1f5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55390/0 (socket says 172.21.15.88:55390) 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.450+0000 7fe481000640 1 -- 172.21.15.88:0/3027568851 learned_addr learned my addr 172.21.15.88:0/3027568851 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481a00640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c0ad0e0 0x7fe47c155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe482400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c160310 0x7fe47c14f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481000640 1 -- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c160310 msgr2=0x7fe47c14f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481000640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c160310 0x7fe47c14f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481000640 1 -- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c0ad0e0 msgr2=0x7fe47c155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481000640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c0ad0e0 0x7fe47c155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481000640 1 -- 172.21.15.88:0/3027568851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe47c155e70 con 0x7fe47c15d760 2024-09-21T07:53:48.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe481000640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1f5600 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe46800d6d0 tx=0x7fe46800db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:48.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468017070 con 0x7fe47c15d760 2024-09-21T07:53:48.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe468004310 con 0x7fe47c15d760 2024-09-21T07:53:48.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe47c156150 con 0x7fe47c15d760 2024-09-21T07:53:48.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.451+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe468009a20 con 0x7fe47c15d760 2024-09-21T07:53:48.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.452+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe47c073290 con 0x7fe47c15d760 2024-09-21T07:53:48.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.453+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe47c1535c0 con 0x7fe47c15d760 2024-09-21T07:53:48.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.454+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fe4680035f0 con 0x7fe47c15d760 2024-09-21T07:53:48.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.455+0000 7fe479600640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe45c079250 0x7fe45c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.455+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7fe46809bcf0 con 0x7fe47c15d760 2024-09-21T07:53:48.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.455+0000 7fe481a00640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe45c079250 0x7fe45c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.456+0000 7fe481a00640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe45c079250 0x7fe45c07b760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe470035000 tx=0x7fe470004ab0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:48.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.459+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe468062fe0 con 0x7fe47c15d760 2024-09-21T07:53:48.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.645+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe47c0cfbe0 con 0x7fe47c15d760 2024-09-21T07:53:48.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.646+0000 7fe479600640 1 -- 172.21.15.88:0/3027568851 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16832 (secure 0 0 0) 0x7fe468067a10 con 0x7fe47c15d760 2024-09-21T07:53:48.649 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:48.649 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":134,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:37.753238+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:53:20.713784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T07:53:21.774768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.34","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.648+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe45c079250 msgr2=0x7fe45c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.648+0000 7fe484400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe45c079250 0x7fe45c07b760 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fe470035000 tx=0x7fe470004ab0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.648+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 msgr2=0x7fe47c1f5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.648+0000 7fe484400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1f5600 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7fe46800d6d0 tx=0x7fe46800db90 comp rx=0 tx=0).stop 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 shutdown_connections 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe45c079250 0x7fe45c07b760 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe47c160310 0x7fe47c14f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe47c15d760 0x7fe47c1f5600 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 --2- 172.21.15.88:0/3027568851 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe47c0ad0e0 0x7fe47c155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 >> 172.21.15.88:0/3027568851 conn(0x7fe47c06d730 msgr2=0x7fe47c06f2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:48.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 shutdown_connections 2024-09-21T07:53:48.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.649+0000 7fe484400640 1 -- 172.21.15.88:0/3027568851 wait complete. 2024-09-21T07:53:48.672 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:53:48.672 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 -- 172.21.15.88:0/898330925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 msgr2=0x7f03d40b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 --2- 172.21.15.88:0/898330925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d40b3960 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f03d4072d60 tx=0x7f03cc02aa30 comp rx=0 tx=0).stop 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 -- 172.21.15.88:0/898330925 shutdown_connections 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 --2- 172.21.15.88:0/898330925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d40b3960 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 --2- 172.21.15.88:0/898330925 >> v2:172.21.15.175:3300/0 conn(0x7f03d40ad0e0 0x7f03d40ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 -- 172.21.15.88:0/898330925 >> 172.21.15.88:0/898330925 conn(0x7f03d406d730 msgr2=0x7f03d40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 -- 172.21.15.88:0/898330925 shutdown_connections 2024-09-21T07:53:48.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.891+0000 7f03db200640 1 -- 172.21.15.88:0/898330925 wait complete. 2024-09-21T07:53:48.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.894+0000 7f03db200640 1 Processor -- start 2024-09-21T07:53:48.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.894+0000 7f03db200640 1 -- start start 2024-09-21T07:53:48.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03db200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d40ad0e0 0x7f03d414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03db200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d4156010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03db200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03d415d760 0x7f03d4156550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03db200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03d40b4da0 con 0x7f03d40b3550 2024-09-21T07:53:48.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03db200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03d40b4c20 con 0x7f03d415d760 2024-09-21T07:53:48.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03db200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03d40b3ea0 con 0x7f03d40ad0e0 2024-09-21T07:53:48.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d4156010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d4156010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55394/0 (socket says 172.21.15.88:55394) 2024-09-21T07:53:48.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.895+0000 7f03cbe00640 1 -- 172.21.15.88:0/278707211 learned_addr learned my addr 172.21.15.88:0/278707211 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03cbe00640 1 -- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d40ad0e0 msgr2=0x7f03d414f170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03cbe00640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d40ad0e0 0x7f03d414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03cbe00640 1 -- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03d415d760 msgr2=0x7f03d4156550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03d9200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03d415d760 0x7f03d4156550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03cbe00640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03d415d760 0x7f03d4156550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:48.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03cbe00640 1 -- 172.21.15.88:0/278707211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03d409d700 con 0x7f03d40b3550 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03cbe00640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d4156010 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00d700 tx=0x7f03bc00dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc017070 con 0x7f03d40b3550 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03bc004310 con 0x7f03d40b3550 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc009a80 con 0x7f03d40b3550 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.896+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03d409d9e0 con 0x7f03d40b3550 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.897+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03d4150ea0 con 0x7f03d40b3550 2024-09-21T07:53:48.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.898+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03d40b4360 con 0x7f03d40b3550 2024-09-21T07:53:48.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.899+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f03bc009c20 con 0x7f03d40b3550 2024-09-21T07:53:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.900+0000 7f03d0c00640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03b0079280 0x7f03b007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.900+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f03bc09bb80 con 0x7f03d40b3550 2024-09-21T07:53:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.900+0000 7f03d3e00640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03b0079280 0x7f03b007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:48.902 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.901+0000 7f03d3e00640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03b0079280 0x7f03b007b790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c40079b0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:48.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:48.905+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03bc063480 con 0x7f03d40b3550 2024-09-21T07:53:49.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.131+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03d415dfd0 con 0x7f03d40b3550 2024-09-21T07:53:49.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.131+0000 7f03d0c00640 1 -- 172.21.15.88:0/278707211 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5233 (secure 0 0 0) 0x7f03bc062d80 con 0x7f03d40b3550 2024-09-21T07:53:49.134 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:49.134 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":90,"btime":"2024-09-21T07:53:33:725994+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:53:22.814801+0000","modified":"2024-09-21T07:53:25.831670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2904821684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2904821684},{"type":"v1","addr":"172.21.15.88:6835","nonce":2904821684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7938,"qdb_cluster":[7938]},"id":10}]} 2024-09-21T07:53:49.134 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 90 2024-09-21T07:53:49.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03b0079280 msgr2=0x7f03b007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03b0079280 0x7f03b007b790 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f03c4000f60 tx=0x7f03c40079b0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 msgr2=0x7f03d4156010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d4156010 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f03bc00d700 tx=0x7f03bc00dbc0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 shutdown_connections 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03b0079280 0x7f03b007b790 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03d415d760 0x7f03d4156550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d40b3550 0x7f03d4156010 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 --2- 172.21.15.88:0/278707211 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d40ad0e0 0x7f03d414f170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 >> 172.21.15.88:0/278707211 conn(0x7f03d406d730 msgr2=0x7f03d4160c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 shutdown_connections 2024-09-21T07:53:49.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.134+0000 7f03db200640 1 -- 172.21.15.88:0/278707211 wait complete. 2024-09-21T07:53:49.158 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:49.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 -- 172.21.15.88:0/2404929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 msgr2=0x7f7f481535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 --2- 172.21.15.88:0/2404929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f481535c0 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f7f48072d90 tx=0x7f7f3802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 -- 172.21.15.88:0/2404929 shutdown_connections 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 --2- 172.21.15.88:0/2404929 >> v2:172.21.15.175:3300/0 conn(0x7f7f48160310 0x7f7f48160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 --2- 172.21.15.88:0/2404929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f480ad0e0 0x7f7f480ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 --2- 172.21.15.88:0/2404929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f481535c0 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 -- 172.21.15.88:0/2404929 >> 172.21.15.88:0/2404929 conn(0x7f7f4806d730 msgr2=0x7f7f480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 -- 172.21.15.88:0/2404929 shutdown_connections 2024-09-21T07:53:49.378 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.375+0000 7f7f4de00640 1 -- 172.21.15.88:0/2404929 wait complete. 2024-09-21T07:53:49.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.378+0000 7f7f4de00640 1 Processor -- start 2024-09-21T07:53:49.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.378+0000 7f7f4de00640 1 -- start start 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f4de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f480ad0e0 0x7f7f48099960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f4de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f48153cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f4de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f48160310 0x7f7f48154230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f4de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480b4da0 con 0x7f7f4815d760 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f4de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f480b4c20 con 0x7f7f48160310 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f4de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7f48160c60 con 0x7f7f480ad0e0 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f3e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f48153cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f3e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f48153cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55406/0 (socket says 172.21.15.88:55406) 2024-09-21T07:53:49.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.379+0000 7f7f3e000640 1 -- 172.21.15.88:0/1279573758 learned_addr learned my addr 172.21.15.88:0/1279573758 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f46a00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f480ad0e0 0x7f7f48099960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f47400640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f48160310 0x7f7f48154230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3e000640 1 -- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f480ad0e0 msgr2=0x7f7f48099960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3e000640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f480ad0e0 0x7f7f48099960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3e000640 1 -- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f48160310 msgr2=0x7f7f48154230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3e000640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f48160310 0x7f7f48154230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3e000640 1 -- 172.21.15.88:0/1279573758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f48099ea0 con 0x7f7f4815d760 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3e000640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f48153cf0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000d700 tx=0x7f7f3000dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.380+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30017070 con 0x7f7f4815d760 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.381+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7f30004310 con 0x7f7f4815d760 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.381+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f48156300 con 0x7f7f4815d760 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.381+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f30009a80 con 0x7f7f4815d760 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.381+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f481568c0 con 0x7f7f4815d760 2024-09-21T07:53:49.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.382+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f48156070 con 0x7f7f4815d760 2024-09-21T07:53:49.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.383+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f7f30009c20 con 0x7f7f4815d760 2024-09-21T07:53:49.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.384+0000 7f7f3fe00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f24079250 0x7f7f2407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.384+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7f7f3009baf0 con 0x7f7f4815d760 2024-09-21T07:53:49.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.384+0000 7f7f46a00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f24079250 0x7f7f2407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.385+0000 7f7f46a00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f24079250 0x7f7f2407b760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f38033000 tx=0x7f7f38004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:49.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.388+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7f30062da0 con 0x7f7f4815d760 2024-09-21T07:53:49.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.609+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f481535c0 con 0x7f7f4815d760 2024-09-21T07:53:49.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.609+0000 7f7f3fe00640 1 -- 172.21.15.88:0/1279573758 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 90 v90) ==== 94+0+5233 (secure 0 0 0) 0x7f7f300677d0 con 0x7f7f4815d760 2024-09-21T07:53:49.611 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:49.612 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":90,"btime":"2024-09-21T07:53:33:725994+0000","default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85}],"filesystems":[{"mdsmap":{"epoch":89,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:53:22.814801+0000","modified":"2024-09-21T07:53:25.831670+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":7938},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7938":{"gid":7938,"name":"a","rank":0,"incarnation":86,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/2904821684","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2904821684},{"type":"v1","addr":"172.21.15.88:6835","nonce":2904821684}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":7938,"qdb_cluster":[7938]},"id":10}]} 2024-09-21T07:53:49.612 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 90 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.611+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f24079250 msgr2=0x7f7f2407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.611+0000 7f7f4de00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f24079250 0x7f7f2407b760 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f7f38033000 tx=0x7f7f38004730 comp rx=0 tx=0).stop 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.611+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 msgr2=0x7f7f48153cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.611+0000 7f7f4de00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f48153cf0 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000d700 tx=0x7f7f3000dbc0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 shutdown_connections 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7f24079250 0x7f7f2407b760 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7f48160310 0x7f7f48154230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f4815d760 0x7f7f48153cf0 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 --2- 172.21.15.88:0/1279573758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f480ad0e0 0x7f7f48099960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 >> 172.21.15.88:0/1279573758 conn(0x7f7f4806d730 msgr2=0x7f7f480b34c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 shutdown_connections 2024-09-21T07:53:49.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.612+0000 7f7f4de00640 1 -- 172.21.15.88:0/1279573758 wait complete. 2024-09-21T07:53:49.635 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:49.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 -- 172.21.15.88:0/1262501614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f00b3550 msgr2=0x7fb2f00b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/1262501614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f00b3550 0x7fb2f00b3960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb2e8005fd0 tx=0x7fb2e802aa10 comp rx=0 tx=0).stop 2024-09-21T07:53:49.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 -- 172.21.15.88:0/1262501614 shutdown_connections 2024-09-21T07:53:49.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/1262501614 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f00b3550 0x7fb2f00b3960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/1262501614 >> v2:172.21.15.175:3300/0 conn(0x7fb2f00ad0e0 0x7fb2f00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 -- 172.21.15.88:0/1262501614 >> 172.21.15.88:0/1262501614 conn(0x7fb2f006d730 msgr2=0x7fb2f00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:49.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 -- 172.21.15.88:0/1262501614 shutdown_connections 2024-09-21T07:53:49.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.851+0000 7fb2f5a00640 1 -- 172.21.15.88:0/1262501614 wait complete. 2024-09-21T07:53:49.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.853+0000 7fb2f5a00640 1 Processor -- start 2024-09-21T07:53:49.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.853+0000 7fb2f5a00640 1 -- start start 2024-09-21T07:53:49.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f00ad0e0 0x7fb2f009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 0x7fb2f01f6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2f5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f015d760 0x7fb2f01f7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2f5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f00b4da0 con 0x7fb2f00b3550 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2f5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb2f00b4c20 con 0x7fb2f00ad0e0 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2f5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb2f00b3ea0 con 0x7fb2f015d760 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 0x7fb2f01f6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2ee000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 0x7fb2f01f6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55416/0 (socket says 172.21.15.88:55416) 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2ee000640 1 -- 172.21.15.88:0/562997737 learned_addr learned my addr 172.21.15.88:0/562997737 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2eea00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f00ad0e0 0x7fb2f009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2ef400640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f015d760 0x7fb2f01f7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2ee000640 1 -- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f015d760 msgr2=0x7fb2f01f7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.855+0000 7fb2ee000640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f015d760 0x7fb2f01f7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2ee000640 1 -- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f00ad0e0 msgr2=0x7fb2f009a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:49.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2ee000640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f00ad0e0 0x7fb2f009a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:49.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2ee000640 1 -- 172.21.15.88:0/562997737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2f009a810 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2ee000640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 0x7fb2f01f6ee0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00e940 tx=0x7fb2dc00ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc00cd80 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb2dc010430 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2dc010880 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2f0159540 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.856+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2f0159b00 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.858+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb2f00b4370 con 0x7fb2f00b3550 2024-09-21T07:53:49.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.859+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fb2dc010a20 con 0x7fb2f00b3550 2024-09-21T07:53:49.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.860+0000 7fb2e6a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2d0079280 0x7fb2d007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:49.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.860+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7fb2dc09c650 con 0x7fb2f00b3550 2024-09-21T07:53:49.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.860+0000 7fb2eea00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2d0079280 0x7fb2d007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:49.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.861+0000 7fb2eea00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2d0079280 0x7fb2d007b790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0001600 tx=0x7fb2e0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:49.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:49.865+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb2dc063f10 con 0x7fb2f00b3550 2024-09-21T07:53:50.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.054+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb2f0152990 con 0x7fb2f00b3550 2024-09-21T07:53:50.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.055+0000 7fb2e6a00640 1 -- 172.21.15.88:0/562997737 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) ==== 74+0+16832 (secure 0 0 0) 0x7fb2dc063810 con 0x7fb2f00b3550 2024-09-21T07:53:50.057 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:50.058 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":134,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:37.753238+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:53:20.713784+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"131","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2024-09-21T07:53:21.774768+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"132","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.6","mappings":[{"from":2,"to":5}]},{"pgid":"21.34","mappings":[{"from":2,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2d0079280 msgr2=0x7fb2d007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2d0079280 0x7fb2d007b790 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7fb2e0001600 tx=0x7fb2e0009040 comp rx=0 tx=0).stop 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 msgr2=0x7fb2f01f6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 0x7fb2f01f6ee0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7fb2dc00e940 tx=0x7fb2dc00ee00 comp rx=0 tx=0).stop 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 shutdown_connections 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb2d0079280 0x7fb2d007b790 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb2f015d760 0x7fb2f01f7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb2f00b3550 0x7fb2f01f6ee0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 --2- 172.21.15.88:0/562997737 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb2f00ad0e0 0x7fb2f009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.057+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 >> 172.21.15.88:0/562997737 conn(0x7fb2f006d730 msgr2=0x7fb2f015e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:50.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.058+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 shutdown_connections 2024-09-21T07:53:50.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.058+0000 7fb2f5a00640 1 -- 172.21.15.88:0/562997737 wait complete. 2024-09-21T07:53:50.081 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6b200640 1 -- 172.21.15.88:0/3238570452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd50005270 msgr2=0x7efd5000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6b200640 1 --2- 172.21.15.88:0/3238570452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd50005270 0x7efd5000b9c0 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7efd6400bb20 tx=0x7efd50005830 comp rx=0 tx=0).stop 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6a800640 1 -- 172.21.15.88:0/3238570452 reap_dead start 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6b200640 1 -- 172.21.15.88:0/3238570452 shutdown_connections 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6b200640 1 -- 172.21.15.88:0/3238570452 >> 172.21.15.88:0/3238570452 conn(0x7efd6c0b1e40 msgr2=0x7efd6c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6b200640 1 -- 172.21.15.88:0/3238570452 shutdown_connections 2024-09-21T07:53:50.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.301+0000 7efd6b200640 1 -- 172.21.15.88:0/3238570452 wait complete. 2024-09-21T07:53:50.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.303+0000 7efd6b200640 1 Processor -- start 2024-09-21T07:53:50.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.304+0000 7efd6b200640 1 -- start start 2024-09-21T07:53:50.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd6b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd6c0caf50 0x7efd6c09dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd6b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 0x7efd6c0ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd6b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd6c155fe0 0x7efd6c0ce620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd6b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c0b2f60 con 0x7efd6c155bd0 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd6b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efd6c06fc00 con 0x7efd6c155fe0 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd6b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efd6c0b30e0 con 0x7efd6c0caf50 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd69400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 0x7efd6c0ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd69400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 0x7efd6c0ce0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55436/0 (socket says 172.21.15.88:55436) 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.305+0000 7efd69400640 1 -- 172.21.15.88:0/3891998652 learned_addr learned my addr 172.21.15.88:0/3891998652 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69e00640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd6c0caf50 0x7efd6c09dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd6a800640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd6c155fe0 0x7efd6c0ce620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69400640 1 -- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd6c0caf50 msgr2=0x7efd6c09dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69400640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd6c0caf50 0x7efd6c09dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69400640 1 -- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd6c155fe0 msgr2=0x7efd6c0ce620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69400640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd6c155fe0 0x7efd6c0ce620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69400640 1 -- 172.21.15.88:0/3891998652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd6c0c93c0 con 0x7efd6c155bd0 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd6a800640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd6c155fe0 0x7efd6c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:50.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd69400640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 0x7efd6c0ce0e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7efd58003f50 tx=0x7efd580016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58011640 con 0x7efd6c155bd0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efd58016020 con 0x7efd6c155bd0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd6c0c96a0 con 0x7efd6c155bd0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd58011a60 con 0x7efd6c155bd0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.306+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd6c0c9c60 con 0x7efd6c155bd0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.308+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd6c15a1d0 con 0x7efd6c155bd0 2024-09-21T07:53:50.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.309+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7efd58011c00 con 0x7efd6c155bd0 2024-09-21T07:53:50.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.309+0000 7efd62000640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd4c079250 0x7efd4c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.310+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(134..134 src has 1..134) ==== 5795+0+0 (secure 0 0 0) 0x7efd5809c6d0 con 0x7efd6c155bd0 2024-09-21T07:53:50.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.310+0000 7efd69e00640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd4c079250 0x7efd4c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.310+0000 7efd69e00640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd4c079250 0x7efd4c07b760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0039f0 tx=0x7efd5c006b20 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:50.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.314+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efd58020020 con 0x7efd6c155bd0 2024-09-21T07:53:50.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.537+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7efd6c06de60 con 0x7efd6c155bd0 2024-09-21T07:53:50.540 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:53:50.538+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:53:50.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.540+0000 7efd62000640 1 -- 172.21.15.88:0/3891998652 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v91) ==== 161+0+0 (secure 0 0 0) 0x7efd580640a0 con 0x7efd6c155bd0 2024-09-21T07:53:50.542 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd4c079250 msgr2=0x7efd4c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd4c079250 0x7efd4c07b760 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7efd5c0039f0 tx=0x7efd5c006b20 comp rx=0 tx=0).stop 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 msgr2=0x7efd6c0ce0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 0x7efd6c0ce0e0 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7efd58003f50 tx=0x7efd580016f0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 shutdown_connections 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efd4c079250 0x7efd4c07b760 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efd6c155fe0 0x7efd6c0ce620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd6c155bd0 0x7efd6c0ce0e0 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 --2- 172.21.15.88:0/3891998652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efd6c0caf50 0x7efd6c09dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.543+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 >> 172.21.15.88:0/3891998652 conn(0x7efd6c0b1e40 msgr2=0x7efd6c0cf280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:50.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.544+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 shutdown_connections 2024-09-21T07:53:50.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.544+0000 7efd6b200640 1 -- 172.21.15.88:0/3891998652 wait complete. 2024-09-21T07:53:50.552 INFO:tasks.ceph.mds.a.smithi088.stderr: -3254> 2024-09-21T07:53:25.289+0000 7fc5e4800640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:53:50.573 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:53:50.675 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 -- 172.21.15.88:0/735471711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 msgr2=0x7fdbc406fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 --2- 172.21.15.88:0/735471711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc406fb40 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8005fd0 tx=0x7fdbb802a670 comp rx=0 tx=0).stop 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 -- 172.21.15.88:0/735471711 shutdown_connections 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 --2- 172.21.15.88:0/735471711 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc4145a40 0x7fdbc4073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 --2- 172.21.15.88:0/735471711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc406fb40 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 -- 172.21.15.88:0/735471711 >> 172.21.15.88:0/735471711 conn(0x7fdbc406d1c0 msgr2=0x7fdbc406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 -- 172.21.15.88:0/735471711 shutdown_connections 2024-09-21T07:53:50.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.797+0000 7fdbcc200640 1 -- 172.21.15.88:0/735471711 wait complete. 2024-09-21T07:53:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.800+0000 7fdbcc200640 1 Processor -- start 2024-09-21T07:53:50.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.800+0000 7fdbcc200640 1 -- start start 2024-09-21T07:53:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbcc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc4073a80 0x7fdbc41fa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbcc200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc40b2520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbcc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc4145a40 0x7fdbc41f9760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbcc200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc4160a30 con 0x7fdbc4145550 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbcc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdbc40ce940 con 0x7fdbc4073a80 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbcc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdbc4160bb0 con 0x7fdbc4145a40 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc40b2520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc40b2520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55472/0 (socket says 172.21.15.88:55472) 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 -- 172.21.15.88:0/1287275022 learned_addr learned my addr 172.21.15.88:0/1287275022 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 -- 172.21.15.88:0/1287275022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc4145a40 msgr2=0x7fdbc41f9760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc4145a40 0x7fdbc41f9760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 -- 172.21.15.88:0/1287275022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc4073a80 msgr2=0x7fdbc41fa300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc4073a80 0x7fdbc41fa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:50.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.802+0000 7fdbc8e00640 1 -- 172.21.15.88:0/1287275022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbc41e6d10 con 0x7fdbc4145550 2024-09-21T07:53:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.803+0000 7fdbc8e00640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc40b2520 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000d6b0 tx=0x7fdbb000db70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:50.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.803+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb0017070 con 0x7fdbc4145550 2024-09-21T07:53:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.803+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdbb0005310 con 0x7fdbc4145550 2024-09-21T07:53:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.803+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbc41e6ff0 con 0x7fdbc4145550 2024-09-21T07:53:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.803+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbb00097c0 con 0x7fdbc4145550 2024-09-21T07:53:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.803+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbc4143f80 con 0x7fdbc4145550 2024-09-21T07:53:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.804+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbc406a7f0 con 0x7fdbc4145550 2024-09-21T07:53:50.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.805+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fdbb0009960 con 0x7fdbc4145550 2024-09-21T07:53:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.806+0000 7fdbc1600640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdba4079280 0x7fdba407b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:50.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.806+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5881+0+0 (secure 0 0 0) 0x7fdbb009bb60 con 0x7fdbc4145550 2024-09-21T07:53:50.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.807+0000 7fdbc9800640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdba4079280 0x7fdba407b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:50.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.807+0000 7fdbc9800640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdba4079280 0x7fdba407b790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8033000 tx=0x7fdbb80046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:50.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:50.811+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdbb0063380 con 0x7fdbc4145550 2024-09-21T07:53:51.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.034+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fdbc415cab0 con 0x7fdbc4145550 2024-09-21T07:53:51.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.542+0000 7fdbc1600640 1 -- 172.21.15.88:0/1287275022 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v92) ==== 104+0+0 (secure 0 0 0) 0x7fdbb0062c80 con 0x7fdbc4145550 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdba4079280 msgr2=0x7fdba407b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdba4079280 0x7fdba407b790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fdbb8033000 tx=0x7fdbb80046c0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 msgr2=0x7fdbc40b2520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc40b2520 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fdbb000d6b0 tx=0x7fdbb000db70 comp rx=0 tx=0).stop 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 shutdown_connections 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdba4079280 0x7fdba407b790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdbc4145a40 0x7fdbc41f9760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdbc4145550 0x7fdbc40b2520 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.545+0000 7fdbcc200640 1 --2- 172.21.15.88:0/1287275022 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdbc4073a80 0x7fdbc41fa300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.546+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 >> 172.21.15.88:0/1287275022 conn(0x7fdbc406d1c0 msgr2=0x7fdbc40ae790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:51.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.546+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 shutdown_connections 2024-09-21T07:53:51.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.546+0000 7fdbcc200640 1 -- 172.21.15.88:0/1287275022 wait complete. 2024-09-21T07:53:51.573 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:53:51.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 -- 172.21.15.88:0/3814216274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 msgr2=0x7fcd74003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 --2- 172.21.15.88:0/3814216274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd74003ec0 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009ed0 tx=0x7fcd74005830 comp rx=0 tx=0).stop 2024-09-21T07:53:51.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 -- 172.21.15.88:0/3814216274 shutdown_connections 2024-09-21T07:53:51.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 --2- 172.21.15.88:0/3814216274 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd74003ec0 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 -- 172.21.15.88:0/3814216274 >> 172.21.15.88:0/3814216274 conn(0x7fcd900b2540 msgr2=0x7fcd900d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:51.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 -- 172.21.15.88:0/3814216274 shutdown_connections 2024-09-21T07:53:51.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.800+0000 7fcd95800640 1 -- 172.21.15.88:0/3814216274 wait complete. 2024-09-21T07:53:51.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.802+0000 7fcd95800640 1 Processor -- start 2024-09-21T07:53:51.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.803+0000 7fcd95800640 1 -- start start 2024-09-21T07:53:51.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd95800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd901f6490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:51.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd95800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd901f6b60 0x7fcd90153260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:51.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd95800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd900987c0 0x7fcd90098bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:51.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd95800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9006d540 con 0x7fcd74003a50 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd95800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd9006d3c0 con 0x7fcd900987c0 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd95800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd9006d6c0 con 0x7fcd901f6b60 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd8ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd901f6490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd8ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd901f6490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55486/0 (socket says 172.21.15.88:55486) 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd8ea00640 1 -- 172.21.15.88:0/992452755 learned_addr learned my addr 172.21.15.88:0/992452755 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd8f400640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd900987c0 0x7fcd90098bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.804+0000 7fcd86000640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd901f6b60 0x7fcd90153260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd8ea00640 1 -- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd901f6b60 msgr2=0x7fcd90153260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd8ea00640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd901f6b60 0x7fcd90153260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd8ea00640 1 -- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd900987c0 msgr2=0x7fcd90098bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd8ea00640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd900987c0 0x7fcd90098bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:51.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd8ea00640 1 -- 172.21.15.88:0/992452755 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd900c33d0 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd8ea00640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd901f6490 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e20 tx=0x7fcd80033cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd80038040 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd80041070 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd90099110 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd8003c690 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.805+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd901f9870 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.806+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd900af800 con 0x7fcd74003a50 2024-09-21T07:53:51.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.808+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fcd8003c830 con 0x7fcd74003a50 2024-09-21T07:53:51.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.809+0000 7fcd87400640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd6c079280 0x7fcd6c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:51.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.809+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(135..135 src has 1..135) ==== 5881+0+0 (secure 0 0 0) 0x7fcd800bca40 con 0x7fcd74003a50 2024-09-21T07:53:51.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.809+0000 7fcd86000640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd6c079280 0x7fcd6c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:51.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.810+0000 7fcd86000640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd6c079280 0x7fcd6c07b790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd901541a0 tx=0x7fcd78006b70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:51.815 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:51.813+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd90099110 con 0x7fcd74003a50 2024-09-21T07:53:52.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.003+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcd900d1030 con 0x7fcd74003a50 2024-09-21T07:53:52.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.885+0000 7fcd87400640 1 -- 172.21.15.88:0/992452755 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v136) ==== 181+0+0 (secure 0 0 0) 0x7fcd800843f0 con 0x7fcd74003a50 2024-09-21T07:53:52.888 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd6c079280 msgr2=0x7fcd6c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd6c079280 0x7fcd6c07b790 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fcd901541a0 tx=0x7fcd78006b70 comp rx=0 tx=0).stop 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 msgr2=0x7fcd901f6490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd901f6490 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e20 tx=0x7fcd80033cd0 comp rx=0 tx=0).stop 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 shutdown_connections 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd6c079280 0x7fcd6c07b790 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd901f6b60 0x7fcd90153260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd900987c0 0x7fcd90098bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:52.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 --2- 172.21.15.88:0/992452755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd74003a50 0x7fcd901f6490 secure :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fcd80009e20 tx=0x7fcd80033cd0 comp rx=0 tx=0).stop 2024-09-21T07:53:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.889+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 >> 172.21.15.88:0/992452755 conn(0x7fcd900b2540 msgr2=0x7fcd9006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.890+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 shutdown_connections 2024-09-21T07:53:52.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:52.890+0000 7fcd95800640 1 -- 172.21.15.88:0/992452755 wait complete. 2024-09-21T07:53:52.915 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:53:53.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 -- 172.21.15.88:0/2457703686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac06d4b0 msgr2=0x7fe7ac0b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 --2- 172.21.15.88:0/2457703686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac06d4b0 0x7fe7ac0b4620 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fe798009830 tx=0x7fe79802a510 comp rx=0 tx=0).stop 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 -- 172.21.15.88:0/2457703686 shutdown_connections 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 --2- 172.21.15.88:0/2457703686 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7ac0af100 0x7fe7ac0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 --2- 172.21.15.88:0/2457703686 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac06d4b0 0x7fe7ac0b4620 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 -- 172.21.15.88:0/2457703686 >> 172.21.15.88:0/2457703686 conn(0x7fe7ac0b2540 msgr2=0x7fe7ac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 -- 172.21.15.88:0/2457703686 shutdown_connections 2024-09-21T07:53:53.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.138+0000 7fe7b3000640 1 -- 172.21.15.88:0/2457703686 wait complete. 2024-09-21T07:53:53.142 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.140+0000 7fe7b3000640 1 Processor -- start 2024-09-21T07:53:53.143 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.141+0000 7fe7b3000640 1 -- start start 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 0x7fe7ac1f53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7ac0cc830 0x7fe7ac155b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7b3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac098dd0 0x7fe7ac156050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7b3000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac0b36a0 con 0x7fe7ac0af100 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7b3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe7ac0b3520 con 0x7fe7ac098dd0 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7b3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe7ac0b3820 con 0x7fe7ac0cc830 2024-09-21T07:53:53.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 0x7fe7ac1f53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7abe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 0x7fe7ac1f53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55498/0 (socket says 172.21.15.88:55498) 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.142+0000 7fe7abe00640 1 -- 172.21.15.88:0/3370741198 learned_addr learned my addr 172.21.15.88:0/3370741198 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7b1000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac098dd0 0x7fe7ac156050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7ab400640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7ac0cc830 0x7fe7ac155b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7abe00640 1 -- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7ac0cc830 msgr2=0x7fe7ac155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7abe00640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7ac0cc830 0x7fe7ac155b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7abe00640 1 -- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac098dd0 msgr2=0x7fe7ac156050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7abe00640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac098dd0 0x7fe7ac156050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7abe00640 1 -- 172.21.15.88:0/3370741198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7ac098be0 con 0x7fe7ac0af100 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7b1000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac098dd0 0x7fe7ac156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:53.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7abe00640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 0x7fe7ac1f53e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c420 tx=0x7fe79c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:53.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c017070 con 0x7fe7ac0af100 2024-09-21T07:53:53.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe79c0035f0 con 0x7fe7ac0af100 2024-09-21T07:53:53.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe79c003a40 con 0x7fe7ac0af100 2024-09-21T07:53:53.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.143+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7ac15fa40 con 0x7fe7ac0af100 2024-09-21T07:53:53.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.144+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7ac15fce0 con 0x7fe7ac0af100 2024-09-21T07:53:53.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.145+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7ac0ca540 con 0x7fe7ac0af100 2024-09-21T07:53:53.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.146+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fe79c01a070 con 0x7fe7ac0af100 2024-09-21T07:53:53.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.147+0000 7fe7a3e00640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe78c079280 0x7fe78c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:53.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.148+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(136..136 src has 1..136) ==== 5443+0+0 (secure 0 0 0) 0x7fe79c09ad60 con 0x7fe7ac0af100 2024-09-21T07:53:53.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.148+0000 7fe7ab400640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe78c079280 0x7fe78c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:53.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.148+0000 7fe7ab400640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe78c079280 0x7fe78c07b790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe798034000 tx=0x7fe798004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:53.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.152+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe7ac0ca540 con 0x7fe7ac0af100 2024-09-21T07:53:53.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.338+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fe7ac0b3d40 con 0x7fe7ac0af100 2024-09-21T07:53:53.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.892+0000 7fe7a3e00640 1 -- 172.21.15.88:0/3370741198 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v137) ==== 169+0+0 (secure 0 0 0) 0x7fe79c0626b0 con 0x7fe7ac0af100 2024-09-21T07:53:53.894 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:53:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.896+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe78c079280 msgr2=0x7fe78c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.896+0000 7fe7b3000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe78c079280 0x7fe78c07b790 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fe798034000 tx=0x7fe798004730 comp rx=0 tx=0).stop 2024-09-21T07:53:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.896+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 msgr2=0x7fe7ac1f53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:53.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.896+0000 7fe7b3000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 0x7fe7ac1f53e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00c420 tx=0x7fe79c00c8e0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 shutdown_connections 2024-09-21T07:53:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe78c079280 0x7fe78c07b790 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe7ac0cc830 0x7fe7ac155b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe7ac098dd0 0x7fe7ac156050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 --2- 172.21.15.88:0/3370741198 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe7ac0af100 0x7fe7ac1f53e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:53.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 >> 172.21.15.88:0/3370741198 conn(0x7fe7ac0b2540 msgr2=0x7fe7ac0b0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:53.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 shutdown_connections 2024-09-21T07:53:53.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:53.897+0000 7fe7b3000640 1 -- 172.21.15.88:0/3370741198 wait complete. 2024-09-21T07:53:53.923 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:53:54.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 -- 172.21.15.88:0/1165167442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5634004800 msgr2=0x7f5634004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 --2- 172.21.15.88:0/1165167442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5634004800 0x7f5634004c10 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f5638009b50 tx=0x7f5634005830 comp rx=0 tx=0).stop 2024-09-21T07:53:54.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 -- 172.21.15.88:0/1165167442 shutdown_connections 2024-09-21T07:53:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 --2- 172.21.15.88:0/1165167442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5634004800 0x7f5634004c10 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 -- 172.21.15.88:0/1165167442 >> 172.21.15.88:0/1165167442 conn(0x7f565006d3d0 msgr2=0x7f56500b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 -- 172.21.15.88:0/1165167442 shutdown_connections 2024-09-21T07:53:54.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.143+0000 7f5657a00640 1 -- 172.21.15.88:0/1165167442 wait complete. 2024-09-21T07:53:54.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.145+0000 7f5657a00640 1 Processor -- start 2024-09-21T07:53:54.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.145+0000 7f5657a00640 1 -- start start 2024-09-21T07:53:54.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5657a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5634004800 0x7f56500ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5657a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 0x7f56500b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5657a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56500b17f0 0x7f56501f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5657a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f565015d240 con 0x7f56500aec90 2024-09-21T07:53:54.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5657a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5650157320 con 0x7f56500b17f0 2024-09-21T07:53:54.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5657a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f565015d3c0 con 0x7f5634004800 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5647e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 0x7f56500b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5647e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 0x7f56500b12b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55520/0 (socket says 172.21.15.88:55520) 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5647e00640 1 -- 172.21.15.88:0/407692188 learned_addr learned my addr 172.21.15.88:0/407692188 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5655000640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5634004800 0x7f56500ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5655a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56500b17f0 0x7f56501f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5647e00640 1 -- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5634004800 msgr2=0x7f56500ae750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.147+0000 7f5647e00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5634004800 0x7f56500ae750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f5647e00640 1 -- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56500b17f0 msgr2=0x7f56501f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f5647e00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56500b17f0 0x7f56501f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f5647e00640 1 -- 172.21.15.88:0/407692188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56500b1c00 con 0x7f56500aec90 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f5647e00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 0x7f56500b12b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5638009a70 tx=0x7f56380336f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638038040 con 0x7f56500aec90 2024-09-21T07:53:54.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56380339d0 con 0x7f56500aec90 2024-09-21T07:53:54.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5638033e20 con 0x7f56500aec90 2024-09-21T07:53:54.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56500ad1c0 con 0x7f56500aec90 2024-09-21T07:53:54.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.148+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56500ad780 con 0x7f56500aec90 2024-09-21T07:53:54.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.149+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56500afa70 con 0x7f56500aec90 2024-09-21T07:53:54.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.151+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f5638043020 con 0x7f56500aec90 2024-09-21T07:53:54.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.151+0000 7f564d600640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f562c079250 0x7f562c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.152+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7f56380bb910 con 0x7f56500aec90 2024-09-21T07:53:54.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.152+0000 7f5655000640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f562c079250 0x7f562c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.152+0000 7f5655000640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f562c079250 0x7f562c07b760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5640000fd0 tx=0x7f56400067d0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:54.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.156+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f563804a020 con 0x7f56500aec90 2024-09-21T07:53:54.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.353+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f56500b3660 con 0x7f56500aec90 2024-09-21T07:53:54.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.358+0000 7f564d600640 1 -- 172.21.15.88:0/407692188 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v47) ==== 71+0+0 (secure 0 0 0) 0x7f56500b1c00 con 0x7f56500aec90 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f562c079250 msgr2=0x7f562c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f562c079250 0x7f562c07b760 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f5640000fd0 tx=0x7f56400067d0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 msgr2=0x7f56500b12b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 0x7f56500b12b0 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f5638009a70 tx=0x7f56380336f0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 shutdown_connections 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f562c079250 0x7f562c07b760 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56500b17f0 0x7f56501f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56500aec90 0x7f56500b12b0 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 --2- 172.21.15.88:0/407692188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5634004800 0x7f56500ae750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 >> 172.21.15.88:0/407692188 conn(0x7f565006d3d0 msgr2=0x7f56500ccc70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 shutdown_connections 2024-09-21T07:53:54.364 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.360+0000 7f5657a00640 1 -- 172.21.15.88:0/407692188 wait complete. 2024-09-21T07:53:54.385 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol' 2024-09-21T07:53:54.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 -- 172.21.15.88:0/3941001087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708072e60 msgr2=0x7f8708073270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 --2- 172.21.15.88:0/3941001087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708072e60 0x7f8708073270 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f86f4009900 tx=0x7f86f402a5b0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 -- 172.21.15.88:0/3941001087 shutdown_connections 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 --2- 172.21.15.88:0/3941001087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708072e60 0x7f8708073270 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 --2- 172.21.15.88:0/3941001087 >> v2:172.21.15.175:3300/0 conn(0x7f87080af050 0x7f87080ae080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 -- 172.21.15.88:0/3941001087 >> 172.21.15.88:0/3941001087 conn(0x7f87080b1e40 msgr2=0x7f87080b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 -- 172.21.15.88:0/3941001087 shutdown_connections 2024-09-21T07:53:54.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.602+0000 7f870d600640 1 -- 172.21.15.88:0/3941001087 wait complete. 2024-09-21T07:53:54.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.604+0000 7f870d600640 1 Processor -- start 2024-09-21T07:53:54.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.605+0000 7f870d600640 1 -- start start 2024-09-21T07:53:54.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f870d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 0x7f870813b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f870d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8708072e60 0x7f8708162d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f870d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87080af050 0x7f8708153fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f870d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87081658b0 con 0x7f8708070000 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f870d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8708165730 con 0x7f8708072e60 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f870d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8708165a30 con 0x7f87080af050 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f8707e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 0x7f870813b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f8707e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 0x7f870813b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55530/0 (socket says 172.21.15.88:55530) 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.606+0000 7f8707e00640 1 -- 172.21.15.88:0/345839901 learned_addr learned my addr 172.21.15.88:0/345839901 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707400640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8708072e60 0x7f8708162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f870cc00640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87080af050 0x7f8708153fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707e00640 1 -- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87080af050 msgr2=0x7f8708153fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707e00640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87080af050 0x7f8708153fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707e00640 1 -- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8708072e60 msgr2=0x7f8708162d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707e00640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8708072e60 0x7f8708162d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707e00640 1 -- 172.21.15.88:0/345839901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870813f110 con 0x7f8708070000 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f870cc00640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87080af050 0x7f8708153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f8707e00640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 0x7f870813b9c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b410 tx=0x7f86f800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8015020 con 0x7f8708070000 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f86f8011940 con 0x7f8708070000 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87081f3b30 con 0x7f8708070000 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.607+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86f8011d90 con 0x7f8708070000 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.608+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87081f40f0 con 0x7f8708070000 2024-09-21T07:53:54.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.609+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f870815dee0 con 0x7f8708070000 2024-09-21T07:53:54.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.610+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f86f8018020 con 0x7f8708070000 2024-09-21T07:53:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.611+0000 7f86ffe00640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f86e8079280 0x7f86e807b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.611+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7f86f809b610 con 0x7f8708070000 2024-09-21T07:53:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.611+0000 7f8707400640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f86e8079280 0x7f86e807b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:54.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.611+0000 7f8707400640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f86e8079280 0x7f86e807b790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86f4033000 tx=0x7f86f4004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:54.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.615+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f870815dee0 con 0x7f8708070000 2024-09-21T07:53:54.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.872+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]} v 0) -- 0x7f8708073270 con 0x7f8708070000 2024-09-21T07:53:54.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.894+0000 7f86ffe00640 1 -- 172.21.15.88:0/345839901 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_ops_on_nonexistent_vol"]}]=0 v440) ==== 151+0+0 (secure 0 0 0) 0x7f86f801f020 con 0x7f8708070000 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f86e8079280 msgr2=0x7f86e807b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f86e8079280 0x7f86e807b790 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f86f4033000 tx=0x7f86f4004730 comp rx=0 tx=0).stop 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 msgr2=0x7f870813b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 0x7f870813b9c0 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f86f800b410 tx=0x7f86f800b8d0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 shutdown_connections 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f86e8079280 0x7f86e807b790 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f87080af050 0x7f8708153fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8708072e60 0x7f8708162d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 --2- 172.21.15.88:0/345839901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8708070000 0x7f870813b9c0 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.896+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 >> 172.21.15.88:0/345839901 conn(0x7f87080b1e40 msgr2=0x7f87081550d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:54.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.897+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 shutdown_connections 2024-09-21T07:53:54.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:54.897+0000 7f870d600640 1 -- 172.21.15.88:0/345839901 wait complete. 2024-09-21T07:53:54.924 INFO:tasks.cephfs_test_runner:test_subvolume_ops_on_nonexistent_vol (tasks.cephfs.test_volumes.TestMisc) ... ok 2024-09-21T07:53:54.924 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- rather than going through a full upgrade cycle, 2024-09-21T07:53:54.924 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_legacy_to_v1 (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:53:54.924 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:53:55.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 -- 172.21.15.88:0/4224253500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48073c10 msgr2=0x7efc4806f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 --2- 172.21.15.88:0/4224253500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48073c10 0x7efc4806f0a0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efc38005fd0 tx=0x7efc3802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 -- 172.21.15.88:0/4224253500 shutdown_connections 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 --2- 172.21.15.88:0/4224253500 >> v2:172.21.15.175:3300/0 conn(0x7efc4806f5e0 0x7efc480b31e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 --2- 172.21.15.88:0/4224253500 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48073c10 0x7efc4806f0a0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 -- 172.21.15.88:0/4224253500 >> 172.21.15.88:0/4224253500 conn(0x7efc4806d730 msgr2=0x7efc480b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 -- 172.21.15.88:0/4224253500 shutdown_connections 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.142+0000 7efc50800640 1 -- 172.21.15.88:0/4224253500 wait complete. 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.144+0000 7efc50800640 1 Processor -- start 2024-09-21T07:53:55.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.144+0000 7efc50800640 1 -- start start 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc50800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 0x7efc48153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc50800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480af630 0x7efc481ea040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc50800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48154470 0x7efc481ea580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc50800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc480b3c50 con 0x7efc4806f5e0 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc50800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efc480b3ad0 con 0x7efc480af630 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc50800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efc480b3dd0 con 0x7efc48154470 2024-09-21T07:53:55.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 0x7efc48153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 0x7efc48153e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55532/0 (socket says 172.21.15.88:55532) 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 -- 172.21.15.88:0/2279658852 learned_addr learned my addr 172.21.15.88:0/2279658852 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 -- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48154470 msgr2=0x7efc481ea580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48154470 0x7efc481ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 -- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480af630 msgr2=0x7efc481ea040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4d400640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480af630 0x7efc481ea040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480af630 0x7efc481ea040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4de00640 1 -- 172.21.15.88:0/2279658852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc481f8d50 con 0x7efc4806f5e0 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.146+0000 7efc4d400640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480af630 0x7efc481ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.147+0000 7efc4de00640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 0x7efc48153e10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c3c0 tx=0x7efc3c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.147+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c017070 con 0x7efc4806f5e0 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.147+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efc3c0035f0 con 0x7efc4806f5e0 2024-09-21T07:53:55.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.147+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc3c003a80 con 0x7efc4806f5e0 2024-09-21T07:53:55.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.147+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc481f9030 con 0x7efc4806f5e0 2024-09-21T07:53:55.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.147+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc481f95f0 con 0x7efc4806f5e0 2024-09-21T07:53:55.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.148+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc480b4280 con 0x7efc4806f5e0 2024-09-21T07:53:55.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.149+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7efc3c01a070 con 0x7efc4806f5e0 2024-09-21T07:53:55.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.150+0000 7efc42000640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc2c079250 0x7efc2c07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.150+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7efc3c09ac20 con 0x7efc4806f5e0 2024-09-21T07:53:55.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.151+0000 7efc4d400640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc2c079250 0x7efc2c07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.151+0000 7efc4d400640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc2c079250 0x7efc2c07b760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc38033000 tx=0x7efc38004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:55.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.155+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7efc3c062750 con 0x7efc4806f5e0 2024-09-21T07:53:55.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.343+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efc480adc90 con 0x7efc4806f5e0 2024-09-21T07:53:55.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.344+0000 7efc42000640 1 -- 172.21.15.88:0/2279658852 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) ==== 74+0+13098 (secure 0 0 0) 0x7efc3c062050 con 0x7efc4806f5e0 2024-09-21T07:53:55.346 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:55.346 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":137,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:53:53.887503+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":133,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":130,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/2904821684":"2024-09-22T07:53:50.539635+0000","172.21.15.88:6833/2904821684":"2024-09-22T07:53:50.539635+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:53:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc2c079250 msgr2=0x7efc2c07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc2c079250 0x7efc2c07b760 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7efc38033000 tx=0x7efc38004730 comp rx=0 tx=0).stop 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 msgr2=0x7efc48153e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 0x7efc48153e10 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7efc3c00c3c0 tx=0x7efc3c00c880 comp rx=0 tx=0).stop 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 shutdown_connections 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efc2c079250 0x7efc2c07b760 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efc48154470 0x7efc481ea580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efc480af630 0x7efc481ea040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 --2- 172.21.15.88:0/2279658852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc4806f5e0 0x7efc48153e10 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 >> 172.21.15.88:0/2279658852 conn(0x7efc4806d730 msgr2=0x7efc480ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 shutdown_connections 2024-09-21T07:53:55.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.346+0000 7efc50800640 1 -- 172.21.15.88:0/2279658852 wait complete. 2024-09-21T07:53:55.373 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:53:55.374 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.594+0000 7fd467a00640 1 -- 172.21.15.88:0/559374720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4600b4f00 msgr2=0x7fd4600704a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.594+0000 7fd467a00640 1 --2- 172.21.15.88:0/559374720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4600b4f00 0x7fd4600704a0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7fd454005fd0 tx=0x7fd45402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.595+0000 7fd467a00640 1 -- 172.21.15.88:0/559374720 shutdown_connections 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.595+0000 7fd467a00640 1 --2- 172.21.15.88:0/559374720 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600ad3d0 0x7fd4600ad7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.595+0000 7fd467a00640 1 --2- 172.21.15.88:0/559374720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd4600b4f00 0x7fd4600704a0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.595+0000 7fd467a00640 1 -- 172.21.15.88:0/559374720 >> 172.21.15.88:0/559374720 conn(0x7fd46006d710 msgr2=0x7fd4600b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:55.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.595+0000 7fd467a00640 1 -- 172.21.15.88:0/559374720 shutdown_connections 2024-09-21T07:53:55.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.595+0000 7fd467a00640 1 -- 172.21.15.88:0/559374720 wait complete. 2024-09-21T07:53:55.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.597+0000 7fd467a00640 1 Processor -- start 2024-09-21T07:53:55.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.597+0000 7fd467a00640 1 -- start start 2024-09-21T07:53:55.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.598+0000 7fd467a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600ad3d0 0x7fd4600cf5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.598+0000 7fd467a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4600b4f00 0x7fd460152f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.598+0000 7fd467a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 0x7fd4601534a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.598+0000 7fd467a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd46006f0f0 con 0x7fd460160490 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.598+0000 7fd467a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd46006ef70 con 0x7fd4600b4f00 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.598+0000 7fd467a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd46006f270 con 0x7fd4600ad3d0 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 0x7fd4601534a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 0x7fd4601534a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55558/0 (socket says 172.21.15.88:55558) 2024-09-21T07:53:55.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 -- 172.21.15.88:0/120263935 learned_addr learned my addr 172.21.15.88:0/120263935 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:55.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465000640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600ad3d0 0x7fd4600cf5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd45fe00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4600b4f00 0x7fd460152f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 -- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600ad3d0 msgr2=0x7fd4600cf5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600ad3d0 0x7fd4600cf5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 -- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4600b4f00 msgr2=0x7fd460152f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4600b4f00 0x7fd460152f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 -- 172.21.15.88:0/120263935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd46014d3f0 con 0x7fd460160490 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.599+0000 7fd465a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 0x7fd4601534a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd45800e990 tx=0x7fd45800ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.600+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458004d20 con 0x7fd460160490 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.600+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd45800caa0 con 0x7fd460160490 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.600+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd46014d6d0 con 0x7fd460160490 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.600+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd458005020 con 0x7fd460160490 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.600+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd46014dc90 con 0x7fd460160490 2024-09-21T07:53:55.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.601+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd46006f6c0 con 0x7fd460160490 2024-09-21T07:53:55.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.602+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fd4580051c0 con 0x7fd460160490 2024-09-21T07:53:55.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.603+0000 7fd45cc00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd440079280 0x7fd44007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:55.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.603+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7fd45809bf40 con 0x7fd460160490 2024-09-21T07:53:55.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.603+0000 7fd465000640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd440079280 0x7fd44007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:55.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.604+0000 7fd465000640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd440079280 0x7fd44007b790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd454033000 tx=0x7fd454004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:55.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.608+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd458016030 con 0x7fd460160490 2024-09-21T07:53:55.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.794+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd46014e650 con 0x7fd460160490 2024-09-21T07:53:55.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.794+0000 7fd45cc00640 1 -- 172.21.15.88:0/120263935 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v137) ==== 93+0+10 (secure 0 0 0) 0x7fd458063570 con 0x7fd460160490 2024-09-21T07:53:55.796 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:53:55.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd440079280 msgr2=0x7fd44007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd440079280 0x7fd44007b790 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fd454033000 tx=0x7fd454004730 comp rx=0 tx=0).stop 2024-09-21T07:53:55.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 msgr2=0x7fd4601534a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:55.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 0x7fd4601534a0 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7fd45800e990 tx=0x7fd45800ee50 comp rx=0 tx=0).stop 2024-09-21T07:53:55.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 shutdown_connections 2024-09-21T07:53:55.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd440079280 0x7fd44007b790 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd460160490 0x7fd4601534a0 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd4600b4f00 0x7fd460152f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 --2- 172.21.15.88:0/120263935 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd4600ad3d0 0x7fd4600cf5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:55.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 >> 172.21.15.88:0/120263935 conn(0x7fd46006d710 msgr2=0x7fd4600b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:55.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 shutdown_connections 2024-09-21T07:53:55.800 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:55.797+0000 7fd467a00640 1 -- 172.21.15.88:0/120263935 wait complete. 2024-09-21T07:53:55.821 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 -- 172.21.15.88:0/3775744268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8310003a50 msgr2=0x7f8310003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 --2- 172.21.15.88:0/3775744268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8310003a50 0x7f8310003ec0 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f831c0016c0 tx=0x7f8310005830 comp rx=0 tx=0).stop 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 -- 172.21.15.88:0/3775744268 shutdown_connections 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 --2- 172.21.15.88:0/3775744268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8310003a50 0x7f8310003ec0 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 -- 172.21.15.88:0/3775744268 >> 172.21.15.88:0/3775744268 conn(0x7f832c06d730 msgr2=0x7f832c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 -- 172.21.15.88:0/3775744268 shutdown_connections 2024-09-21T07:53:56.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.041+0000 7f8330c00640 1 -- 172.21.15.88:0/3775744268 wait complete. 2024-09-21T07:53:56.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.044+0000 7f8330c00640 1 Processor -- start 2024-09-21T07:53:56.046 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.044+0000 7f8330c00640 1 -- start start 2024-09-21T07:53:56.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8330c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8310003a50 0x7f832c09bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8330c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 0x7f832c0cf170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8330c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c0cf6b0 0x7f832c09a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8330c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f832c0b2750 con 0x7f832c14f860 2024-09-21T07:53:56.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8330c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f832c15f6f0 con 0x7f8310003a50 2024-09-21T07:53:56.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8330c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f832c0b28d0 con 0x7f832c0cf6b0 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.045+0000 7f8329400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 0x7f832c0cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 0x7f832c0cf170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54308/0 (socket says 172.21.15.88:54308) 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 -- 172.21.15.88:0/1892736083 learned_addr learned my addr 172.21.15.88:0/1892736083 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329e00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8310003a50 0x7f832c09bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f832a800640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c0cf6b0 0x7f832c09a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 -- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c0cf6b0 msgr2=0x7f832c09a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c0cf6b0 0x7f832c09a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 -- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8310003a50 msgr2=0x7f832c09bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8310003a50 0x7f832c09bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 -- 172.21.15.88:0/1892736083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f832c0cfac0 con 0x7f832c14f860 2024-09-21T07:53:56.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.046+0000 7f8329400640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 0x7f832c0cf170 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8318011780 tx=0x7f8318011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.047+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318004230 con 0x7f832c14f860 2024-09-21T07:53:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.047+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f831800f940 con 0x7f832c14f860 2024-09-21T07:53:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.047+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f832c1f6bf0 con 0x7f832c14f860 2024-09-21T07:53:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.047+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83180044c0 con 0x7f832c14f860 2024-09-21T07:53:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.047+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f832c1f71b0 con 0x7f832c14f860 2024-09-21T07:53:56.050 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.048+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f832c09c4e0 con 0x7f832c14f860 2024-09-21T07:53:56.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.049+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f83180035f0 con 0x7f832c14f860 2024-09-21T07:53:56.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.050+0000 7f8322000640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f830c079280 0x7f830c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:56.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.050+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7f831809c030 con 0x7f832c14f860 2024-09-21T07:53:56.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.050+0000 7f8329e00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f830c079280 0x7f830c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:56.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.051+0000 7f8329e00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f830c079280 0x7f830c07b790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f831c032000 tx=0x7f831c02c040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:56.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.054+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f83180637b0 con 0x7f832c14f860 2024-09-21T07:53:56.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.310+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f832c15bd90 con 0x7f832c14f860 2024-09-21T07:53:56.904 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.902+0000 7f8322000640 1 -- 172.21.15.88:0/1892736083 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v442) ==== 152+0+0 (secure 0 0 0) 0x7f83180681e0 con 0x7f832c14f860 2024-09-21T07:53:56.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f830c079280 msgr2=0x7f830c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.907 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f830c079280 0x7f830c07b790 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f831c032000 tx=0x7f831c02c040 comp rx=0 tx=0).stop 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 msgr2=0x7f832c0cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 0x7f832c0cf170 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f8318011780 tx=0x7f8318011b80 comp rx=0 tx=0).stop 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 shutdown_connections 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f830c079280 0x7f830c07b790 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f832c0cf6b0 0x7f832c09a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f832c14f860 0x7f832c0cf170 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 --2- 172.21.15.88:0/1892736083 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8310003a50 0x7f832c09bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 >> 172.21.15.88:0/1892736083 conn(0x7f832c06d730 msgr2=0x7f832c150030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 shutdown_connections 2024-09-21T07:53:56.908 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:56.905+0000 7f8330c00640 1 -- 172.21.15.88:0/1892736083 wait complete. 2024-09-21T07:53:56.930 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.148+0000 7fc43dc00640 1 -- 172.21.15.88:0/3493220527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380aebc0 msgr2=0x7fc438152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.148+0000 7fc43dc00640 1 --2- 172.21.15.88:0/3493220527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380aebc0 0x7fc438152920 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7fc428005fd0 tx=0x7fc42802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.148+0000 7fc43dc00640 1 -- 172.21.15.88:0/3493220527 shutdown_connections 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.148+0000 7fc43dc00640 1 --2- 172.21.15.88:0/3493220527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380aebc0 0x7fc438152920 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.148+0000 7fc43dc00640 1 -- 172.21.15.88:0/3493220527 >> 172.21.15.88:0/3493220527 conn(0x7fc43806d730 msgr2=0x7fc4380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.149+0000 7fc43dc00640 1 -- 172.21.15.88:0/3493220527 shutdown_connections 2024-09-21T07:53:57.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.149+0000 7fc43dc00640 1 -- 172.21.15.88:0/3493220527 wait complete. 2024-09-21T07:53:57.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.151+0000 7fc43dc00640 1 Processor -- start 2024-09-21T07:53:57.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.151+0000 7fc43dc00640 1 -- start start 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.152+0000 7fc43dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4380aebc0 0x7fc4380cf600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.152+0000 7fc43dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 0x7fc438154fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.152+0000 7fc43dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4381f8a00 0x7fc4380b0cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.152+0000 7fc43dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4380b4360 con 0x7fc4380b2750 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.152+0000 7fc43dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc4380b41e0 con 0x7fc4380aebc0 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.152+0000 7fc43dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc4380b44e0 con 0x7fc4381f8a00 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 0x7fc438154fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 0x7fc438154fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54326/0 (socket says 172.21.15.88:54326) 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 -- 172.21.15.88:0/1537528835 learned_addr learned my addr 172.21.15.88:0/1537528835 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:57.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436a00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4380aebc0 0x7fc4380cf600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 -- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4381f8a00 msgr2=0x7fc4380b0cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4381f8a00 0x7fc4380b0cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 -- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4380aebc0 msgr2=0x7fc4380cf600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4380aebc0 0x7fc4380cf600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 -- 172.21.15.88:0/1537528835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc438155500 con 0x7fc4380b2750 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436a00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4380aebc0 0x7fc4380cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.153+0000 7fc436000640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 0x7fc438154fc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc42400c3a0 tx=0x7fc42400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.154+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424017070 con 0x7fc4380b2750 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.154+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc4240053c0 con 0x7fc4380b2750 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.154+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc424003850 con 0x7fc4380b2750 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.154+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4381f8fe0 con 0x7fc4380b2750 2024-09-21T07:53:57.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.154+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4381f95a0 con 0x7fc4380b2750 2024-09-21T07:53:57.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.155+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4380b4990 con 0x7fc4380b2750 2024-09-21T07:53:57.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.156+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fc42401a070 con 0x7fc4380b2750 2024-09-21T07:53:57.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.157+0000 7fc42ea00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc418079250 0x7fc41807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.157+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7fc42409abc0 con 0x7fc4380b2750 2024-09-21T07:53:57.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.158+0000 7fc436a00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc418079250 0x7fc41807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.158+0000 7fc436a00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc418079250 0x7fc41807b760 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc428033000 tx=0x7fc428004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.162+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc424062770 con 0x7fc4380b2750 2024-09-21T07:53:57.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.349+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fc438070020 con 0x7fc4380b2750 2024-09-21T07:53:57.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.349+0000 7fc42ea00640 1 -- 172.21.15.88:0/1537528835 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v47) ==== 59+0+2294 (secure 0 0 0) 0x7fc424062070 con 0x7fc4380b2750 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:53:57.352 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:53:57.353 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:53:57.354 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:53:57.354 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:53:57.354 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc418079250 msgr2=0x7fc41807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc418079250 0x7fc41807b760 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fc428033000 tx=0x7fc428004730 comp rx=0 tx=0).stop 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 msgr2=0x7fc438154fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 0x7fc438154fc0 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7fc42400c3a0 tx=0x7fc42400c860 comp rx=0 tx=0).stop 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 shutdown_connections 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc418079250 0x7fc41807b760 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc4381f8a00 0x7fc4380b0cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4380b2750 0x7fc438154fc0 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 --2- 172.21.15.88:0/1537528835 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc4380aebc0 0x7fc4380cf600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 >> 172.21.15.88:0/1537528835 conn(0x7fc43806d730 msgr2=0x7fc438151d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 shutdown_connections 2024-09-21T07:53:57.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.352+0000 7fc43dc00640 1 -- 172.21.15.88:0/1537528835 wait complete. 2024-09-21T07:53:57.376 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:53:57.420 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:57.425 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:53:57.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.597+0000 7faa5f800640 1 -- 172.21.15.88:0/807855507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 msgr2=0x7faa58073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.597+0000 7faa5f800640 1 --2- 172.21.15.88:0/807855507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa58073540 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7faa58072bd0 tx=0x7faa4002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 -- 172.21.15.88:0/807855507 shutdown_connections 2024-09-21T07:53:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 --2- 172.21.15.88:0/807855507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa58073a80 0x7faa58160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 --2- 172.21.15.88:0/807855507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa58073540 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 --2- 172.21.15.88:0/807855507 >> v2:172.21.15.175:3300/0 conn(0x7faa58145550 0x7faa5806fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 -- 172.21.15.88:0/807855507 >> 172.21.15.88:0/807855507 conn(0x7faa5806d1c0 msgr2=0x7faa5806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 -- 172.21.15.88:0/807855507 shutdown_connections 2024-09-21T07:53:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.598+0000 7faa5f800640 1 -- 172.21.15.88:0/807855507 wait complete. 2024-09-21T07:53:57.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.600+0000 7faa5f800640 1 Processor -- start 2024-09-21T07:53:57.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.600+0000 7faa5f800640 1 -- start start 2024-09-21T07:53:57.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5f800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa58073a80 0x7faa581e9690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5f800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa58145550 0x7faa5815e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa5812d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa58160c30 con 0x7faa58145a40 2024-09-21T07:53:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5f800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faa58160ab0 con 0x7faa58145550 2024-09-21T07:53:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5f800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faa58160db0 con 0x7faa58073a80 2024-09-21T07:53:57.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa5812d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa5812d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54350/0 (socket says 172.21.15.88:54350) 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 -- 172.21.15.88:0/2453620929 learned_addr learned my addr 172.21.15.88:0/2453620929 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5ce00640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa58073a80 0x7faa581e9690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa4fe00640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa58145550 0x7faa5815e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 -- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa58073a80 msgr2=0x7faa581e9690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa58073a80 0x7faa581e9690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 -- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa58145550 msgr2=0x7faa5815e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa58145550 0x7faa5815e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.602+0000 7faa5d800640 1 -- 172.21.15.88:0/2453620929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa5815efc0 con 0x7faa58145a40 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.603+0000 7faa5d800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa5812d4f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7faa5000e960 tx=0x7faa5000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.603+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50004d20 con 0x7faa58145a40 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.603+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faa5000ca60 con 0x7faa58145a40 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.603+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa50005020 con 0x7faa58145a40 2024-09-21T07:53:57.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.603+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa5815f2a0 con 0x7faa58145a40 2024-09-21T07:53:57.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.603+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa581e7f40 con 0x7faa58145a40 2024-09-21T07:53:57.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.604+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa581e67c0 con 0x7faa58145a40 2024-09-21T07:53:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.605+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7faa500051c0 con 0x7faa58145a40 2024-09-21T07:53:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.606+0000 7faa55600640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faa34079250 0x7faa3407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.606+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7faa5009c010 con 0x7faa58145a40 2024-09-21T07:53:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.607+0000 7faa5ce00640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faa34079250 0x7faa3407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:57.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.607+0000 7faa5ce00640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faa34079250 0x7faa3407b760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faa4800bb50 tx=0x7faa4800b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:57.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.611+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faa581e67c0 con 0x7faa58145a40 2024-09-21T07:53:57.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.800+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7faa5815ed50 con 0x7faa58145a40 2024-09-21T07:53:57.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.805+0000 7faa55600640 1 -- 172.21.15.88:0/2453620929 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7faa50016030 con 0x7faa58145a40 2024-09-21T07:53:57.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faa34079250 msgr2=0x7faa3407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faa34079250 0x7faa3407b760 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7faa4800bb50 tx=0x7faa4800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 msgr2=0x7faa5812d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa5812d4f0 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7faa5000e960 tx=0x7faa5000ee20 comp rx=0 tx=0).stop 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 shutdown_connections 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faa34079250 0x7faa3407b760 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa58145a40 0x7faa5812d4f0 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faa58145550 0x7faa5815e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 --2- 172.21.15.88:0/2453620929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faa58073a80 0x7faa581e9690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 >> 172.21.15.88:0/2453620929 conn(0x7faa5806d1c0 msgr2=0x7faa5813e900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 shutdown_connections 2024-09-21T07:53:57.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:57.808+0000 7faa5f800640 1 -- 172.21.15.88:0/2453620929 wait complete. 2024-09-21T07:53:57.837 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:53:57.837 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:57.932 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:53:58.031 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:53:58.031 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:58.031 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:58.067 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:58.067 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:58.067 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:58.104 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:53:58.104 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:53:58.104 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:58.142 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:53:58.142 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:53:58.142 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:53:58.180 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:53:58.180 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:53:58.180 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:53:58.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.400+0000 7face3800640 1 -- 172.21.15.88:0/1372514714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc4004800 msgr2=0x7facc4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.400+0000 7face3800640 1 --2- 172.21.15.88:0/1372514714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facc4004800 0x7facc4004c10 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7facdc071180 tx=0x7facc4005830 comp rx=0 tx=0).stop 2024-09-21T07:53:58.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.400+0000 7face1800640 1 -- 172.21.15.88:0/1372514714 reap_dead start 2024-09-21T07:53:58.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.400+0000 7face3800640 1 -- 172.21.15.88:0/1372514714 shutdown_connections 2024-09-21T07:53:58.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.400+0000 7face3800640 1 -- 172.21.15.88:0/1372514714 >> 172.21.15.88:0/1372514714 conn(0x7facdc0d0b50 msgr2=0x7facdc0d0f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:58.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.401+0000 7face3800640 1 -- 172.21.15.88:0/1372514714 shutdown_connections 2024-09-21T07:53:58.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.401+0000 7face3800640 1 -- 172.21.15.88:0/1372514714 wait complete. 2024-09-21T07:53:58.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.403+0000 7face3800640 1 Processor -- start 2024-09-21T07:53:58.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.403+0000 7face3800640 1 -- start start 2024-09-21T07:53:58.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.404+0000 7face3800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc4004800 0x7facdc159090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.404+0000 7face3800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 0x7facdc1595d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7face3800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc1f3470 0x7facdc159b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7face3800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7facdc071e90 con 0x7facdc13aa90 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7face3800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7facdc071d10 con 0x7facc4004800 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7face3800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7facdc072010 con 0x7facdc1f3470 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 0x7facdc1595d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 0x7facdc1595d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54370/0 (socket says 172.21.15.88:54370) 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 -- 172.21.15.88:0/1860847620 learned_addr learned my addr 172.21.15.88:0/1860847620 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7face0e00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc4004800 0x7facdc159090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7face1800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc1f3470 0x7facdc159b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 -- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc1f3470 msgr2=0x7facdc159b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc1f3470 0x7facdc159b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 -- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc4004800 msgr2=0x7facdc159090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc4004800 0x7facdc159090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 -- 172.21.15.88:0/1860847620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7facdc0b0430 con 0x7facdc13aa90 2024-09-21T07:53:58.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.405+0000 7facdbe00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 0x7facdc1595d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7facc8000c00 tx=0x7facc8034700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:58.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.406+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8040020 con 0x7facdc13aa90 2024-09-21T07:53:58.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.406+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7facc80348f0 con 0x7facdc13aa90 2024-09-21T07:53:58.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.406+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facc8038040 con 0x7facdc13aa90 2024-09-21T07:53:58.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.406+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7facdc157ff0 con 0x7facdc13aa90 2024-09-21T07:53:58.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.406+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7facdc1585b0 con 0x7facdc13aa90 2024-09-21T07:53:58.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.407+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7facdc06d1b0 con 0x7facdc13aa90 2024-09-21T07:53:58.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.408+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7facc8043020 con 0x7facdc13aa90 2024-09-21T07:53:58.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.410+0000 7facd8c00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7facbc079250 0x7facbc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.410+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7facc80bbeb0 con 0x7facdc13aa90 2024-09-21T07:53:58.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.410+0000 7face0e00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7facbc079250 0x7facbc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.411+0000 7face0e00640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7facbc079250 0x7facbc07b760 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7facdc142fe0 tx=0x7facd0006cf0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:58.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.414+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7facc804a030 con 0x7facdc13aa90 2024-09-21T07:53:58.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.635+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7facdc0cd390 con 0x7facdc13aa90 2024-09-21T07:53:58.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.636+0000 7facd8c00640 1 -- 172.21.15.88:0/1860847620 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 92 v92) ==== 94+0+3917 (secure 0 0 0) 0x7facc8083520 con 0x7facdc13aa90 2024-09-21T07:53:58.638 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:53:58.638 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":92,"btime":"2024-09-21T07:53:51:541981+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6722,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2058248277","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2058248277},{"type":"v1","addr":"172.21.15.175:6801","nonce":2058248277}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":84},{"gid":6731,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/3905088679","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":3905088679},{"type":"v1","addr":"172.21.15.175:6835","nonce":3905088679}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":7935,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/2840865687","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":2840865687},{"type":"v1","addr":"172.21.15.88:6834","nonce":2840865687}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":85},{"gid":8310,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/2315062319","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":2315062319},{"type":"v1","addr":"172.21.15.88:6835","nonce":2315062319}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":92}],"filesystems":[]} 2024-09-21T07:53:58.639 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 92 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.638+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7facbc079250 msgr2=0x7facbc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.638+0000 7face3800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7facbc079250 0x7facbc07b760 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7facdc142fe0 tx=0x7facd0006cf0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 msgr2=0x7facdc1595d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 0x7facdc1595d0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7facc8000c00 tx=0x7facc8034700 comp rx=0 tx=0).stop 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 shutdown_connections 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7facbc079250 0x7facbc07b760 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7facdc1f3470 0x7facdc159b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7facdc13aa90 0x7facdc1595d0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 --2- 172.21.15.88:0/1860847620 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7facc4004800 0x7facdc159090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 >> 172.21.15.88:0/1860847620 conn(0x7facdc0d0b50 msgr2=0x7facdc073900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:58.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 shutdown_connections 2024-09-21T07:53:58.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.639+0000 7face3800640 1 -- 172.21.15.88:0/1860847620 wait complete. 2024-09-21T07:53:58.663 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:53:58.663 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:53:58.664 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:53:58.664 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:58.664 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:53:58.664 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:53:58.664 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:53:58.664 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:58.664 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:53:58.664 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:53:58.665 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:53:58.665 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:58.665 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:53:58.665 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:53:58.665 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:53:58.665 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:53:58.687 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:53:58.687 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:53:58.689 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:53:58.690 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:53:58.690 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:53:58.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:53:58.691 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:53:58.691 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:53:58.733 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:53:58.733 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:58.735 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:53:58.735 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:58.736 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:53:58.736 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:58.737 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:53:58.737 DEBUG:teuthology.parallel:result is None 2024-09-21T07:53:58.738 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:53:58.930 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:53:58.930 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:53:58.934 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:53:58.937 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:53:58.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 -- 172.21.15.88:0/2446725852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e640715d0 msgr2=0x7f5e640719e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 --2- 172.21.15.88:0/2446725852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e640715d0 0x7f5e640719e0 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009830 tx=0x7f5e5c02a530 comp rx=0 tx=0).stop 2024-09-21T07:53:58.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 -- 172.21.15.88:0/2446725852 shutdown_connections 2024-09-21T07:53:58.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 --2- 172.21.15.88:0/2446725852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e640715d0 0x7f5e640719e0 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.975 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 -- 172.21.15.88:0/2446725852 >> 172.21.15.88:0/2446725852 conn(0x7f5e6406d1c0 msgr2=0x7f5e6406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:58.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 -- 172.21.15.88:0/2446725852 shutdown_connections 2024-09-21T07:53:58.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.973+0000 7f5e69a00640 1 -- 172.21.15.88:0/2446725852 wait complete. 2024-09-21T07:53:58.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.975+0000 7f5e69a00640 1 Processor -- start 2024-09-21T07:53:58.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.975+0000 7f5e69a00640 1 -- start start 2024-09-21T07:53:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e6406fc30 0x7f5e64157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e640715d0 0x7f5e64157cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 0x7f5e641fd1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6406eb40 con 0x7f5e6415fad0 2024-09-21T07:53:58.979 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5e6406e9c0 con 0x7f5e6406fc30 2024-09-21T07:53:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5e6406ecc0 con 0x7f5e640715d0 2024-09-21T07:53:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 0x7f5e641fd1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 0x7f5e641fd1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54406/0 (socket says 172.21.15.88:54406) 2024-09-21T07:53:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69000640 1 -- 172.21.15.88:0/3031642699 learned_addr learned my addr 172.21.15.88:0/3031642699 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:53:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e63e00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e6406fc30 0x7f5e64157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e63400640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e640715d0 0x7f5e64157cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69000640 1 -- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e640715d0 msgr2=0x7f5e64157cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69000640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e640715d0 0x7f5e64157cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.977+0000 7f5e69000640 1 -- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e6406fc30 msgr2=0x7f5e64157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e69000640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e6406fc30 0x7f5e64157790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:58.981 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e69000640 1 -- 172.21.15.88:0/3031642699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e640b34a0 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e69000640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 0x7f5e641fd1a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009950 tx=0x7f5e5c033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c031b50 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e5c031de0 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e5c005240 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e640c5e20 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.978+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e640c63e0 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.979+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e640b3210 con 0x7f5e6415fad0 2024-09-21T07:53:58.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.980+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f5e5c0053e0 con 0x7f5e6415fad0 2024-09-21T07:53:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.981+0000 7f5e5be00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e440792a0 0x7f5e4407b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:53:58.983 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.982+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(137..137 src has 1..137) ==== 4971+0+0 (secure 0 0 0) 0x7f5e5c0bc6d0 con 0x7f5e6415fad0 2024-09-21T07:53:58.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.982+0000 7f5e63e00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e440792a0 0x7f5e4407b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:53:58.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.982+0000 7f5e63e00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e440792a0 0x7f5e4407b7b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e54001770 tx=0x7f5e54001e70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:53:58.988 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:58.986+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e5c03f030 con 0x7f5e6415fad0 2024-09-21T07:53:59.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.186+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f5e64159d90 con 0x7f5e6415fad0 2024-09-21T07:53:59.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.926+0000 7f5e5be00640 1 -- 172.21.15.88:0/3031642699 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v139) ==== 97+0+0 (secure 0 0 0) 0x7f5e5c031770 con 0x7f5e6415fad0 2024-09-21T07:53:59.928 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:53:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e440792a0 msgr2=0x7f5e4407b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e440792a0 0x7f5e4407b7b0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e54001770 tx=0x7f5e54001e70 comp rx=0 tx=0).stop 2024-09-21T07:53:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 msgr2=0x7f5e641fd1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:53:59.931 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 0x7f5e641fd1a0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c009950 tx=0x7f5e5c033ba0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 shutdown_connections 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e440792a0 0x7f5e4407b7b0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5e640715d0 0x7f5e64157cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e6415fad0 0x7f5e641fd1a0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 --2- 172.21.15.88:0/3031642699 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5e6406fc30 0x7f5e64157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 >> 172.21.15.88:0/3031642699 conn(0x7f5e6406d1c0 msgr2=0x7f5e6415b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 shutdown_connections 2024-09-21T07:53:59.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:53:59.929+0000 7f5e69a00640 1 -- 172.21.15.88:0/3031642699 wait complete. 2024-09-21T07:53:59.959 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:54:00.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 -- 172.21.15.88:0/140037743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc0b3550 msgr2=0x7fe6dc0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 --2- 172.21.15.88:0/140037743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc0b3550 0x7fe6dc0b3960 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4005fd0 tx=0x7fe6d402a980 comp rx=0 tx=0).stop 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 -- 172.21.15.88:0/140037743 shutdown_connections 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 --2- 172.21.15.88:0/140037743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc0b3550 0x7fe6dc0b3960 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 --2- 172.21.15.88:0/140037743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6dc0ad0e0 0x7fe6dc0ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 -- 172.21.15.88:0/140037743 >> 172.21.15.88:0/140037743 conn(0x7fe6dc06d730 msgr2=0x7fe6dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 -- 172.21.15.88:0/140037743 shutdown_connections 2024-09-21T07:54:00.181 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.178+0000 7fe6e1200640 1 -- 172.21.15.88:0/140037743 wait complete. 2024-09-21T07:54:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.180+0000 7fe6e1200640 1 Processor -- start 2024-09-21T07:54:00.182 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.181+0000 7fe6e1200640 1 -- start start 2024-09-21T07:54:00.185 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6e1200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6dc0ad0e0 0x7fe6dc073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6e1200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6dc0b3550 0x7fe6dc148710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6e1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 0x7fe6dc1548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6e1200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc0b4da0 con 0x7fe6dc15d760 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6e1200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe6dc0b4c20 con 0x7fe6dc0b3550 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6e1200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe6dc0b3ea0 con 0x7fe6dc0ad0e0 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6da800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 0x7fe6dc1548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6da800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 0x7fe6dc1548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54432/0 (socket says 172.21.15.88:54432) 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.182+0000 7fe6da800640 1 -- 172.21.15.88:0/308964839 learned_addr learned my addr 172.21.15.88:0/308964839 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6d9e00640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6dc0ad0e0 0x7fe6dc073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6d9400640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6dc0b3550 0x7fe6dc148710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6da800640 1 -- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6dc0ad0e0 msgr2=0x7fe6dc073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6da800640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6dc0ad0e0 0x7fe6dc073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.186 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6da800640 1 -- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6dc0b3550 msgr2=0x7fe6dc148710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6da800640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6dc0b3550 0x7fe6dc148710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6da800640 1 -- 172.21.15.88:0/308964839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6dc153e40 con 0x7fe6dc15d760 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6da800640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 0x7fe6dc1548e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009d90 tx=0x7fe6d4003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4034040 con 0x7fe6dc15d760 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe6d4004540 con 0x7fe6dc15d760 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.183+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6d4045480 con 0x7fe6dc15d760 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.184+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6dc15ba60 con 0x7fe6dc15d760 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.184+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6dc1540e0 con 0x7fe6dc15d760 2024-09-21T07:54:00.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.185+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6dc0b4350 con 0x7fe6dc15d760 2024-09-21T07:54:00.188 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.186+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fe6d4037020 con 0x7fe6dc15d760 2024-09-21T07:54:00.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.187+0000 7fe6d2000640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe6bc079250 0x7fe6bc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.187+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4885+0+0 (secure 0 0 0) 0x7fe6d40bcfe0 con 0x7fe6dc15d760 2024-09-21T07:54:00.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.187+0000 7fe6d9e00640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe6bc079250 0x7fe6bc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.188+0000 7fe6d9e00640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe6bc079250 0x7fe6bc07b760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc006fd0 tx=0x7fe6cc009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:00.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.192+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe6dc0ada30 con 0x7fe6dc15d760 2024-09-21T07:54:00.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.447+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7fe6dc1537b0 con 0x7fe6dc15d760 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.448+0000 7fe6d2000640 1 -- 172.21.15.88:0/308964839 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) ==== 80+0+5304 (secure 0 0 0) 0x7fe6d4084be0 con 0x7fe6dc15d760 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:54:00.450 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:00.451 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.452 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.453 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.454 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.455 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.456 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:54:00.457 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.458 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:54:00.459 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:54:00.460 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.461 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.462 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.463 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.464 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:54:00.464 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:54:00.464 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.450+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe6bc079250 msgr2=0x7fe6bc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.450+0000 7fe6e1200640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe6bc079250 0x7fe6bc07b760 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe6cc006fd0 tx=0x7fe6cc009060 comp rx=0 tx=0).stop 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.450+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 msgr2=0x7fe6dc1548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.450+0000 7fe6e1200640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 0x7fe6dc1548e0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7fe6d4009d90 tx=0x7fe6d4003f70 comp rx=0 tx=0).stop 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 shutdown_connections 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe6bc079250 0x7fe6bc07b760 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6dc15d760 0x7fe6dc1548e0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe6dc0b3550 0x7fe6dc148710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 --2- 172.21.15.88:0/308964839 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe6dc0ad0e0 0x7fe6dc073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 >> 172.21.15.88:0/308964839 conn(0x7fe6dc06d730 msgr2=0x7fe6dc15f8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 shutdown_connections 2024-09-21T07:54:00.465 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.451+0000 7fe6e1200640 1 -- 172.21.15.88:0/308964839 wait complete. 2024-09-21T07:54:00.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.692+0000 7fb37d400640 1 -- 172.21.15.88:0/612571678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780aebc0 msgr2=0x7fb378152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.692+0000 7fb37d400640 1 --2- 172.21.15.88:0/612571678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780aebc0 0x7fb378152920 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fb378160420 tx=0x7fb36802a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.692+0000 7fb37d400640 1 -- 172.21.15.88:0/612571678 shutdown_connections 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.692+0000 7fb37d400640 1 --2- 172.21.15.88:0/612571678 >> v2:172.21.15.175:3300/0 conn(0x7fb37815fae0 0x7fb3780b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.692+0000 7fb37d400640 1 --2- 172.21.15.88:0/612571678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780aebc0 0x7fb378152920 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.692+0000 7fb37d400640 1 -- 172.21.15.88:0/612571678 >> 172.21.15.88:0/612571678 conn(0x7fb37806d730 msgr2=0x7fb3780b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.693+0000 7fb37d400640 1 -- 172.21.15.88:0/612571678 shutdown_connections 2024-09-21T07:54:00.695 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.693+0000 7fb37d400640 1 -- 172.21.15.88:0/612571678 wait complete. 2024-09-21T07:54:00.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.695+0000 7fb37d400640 1 Processor -- start 2024-09-21T07:54:00.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.695+0000 7fb37d400640 1 -- start start 2024-09-21T07:54:00.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.696+0000 7fb37d400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3780aebc0 0x7fb3780ced00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.696+0000 7fb37d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 0x7fb3780cd930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.696+0000 7fb37d400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378145800 0x7fb378145c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.696+0000 7fb37d400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780b40d0 con 0x7fb3780b2750 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.696+0000 7fb37d400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb3780b3f50 con 0x7fb378145800 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.696+0000 7fb37d400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb3780b4250 con 0x7fb3780aebc0 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 0x7fb3780cd930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 0x7fb3780cd930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54446/0 (socket says 172.21.15.88:54446) 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 -- 172.21.15.88:0/1265104562 learned_addr learned my addr 172.21.15.88:0/1265104562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb376800640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3780aebc0 0x7fb3780ced00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb377200640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378145800 0x7fb378145c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 -- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3780aebc0 msgr2=0x7fb3780ced00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3780aebc0 0x7fb3780ced00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 -- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378145800 msgr2=0x7fb378145c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378145800 0x7fb378145c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 -- 172.21.15.88:0/1265104562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb378146150 con 0x7fb3780b2750 2024-09-21T07:54:00.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.697+0000 7fb36de00640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 0x7fb3780cd930 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c3c0 tx=0x7fb36000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:00.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.698+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360017070 con 0x7fb3780b2750 2024-09-21T07:54:00.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.698+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb3600035f0 con 0x7fb3780b2750 2024-09-21T07:54:00.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.698+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3781f56d0 con 0x7fb3780b2750 2024-09-21T07:54:00.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.698+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb360003a40 con 0x7fb3780b2750 2024-09-21T07:54:00.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.698+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3780cdf40 con 0x7fb3780b2750 2024-09-21T07:54:00.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.699+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3780b4770 con 0x7fb3780b2750 2024-09-21T07:54:00.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.700+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fb36001a070 con 0x7fb3780b2750 2024-09-21T07:54:00.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.701+0000 7fb36f400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb354079250 0x7fb35407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:00.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.701+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4885+0+0 (secure 0 0 0) 0x7fb36009aa80 con 0x7fb3780b2750 2024-09-21T07:54:00.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.701+0000 7fb376800640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb354079250 0x7fb35407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:00.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.702+0000 7fb376800640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb354079250 0x7fb35407b760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb368004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:00.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.705+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb360062030 con 0x7fb3780b2750 2024-09-21T07:54:00.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.892+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fb37815ec00 con 0x7fb3780b2750 2024-09-21T07:54:00.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.893+0000 7fb36f400640 1 -- 172.21.15.88:0/1265104562 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v141) ==== 84+0+65 (secure 0 0 0) 0x7fb360066a60 con 0x7fb3780b2750 2024-09-21T07:54:00.897 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:00.898 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:54:00.898 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:00.898 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:54:00.898 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:54:00.898 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:00.898 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb354079250 msgr2=0x7fb35407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb354079250 0x7fb35407b760 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fb368034000 tx=0x7fb368004730 comp rx=0 tx=0).stop 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 msgr2=0x7fb3780cd930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 0x7fb3780cd930 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fb36000c3c0 tx=0x7fb36000c880 comp rx=0 tx=0).stop 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 shutdown_connections 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb354079250 0x7fb35407b760 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb378145800 0x7fb378145c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3780b2750 0x7fb3780cd930 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 --2- 172.21.15.88:0/1265104562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb3780aebc0 0x7fb3780ced00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:00.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 >> 172.21.15.88:0/1265104562 conn(0x7fb37806d730 msgr2=0x7fb3780d0290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:00.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 shutdown_connections 2024-09-21T07:54:00.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:00.895+0000 7fb37d400640 1 -- 172.21.15.88:0/1265104562 wait complete. 2024-09-21T07:54:00.921 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:54:00.922 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 -- 172.21.15.88:0/2319344718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00b3ec0 msgr2=0x7f18f00b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 --2- 172.21.15.88:0/2319344718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00b3ec0 0x7f18f00b47c0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7f18e0009830 tx=0x7f18e002a480 comp rx=0 tx=0).stop 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 -- 172.21.15.88:0/2319344718 shutdown_connections 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 --2- 172.21.15.88:0/2319344718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18f00af3c0 0x7f18f01602b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 --2- 172.21.15.88:0/2319344718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00b3ec0 0x7f18f00b47c0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 -- 172.21.15.88:0/2319344718 >> 172.21.15.88:0/2319344718 conn(0x7f18f00b1dd0 msgr2=0x7f18f00b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 -- 172.21.15.88:0/2319344718 shutdown_connections 2024-09-21T07:54:01.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.144+0000 7f18f5a00640 1 -- 172.21.15.88:0/2319344718 wait complete. 2024-09-21T07:54:01.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.147+0000 7f18f5a00640 1 Processor -- start 2024-09-21T07:54:01.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.147+0000 7f18f5a00640 1 -- start start 2024-09-21T07:54:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18f5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 0x7f18f0074210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18f5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f00af3c0 0x7f18f009c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18f5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18f00b3ec0 0x7f18f01f4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18f5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0160b80 con 0x7f18f00705b0 2024-09-21T07:54:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18f5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0160a00 con 0x7f18f00af3c0 2024-09-21T07:54:01.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18f5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f18f015cc60 con 0x7f18f00b3ec0 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.148+0000 7f18efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 0x7f18f0074210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 0x7f18f0074210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54450/0 (socket says 172.21.15.88:54450) 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 -- 172.21.15.88:0/3043123828 learned_addr learned my addr 172.21.15.88:0/3043123828 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 -- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18f00b3ec0 msgr2=0x7f18f01f4de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18f00b3ec0 0x7f18f01f4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 -- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f00af3c0 msgr2=0x7f18f009c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18ef400640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f00af3c0 0x7f18f009c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f00af3c0 0x7f18f009c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 -- 172.21.15.88:0/3043123828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f009aa50 con 0x7f18f00705b0 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18ef400640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f00af3c0 0x7f18f009c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:01.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.149+0000 7f18efe00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 0x7f18f0074210 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f18e002aa70 tx=0x7f18e0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:01.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.150+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0031d60 con 0x7f18f00705b0 2024-09-21T07:54:01.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.150+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f18e0033e10 con 0x7f18f00705b0 2024-09-21T07:54:01.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.150+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0005260 con 0x7f18f00705b0 2024-09-21T07:54:01.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.150+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f009acf0 con 0x7f18f00705b0 2024-09-21T07:54:01.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.150+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f0073a10 con 0x7f18f00705b0 2024-09-21T07:54:01.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.151+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18f0073830 con 0x7f18f00705b0 2024-09-21T07:54:01.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.152+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f18e0005400 con 0x7f18f00705b0 2024-09-21T07:54:01.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.153+0000 7f18e7e00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f18d0079250 0x7f18d007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:01.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.153+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(141..141 src has 1..141) ==== 4885+0+0 (secure 0 0 0) 0x7f18e00bcaa0 con 0x7f18f00705b0 2024-09-21T07:54:01.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.153+0000 7f18ef400640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f18d0079250 0x7f18d007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:01.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.154+0000 7f18ef400640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f18d0079250 0x7f18d007b760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f18f01f9d00 tx=0x7f18dc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:01.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.157+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f18f0073830 con 0x7f18f00705b0 2024-09-21T07:54:01.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.342+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f18f00b0c80 con 0x7f18f00705b0 2024-09-21T07:54:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.928+0000 7f18e7e00640 1 -- 172.21.15.88:0/3043123828 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v142) ==== 138+0+0 (secure 0 0 0) 0x7f18e003f030 con 0x7f18f00705b0 2024-09-21T07:54:01.930 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:54:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f18d0079250 msgr2=0x7f18d007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f18d0079250 0x7f18d007b760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f18f01f9d00 tx=0x7f18dc009040 comp rx=0 tx=0).stop 2024-09-21T07:54:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 msgr2=0x7f18f0074210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 0x7f18f0074210 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7f18e002aa70 tx=0x7f18e0033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 shutdown_connections 2024-09-21T07:54:01.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f18d0079250 0x7f18d007b760 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f18f00b3ec0 0x7f18f01f4de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f18f00af3c0 0x7f18f009c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 --2- 172.21.15.88:0/3043123828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f18f00705b0 0x7f18f0074210 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 >> 172.21.15.88:0/3043123828 conn(0x7f18f00b1dd0 msgr2=0x7f18f00b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 shutdown_connections 2024-09-21T07:54:01.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:01.931+0000 7f18f5a00640 1 -- 172.21.15.88:0/3043123828 wait complete. 2024-09-21T07:54:01.960 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 -- 172.21.15.88:0/1548630536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160780 msgr2=0x7f81b80cd070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 --2- 172.21.15.88:0/1548630536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160780 0x7f81b80cd070 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7f81b81613e0 tx=0x7f81b002aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 -- 172.21.15.88:0/1548630536 shutdown_connections 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 --2- 172.21.15.88:0/1548630536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160780 0x7f81b80cd070 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 --2- 172.21.15.88:0/1548630536 >> v2:172.21.15.175:3300/0 conn(0x7f81b8160370 0x7f81b80ccb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 -- 172.21.15.88:0/1548630536 >> 172.21.15.88:0/1548630536 conn(0x7f81b80b1dd0 msgr2=0x7f81b80b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 -- 172.21.15.88:0/1548630536 shutdown_connections 2024-09-21T07:54:02.189 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.186+0000 7f81bd000640 1 -- 172.21.15.88:0/1548630536 wait complete. 2024-09-21T07:54:02.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.188+0000 7f81bd000640 1 Processor -- start 2024-09-21T07:54:02.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.189+0000 7f81bd000640 1 -- start start 2024-09-21T07:54:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81bd000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80af4e0 0x7f81b81f98c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81bd000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 0x7f81b815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81bd000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b8160780 0x7f81b81f28c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81bd000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81b80b4390 con 0x7f81b8160370 2024-09-21T07:54:02.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81bd000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81b80b4210 con 0x7f81b80af4e0 2024-09-21T07:54:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81bd000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81b80b4510 con 0x7f81b8160780 2024-09-21T07:54:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81aea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 0x7f81b815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81aea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 0x7f81b815e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54466/0 (socket says 172.21.15.88:54466) 2024-09-21T07:54:02.193 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.190+0000 7f81aea00640 1 -- 172.21.15.88:0/676240676 learned_addr learned my addr 172.21.15.88:0/676240676 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81b7400640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80af4e0 0x7f81b81f98c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81b7e00640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b8160780 0x7f81b81f28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81aea00640 1 -- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b8160780 msgr2=0x7f81b81f28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81aea00640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b8160780 0x7f81b81f28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81aea00640 1 -- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80af4e0 msgr2=0x7f81b81f98c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81aea00640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80af4e0 0x7f81b81f98c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81aea00640 1 -- 172.21.15.88:0/676240676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b81f9e00 con 0x7f81b8160370 2024-09-21T07:54:02.194 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81aea00640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 0x7f81b815e930 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f81a000e790 tx=0x7f81a000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a000eea0 con 0x7f81b8160370 2024-09-21T07:54:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81a000cdb0 con 0x7f81b8160370 2024-09-21T07:54:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b8146cd0 con 0x7f81b8160370 2024-09-21T07:54:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.191+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a0010690 con 0x7f81b8160370 2024-09-21T07:54:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.192+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b8149090 con 0x7f81b8160370 2024-09-21T07:54:02.195 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.193+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b80b49c0 con 0x7f81b8160370 2024-09-21T07:54:02.196 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.194+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f81a0010830 con 0x7f81b8160370 2024-09-21T07:54:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.196+0000 7f81afe00640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8194079250 0x7f819407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.196+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(142..142 src has 1..142) ==== 5255+0+0 (secure 0 0 0) 0x7f81a009c3f0 con 0x7f81b8160370 2024-09-21T07:54:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.196+0000 7f81b7400640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8194079250 0x7f819407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:02.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.196+0000 7f81b7400640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8194079250 0x7f819407b760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f81a80015e0 tx=0x7f81a8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:02.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.201+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81b81f9e00 con 0x7f81b8160370 2024-09-21T07:54:02.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.387+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f81b815c860 con 0x7f81b8160370 2024-09-21T07:54:02.939 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.936+0000 7f81afe00640 1 -- 172.21.15.88:0/676240676 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v143) ==== 170+0+0 (secure 0 0 0) 0x7f81a0063e80 con 0x7f81b8160370 2024-09-21T07:54:02.939 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:54:02.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8194079250 msgr2=0x7f819407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:02.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8194079250 0x7f819407b760 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f81a80015e0 tx=0x7f81a8009040 comp rx=0 tx=0).stop 2024-09-21T07:54:02.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 msgr2=0x7f81b815e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:02.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 0x7f81b815e930 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7f81a000e790 tx=0x7f81a000ec50 comp rx=0 tx=0).stop 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 shutdown_connections 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8194079250 0x7f819407b760 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81b8160780 0x7f81b81f28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b8160370 0x7f81b815e930 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 --2- 172.21.15.88:0/676240676 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81b80af4e0 0x7f81b81f98c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 >> 172.21.15.88:0/676240676 conn(0x7f81b80b1dd0 msgr2=0x7f81b8073770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.940+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 shutdown_connections 2024-09-21T07:54:02.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:02.941+0000 7f81bd000640 1 -- 172.21.15.88:0/676240676 wait complete. 2024-09-21T07:54:02.981 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:54:03.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 -- 172.21.15.88:0/2667435977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40b42a0 msgr2=0x7f73d41624b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2667435977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40b42a0 0x7f73d41624b0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f73d4159480 tx=0x7f73bc02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 -- 172.21.15.88:0/2667435977 shutdown_connections 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2667435977 >> v2:172.21.15.175:3300/0 conn(0x7f73d40cf2c0 0x7f73d40cf6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2667435977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40b42a0 0x7f73d41624b0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 -- 172.21.15.88:0/2667435977 >> 172.21.15.88:0/2667435977 conn(0x7f73d406d620 msgr2=0x7f73d40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 -- 172.21.15.88:0/2667435977 shutdown_connections 2024-09-21T07:54:03.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.202+0000 7f73d3e00640 1 -- 172.21.15.88:0/2667435977 wait complete. 2024-09-21T07:54:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.204+0000 7f73d3e00640 1 Processor -- start 2024-09-21T07:54:03.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.205+0000 7f73d3e00640 1 -- start start 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d40742b0 0x7f73d4147750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d40b42a0 0x7f73d4147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 0x7f73d40b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4073030 con 0x7f73d40cf2c0 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f73d4072eb0 con 0x7f73d40742b0 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f73d40731b0 con 0x7f73d40b42a0 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 0x7f73d40b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 0x7f73d40b0e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54480/0 (socket says 172.21.15.88:54480) 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d3400640 1 -- 172.21.15.88:0/2699888882 learned_addr learned my addr 172.21.15.88:0/2699888882 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:03.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73ca000640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d40b42a0 0x7f73d4147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.206+0000 7f73d2a00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d40742b0 0x7f73d4147750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3400640 1 -- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d40b42a0 msgr2=0x7f73d4147c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3400640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d40b42a0 0x7f73d4147c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3400640 1 -- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d40742b0 msgr2=0x7f73d4147750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3400640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d40742b0 0x7f73d4147750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3400640 1 -- 172.21.15.88:0/2699888882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f73d4148690 con 0x7f73d40cf2c0 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d2a00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d40742b0 0x7f73d4147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:03.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3400640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 0x7f73d40b0e20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00caa0 tx=0x7f73cc00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00adf0 con 0x7f73d40cf2c0 2024-09-21T07:54:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f73cc00f540 con 0x7f73d40cf2c0 2024-09-21T07:54:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73cc00f990 con 0x7f73d40cf2c0 2024-09-21T07:54:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f73d40b0060 con 0x7f73d40cf2c0 2024-09-21T07:54:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.207+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f73d40b0620 con 0x7f73d40cf2c0 2024-09-21T07:54:03.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.209+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f73d4073740 con 0x7f73d40cf2c0 2024-09-21T07:54:03.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.210+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f73cc00fb30 con 0x7f73d40cf2c0 2024-09-21T07:54:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.211+0000 7f73cb400640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f73b0079280 0x7f73b007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.211+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(143..143 src has 1..143) ==== 5637+0+0 (secure 0 0 0) 0x7f73cc09c4b0 con 0x7f73d40cf2c0 2024-09-21T07:54:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.211+0000 7f73d2a00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f73b0079280 0x7f73b007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:03.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.212+0000 7f73d2a00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f73b0079280 0x7f73b007b790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f73c40016a0 tx=0x7f73c4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:03.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.216+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f73cc014030 con 0x7f73d40cf2c0 2024-09-21T07:54:03.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.439+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f73d40b2ab0 con 0x7f73d40cf2c0 2024-09-21T07:54:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.946+0000 7f73cb400640 1 -- 172.21.15.88:0/2699888882 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v95) ==== 174+0+0 (secure 0 0 0) 0x7f73cc00fe20 con 0x7f73d40cf2c0 2024-09-21T07:54:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 22 and data pool 23 2024-09-21T07:54:03.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.956+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f73b0079280 msgr2=0x7f73b007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:03.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.956+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f73b0079280 0x7f73b007b790 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f73c40016a0 tx=0x7f73c4009040 comp rx=0 tx=0).stop 2024-09-21T07:54:03.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.956+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 msgr2=0x7f73d40b0e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:03.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.956+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 0x7f73d40b0e20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f73cc00caa0 tx=0x7f73cc00cf60 comp rx=0 tx=0).stop 2024-09-21T07:54:03.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.957+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 shutdown_connections 2024-09-21T07:54:03.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.957+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f73b0079280 0x7f73b007b790 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.957+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f73d40b42a0 0x7f73d4147c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.957+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f73d40cf2c0 0x7f73d40b0e20 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.957+0000 7f73d3e00640 1 --2- 172.21.15.88:0/2699888882 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f73d40742b0 0x7f73d4147750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:03.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.957+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 >> 172.21.15.88:0/2699888882 conn(0x7f73d406d620 msgr2=0x7f73d4162260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:03.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.958+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 shutdown_connections 2024-09-21T07:54:03.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:03.958+0000 7f73d3e00640 1 -- 172.21.15.88:0/2699888882 wait complete. 2024-09-21T07:54:03.999 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 -- 172.21.15.88:0/368946485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c15d760 msgr2=0x7f558c1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 --2- 172.21.15.88:0/368946485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c15d760 0x7f558c1535c0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f5580005fd0 tx=0x7f558002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 -- 172.21.15.88:0/368946485 shutdown_connections 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 --2- 172.21.15.88:0/368946485 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f558c0b3550 0x7f558c0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 --2- 172.21.15.88:0/368946485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c15d760 0x7f558c1535c0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 -- 172.21.15.88:0/368946485 >> 172.21.15.88:0/368946485 conn(0x7f558c06d730 msgr2=0x7f558c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 -- 172.21.15.88:0/368946485 shutdown_connections 2024-09-21T07:54:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.217+0000 7f5593400640 1 -- 172.21.15.88:0/368946485 wait complete. 2024-09-21T07:54:04.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.219+0000 7f5593400640 1 Processor -- start 2024-09-21T07:54:04.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.220+0000 7f5593400640 1 -- start start 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5593400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 0x7f558c14f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5593400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f558c0b3550 0x7f558c099880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5593400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f558c15d760 0x7f558c149e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5593400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f558c06f240 con 0x7f558c0ad0e0 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5593400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f558c0b4e30 con 0x7f558c15d760 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5593400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f558c0b3ea0 con 0x7f558c0b3550 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5590a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 0x7f558c14f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5590a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 0x7f558c14f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54496/0 (socket says 172.21.15.88:54496) 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f5590a00640 1 -- 172.21.15.88:0/343467151 learned_addr learned my addr 172.21.15.88:0/343467151 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.221+0000 7f558be00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f558c0b3550 0x7f558c099880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5591400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f558c15d760 0x7f558c149e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5590a00640 1 -- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f558c0b3550 msgr2=0x7f558c099880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5590a00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f558c0b3550 0x7f558c099880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5590a00640 1 -- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f558c15d760 msgr2=0x7f558c149e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5590a00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f558c15d760 0x7f558c149e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5590a00640 1 -- 172.21.15.88:0/343467151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f558c14e870 con 0x7f558c0ad0e0 2024-09-21T07:54:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5590a00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 0x7f558c14f910 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5580034040 tx=0x7f5580003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.222+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f558003e030 con 0x7f558c0ad0e0 2024-09-21T07:54:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.223+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5580003e00 con 0x7f558c0ad0e0 2024-09-21T07:54:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.223+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5580035040 con 0x7f558c0ad0e0 2024-09-21T07:54:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.223+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f558c14eb50 con 0x7f558c0ad0e0 2024-09-21T07:54:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.223+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f558c14f110 con 0x7f558c0ad0e0 2024-09-21T07:54:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.224+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f558c0b42f0 con 0x7f558c0ad0e0 2024-09-21T07:54:04.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.225+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f5580041030 con 0x7f558c0ad0e0 2024-09-21T07:54:04.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.226+0000 7f5588c00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f556c079280 0x7f556c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.226+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5737+0+0 (secure 0 0 0) 0x7f55800bc160 con 0x7f558c0ad0e0 2024-09-21T07:54:04.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.226+0000 7f558be00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f556c079280 0x7f556c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.227+0000 7f558be00640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f556c079280 0x7f556c07b790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f558c150310 tx=0x7f5578009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.230+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55800834e0 con 0x7f558c0ad0e0 2024-09-21T07:54:04.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.416+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f558c0add40 con 0x7f558c0ad0e0 2024-09-21T07:54:04.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.417+0000 7f5588c00640 1 -- 172.21.15.88:0/343467151 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16808 (secure 0 0 0) 0x7f5580087f10 con 0x7f558c0ad0e0 2024-09-21T07:54:04.420 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:04.420 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":144,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:03.441008+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:01.356225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:02.401381+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:04.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f556c079280 msgr2=0x7f556c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f556c079280 0x7f556c07b790 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f558c150310 tx=0x7f5578009060 comp rx=0 tx=0).stop 2024-09-21T07:54:04.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 msgr2=0x7f558c14f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 0x7f558c14f910 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f5580034040 tx=0x7f5580003910 comp rx=0 tx=0).stop 2024-09-21T07:54:04.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 shutdown_connections 2024-09-21T07:54:04.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f556c079280 0x7f556c07b790 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f558c0b3550 0x7f558c099880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f558c15d760 0x7f558c149e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 --2- 172.21.15.88:0/343467151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f558c0ad0e0 0x7f558c14f910 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 >> 172.21.15.88:0/343467151 conn(0x7f558c06d730 msgr2=0x7f558c0b12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 shutdown_connections 2024-09-21T07:54:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.420+0000 7f5593400640 1 -- 172.21.15.88:0/343467151 wait complete. 2024-09-21T07:54:04.448 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f6956200640 1 -- 172.21.15.88:0/3523253988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 msgr2=0x7f693400b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f6956200640 1 --2- 172.21.15.88:0/3523253988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f693400b9c0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7f6950159570 tx=0x7f6934005830 comp rx=0 tx=0).stop 2024-09-21T07:54:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f694fe00640 1 -- 172.21.15.88:0/3523253988 reap_dead start 2024-09-21T07:54:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f6956200640 1 -- 172.21.15.88:0/3523253988 shutdown_connections 2024-09-21T07:54:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f6956200640 1 -- 172.21.15.88:0/3523253988 >> 172.21.15.88:0/3523253988 conn(0x7f695006d730 msgr2=0x7f69500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f6956200640 1 -- 172.21.15.88:0/3523253988 shutdown_connections 2024-09-21T07:54:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.668+0000 7f6956200640 1 -- 172.21.15.88:0/3523253988 wait complete. 2024-09-21T07:54:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.670+0000 7f6956200640 1 Processor -- start 2024-09-21T07:54:04.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.671+0000 7f6956200640 1 -- start start 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6956200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f69500c9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6956200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695014a8a0 0x7f695015a380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6956200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695014acb0 0x7f695015a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6956200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69500b38f0 con 0x7f6934005150 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6956200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6950073880 con 0x7f695014acb0 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6956200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69500b3a70 con 0x7f695014a8a0 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f694f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f69500c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f694f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f69500c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54522/0 (socket says 172.21.15.88:54522) 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f694f400640 1 -- 172.21.15.88:0/3100098601 learned_addr learned my addr 172.21.15.88:0/3100098601 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.672+0000 7f6946a00640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695014a8a0 0x7f695015a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694fe00640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695014acb0 0x7f695015a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694f400640 1 -- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695014a8a0 msgr2=0x7f695015a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694f400640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695014a8a0 0x7f695015a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694f400640 1 -- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695014acb0 msgr2=0x7f695015a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694f400640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695014acb0 0x7f695015a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694f400640 1 -- 172.21.15.88:0/3100098601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69501f8ca0 con 0x7f6934005150 2024-09-21T07:54:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f694f400640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f69500c9650 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6940011a30 tx=0x7f6940011e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f694000fcd0 con 0x7f6934005150 2024-09-21T07:54:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6940005020 con 0x7f6934005150 2024-09-21T07:54:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.673+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6940005470 con 0x7f6934005150 2024-09-21T07:54:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.674+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69501f8f80 con 0x7f6934005150 2024-09-21T07:54:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.674+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f695009b240 con 0x7f6934005150 2024-09-21T07:54:04.677 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.675+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69500b4000 con 0x7f6934005150 2024-09-21T07:54:04.678 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.676+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f69400047a0 con 0x7f6934005150 2024-09-21T07:54:04.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.677+0000 7f6947e00640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f692c079280 0x7f692c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:04.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.677+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(144..144 src has 1..144) ==== 5737+0+0 (secure 0 0 0) 0x7f694009cc70 con 0x7f6934005150 2024-09-21T07:54:04.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.677+0000 7f6946a00640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f692c079280 0x7f692c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:04.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.678+0000 7f6946a00640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f692c079280 0x7f692c07b790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f69500b24c0 tx=0x7f6938006060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:04.682 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.680+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6940064200 con 0x7f6934005150 2024-09-21T07:54:04.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.868+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f69500aeb00 con 0x7f6934005150 2024-09-21T07:54:04.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.870+0000 7f6947e00640 1 -- 172.21.15.88:0/3100098601 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v144) ==== 74+0+16808 (secure 0 0 0) 0x7f6940068c30 con 0x7f6934005150 2024-09-21T07:54:04.872 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:04.872 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":144,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:03.441008+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:01.356225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:02.401381+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:04.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f692c079280 msgr2=0x7f692c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f692c079280 0x7f692c07b790 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f69500b24c0 tx=0x7f6938006060 comp rx=0 tx=0).stop 2024-09-21T07:54:04.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 msgr2=0x7f69500c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:04.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f69500c9650 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7f6940011a30 tx=0x7f6940011e30 comp rx=0 tx=0).stop 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 shutdown_connections 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f692c079280 0x7f692c07b790 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f695014a8a0 0x7f695015a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f695014acb0 0x7f695015a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 --2- 172.21.15.88:0/3100098601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6934005150 0x7f69500c9650 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 >> 172.21.15.88:0/3100098601 conn(0x7f695006d730 msgr2=0x7f69500b0ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 shutdown_connections 2024-09-21T07:54:04.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:04.872+0000 7f6956200640 1 -- 172.21.15.88:0/3100098601 wait complete. 2024-09-21T07:54:04.896 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 -- 172.21.15.88:0/2817504686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f015d6e0 msgr2=0x7f49f00ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 --2- 172.21.15.88:0/2817504686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f015d6e0 0x7f49f00ccac0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7f49f0075230 tx=0x7f49d802a530 comp rx=0 tx=0).stop 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 -- 172.21.15.88:0/2817504686 shutdown_connections 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 --2- 172.21.15.88:0/2817504686 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f015d6e0 0x7f49f00ccac0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 --2- 172.21.15.88:0/2817504686 >> v2:172.21.15.175:3300/0 conn(0x7f49f00b3f30 0x7f49f0156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 -- 172.21.15.88:0/2817504686 >> 172.21.15.88:0/2817504686 conn(0x7f49f006d620 msgr2=0x7f49f00b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 -- 172.21.15.88:0/2817504686 shutdown_connections 2024-09-21T07:54:05.125 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.122+0000 7f49f4a00640 1 -- 172.21.15.88:0/2817504686 wait complete. 2024-09-21T07:54:05.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.124+0000 7f49f4a00640 1 Processor -- start 2024-09-21T07:54:05.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.125+0000 7f49f4a00640 1 -- start start 2024-09-21T07:54:05.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49f4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f00b3f30 0x7f49f009a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49f4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 0x7f49f0072eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49f4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49f015d6e0 0x7f49f01f50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49f4a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00af960 con 0x7f49f00cd000 2024-09-21T07:54:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49f4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f49f00af7e0 con 0x7f49f00b3f30 2024-09-21T07:54:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49f4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f49f00afae0 con 0x7f49f015d6e0 2024-09-21T07:54:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 0x7f49f0072eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49e6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 0x7f49f0072eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:54530/0 (socket says 172.21.15.88:54530) 2024-09-21T07:54:05.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.126+0000 7f49e6a00640 1 -- 172.21.15.88:0/3831604658 learned_addr learned my addr 172.21.15.88:0/3831604658 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e6a00640 1 -- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49f015d6e0 msgr2=0x7f49f01f50e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49efe00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49f015d6e0 0x7f49f01f50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49ef400640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f00b3f30 0x7f49f009a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e6a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49f015d6e0 0x7f49f01f50e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e6a00640 1 -- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f00b3f30 msgr2=0x7f49f009a1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e6a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f00b3f30 0x7f49f009a1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e6a00640 1 -- 172.21.15.88:0/3831604658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49f015c070 con 0x7f49f00cd000 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49efe00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49f015d6e0 0x7f49f01f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49ef400640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f00b3f30 0x7f49f009a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e6a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 0x7f49f0072eb0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f49d80098f0 tx=0x7f49d8033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8005c90 con 0x7f49f00cd000 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f49d8031e50 con 0x7f49f00cd000 2024-09-21T07:54:05.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49f015c350 con 0x7f49f00cd000 2024-09-21T07:54:05.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49d8005260 con 0x7f49f00cd000 2024-09-21T07:54:05.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.127+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49f009cff0 con 0x7f49f00cd000 2024-09-21T07:54:05.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.128+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49f015bde0 con 0x7f49f00cd000 2024-09-21T07:54:05.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.130+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f49d8005400 con 0x7f49f00cd000 2024-09-21T07:54:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.131+0000 7f49e7e00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49cc079250 0x7f49cc07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.131+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f49d80bca30 con 0x7f49f00cd000 2024-09-21T07:54:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.131+0000 7f49ef400640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49cc079250 0x7f49cc07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:05.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.132+0000 7f49ef400640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49cc079250 0x7f49cc07b760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009ab0 tx=0x7f49e0005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:05.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.135+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f49d803f030 con 0x7f49f00cd000 2024-09-21T07:54:05.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.358+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f49f0156ca0 con 0x7f49f00cd000 2024-09-21T07:54:05.931 INFO:tasks.ceph.mds.a.smithi088.stderr:2024-09-21T07:54:05.929+0000 7f0065e00640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:54:05.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.960+0000 7f49e7e00640 1 -- 172.21.15.88:0/3831604658 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v98) ==== 118+0+0 (secure 0 0 0) 0x7f49d8084470 con 0x7f49f00cd000 2024-09-21T07:54:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49cc079250 msgr2=0x7f49cc07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49cc079250 0x7f49cc07b760 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f49e0009ab0 tx=0x7f49e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 msgr2=0x7f49f0072eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:05.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 0x7f49f0072eb0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f49d80098f0 tx=0x7f49d8033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 shutdown_connections 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f49cc079250 0x7f49cc07b760 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f49f015d6e0 0x7f49f01f50e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f49f00cd000 0x7f49f0072eb0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 --2- 172.21.15.88:0/3831604658 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f49f00b3f30 0x7f49f009a1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 >> 172.21.15.88:0/3831604658 conn(0x7f49f006d620 msgr2=0x7f49f00aee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 shutdown_connections 2024-09-21T07:54:05.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:05.963+0000 7f49f4a00640 1 -- 172.21.15.88:0/3831604658 wait complete. 2024-09-21T07:54:05.993 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:54:05.993 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.214+0000 7fde56c00640 1 -- 172.21.15.88:0/3300678723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50073c50 msgr2=0x7fde5006f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.214+0000 7fde56c00640 1 --2- 172.21.15.88:0/3300678723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50073c50 0x7fde5006f7d0 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7fde48005fd0 tx=0x7fde4802a820 comp rx=0 tx=0).stop 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.214+0000 7fde56c00640 1 -- 172.21.15.88:0/3300678723 shutdown_connections 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.214+0000 7fde56c00640 1 --2- 172.21.15.88:0/3300678723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde50073c50 0x7fde5006f7d0 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.214+0000 7fde56c00640 1 -- 172.21.15.88:0/3300678723 >> 172.21.15.88:0/3300678723 conn(0x7fde5006d1c0 msgr2=0x7fde5006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.215+0000 7fde56c00640 1 -- 172.21.15.88:0/3300678723 shutdown_connections 2024-09-21T07:54:06.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.215+0000 7fde56c00640 1 -- 172.21.15.88:0/3300678723 wait complete. 2024-09-21T07:54:06.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.217+0000 7fde56c00640 1 Processor -- start 2024-09-21T07:54:06.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.217+0000 7fde56c00640 1 -- start start 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde50147b90 0x7fde5015e5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 0x7fde5013d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5013f9d0 0x7fde5013fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde5006fe90 con 0x7fde5015f9e0 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde5006fd10 con 0x7fde50147b90 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde50070010 con 0x7fde5013f9d0 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 0x7fde5013d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 0x7fde5013d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51374/0 (socket says 172.21.15.88:51374) 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 -- 172.21.15.88:0/3624715893 learned_addr learned my addr 172.21.15.88:0/3624715893 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde4fe00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde50147b90 0x7fde5015e5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde54c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5013f9d0 0x7fde5013fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 -- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5013f9d0 msgr2=0x7fde5013fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5013f9d0 0x7fde5013fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 -- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde50147b90 msgr2=0x7fde5015e5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde50147b90 0x7fde5015e5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 -- 172.21.15.88:0/3624715893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde50140320 con 0x7fde5015f9e0 2024-09-21T07:54:06.222 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde47400640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 0x7fde5013d3a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c400 tx=0x7fde3800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:06.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38017070 con 0x7fde5015f9e0 2024-09-21T07:54:06.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde380053c0 con 0x7fde5015f9e0 2024-09-21T07:54:06.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde38003850 con 0x7fde5015f9e0 2024-09-21T07:54:06.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.219+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde500cf190 con 0x7fde5015f9e0 2024-09-21T07:54:06.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.220+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde500b2700 con 0x7fde5015f9e0 2024-09-21T07:54:06.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.221+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde5006f7d0 con 0x7fde5015f9e0 2024-09-21T07:54:06.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.222+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fde3801a070 con 0x7fde5015f9e0 2024-09-21T07:54:06.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.223+0000 7fde4cc00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde2c079280 0x7fde2c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:06.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.223+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7fde3809aef0 con 0x7fde5015f9e0 2024-09-21T07:54:06.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.223+0000 7fde4fe00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde2c079280 0x7fde2c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:06.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.224+0000 7fde4fe00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde2c079280 0x7fde2c07b790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fde40000f60 tx=0x7fde40009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:06.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.228+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde50140320 con 0x7fde5015f9e0 2024-09-21T07:54:06.451 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.449+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fde500b4190 con 0x7fde5015f9e0 2024-09-21T07:54:06.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.963+0000 7fde4cc00640 1 -- 172.21.15.88:0/3624715893 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v99) ==== 122+0+0 (secure 0 0 0) 0x7fde380627e0 con 0x7fde5015f9e0 2024-09-21T07:54:06.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde2c079280 msgr2=0x7fde2c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde2c079280 0x7fde2c07b790 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fde40000f60 tx=0x7fde40009040 comp rx=0 tx=0).stop 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 msgr2=0x7fde5013d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 0x7fde5013d3a0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fde3800c400 tx=0x7fde3800c8c0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 shutdown_connections 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde2c079280 0x7fde2c07b790 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde5013f9d0 0x7fde5013fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde5015f9e0 0x7fde5013d3a0 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 --2- 172.21.15.88:0/3624715893 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde50147b90 0x7fde5015e5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 >> 172.21.15.88:0/3624715893 conn(0x7fde5006d1c0 msgr2=0x7fde50160100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 shutdown_connections 2024-09-21T07:54:06.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:06.966+0000 7fde56c00640 1 -- 172.21.15.88:0/3624715893 wait complete. 2024-09-21T07:54:06.990 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:54:06.991 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:07.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae3e600640 1 -- 172.21.15.88:0/3608556977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae1c004d40 msgr2=0x7fae1c0051d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae3e600640 1 --2- 172.21.15.88:0/3608556977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae1c004d40 0x7fae1c0051d0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7fae300099a0 tx=0x7fae1c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:07.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae37e00640 1 -- 172.21.15.88:0/3608556977 reap_dead start 2024-09-21T07:54:07.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae3e600640 1 -- 172.21.15.88:0/3608556977 shutdown_connections 2024-09-21T07:54:07.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae3e600640 1 -- 172.21.15.88:0/3608556977 >> 172.21.15.88:0/3608556977 conn(0x7fae3806d730 msgr2=0x7fae380b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:07.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae3e600640 1 -- 172.21.15.88:0/3608556977 shutdown_connections 2024-09-21T07:54:07.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.210+0000 7fae3e600640 1 -- 172.21.15.88:0/3608556977 wait complete. 2024-09-21T07:54:07.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.212+0000 7fae3e600640 1 Processor -- start 2024-09-21T07:54:07.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.212+0000 7fae3e600640 1 -- start start 2024-09-21T07:54:07.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae3e600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c004d40 0x7fae380999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae3e600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae38148ad0 0x7fae381f6d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae3e600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 0x7fae381f76b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae3e600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fae380b3ea0 con 0x7fae381f72a0 2024-09-21T07:54:07.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae3e600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fae380b4e30 con 0x7fae1c004d40 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae3e600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fae380b4020 con 0x7fae38148ad0 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 0x7fae381f76b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 0x7fae381f76b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51388/0 (socket says 172.21.15.88:51388) 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37e00640 1 -- 172.21.15.88:0/2905601849 learned_addr learned my addr 172.21.15.88:0/2905601849 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37e00640 1 -- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae38148ad0 msgr2=0x7fae381f6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37e00640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae38148ad0 0x7fae381f6d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37e00640 1 -- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c004d40 msgr2=0x7fae380999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.214+0000 7fae37400640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c004d40 0x7fae380999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae37e00640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c004d40 0x7fae380999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae37e00640 1 -- 172.21.15.88:0/2905601849 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fae38099ee0 con 0x7fae381f72a0 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae37400640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c004d40 0x7fae380999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:07.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae37e00640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 0x7fae381f76b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fae3002a970 tx=0x7fae300047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30037040 con 0x7fae381f72a0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fae30003d10 con 0x7fae381f72a0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fae3814fd70 con 0x7fae381f72a0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fae30005be0 con 0x7fae381f72a0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.215+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fae3809c8a0 con 0x7fae381f72a0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.216+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fae3815f650 con 0x7fae381f72a0 2024-09-21T07:54:07.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.217+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fae30032850 con 0x7fae381f72a0 2024-09-21T07:54:07.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.218+0000 7fae2fe00640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fae140792b0 0x7fae1407b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.218+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7fae300bb6a0 con 0x7fae381f72a0 2024-09-21T07:54:07.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.219+0000 7fae37400640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fae140792b0 0x7fae1407b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.219+0000 7fae37400640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fae140792b0 0x7fae1407b7c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae28005fd0 tx=0x7fae28009a90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:07.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.223+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fae3815f650 con 0x7fae381f72a0 2024-09-21T07:54:07.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.447+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fae380b3960 con 0x7fae381f72a0 2024-09-21T07:54:07.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.448+0000 7fae2fe00640 1 -- 172.21.15.88:0/2905601849 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5233 (secure 0 0 0) 0x7fae38099ee0 con 0x7fae381f72a0 2024-09-21T07:54:07.449 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:07.450 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":99,"btime":"2024-09-21T07:54:06:961559+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:03.440969+0000","modified":"2024-09-21T07:54:06.451584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3712798522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3712798522},{"type":"v1","addr":"172.21.15.88:6835","nonce":3712798522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":11}]} 2024-09-21T07:54:07.450 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 99 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fae140792b0 msgr2=0x7fae1407b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fae140792b0 0x7fae1407b7c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fae28005fd0 tx=0x7fae28009a90 comp rx=0 tx=0).stop 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 msgr2=0x7fae381f76b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 0x7fae381f76b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7fae3002a970 tx=0x7fae300047d0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 shutdown_connections 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fae140792b0 0x7fae1407b7c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fae38148ad0 0x7fae381f6d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fae381f72a0 0x7fae381f76b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 --2- 172.21.15.88:0/2905601849 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fae1c004d40 0x7fae380999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 >> 172.21.15.88:0/2905601849 conn(0x7fae3806d730 msgr2=0x7fae380b18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 shutdown_connections 2024-09-21T07:54:07.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.451+0000 7fae3e600640 1 -- 172.21.15.88:0/2905601849 wait complete. 2024-09-21T07:54:07.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:07.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.694+0000 7feb35e00640 1 -- 172.21.15.88:0/3056734758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb300b3550 msgr2=0x7feb300b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.694+0000 7feb35e00640 1 --2- 172.21.15.88:0/3056734758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb300b3550 0x7feb300b3960 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7feb28005fd0 tx=0x7feb2802a820 comp rx=0 tx=0).stop 2024-09-21T07:54:07.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.695+0000 7feb35e00640 1 -- 172.21.15.88:0/3056734758 shutdown_connections 2024-09-21T07:54:07.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.695+0000 7feb35e00640 1 --2- 172.21.15.88:0/3056734758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb300b3550 0x7feb300b3960 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.695+0000 7feb35e00640 1 --2- 172.21.15.88:0/3056734758 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb300ad0e0 0x7feb300ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.695+0000 7feb35e00640 1 -- 172.21.15.88:0/3056734758 >> 172.21.15.88:0/3056734758 conn(0x7feb3006d730 msgr2=0x7feb300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:07.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.695+0000 7feb35e00640 1 -- 172.21.15.88:0/3056734758 shutdown_connections 2024-09-21T07:54:07.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.695+0000 7feb35e00640 1 -- 172.21.15.88:0/3056734758 wait complete. 2024-09-21T07:54:07.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.697+0000 7feb35e00640 1 Processor -- start 2024-09-21T07:54:07.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.697+0000 7feb35e00640 1 -- start start 2024-09-21T07:54:07.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.698+0000 7feb35e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb300ad0e0 0x7feb30154800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.698+0000 7feb35e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb300b3550 0x7feb301f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.698+0000 7feb35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 0x7feb300b1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.701 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.698+0000 7feb35e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b4da0 con 0x7feb3015d760 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.698+0000 7feb35e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7feb300b4c20 con 0x7feb300ad0e0 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.698+0000 7feb35e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7feb300b3ea0 con 0x7feb300b3550 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 0x7feb300b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 0x7feb300b1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51402/0 (socket says 172.21.15.88:51402) 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 -- 172.21.15.88:0/1419327001 learned_addr learned my addr 172.21.15.88:0/1419327001 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2e000640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb300b3550 0x7feb301f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2ea00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb300ad0e0 0x7feb30154800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 -- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb300b3550 msgr2=0x7feb301f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb300b3550 0x7feb301f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 -- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb300ad0e0 msgr2=0x7feb30154800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb300ad0e0 0x7feb30154800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2f400640 1 -- 172.21.15.88:0/1419327001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb301551c0 con 0x7feb3015d760 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.699+0000 7feb2ea00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb300ad0e0 0x7feb30154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.700+0000 7feb2f400640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 0x7feb300b1970 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7feb28009cd0 tx=0x7feb28032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.700+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28037040 con 0x7feb3015d760 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.700+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7feb28042070 con 0x7feb3015d760 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.700+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb28003e60 con 0x7feb3015d760 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.700+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb301554a0 con 0x7feb3015d760 2024-09-21T07:54:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.700+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb30073040 con 0x7feb3015d760 2024-09-21T07:54:07.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.701+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb300b43c0 con 0x7feb3015d760 2024-09-21T07:54:07.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.702+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7feb28004280 con 0x7feb3015d760 2024-09-21T07:54:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.703+0000 7feb27400640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb10079280 0x7feb1007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.703+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7feb280bc560 con 0x7feb3015d760 2024-09-21T07:54:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.704+0000 7feb2ea00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb10079280 0x7feb1007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:07.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.704+0000 7feb2ea00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb10079280 0x7feb1007b790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feb200013f0 tx=0x7feb20009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:07.709 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.708+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7feb28084030 con 0x7feb3015d760 2024-09-21T07:54:07.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.895+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7feb30154f50 con 0x7feb3015d760 2024-09-21T07:54:07.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.896+0000 7feb27400640 1 -- 172.21.15.88:0/1419327001 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16795 (secure 0 0 0) 0x7feb28083930 con 0x7feb3015d760 2024-09-21T07:54:07.898 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:07.898 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":145,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:04.947773+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:01.356225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:02.401381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.898+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb10079280 msgr2=0x7feb1007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.898+0000 7feb35e00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb10079280 0x7feb1007b790 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7feb200013f0 tx=0x7feb20009040 comp rx=0 tx=0).stop 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.898+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 msgr2=0x7feb300b1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.898+0000 7feb35e00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 0x7feb300b1970 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7feb28009cd0 tx=0x7feb28032c60 comp rx=0 tx=0).stop 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 shutdown_connections 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7feb10079280 0x7feb1007b790 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7feb300b3550 0x7feb301f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb3015d760 0x7feb300b1970 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 --2- 172.21.15.88:0/1419327001 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feb300ad0e0 0x7feb30154800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:07.900 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 >> 172.21.15.88:0/1419327001 conn(0x7feb3006d730 msgr2=0x7feb30152b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 shutdown_connections 2024-09-21T07:54:07.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:07.899+0000 7feb35e00640 1 -- 172.21.15.88:0/1419327001 wait complete. 2024-09-21T07:54:07.923 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:54:07.923 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.142+0000 7f1653200640 1 -- 172.21.15.88:0/3705621858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 msgr2=0x7f1630004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.142+0000 7f1653200640 1 --2- 172.21.15.88:0/3705621858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f1630004800 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f1638001710 tx=0x7f1630005830 comp rx=0 tx=0).stop 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.142+0000 7f1651200640 1 -- 172.21.15.88:0/3705621858 reap_dead start 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.143+0000 7f1653200640 1 -- 172.21.15.88:0/3705621858 shutdown_connections 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.143+0000 7f1653200640 1 -- 172.21.15.88:0/3705621858 >> 172.21.15.88:0/3705621858 conn(0x7f164c06d730 msgr2=0x7f164c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.143+0000 7f1653200640 1 -- 172.21.15.88:0/3705621858 shutdown_connections 2024-09-21T07:54:08.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.143+0000 7f1653200640 1 -- 172.21.15.88:0/3705621858 wait complete. 2024-09-21T07:54:08.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.145+0000 7f1653200640 1 Processor -- start 2024-09-21T07:54:08.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.145+0000 7f1653200640 1 -- start start 2024-09-21T07:54:08.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.146+0000 7f1653200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f164c074c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.146+0000 7f1653200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f164c1515b0 0x7f164c0d0a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.146+0000 7f1653200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f164c074760 0x7f164c147900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.146+0000 7f1653200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f164c06f260 con 0x7f1630003f90 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.146+0000 7f1653200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f164c0b4e30 con 0x7f164c1515b0 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.146+0000 7f1653200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f164c165890 con 0x7f164c074760 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f164c074c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f164c074c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51412/0 (socket says 172.21.15.88:51412) 2024-09-21T07:54:08.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 -- 172.21.15.88:0/3511971406 learned_addr learned my addr 172.21.15.88:0/3511971406 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:08.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164b400640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f164c1515b0 0x7f164c0d0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f1651200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f164c074760 0x7f164c147900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 -- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f164c074760 msgr2=0x7f164c147900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f164c074760 0x7f164c147900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 -- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f164c1515b0 msgr2=0x7f164c0d0a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f164c1515b0 0x7f164c0d0a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 -- 172.21.15.88:0/3511971406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f164c070010 con 0x7f1630003f90 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f1651200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f164c074760 0x7f164c147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.147+0000 7f164be00640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f164c074c40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f163c006450 tx=0x7f163c0039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.148+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c015070 con 0x7f1630003f90 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.148+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f163c011900 con 0x7f1630003f90 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.148+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163c011d50 con 0x7f1630003f90 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.148+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f164c0702b0 con 0x7f1630003f90 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.148+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f164c1f9a00 con 0x7f1630003f90 2024-09-21T07:54:08.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.149+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f164c1535c0 con 0x7f1630003f90 2024-09-21T07:54:08.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.150+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f163c018070 con 0x7f1630003f90 2024-09-21T07:54:08.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.151+0000 7f1643e00640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f162c079280 0x7f162c07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.151+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f163c09c480 con 0x7f1630003f90 2024-09-21T07:54:08.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.151+0000 7f164b400640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f162c079280 0x7f162c07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.152+0000 7f164b400640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f162c079280 0x7f162c07b790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f164c15cd60 tx=0x7f1638001270 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:08.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.156+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f163c063dd0 con 0x7f1630003f90 2024-09-21T07:54:08.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.338+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f164c151060 con 0x7f162c079280 2024-09-21T07:54:08.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.343+0000 7f1643e00640 1 -- 172.21.15.88:0/3511971406 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f163807fa60 con 0x7f162c079280 2024-09-21T07:54:08.347 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:08.349 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f162c079280 msgr2=0x7f162c07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f162c079280 0x7f162c07b790 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f164c15cd60 tx=0x7f1638001270 comp rx=0 tx=0).stop 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 msgr2=0x7f164c074c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f164c074c40 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f163c006450 tx=0x7f163c0039c0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 shutdown_connections 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f162c079280 0x7f162c07b790 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f164c074760 0x7f164c147900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f164c1515b0 0x7f164c0d0a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 --2- 172.21.15.88:0/3511971406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1630003f90 0x7f164c074c40 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 >> 172.21.15.88:0/3511971406 conn(0x7f164c06d730 msgr2=0x7f164c149860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 shutdown_connections 2024-09-21T07:54:08.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.347+0000 7f1653200640 1 -- 172.21.15.88:0/3511971406 wait complete. 2024-09-21T07:54:08.371 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-21T07:54:07.741660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264896,"kb_used_data":15356,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709824,"statfs":{"total":773094113280,"available":772822859776,"internally_reserved":0,"allocated":15724544,"data_stored":7582556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.811461"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968860+0000","last_change":"2024-09-21T07:54:03.967776+0000","last_active":"2024-09-21T07:54:04.968860+0000","last_peered":"2024-09-21T07:54:04.968860+0000","last_clean":"2024-09-21T07:54:04.968860+0000","last_became_active":"2024-09-21T07:54:03.967239+0000","last_became_peered":"2024-09-21T07:54:03.967239+0000","last_unstale":"2024-09-21T07:54:04.968860+0000","last_undegraded":"2024-09-21T07:54:04.968860+0000","last_fullsized":"2024-09-21T07:54:04.968860+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:21.500284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.969070+0000","last_change":"2024-09-21T07:54:03.972467+0000","last_active":"2024-09-21T07:54:04.969070+0000","last_peered":"2024-09-21T07:54:04.969070+0000","last_clean":"2024-09-21T07:54:04.969070+0000","last_became_active":"2024-09-21T07:54:03.972169+0000","last_became_peered":"2024-09-21T07:54:03.972169+0000","last_unstale":"2024-09-21T07:54:04.969070+0000","last_undegraded":"2024-09-21T07:54:04.969070+0000","last_fullsized":"2024-09-21T07:54:04.969070+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:02:00.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968753+0000","last_change":"2024-09-21T07:54:03.979603+0000","last_active":"2024-09-21T07:54:04.968753+0000","last_peered":"2024-09-21T07:54:04.968753+0000","last_clean":"2024-09-21T07:54:04.968753+0000","last_became_active":"2024-09-21T07:54:03.979004+0000","last_became_peered":"2024-09-21T07:54:03.979004+0000","last_unstale":"2024-09-21T07:54:04.968753+0000","last_undegraded":"2024-09-21T07:54:04.968753+0000","last_fullsized":"2024-09-21T07:54:04.968753+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:35.951838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957865+0000","last_change":"2024-09-21T07:54:03.965065+0000","last_active":"2024-09-21T07:54:04.957865+0000","last_peered":"2024-09-21T07:54:04.957865+0000","last_clean":"2024-09-21T07:54:04.957865+0000","last_became_active":"2024-09-21T07:54:03.964387+0000","last_became_peered":"2024-09-21T07:54:03.964387+0000","last_unstale":"2024-09-21T07:54:04.957865+0000","last_undegraded":"2024-09-21T07:54:04.957865+0000","last_fullsized":"2024-09-21T07:54:04.957865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:58.614172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959116+0000","last_change":"2024-09-21T07:54:03.957510+0000","last_active":"2024-09-21T07:54:04.959116+0000","last_peered":"2024-09-21T07:54:04.959116+0000","last_clean":"2024-09-21T07:54:04.959116+0000","last_became_active":"2024-09-21T07:54:03.956596+0000","last_became_peered":"2024-09-21T07:54:03.956596+0000","last_unstale":"2024-09-21T07:54:04.959116+0000","last_undegraded":"2024-09-21T07:54:04.959116+0000","last_fullsized":"2024-09-21T07:54:04.959116+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:39.498696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366860+0000","last_change":"2024-09-21T07:54:03.972831+0000","last_active":"2024-09-21T07:54:06.366860+0000","last_peered":"2024-09-21T07:54:06.366860+0000","last_clean":"2024-09-21T07:54:06.366860+0000","last_became_active":"2024-09-21T07:54:03.971051+0000","last_became_peered":"2024-09-21T07:54:03.971051+0000","last_unstale":"2024-09-21T07:54:06.366860+0000","last_undegraded":"2024-09-21T07:54:06.366860+0000","last_fullsized":"2024-09-21T07:54:06.366860+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:57.496456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900223+0000","last_change":"2024-09-21T07:54:03.966873+0000","last_active":"2024-09-21T07:54:05.900223+0000","last_peered":"2024-09-21T07:54:05.900223+0000","last_clean":"2024-09-21T07:54:05.900223+0000","last_became_active":"2024-09-21T07:54:03.966550+0000","last_became_peered":"2024-09-21T07:54:03.966550+0000","last_unstale":"2024-09-21T07:54:05.900223+0000","last_undegraded":"2024-09-21T07:54:05.900223+0000","last_fullsized":"2024-09-21T07:54:05.900223+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:24.800547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038302+0000","last_change":"2024-09-21T07:54:03.979521+0000","last_active":"2024-09-21T07:54:06.038302+0000","last_peered":"2024-09-21T07:54:06.038302+0000","last_clean":"2024-09-21T07:54:06.038302+0000","last_became_active":"2024-09-21T07:54:03.979199+0000","last_became_peered":"2024-09-21T07:54:03.979199+0000","last_unstale":"2024-09-21T07:54:06.038302+0000","last_undegraded":"2024-09-21T07:54:06.038302+0000","last_fullsized":"2024-09-21T07:54:06.038302+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:47.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038658+0000","last_change":"2024-09-21T07:54:03.952233+0000","last_active":"2024-09-21T07:54:06.038658+0000","last_peered":"2024-09-21T07:54:06.038658+0000","last_clean":"2024-09-21T07:54:06.038658+0000","last_became_active":"2024-09-21T07:54:03.949630+0000","last_became_peered":"2024-09-21T07:54:03.949630+0000","last_unstale":"2024-09-21T07:54:06.038658+0000","last_undegraded":"2024-09-21T07:54:06.038658+0000","last_fullsized":"2024-09-21T07:54:06.038658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:56.835444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038567+0000","last_change":"2024-09-21T07:54:03.958744+0000","last_active":"2024-09-21T07:54:06.038567+0000","last_peered":"2024-09-21T07:54:06.038567+0000","last_clean":"2024-09-21T07:54:06.038567+0000","last_became_active":"2024-09-21T07:54:03.958123+0000","last_became_peered":"2024-09-21T07:54:03.958123+0000","last_unstale":"2024-09-21T07:54:06.038567+0000","last_undegraded":"2024-09-21T07:54:06.038567+0000","last_fullsized":"2024-09-21T07:54:06.038567+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:43.139253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958890+0000","last_change":"2024-09-21T07:54:03.960751+0000","last_active":"2024-09-21T07:54:04.958890+0000","last_peered":"2024-09-21T07:54:04.958890+0000","last_clean":"2024-09-21T07:54:04.958890+0000","last_became_active":"2024-09-21T07:54:03.960213+0000","last_became_peered":"2024-09-21T07:54:03.960213+0000","last_unstale":"2024-09-21T07:54:04.958890+0000","last_undegraded":"2024-09-21T07:54:04.958890+0000","last_fullsized":"2024-09-21T07:54:04.958890+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:18.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964102+0000","last_change":"2024-09-21T07:54:03.974927+0000","last_active":"2024-09-21T07:54:04.964102+0000","last_peered":"2024-09-21T07:54:04.964102+0000","last_clean":"2024-09-21T07:54:04.964102+0000","last_became_active":"2024-09-21T07:54:03.974635+0000","last_became_peered":"2024-09-21T07:54:03.974635+0000","last_unstale":"2024-09-21T07:54:04.964102+0000","last_undegraded":"2024-09-21T07:54:04.964102+0000","last_fullsized":"2024-09-21T07:54:04.964102+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:58.461584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964047+0000","last_change":"2024-09-21T07:54:03.970845+0000","last_active":"2024-09-21T07:54:04.964047+0000","last_peered":"2024-09-21T07:54:04.964047+0000","last_clean":"2024-09-21T07:54:04.964047+0000","last_became_active":"2024-09-21T07:54:03.970084+0000","last_became_peered":"2024-09-21T07:54:03.970084+0000","last_unstale":"2024-09-21T07:54:04.964047+0000","last_undegraded":"2024-09-21T07:54:04.964047+0000","last_fullsized":"2024-09-21T07:54:04.964047+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:20.271595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038430+0000","last_change":"2024-09-21T07:54:03.956973+0000","last_active":"2024-09-21T07:54:06.038430+0000","last_peered":"2024-09-21T07:54:06.038430+0000","last_clean":"2024-09-21T07:54:06.038430+0000","last_became_active":"2024-09-21T07:54:03.956642+0000","last_became_peered":"2024-09-21T07:54:03.956642+0000","last_unstale":"2024-09-21T07:54:06.038430+0000","last_undegraded":"2024-09-21T07:54:06.038430+0000","last_fullsized":"2024-09-21T07:54:06.038430+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:40.718444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038835+0000","last_change":"2024-09-21T07:54:03.963106+0000","last_active":"2024-09-21T07:54:06.038835+0000","last_peered":"2024-09-21T07:54:06.038835+0000","last_clean":"2024-09-21T07:54:06.038835+0000","last_became_active":"2024-09-21T07:54:03.962213+0000","last_became_peered":"2024-09-21T07:54:03.962213+0000","last_unstale":"2024-09-21T07:54:06.038835+0000","last_undegraded":"2024-09-21T07:54:06.038835+0000","last_fullsized":"2024-09-21T07:54:06.038835+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.785305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900221+0000","last_change":"2024-09-21T07:54:03.978295+0000","last_active":"2024-09-21T07:54:05.900221+0000","last_peered":"2024-09-21T07:54:05.900221+0000","last_clean":"2024-09-21T07:54:05.900221+0000","last_became_active":"2024-09-21T07:54:03.977951+0000","last_became_peered":"2024-09-21T07:54:03.977951+0000","last_unstale":"2024-09-21T07:54:05.900221+0000","last_undegraded":"2024-09-21T07:54:05.900221+0000","last_fullsized":"2024-09-21T07:54:05.900221+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:59.863067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.367119+0000","last_change":"2024-09-21T07:54:03.973172+0000","last_active":"2024-09-21T07:54:06.367119+0000","last_peered":"2024-09-21T07:54:06.367119+0000","last_clean":"2024-09-21T07:54:06.367119+0000","last_became_active":"2024-09-21T07:54:03.972809+0000","last_became_peered":"2024-09-21T07:54:03.972809+0000","last_unstale":"2024-09-21T07:54:06.367119+0000","last_undegraded":"2024-09-21T07:54:06.367119+0000","last_fullsized":"2024-09-21T07:54:06.367119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:33.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959559+0000","last_change":"2024-09-21T07:54:03.969375+0000","last_active":"2024-09-21T07:54:04.959559+0000","last_peered":"2024-09-21T07:54:04.959559+0000","last_clean":"2024-09-21T07:54:04.959559+0000","last_became_active":"2024-09-21T07:54:03.969005+0000","last_became_peered":"2024-09-21T07:54:03.969005+0000","last_unstale":"2024-09-21T07:54:04.959559+0000","last_undegraded":"2024-09-21T07:54:04.959559+0000","last_fullsized":"2024-09-21T07:54:04.959559+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:28.249397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900691+0000","last_change":"2024-09-21T07:54:03.964077+0000","last_active":"2024-09-21T07:54:05.900691+0000","last_peered":"2024-09-21T07:54:05.900691+0000","last_clean":"2024-09-21T07:54:05.900691+0000","last_became_active":"2024-09-21T07:54:03.961800+0000","last_became_peered":"2024-09-21T07:54:03.961800+0000","last_unstale":"2024-09-21T07:54:05.900691+0000","last_undegraded":"2024-09-21T07:54:05.900691+0000","last_fullsized":"2024-09-21T07:54:05.900691+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.270102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964124+0000","last_change":"2024-09-21T07:54:03.982366+0000","last_active":"2024-09-21T07:54:04.964124+0000","last_peered":"2024-09-21T07:54:04.964124+0000","last_clean":"2024-09-21T07:54:04.964124+0000","last_became_active":"2024-09-21T07:54:03.978306+0000","last_became_peered":"2024-09-21T07:54:03.978306+0000","last_unstale":"2024-09-21T07:54:04.964124+0000","last_undegraded":"2024-09-21T07:54:04.964124+0000","last_fullsized":"2024-09-21T07:54:04.964124+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:21.416321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366738+0000","last_change":"2024-09-21T07:54:03.979276+0000","last_active":"2024-09-21T07:54:06.366738+0000","last_peered":"2024-09-21T07:54:06.366738+0000","last_clean":"2024-09-21T07:54:06.366738+0000","last_became_active":"2024-09-21T07:54:03.978909+0000","last_became_peered":"2024-09-21T07:54:03.978909+0000","last_unstale":"2024-09-21T07:54:06.366738+0000","last_undegraded":"2024-09-21T07:54:06.366738+0000","last_fullsized":"2024-09-21T07:54:06.366738+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:47.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958712+0000","last_change":"2024-09-21T07:54:03.961010+0000","last_active":"2024-09-21T07:54:04.958712+0000","last_peered":"2024-09-21T07:54:04.958712+0000","last_clean":"2024-09-21T07:54:04.958712+0000","last_became_active":"2024-09-21T07:54:03.960727+0000","last_became_peered":"2024-09-21T07:54:03.960727+0000","last_unstale":"2024-09-21T07:54:04.958712+0000","last_undegraded":"2024-09-21T07:54:04.958712+0000","last_fullsized":"2024-09-21T07:54:04.958712+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:40.969184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959104+0000","last_change":"2024-09-21T07:54:03.972825+0000","last_active":"2024-09-21T07:54:04.959104+0000","last_peered":"2024-09-21T07:54:04.959104+0000","last_clean":"2024-09-21T07:54:04.959104+0000","last_became_active":"2024-09-21T07:54:03.972542+0000","last_became_peered":"2024-09-21T07:54:03.972542+0000","last_unstale":"2024-09-21T07:54:04.959104+0000","last_undegraded":"2024-09-21T07:54:04.959104+0000","last_fullsized":"2024-09-21T07:54:04.959104+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:37.766666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958576+0000","last_change":"2024-09-21T07:54:03.950116+0000","last_active":"2024-09-21T07:54:04.958576+0000","last_peered":"2024-09-21T07:54:04.958576+0000","last_clean":"2024-09-21T07:54:04.958576+0000","last_became_active":"2024-09-21T07:54:03.949816+0000","last_became_peered":"2024-09-21T07:54:03.949816+0000","last_unstale":"2024-09-21T07:54:04.958576+0000","last_undegraded":"2024-09-21T07:54:04.958576+0000","last_fullsized":"2024-09-21T07:54:04.958576+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:24.910790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958613+0000","last_change":"2024-09-21T07:54:03.952909+0000","last_active":"2024-09-21T07:54:04.958613+0000","last_peered":"2024-09-21T07:54:04.958613+0000","last_clean":"2024-09-21T07:54:04.958613+0000","last_became_active":"2024-09-21T07:54:03.952608+0000","last_became_peered":"2024-09-21T07:54:03.952608+0000","last_unstale":"2024-09-21T07:54:04.958613+0000","last_undegraded":"2024-09-21T07:54:04.958613+0000","last_fullsized":"2024-09-21T07:54:04.958613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:28.510634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959642+0000","last_change":"2024-09-21T07:54:03.978147+0000","last_active":"2024-09-21T07:54:04.959642+0000","last_peered":"2024-09-21T07:54:04.959642+0000","last_clean":"2024-09-21T07:54:04.959642+0000","last_became_active":"2024-09-21T07:54:03.977835+0000","last_became_peered":"2024-09-21T07:54:03.977835+0000","last_unstale":"2024-09-21T07:54:04.959642+0000","last_undegraded":"2024-09-21T07:54:04.959642+0000","last_fullsized":"2024-09-21T07:54:04.959642+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:08.623383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959080+0000","last_change":"2024-09-21T07:54:03.961202+0000","last_active":"2024-09-21T07:54:04.959080+0000","last_peered":"2024-09-21T07:54:04.959080+0000","last_clean":"2024-09-21T07:54:04.959080+0000","last_became_active":"2024-09-21T07:54:03.960869+0000","last_became_peered":"2024-09-21T07:54:03.960869+0000","last_unstale":"2024-09-21T07:54:04.959080+0000","last_undegraded":"2024-09-21T07:54:04.959080+0000","last_fullsized":"2024-09-21T07:54:04.959080+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:17.680155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900585+0000","last_change":"2024-09-21T07:54:03.962961+0000","last_active":"2024-09-21T07:54:05.900585+0000","last_peered":"2024-09-21T07:54:05.900585+0000","last_clean":"2024-09-21T07:54:05.900585+0000","last_became_active":"2024-09-21T07:54:03.961993+0000","last_became_peered":"2024-09-21T07:54:03.961993+0000","last_unstale":"2024-09-21T07:54:05.900585+0000","last_undegraded":"2024-09-21T07:54:05.900585+0000","last_fullsized":"2024-09-21T07:54:05.900585+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:18.411941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968589+0000","last_change":"2024-09-21T07:54:03.978915+0000","last_active":"2024-09-21T07:54:04.968589+0000","last_peered":"2024-09-21T07:54:04.968589+0000","last_clean":"2024-09-21T07:54:04.968589+0000","last_became_active":"2024-09-21T07:54:03.978592+0000","last_became_peered":"2024-09-21T07:54:03.978592+0000","last_unstale":"2024-09-21T07:54:04.968589+0000","last_undegraded":"2024-09-21T07:54:04.968589+0000","last_fullsized":"2024-09-21T07:54:04.968589+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:02.296991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038310+0000","last_change":"2024-09-21T07:54:03.955664+0000","last_active":"2024-09-21T07:54:06.038310+0000","last_peered":"2024-09-21T07:54:06.038310+0000","last_clean":"2024-09-21T07:54:06.038310+0000","last_became_active":"2024-09-21T07:54:03.955362+0000","last_became_peered":"2024-09-21T07:54:03.955362+0000","last_unstale":"2024-09-21T07:54:06.038310+0000","last_undegraded":"2024-09-21T07:54:06.038310+0000","last_fullsized":"2024-09-21T07:54:06.038310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:51.717790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968316+0000","last_change":"2024-09-21T07:54:03.964613+0000","last_active":"2024-09-21T07:54:04.968316+0000","last_peered":"2024-09-21T07:54:04.968316+0000","last_clean":"2024-09-21T07:54:04.968316+0000","last_became_active":"2024-09-21T07:54:03.964282+0000","last_became_peered":"2024-09-21T07:54:03.964282+0000","last_unstale":"2024-09-21T07:54:04.968316+0000","last_undegraded":"2024-09-21T07:54:04.968316+0000","last_fullsized":"2024-09-21T07:54:04.968316+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.785305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366577+0000","last_change":"2024-09-21T07:54:03.970963+0000","last_active":"2024-09-21T07:54:06.366577+0000","last_peered":"2024-09-21T07:54:06.366577+0000","last_clean":"2024-09-21T07:54:06.366577+0000","last_became_active":"2024-09-21T07:54:03.969094+0000","last_became_peered":"2024-09-21T07:54:03.969094+0000","last_unstale":"2024-09-21T07:54:06.366577+0000","last_undegraded":"2024-09-21T07:54:06.366577+0000","last_fullsized":"2024-09-21T07:54:06.366577+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:40.604013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959710+0000","last_change":"2024-09-21T07:54:03.969116+0000","last_active":"2024-09-21T07:54:04.959710+0000","last_peered":"2024-09-21T07:54:04.959710+0000","last_clean":"2024-09-21T07:54:04.959710+0000","last_became_active":"2024-09-21T07:54:03.968512+0000","last_became_peered":"2024-09-21T07:54:03.968512+0000","last_unstale":"2024-09-21T07:54:04.959710+0000","last_undegraded":"2024-09-21T07:54:04.959710+0000","last_fullsized":"2024-09-21T07:54:04.959710+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:21.288740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":19,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.969098+0000","last_change":"2024-09-21T07:54:02.963513+0000","last_active":"2024-09-21T07:54:04.969098+0000","last_peered":"2024-09-21T07:54:04.969098+0000","last_clean":"2024-09-21T07:54:04.969098+0000","last_became_active":"2024-09-21T07:54:02.962145+0000","last_became_peered":"2024-09-21T07:54:02.962145+0000","last_unstale":"2024-09-21T07:54:04.969098+0000","last_undegraded":"2024-09-21T07:54:04.969098+0000","last_fullsized":"2024-09-21T07:54:04.969098+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:56.239396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.969182+0000","last_change":"2024-09-21T07:54:03.962723+0000","last_active":"2024-09-21T07:54:04.969182+0000","last_peered":"2024-09-21T07:54:04.969182+0000","last_clean":"2024-09-21T07:54:04.969182+0000","last_became_active":"2024-09-21T07:54:03.962425+0000","last_became_peered":"2024-09-21T07:54:03.962425+0000","last_unstale":"2024-09-21T07:54:04.969182+0000","last_undegraded":"2024-09-21T07:54:04.969182+0000","last_fullsized":"2024-09-21T07:54:04.969182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:37.159884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038602+0000","last_change":"2024-09-21T07:54:02.957220+0000","last_active":"2024-09-21T07:54:06.038602+0000","last_peered":"2024-09-21T07:54:06.038602+0000","last_clean":"2024-09-21T07:54:06.038602+0000","last_became_active":"2024-09-21T07:54:02.956920+0000","last_became_peered":"2024-09-21T07:54:02.956920+0000","last_unstale":"2024-09-21T07:54:06.038602+0000","last_undegraded":"2024-09-21T07:54:06.038602+0000","last_fullsized":"2024-09-21T07:54:06.038602+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:18.227182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366973+0000","last_change":"2024-09-21T07:54:03.964431+0000","last_active":"2024-09-21T07:54:06.366973+0000","last_peered":"2024-09-21T07:54:06.366973+0000","last_clean":"2024-09-21T07:54:06.366973+0000","last_became_active":"2024-09-21T07:54:03.964101+0000","last_became_peered":"2024-09-21T07:54:03.964101+0000","last_unstale":"2024-09-21T07:54:06.366973+0000","last_undegraded":"2024-09-21T07:54:06.366973+0000","last_fullsized":"2024-09-21T07:54:06.366973+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:29.556167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366656+0000","last_change":"2024-09-21T07:54:02.966445+0000","last_active":"2024-09-21T07:54:06.366656+0000","last_peered":"2024-09-21T07:54:06.366656+0000","last_clean":"2024-09-21T07:54:06.366656+0000","last_became_active":"2024-09-21T07:54:02.965438+0000","last_became_peered":"2024-09-21T07:54:02.965438+0000","last_unstale":"2024-09-21T07:54:06.366656+0000","last_undegraded":"2024-09-21T07:54:06.366656+0000","last_fullsized":"2024-09-21T07:54:06.366656+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:18.199582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038681+0000","last_change":"2024-09-21T07:54:03.962416+0000","last_active":"2024-09-21T07:54:06.038681+0000","last_peered":"2024-09-21T07:54:06.038681+0000","last_clean":"2024-09-21T07:54:06.038681+0000","last_became_active":"2024-09-21T07:54:03.962001+0000","last_became_peered":"2024-09-21T07:54:03.962001+0000","last_unstale":"2024-09-21T07:54:06.038681+0000","last_undegraded":"2024-09-21T07:54:06.038681+0000","last_fullsized":"2024-09-21T07:54:06.038681+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:37.766666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958891+0000","last_change":"2024-09-21T07:54:02.958593+0000","last_active":"2024-09-21T07:54:04.958891+0000","last_peered":"2024-09-21T07:54:04.958891+0000","last_clean":"2024-09-21T07:54:04.958891+0000","last_became_active":"2024-09-21T07:54:02.958211+0000","last_became_peered":"2024-09-21T07:54:02.958211+0000","last_unstale":"2024-09-21T07:54:04.958891+0000","last_undegraded":"2024-09-21T07:54:04.958891+0000","last_fullsized":"2024-09-21T07:54:04.958891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:31.711026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958016+0000","last_change":"2024-09-21T07:54:03.967668+0000","last_active":"2024-09-21T07:54:04.958016+0000","last_peered":"2024-09-21T07:54:04.958016+0000","last_clean":"2024-09-21T07:54:04.958016+0000","last_became_active":"2024-09-21T07:54:03.967343+0000","last_became_peered":"2024-09-21T07:54:03.967343+0000","last_unstale":"2024-09-21T07:54:04.958016+0000","last_undegraded":"2024-09-21T07:54:04.958016+0000","last_fullsized":"2024-09-21T07:54:04.958016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:07.942496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968550+0000","last_change":"2024-09-21T07:54:02.961138+0000","last_active":"2024-09-21T07:54:04.968550+0000","last_peered":"2024-09-21T07:54:04.968550+0000","last_clean":"2024-09-21T07:54:04.968550+0000","last_became_active":"2024-09-21T07:54:02.960815+0000","last_became_peered":"2024-09-21T07:54:02.960815+0000","last_unstale":"2024-09-21T07:54:04.968550+0000","last_undegraded":"2024-09-21T07:54:04.968550+0000","last_fullsized":"2024-09-21T07:54:04.968550+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:43.585849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366880+0000","last_change":"2024-09-21T07:54:03.970083+0000","last_active":"2024-09-21T07:54:06.366880+0000","last_peered":"2024-09-21T07:54:06.366880+0000","last_clean":"2024-09-21T07:54:06.366880+0000","last_became_active":"2024-09-21T07:54:03.969739+0000","last_became_peered":"2024-09-21T07:54:03.969739+0000","last_unstale":"2024-09-21T07:54:06.366880+0000","last_undegraded":"2024-09-21T07:54:06.366880+0000","last_fullsized":"2024-09-21T07:54:06.366880+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:07.992623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958160+0000","last_change":"2024-09-21T07:54:02.962660+0000","last_active":"2024-09-21T07:54:04.958160+0000","last_peered":"2024-09-21T07:54:04.958160+0000","last_clean":"2024-09-21T07:54:04.958160+0000","last_became_active":"2024-09-21T07:54:02.962322+0000","last_became_peered":"2024-09-21T07:54:02.962322+0000","last_unstale":"2024-09-21T07:54:04.958160+0000","last_undegraded":"2024-09-21T07:54:04.958160+0000","last_fullsized":"2024-09-21T07:54:04.958160+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:34.944411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958237+0000","last_change":"2024-09-21T07:54:03.970486+0000","last_active":"2024-09-21T07:54:04.958237+0000","last_peered":"2024-09-21T07:54:04.958237+0000","last_clean":"2024-09-21T07:54:04.958237+0000","last_became_active":"2024-09-21T07:54:03.966867+0000","last_became_peered":"2024-09-21T07:54:03.966867+0000","last_unstale":"2024-09-21T07:54:04.958237+0000","last_undegraded":"2024-09-21T07:54:04.958237+0000","last_fullsized":"2024-09-21T07:54:04.958237+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:43.139253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968488+0000","last_change":"2024-09-21T07:54:02.962550+0000","last_active":"2024-09-21T07:54:04.968488+0000","last_peered":"2024-09-21T07:54:04.968488+0000","last_clean":"2024-09-21T07:54:04.968488+0000","last_became_active":"2024-09-21T07:54:02.961969+0000","last_became_peered":"2024-09-21T07:54:02.961969+0000","last_unstale":"2024-09-21T07:54:04.968488+0000","last_undegraded":"2024-09-21T07:54:04.968488+0000","last_fullsized":"2024-09-21T07:54:04.968488+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:02.980966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957714+0000","last_change":"2024-09-21T07:54:03.962509+0000","last_active":"2024-09-21T07:54:04.957714+0000","last_peered":"2024-09-21T07:54:04.957714+0000","last_clean":"2024-09-21T07:54:04.957714+0000","last_became_active":"2024-09-21T07:54:03.962188+0000","last_became_peered":"2024-09-21T07:54:03.962188+0000","last_unstale":"2024-09-21T07:54:04.957714+0000","last_undegraded":"2024-09-21T07:54:04.957714+0000","last_fullsized":"2024-09-21T07:54:04.957714+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.715935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957793+0000","last_change":"2024-09-21T07:54:02.962051+0000","last_active":"2024-09-21T07:54:04.957793+0000","last_peered":"2024-09-21T07:54:04.957793+0000","last_clean":"2024-09-21T07:54:04.957793+0000","last_became_active":"2024-09-21T07:54:02.961633+0000","last_became_peered":"2024-09-21T07:54:02.961633+0000","last_unstale":"2024-09-21T07:54:04.957793+0000","last_undegraded":"2024-09-21T07:54:04.957793+0000","last_fullsized":"2024-09-21T07:54:04.957793+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:38.462224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900477+0000","last_change":"2024-09-21T07:54:03.960976+0000","last_active":"2024-09-21T07:54:05.900477+0000","last_peered":"2024-09-21T07:54:05.900477+0000","last_clean":"2024-09-21T07:54:05.900477+0000","last_became_active":"2024-09-21T07:54:03.960025+0000","last_became_peered":"2024-09-21T07:54:03.960025+0000","last_unstale":"2024-09-21T07:54:05.900477+0000","last_undegraded":"2024-09-21T07:54:05.900477+0000","last_fullsized":"2024-09-21T07:54:05.900477+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:00.560494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.367135+0000","last_change":"2024-09-21T07:54:02.967133+0000","last_active":"2024-09-21T07:54:06.367135+0000","last_peered":"2024-09-21T07:54:06.367135+0000","last_clean":"2024-09-21T07:54:06.367135+0000","last_became_active":"2024-09-21T07:54:02.966787+0000","last_became_peered":"2024-09-21T07:54:02.966787+0000","last_unstale":"2024-09-21T07:54:06.367135+0000","last_undegraded":"2024-09-21T07:54:06.367135+0000","last_fullsized":"2024-09-21T07:54:06.367135+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:45.242747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900683+0000","last_change":"2024-09-21T07:54:03.961976+0000","last_active":"2024-09-21T07:54:05.900683+0000","last_peered":"2024-09-21T07:54:05.900683+0000","last_clean":"2024-09-21T07:54:05.900683+0000","last_became_active":"2024-09-21T07:54:03.959046+0000","last_became_peered":"2024-09-21T07:54:03.959046+0000","last_unstale":"2024-09-21T07:54:05.900683+0000","last_undegraded":"2024-09-21T07:54:05.900683+0000","last_fullsized":"2024-09-21T07:54:05.900683+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:51.717790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959294+0000","last_change":"2024-09-21T07:54:02.962199+0000","last_active":"2024-09-21T07:54:04.959294+0000","last_peered":"2024-09-21T07:54:04.959294+0000","last_clean":"2024-09-21T07:54:04.959294+0000","last_became_active":"2024-09-21T07:54:02.961858+0000","last_became_peered":"2024-09-21T07:54:02.961858+0000","last_unstale":"2024-09-21T07:54:04.959294+0000","last_undegraded":"2024-09-21T07:54:04.959294+0000","last_fullsized":"2024-09-21T07:54:04.959294+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:08.430367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968833+0000","last_change":"2024-09-21T07:54:03.979293+0000","last_active":"2024-09-21T07:54:04.968833+0000","last_peered":"2024-09-21T07:54:04.968833+0000","last_clean":"2024-09-21T07:54:04.968833+0000","last_became_active":"2024-09-21T07:54:03.978572+0000","last_became_peered":"2024-09-21T07:54:03.978572+0000","last_unstale":"2024-09-21T07:54:04.968833+0000","last_undegraded":"2024-09-21T07:54:04.968833+0000","last_fullsized":"2024-09-21T07:54:04.968833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.826345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959146+0000","last_change":"2024-09-21T07:54:02.966382+0000","last_active":"2024-09-21T07:54:04.959146+0000","last_peered":"2024-09-21T07:54:04.959146+0000","last_clean":"2024-09-21T07:54:04.959146+0000","last_became_active":"2024-09-21T07:54:02.966060+0000","last_became_peered":"2024-09-21T07:54:02.966060+0000","last_unstale":"2024-09-21T07:54:04.959146+0000","last_undegraded":"2024-09-21T07:54:04.959146+0000","last_fullsized":"2024-09-21T07:54:04.959146+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:50.372027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900231+0000","last_change":"2024-09-21T07:54:03.976974+0000","last_active":"2024-09-21T07:54:05.900231+0000","last_peered":"2024-09-21T07:54:05.900231+0000","last_clean":"2024-09-21T07:54:05.900231+0000","last_became_active":"2024-09-21T07:54:03.976570+0000","last_became_peered":"2024-09-21T07:54:03.976570+0000","last_unstale":"2024-09-21T07:54:05.900231+0000","last_undegraded":"2024-09-21T07:54:05.900231+0000","last_fullsized":"2024-09-21T07:54:05.900231+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:25.013417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958438+0000","last_change":"2024-09-21T07:54:02.962536+0000","last_active":"2024-09-21T07:54:04.958438+0000","last_peered":"2024-09-21T07:54:04.958438+0000","last_clean":"2024-09-21T07:54:04.958438+0000","last_became_active":"2024-09-21T07:54:02.962092+0000","last_became_peered":"2024-09-21T07:54:02.962092+0000","last_unstale":"2024-09-21T07:54:04.958438+0000","last_undegraded":"2024-09-21T07:54:04.958438+0000","last_fullsized":"2024-09-21T07:54:04.958438+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:25.218982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958869+0000","last_change":"2024-09-21T07:54:03.977300+0000","last_active":"2024-09-21T07:54:04.958869+0000","last_peered":"2024-09-21T07:54:04.958869+0000","last_clean":"2024-09-21T07:54:04.958869+0000","last_became_active":"2024-09-21T07:54:03.976964+0000","last_became_peered":"2024-09-21T07:54:03.976964+0000","last_unstale":"2024-09-21T07:54:04.958869+0000","last_undegraded":"2024-09-21T07:54:04.958869+0000","last_fullsized":"2024-09-21T07:54:04.958869+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:56.835444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.367039+0000","last_change":"2024-09-21T07:54:02.966964+0000","last_active":"2024-09-21T07:54:06.367039+0000","last_peered":"2024-09-21T07:54:06.367039+0000","last_clean":"2024-09-21T07:54:06.367039+0000","last_became_active":"2024-09-21T07:54:02.966459+0000","last_became_peered":"2024-09-21T07:54:02.966459+0000","last_unstale":"2024-09-21T07:54:06.367039+0000","last_undegraded":"2024-09-21T07:54:06.367039+0000","last_fullsized":"2024-09-21T07:54:06.367039+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:11.679312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366704+0000","last_change":"2024-09-21T07:54:03.972605+0000","last_active":"2024-09-21T07:54:06.366704+0000","last_peered":"2024-09-21T07:54:06.366704+0000","last_clean":"2024-09-21T07:54:06.366704+0000","last_became_active":"2024-09-21T07:54:03.971880+0000","last_became_peered":"2024-09-21T07:54:03.971880+0000","last_unstale":"2024-09-21T07:54:06.366704+0000","last_undegraded":"2024-09-21T07:54:06.366704+0000","last_fullsized":"2024-09-21T07:54:06.366704+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:44.780322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900460+0000","last_change":"2024-09-21T07:54:02.961634+0000","last_active":"2024-09-21T07:54:05.900460+0000","last_peered":"2024-09-21T07:54:05.900460+0000","last_clean":"2024-09-21T07:54:05.900460+0000","last_became_active":"2024-09-21T07:54:02.960524+0000","last_became_peered":"2024-09-21T07:54:02.960524+0000","last_unstale":"2024-09-21T07:54:05.900460+0000","last_undegraded":"2024-09-21T07:54:05.900460+0000","last_fullsized":"2024-09-21T07:54:05.900460+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:54.217436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038297+0000","last_change":"2024-09-21T07:54:03.962547+0000","last_active":"2024-09-21T07:54:06.038297+0000","last_peered":"2024-09-21T07:54:06.038297+0000","last_clean":"2024-09-21T07:54:06.038297+0000","last_became_active":"2024-09-21T07:54:03.961672+0000","last_became_peered":"2024-09-21T07:54:03.961672+0000","last_unstale":"2024-09-21T07:54:06.038297+0000","last_undegraded":"2024-09-21T07:54:06.038297+0000","last_fullsized":"2024-09-21T07:54:06.038297+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.270102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958319+0000","last_change":"2024-09-21T07:54:02.967430+0000","last_active":"2024-09-21T07:54:04.958319+0000","last_peered":"2024-09-21T07:54:04.958319+0000","last_clean":"2024-09-21T07:54:04.958319+0000","last_became_active":"2024-09-21T07:54:02.966983+0000","last_became_peered":"2024-09-21T07:54:02.966983+0000","last_unstale":"2024-09-21T07:54:04.958319+0000","last_undegraded":"2024-09-21T07:54:04.958319+0000","last_fullsized":"2024-09-21T07:54:04.958319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:08.179354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900854+0000","last_change":"2024-09-21T07:54:03.964969+0000","last_active":"2024-09-21T07:54:05.900854+0000","last_peered":"2024-09-21T07:54:05.900854+0000","last_clean":"2024-09-21T07:54:05.900854+0000","last_became_active":"2024-09-21T07:54:03.964136+0000","last_became_peered":"2024-09-21T07:54:03.964136+0000","last_unstale":"2024-09-21T07:54:05.900854+0000","last_undegraded":"2024-09-21T07:54:05.900854+0000","last_fullsized":"2024-09-21T07:54:05.900854+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:55.776930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959064+0000","last_change":"2024-09-21T07:54:02.965358+0000","last_active":"2024-09-21T07:54:04.959064+0000","last_peered":"2024-09-21T07:54:04.959064+0000","last_clean":"2024-09-21T07:54:04.959064+0000","last_became_active":"2024-09-21T07:54:02.965047+0000","last_became_peered":"2024-09-21T07:54:02.965047+0000","last_unstale":"2024-09-21T07:54:04.959064+0000","last_undegraded":"2024-09-21T07:54:04.959064+0000","last_fullsized":"2024-09-21T07:54:04.959064+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:31.297113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959968+0000","last_change":"2024-09-21T07:54:03.962307+0000","last_active":"2024-09-21T07:54:04.959968+0000","last_peered":"2024-09-21T07:54:04.959968+0000","last_clean":"2024-09-21T07:54:04.959968+0000","last_became_active":"2024-09-21T07:54:03.961867+0000","last_became_peered":"2024-09-21T07:54:03.961867+0000","last_unstale":"2024-09-21T07:54:04.959968+0000","last_undegraded":"2024-09-21T07:54:04.959968+0000","last_fullsized":"2024-09-21T07:54:04.959968+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:50.043563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038656+0000","last_change":"2024-09-21T07:54:02.959157+0000","last_active":"2024-09-21T07:54:06.038656+0000","last_peered":"2024-09-21T07:54:06.038656+0000","last_clean":"2024-09-21T07:54:06.038656+0000","last_became_active":"2024-09-21T07:54:02.958842+0000","last_became_peered":"2024-09-21T07:54:02.958842+0000","last_unstale":"2024-09-21T07:54:06.038656+0000","last_undegraded":"2024-09-21T07:54:06.038656+0000","last_fullsized":"2024-09-21T07:54:06.038656+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:59.905148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958053+0000","last_change":"2024-09-21T07:54:03.965420+0000","last_active":"2024-09-21T07:54:04.958053+0000","last_peered":"2024-09-21T07:54:04.958053+0000","last_clean":"2024-09-21T07:54:04.958053+0000","last_became_active":"2024-09-21T07:54:03.964768+0000","last_became_peered":"2024-09-21T07:54:03.964768+0000","last_unstale":"2024-09-21T07:54:04.958053+0000","last_undegraded":"2024-09-21T07:54:04.958053+0000","last_fullsized":"2024-09-21T07:54:04.958053+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:05.973590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958958+0000","last_change":"2024-09-21T07:54:02.961479+0000","last_active":"2024-09-21T07:54:04.958958+0000","last_peered":"2024-09-21T07:54:04.958958+0000","last_clean":"2024-09-21T07:54:04.958958+0000","last_became_active":"2024-09-21T07:54:02.961164+0000","last_became_peered":"2024-09-21T07:54:02.961164+0000","last_unstale":"2024-09-21T07:54:04.958958+0000","last_undegraded":"2024-09-21T07:54:04.958958+0000","last_fullsized":"2024-09-21T07:54:04.958958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:44.093493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959666+0000","last_change":"2024-09-21T07:54:03.968709+0000","last_active":"2024-09-21T07:54:04.959666+0000","last_peered":"2024-09-21T07:54:04.959666+0000","last_clean":"2024-09-21T07:54:04.959666+0000","last_became_active":"2024-09-21T07:54:03.968406+0000","last_became_peered":"2024-09-21T07:54:03.968406+0000","last_unstale":"2024-09-21T07:54:04.959666+0000","last_undegraded":"2024-09-21T07:54:04.959666+0000","last_fullsized":"2024-09-21T07:54:04.959666+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:40.213473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964224+0000","last_change":"2024-09-21T07:54:02.960646+0000","last_active":"2024-09-21T07:54:04.964224+0000","last_peered":"2024-09-21T07:54:04.964224+0000","last_clean":"2024-09-21T07:54:04.964224+0000","last_became_active":"2024-09-21T07:54:02.959784+0000","last_became_peered":"2024-09-21T07:54:02.959784+0000","last_unstale":"2024-09-21T07:54:04.964224+0000","last_undegraded":"2024-09-21T07:54:04.964224+0000","last_fullsized":"2024-09-21T07:54:04.964224+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:50.206378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968339+0000","last_change":"2024-09-21T07:54:03.955882+0000","last_active":"2024-09-21T07:54:04.968339+0000","last_peered":"2024-09-21T07:54:04.968339+0000","last_clean":"2024-09-21T07:54:04.968339+0000","last_became_active":"2024-09-21T07:54:03.955600+0000","last_became_peered":"2024-09-21T07:54:03.955600+0000","last_unstale":"2024-09-21T07:54:04.968339+0000","last_undegraded":"2024-09-21T07:54:04.968339+0000","last_fullsized":"2024-09-21T07:54:04.968339+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:10.424899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959121+0000","last_change":"2024-09-21T07:54:02.954640+0000","last_active":"2024-09-21T07:54:04.959121+0000","last_peered":"2024-09-21T07:54:04.959121+0000","last_clean":"2024-09-21T07:54:04.959121+0000","last_became_active":"2024-09-21T07:54:02.954337+0000","last_became_peered":"2024-09-21T07:54:02.954337+0000","last_unstale":"2024-09-21T07:54:04.959121+0000","last_undegraded":"2024-09-21T07:54:04.959121+0000","last_fullsized":"2024-09-21T07:54:04.959121+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:23.793120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959125+0000","last_change":"2024-09-21T07:54:03.968291+0000","last_active":"2024-09-21T07:54:04.959125+0000","last_peered":"2024-09-21T07:54:04.959125+0000","last_clean":"2024-09-21T07:54:04.959125+0000","last_became_active":"2024-09-21T07:54:03.967801+0000","last_became_peered":"2024-09-21T07:54:03.967801+0000","last_unstale":"2024-09-21T07:54:04.959125+0000","last_undegraded":"2024-09-21T07:54:04.959125+0000","last_fullsized":"2024-09-21T07:54:04.959125+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.785305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.963791+0000","last_change":"2024-09-21T07:54:02.959629+0000","last_active":"2024-09-21T07:54:04.963791+0000","last_peered":"2024-09-21T07:54:04.963791+0000","last_clean":"2024-09-21T07:54:04.963791+0000","last_became_active":"2024-09-21T07:54:02.958113+0000","last_became_peered":"2024-09-21T07:54:02.958113+0000","last_unstale":"2024-09-21T07:54:04.963791+0000","last_undegraded":"2024-09-21T07:54:04.963791+0000","last_fullsized":"2024-09-21T07:54:04.963791+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:28.662913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968244+0000","last_change":"2024-09-21T07:54:03.975168+0000","last_active":"2024-09-21T07:54:04.968244+0000","last_peered":"2024-09-21T07:54:04.968244+0000","last_clean":"2024-09-21T07:54:04.968244+0000","last_became_active":"2024-09-21T07:54:03.974786+0000","last_became_peered":"2024-09-21T07:54:03.974786+0000","last_unstale":"2024-09-21T07:54:04.968244+0000","last_undegraded":"2024-09-21T07:54:04.968244+0000","last_fullsized":"2024-09-21T07:54:04.968244+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:37.766666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957927+0000","last_change":"2024-09-21T07:54:02.967784+0000","last_active":"2024-09-21T07:54:04.957927+0000","last_peered":"2024-09-21T07:54:04.957927+0000","last_clean":"2024-09-21T07:54:04.957927+0000","last_became_active":"2024-09-21T07:54:02.967465+0000","last_became_peered":"2024-09-21T07:54:02.967465+0000","last_unstale":"2024-09-21T07:54:04.957927+0000","last_undegraded":"2024-09-21T07:54:04.957927+0000","last_fullsized":"2024-09-21T07:54:04.957927+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:01.289564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900331+0000","last_change":"2024-09-21T07:54:03.957594+0000","last_active":"2024-09-21T07:54:05.900331+0000","last_peered":"2024-09-21T07:54:05.900331+0000","last_clean":"2024-09-21T07:54:05.900331+0000","last_became_active":"2024-09-21T07:54:03.957233+0000","last_became_peered":"2024-09-21T07:54:03.957233+0000","last_unstale":"2024-09-21T07:54:05.900331+0000","last_undegraded":"2024-09-21T07:54:05.900331+0000","last_fullsized":"2024-09-21T07:54:05.900331+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:14.602178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.963771+0000","last_change":"2024-09-21T07:54:02.957009+0000","last_active":"2024-09-21T07:54:04.963771+0000","last_peered":"2024-09-21T07:54:04.963771+0000","last_clean":"2024-09-21T07:54:04.963771+0000","last_became_active":"2024-09-21T07:54:02.956077+0000","last_became_peered":"2024-09-21T07:54:02.956077+0000","last_unstale":"2024-09-21T07:54:04.963771+0000","last_undegraded":"2024-09-21T07:54:04.963771+0000","last_fullsized":"2024-09-21T07:54:04.963771+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:26.957303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959407+0000","last_change":"2024-09-21T07:54:03.979237+0000","last_active":"2024-09-21T07:54:04.959407+0000","last_peered":"2024-09-21T07:54:04.959407+0000","last_clean":"2024-09-21T07:54:04.959407+0000","last_became_active":"2024-09-21T07:54:03.977294+0000","last_became_peered":"2024-09-21T07:54:03.977294+0000","last_unstale":"2024-09-21T07:54:04.959407+0000","last_undegraded":"2024-09-21T07:54:04.959407+0000","last_fullsized":"2024-09-21T07:54:04.959407+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:10.424899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366999+0000","last_change":"2024-09-21T07:54:02.965440+0000","last_active":"2024-09-21T07:54:06.366999+0000","last_peered":"2024-09-21T07:54:06.366999+0000","last_clean":"2024-09-21T07:54:06.366999+0000","last_became_active":"2024-09-21T07:54:02.965078+0000","last_became_peered":"2024-09-21T07:54:02.965078+0000","last_unstale":"2024-09-21T07:54:06.366999+0000","last_undegraded":"2024-09-21T07:54:06.366999+0000","last_fullsized":"2024-09-21T07:54:06.366999+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:43.073196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959964+0000","last_change":"2024-09-21T07:54:03.970694+0000","last_active":"2024-09-21T07:54:04.959964+0000","last_peered":"2024-09-21T07:54:04.959964+0000","last_clean":"2024-09-21T07:54:04.959964+0000","last_became_active":"2024-09-21T07:54:03.969861+0000","last_became_peered":"2024-09-21T07:54:03.969861+0000","last_unstale":"2024-09-21T07:54:04.959964+0000","last_undegraded":"2024-09-21T07:54:04.959964+0000","last_fullsized":"2024-09-21T07:54:04.959964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:48.151079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.965432+0000","last_change":"2024-09-21T07:54:02.955571+0000","last_active":"2024-09-21T07:54:04.965432+0000","last_peered":"2024-09-21T07:54:04.965432+0000","last_clean":"2024-09-21T07:54:04.965432+0000","last_became_active":"2024-09-21T07:54:02.955159+0000","last_became_peered":"2024-09-21T07:54:02.955159+0000","last_unstale":"2024-09-21T07:54:04.965432+0000","last_undegraded":"2024-09-21T07:54:04.965432+0000","last_fullsized":"2024-09-21T07:54:04.965432+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:04.619726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958160+0000","last_change":"2024-09-21T07:54:03.966124+0000","last_active":"2024-09-21T07:54:04.958160+0000","last_peered":"2024-09-21T07:54:04.958160+0000","last_clean":"2024-09-21T07:54:04.958160+0000","last_became_active":"2024-09-21T07:54:03.965809+0000","last_became_peered":"2024-09-21T07:54:03.965809+0000","last_unstale":"2024-09-21T07:54:04.958160+0000","last_undegraded":"2024-09-21T07:54:04.958160+0000","last_fullsized":"2024-09-21T07:54:04.958160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:12.134848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968943+0000","last_change":"2024-09-21T07:54:02.963499+0000","last_active":"2024-09-21T07:54:04.968943+0000","last_peered":"2024-09-21T07:54:04.968943+0000","last_clean":"2024-09-21T07:54:04.968943+0000","last_became_active":"2024-09-21T07:54:02.963167+0000","last_became_peered":"2024-09-21T07:54:02.963167+0000","last_unstale":"2024-09-21T07:54:04.968943+0000","last_undegraded":"2024-09-21T07:54:04.968943+0000","last_fullsized":"2024-09-21T07:54:04.968943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:17.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959862+0000","last_change":"2024-09-21T07:54:03.955452+0000","last_active":"2024-09-21T07:54:04.959862+0000","last_peered":"2024-09-21T07:54:04.959862+0000","last_clean":"2024-09-21T07:54:04.959862+0000","last_became_active":"2024-09-21T07:54:03.955071+0000","last_became_peered":"2024-09-21T07:54:03.955071+0000","last_unstale":"2024-09-21T07:54:04.959862+0000","last_undegraded":"2024-09-21T07:54:04.959862+0000","last_fullsized":"2024-09-21T07:54:04.959862+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.921075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958963+0000","last_change":"2024-09-21T07:54:02.952969+0000","last_active":"2024-09-21T07:54:04.958963+0000","last_peered":"2024-09-21T07:54:04.958963+0000","last_clean":"2024-09-21T07:54:04.958963+0000","last_became_active":"2024-09-21T07:54:02.952626+0000","last_became_peered":"2024-09-21T07:54:02.952626+0000","last_unstale":"2024-09-21T07:54:04.958963+0000","last_undegraded":"2024-09-21T07:54:04.958963+0000","last_fullsized":"2024-09-21T07:54:04.958963+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:28.850088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366774+0000","last_change":"2024-09-21T07:54:03.969012+0000","last_active":"2024-09-21T07:54:06.366774+0000","last_peered":"2024-09-21T07:54:06.366774+0000","last_clean":"2024-09-21T07:54:06.366774+0000","last_became_active":"2024-09-21T07:54:03.967345+0000","last_became_peered":"2024-09-21T07:54:03.967345+0000","last_unstale":"2024-09-21T07:54:06.366774+0000","last_undegraded":"2024-09-21T07:54:06.366774+0000","last_fullsized":"2024-09-21T07:54:06.366774+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:28.542602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958594+0000","last_change":"2024-09-21T07:54:02.959907+0000","last_active":"2024-09-21T07:54:04.958594+0000","last_peered":"2024-09-21T07:54:04.958594+0000","last_clean":"2024-09-21T07:54:04.958594+0000","last_became_active":"2024-09-21T07:54:02.959583+0000","last_became_peered":"2024-09-21T07:54:02.959583+0000","last_unstale":"2024-09-21T07:54:04.958594+0000","last_undegraded":"2024-09-21T07:54:04.958594+0000","last_fullsized":"2024-09-21T07:54:04.958594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:37.103572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968410+0000","last_change":"2024-09-21T07:54:03.963537+0000","last_active":"2024-09-21T07:54:04.968410+0000","last_peered":"2024-09-21T07:54:04.968410+0000","last_clean":"2024-09-21T07:54:04.968410+0000","last_became_active":"2024-09-21T07:54:03.963239+0000","last_became_peered":"2024-09-21T07:54:03.963239+0000","last_unstale":"2024-09-21T07:54:04.968410+0000","last_undegraded":"2024-09-21T07:54:04.968410+0000","last_fullsized":"2024-09-21T07:54:04.968410+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:08.623383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900424+0000","last_change":"2024-09-21T07:54:02.957031+0000","last_active":"2024-09-21T07:54:05.900424+0000","last_peered":"2024-09-21T07:54:05.900424+0000","last_clean":"2024-09-21T07:54:05.900424+0000","last_became_active":"2024-09-21T07:54:02.955404+0000","last_became_peered":"2024-09-21T07:54:02.955404+0000","last_unstale":"2024-09-21T07:54:05.900424+0000","last_undegraded":"2024-09-21T07:54:05.900424+0000","last_fullsized":"2024-09-21T07:54:05.900424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:45.509611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900143+0000","last_change":"2024-09-21T07:54:03.972749+0000","last_active":"2024-09-21T07:54:05.900143+0000","last_peered":"2024-09-21T07:54:05.900143+0000","last_clean":"2024-09-21T07:54:05.900143+0000","last_became_active":"2024-09-21T07:54:03.972418+0000","last_became_peered":"2024-09-21T07:54:03.972418+0000","last_unstale":"2024-09-21T07:54:05.900143+0000","last_undegraded":"2024-09-21T07:54:05.900143+0000","last_fullsized":"2024-09-21T07:54:05.900143+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:57.246823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968560+0000","last_change":"2024-09-21T07:54:02.966502+0000","last_active":"2024-09-21T07:54:04.968560+0000","last_peered":"2024-09-21T07:54:04.968560+0000","last_clean":"2024-09-21T07:54:04.968560+0000","last_became_active":"2024-09-21T07:54:02.966190+0000","last_became_peered":"2024-09-21T07:54:02.966190+0000","last_unstale":"2024-09-21T07:54:04.968560+0000","last_undegraded":"2024-09-21T07:54:04.968560+0000","last_fullsized":"2024-09-21T07:54:04.968560+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:42.665964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968435+0000","last_change":"2024-09-21T07:54:03.960880+0000","last_active":"2024-09-21T07:54:04.968435+0000","last_peered":"2024-09-21T07:54:04.968435+0000","last_clean":"2024-09-21T07:54:04.968435+0000","last_became_active":"2024-09-21T07:54:03.959027+0000","last_became_peered":"2024-09-21T07:54:03.959027+0000","last_unstale":"2024-09-21T07:54:04.968435+0000","last_undegraded":"2024-09-21T07:54:04.968435+0000","last_fullsized":"2024-09-21T07:54:04.968435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:26.226409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968139+0000","last_change":"2024-09-21T07:54:02.963984+0000","last_active":"2024-09-21T07:54:04.968139+0000","last_peered":"2024-09-21T07:54:04.968139+0000","last_clean":"2024-09-21T07:54:04.968139+0000","last_became_active":"2024-09-21T07:54:02.961988+0000","last_became_peered":"2024-09-21T07:54:02.961988+0000","last_unstale":"2024-09-21T07:54:04.968139+0000","last_undegraded":"2024-09-21T07:54:04.968139+0000","last_fullsized":"2024-09-21T07:54:04.968139+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:37.630887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900760+0000","last_change":"2024-09-21T07:54:03.959690+0000","last_active":"2024-09-21T07:54:05.900760+0000","last_peered":"2024-09-21T07:54:05.900760+0000","last_clean":"2024-09-21T07:54:05.900760+0000","last_became_active":"2024-09-21T07:54:03.958796+0000","last_became_peered":"2024-09-21T07:54:03.958796+0000","last_unstale":"2024-09-21T07:54:05.900760+0000","last_undegraded":"2024-09-21T07:54:05.900760+0000","last_fullsized":"2024-09-21T07:54:05.900760+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:07.430818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958728+0000","last_change":"2024-09-21T07:54:02.963980+0000","last_active":"2024-09-21T07:54:04.958728+0000","last_peered":"2024-09-21T07:54:04.958728+0000","last_clean":"2024-09-21T07:54:04.958728+0000","last_became_active":"2024-09-21T07:54:02.963687+0000","last_became_peered":"2024-09-21T07:54:02.963687+0000","last_unstale":"2024-09-21T07:54:04.958728+0000","last_undegraded":"2024-09-21T07:54:04.958728+0000","last_fullsized":"2024-09-21T07:54:04.958728+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:27.748680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":372,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958394+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:54:04.958394+0000","last_peered":"2024-09-21T07:54:04.958394+0000","last_clean":"2024-09-21T07:54:04.958394+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:54:04.958394+0000","last_undegraded":"2024-09-21T07:54:04.958394+0000","last_fullsized":"2024-09-21T07:54:04.958394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1798144,"data_stored":794088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":800450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33552,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338288,"statfs":{"total":96636764160,"available":96602406912,"internally_reserved":0,"allocated":2412544,"data_stored":1391456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1814528,"data_stored":800450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476840,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1826816,"data_stored":802050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476840,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1814528,"data_stored":800998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476837,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33568,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338272,"statfs":{"total":96636764160,"available":96602390528,"internally_reserved":0,"allocated":2428928,"data_stored":1392578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476837,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":800486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:54:08.375 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:54:08.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 -- 172.21.15.88:0/601507484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 msgr2=0x7f02b4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 --2- 172.21.15.88:0/601507484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02b4003e30 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7f02c0001990 tx=0x7f02b4005830 comp rx=0 tx=0).stop 2024-09-21T07:54:08.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 -- 172.21.15.88:0/601507484 shutdown_connections 2024-09-21T07:54:08.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 --2- 172.21.15.88:0/601507484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02b4003e30 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 -- 172.21.15.88:0/601507484 >> 172.21.15.88:0/601507484 conn(0x7f02d006d730 msgr2=0x7f02d00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:08.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 -- 172.21.15.88:0/601507484 shutdown_connections 2024-09-21T07:54:08.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.630+0000 7f02d4a00640 1 -- 172.21.15.88:0/601507484 wait complete. 2024-09-21T07:54:08.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.632+0000 7f02d4a00640 1 Processor -- start 2024-09-21T07:54:08.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.632+0000 7f02d4a00640 1 -- start start 2024-09-21T07:54:08.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02d4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02d014f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02d4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00732c0 0x7f02d00ceda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02d4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d0158150 0x7f02d0158560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02d4a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02d00b2750 con 0x7f02b4003a00 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02d4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02d015f6f0 con 0x7f02d00732c0 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02d4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02d00b28d0 con 0x7f02d0158150 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02cde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02d014f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02cde00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02d014f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51436/0 (socket says 172.21.15.88:51436) 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02cde00640 1 -- 172.21.15.88:0/1004786356 learned_addr learned my addr 172.21.15.88:0/1004786356 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02c7e00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00732c0 0x7f02d00ceda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.634+0000 7f02ce800640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d0158150 0x7f02d0158560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02cde00640 1 -- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d0158150 msgr2=0x7f02d0158560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02cde00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d0158150 0x7f02d0158560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02cde00640 1 -- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00732c0 msgr2=0x7f02d00ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02cde00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00732c0 0x7f02d00ceda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02cde00640 1 -- 172.21.15.88:0/1004786356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d0158aa0 con 0x7f02b4003a00 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02c7e00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00732c0 0x7f02d00ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02cde00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02d014f880 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f02c0001990 tx=0x7f02c00061f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c0037040 con 0x7f02b4003a00 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02c0006420 con 0x7f02b4003a00 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d0157390 con 0x7f02b4003a00 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.635+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c00048a0 con 0x7f02b4003a00 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.636+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d0157950 con 0x7f02b4003a00 2024-09-21T07:54:08.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.637+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02d015e350 con 0x7f02b4003a00 2024-09-21T07:54:08.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.638+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f02c0045450 con 0x7f02b4003a00 2024-09-21T07:54:08.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.639+0000 7f02c6000640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02ac079250 0x7f02ac07b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:08.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.639+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f02c00bb690 con 0x7f02b4003a00 2024-09-21T07:54:08.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.639+0000 7f02c7e00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02ac079250 0x7f02ac07b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:08.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.640+0000 7f02c7e00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02ac079250 0x7f02ac07b760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005eb0 tx=0x7f02b8005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:08.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.643+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02d015e350 con 0x7f02b4003a00 2024-09-21T07:54:08.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.824+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f02d0157140 con 0x7f02ac079250 2024-09-21T07:54:08.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.829+0000 7f02c6000640 1 -- 172.21.15.88:0/1004786356 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252481 (secure 0 0 0) 0x7f02d0157140 con 0x7f02ac079250 2024-09-21T07:54:08.833 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:08.834 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02ac079250 msgr2=0x7f02ac07b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02ac079250 0x7f02ac07b760 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005eb0 tx=0x7f02b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 msgr2=0x7f02d014f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02d014f880 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f02c0001990 tx=0x7f02c00061f0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 shutdown_connections 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02ac079250 0x7f02ac07b760 secure :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f02b8005eb0 tx=0x7f02b8005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02d0158150 0x7f02d0158560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02d00732c0 0x7f02d00ceda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 --2- 172.21.15.88:0/1004786356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b4003a00 0x7f02d014f880 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:08.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 >> 172.21.15.88:0/1004786356 conn(0x7f02d006d730 msgr2=0x7f02d00d0230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:08.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 shutdown_connections 2024-09-21T07:54:08.837 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:08.833+0000 7f02d4a00640 1 -- 172.21.15.88:0/1004786356 wait complete. 2024-09-21T07:54:08.860 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-09-21T07:54:07.741660+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":186,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":264896,"kb_used_data":15356,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754709824,"statfs":{"total":773094113280,"available":772822859776,"internally_reserved":0,"allocated":15724544,"data_stored":7582556,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.811461"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968860+0000","last_change":"2024-09-21T07:54:03.967776+0000","last_active":"2024-09-21T07:54:04.968860+0000","last_peered":"2024-09-21T07:54:04.968860+0000","last_clean":"2024-09-21T07:54:04.968860+0000","last_became_active":"2024-09-21T07:54:03.967239+0000","last_became_peered":"2024-09-21T07:54:03.967239+0000","last_unstale":"2024-09-21T07:54:04.968860+0000","last_undegraded":"2024-09-21T07:54:04.968860+0000","last_fullsized":"2024-09-21T07:54:04.968860+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:02:21.500284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.969070+0000","last_change":"2024-09-21T07:54:03.972467+0000","last_active":"2024-09-21T07:54:04.969070+0000","last_peered":"2024-09-21T07:54:04.969070+0000","last_clean":"2024-09-21T07:54:04.969070+0000","last_became_active":"2024-09-21T07:54:03.972169+0000","last_became_peered":"2024-09-21T07:54:03.972169+0000","last_unstale":"2024-09-21T07:54:04.969070+0000","last_undegraded":"2024-09-21T07:54:04.969070+0000","last_fullsized":"2024-09-21T07:54:04.969070+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:02:00.705151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968753+0000","last_change":"2024-09-21T07:54:03.979603+0000","last_active":"2024-09-21T07:54:04.968753+0000","last_peered":"2024-09-21T07:54:04.968753+0000","last_clean":"2024-09-21T07:54:04.968753+0000","last_became_active":"2024-09-21T07:54:03.979004+0000","last_became_peered":"2024-09-21T07:54:03.979004+0000","last_unstale":"2024-09-21T07:54:04.968753+0000","last_undegraded":"2024-09-21T07:54:04.968753+0000","last_fullsized":"2024-09-21T07:54:04.968753+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:35.951838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957865+0000","last_change":"2024-09-21T07:54:03.965065+0000","last_active":"2024-09-21T07:54:04.957865+0000","last_peered":"2024-09-21T07:54:04.957865+0000","last_clean":"2024-09-21T07:54:04.957865+0000","last_became_active":"2024-09-21T07:54:03.964387+0000","last_became_peered":"2024-09-21T07:54:03.964387+0000","last_unstale":"2024-09-21T07:54:04.957865+0000","last_undegraded":"2024-09-21T07:54:04.957865+0000","last_fullsized":"2024-09-21T07:54:04.957865+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:07:58.614172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959116+0000","last_change":"2024-09-21T07:54:03.957510+0000","last_active":"2024-09-21T07:54:04.959116+0000","last_peered":"2024-09-21T07:54:04.959116+0000","last_clean":"2024-09-21T07:54:04.959116+0000","last_became_active":"2024-09-21T07:54:03.956596+0000","last_became_peered":"2024-09-21T07:54:03.956596+0000","last_unstale":"2024-09-21T07:54:04.959116+0000","last_undegraded":"2024-09-21T07:54:04.959116+0000","last_fullsized":"2024-09-21T07:54:04.959116+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:44:39.498696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366860+0000","last_change":"2024-09-21T07:54:03.972831+0000","last_active":"2024-09-21T07:54:06.366860+0000","last_peered":"2024-09-21T07:54:06.366860+0000","last_clean":"2024-09-21T07:54:06.366860+0000","last_became_active":"2024-09-21T07:54:03.971051+0000","last_became_peered":"2024-09-21T07:54:03.971051+0000","last_unstale":"2024-09-21T07:54:06.366860+0000","last_undegraded":"2024-09-21T07:54:06.366860+0000","last_fullsized":"2024-09-21T07:54:06.366860+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:57.496456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900223+0000","last_change":"2024-09-21T07:54:03.966873+0000","last_active":"2024-09-21T07:54:05.900223+0000","last_peered":"2024-09-21T07:54:05.900223+0000","last_clean":"2024-09-21T07:54:05.900223+0000","last_became_active":"2024-09-21T07:54:03.966550+0000","last_became_peered":"2024-09-21T07:54:03.966550+0000","last_unstale":"2024-09-21T07:54:05.900223+0000","last_undegraded":"2024-09-21T07:54:05.900223+0000","last_fullsized":"2024-09-21T07:54:05.900223+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:24.800547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038302+0000","last_change":"2024-09-21T07:54:03.979521+0000","last_active":"2024-09-21T07:54:06.038302+0000","last_peered":"2024-09-21T07:54:06.038302+0000","last_clean":"2024-09-21T07:54:06.038302+0000","last_became_active":"2024-09-21T07:54:03.979199+0000","last_became_peered":"2024-09-21T07:54:03.979199+0000","last_unstale":"2024-09-21T07:54:06.038302+0000","last_undegraded":"2024-09-21T07:54:06.038302+0000","last_fullsized":"2024-09-21T07:54:06.038302+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:47.583215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038658+0000","last_change":"2024-09-21T07:54:03.952233+0000","last_active":"2024-09-21T07:54:06.038658+0000","last_peered":"2024-09-21T07:54:06.038658+0000","last_clean":"2024-09-21T07:54:06.038658+0000","last_became_active":"2024-09-21T07:54:03.949630+0000","last_became_peered":"2024-09-21T07:54:03.949630+0000","last_unstale":"2024-09-21T07:54:06.038658+0000","last_undegraded":"2024-09-21T07:54:06.038658+0000","last_fullsized":"2024-09-21T07:54:06.038658+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:56.835444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038567+0000","last_change":"2024-09-21T07:54:03.958744+0000","last_active":"2024-09-21T07:54:06.038567+0000","last_peered":"2024-09-21T07:54:06.038567+0000","last_clean":"2024-09-21T07:54:06.038567+0000","last_became_active":"2024-09-21T07:54:03.958123+0000","last_became_peered":"2024-09-21T07:54:03.958123+0000","last_unstale":"2024-09-21T07:54:06.038567+0000","last_undegraded":"2024-09-21T07:54:06.038567+0000","last_fullsized":"2024-09-21T07:54:06.038567+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:43.139253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958890+0000","last_change":"2024-09-21T07:54:03.960751+0000","last_active":"2024-09-21T07:54:04.958890+0000","last_peered":"2024-09-21T07:54:04.958890+0000","last_clean":"2024-09-21T07:54:04.958890+0000","last_became_active":"2024-09-21T07:54:03.960213+0000","last_became_peered":"2024-09-21T07:54:03.960213+0000","last_unstale":"2024-09-21T07:54:04.958890+0000","last_undegraded":"2024-09-21T07:54:04.958890+0000","last_fullsized":"2024-09-21T07:54:04.958890+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:41:18.963314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964102+0000","last_change":"2024-09-21T07:54:03.974927+0000","last_active":"2024-09-21T07:54:04.964102+0000","last_peered":"2024-09-21T07:54:04.964102+0000","last_clean":"2024-09-21T07:54:04.964102+0000","last_became_active":"2024-09-21T07:54:03.974635+0000","last_became_peered":"2024-09-21T07:54:03.974635+0000","last_unstale":"2024-09-21T07:54:04.964102+0000","last_undegraded":"2024-09-21T07:54:04.964102+0000","last_fullsized":"2024-09-21T07:54:04.964102+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:07:58.461584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964047+0000","last_change":"2024-09-21T07:54:03.970845+0000","last_active":"2024-09-21T07:54:04.964047+0000","last_peered":"2024-09-21T07:54:04.964047+0000","last_clean":"2024-09-21T07:54:04.964047+0000","last_became_active":"2024-09-21T07:54:03.970084+0000","last_became_peered":"2024-09-21T07:54:03.970084+0000","last_unstale":"2024-09-21T07:54:04.964047+0000","last_undegraded":"2024-09-21T07:54:04.964047+0000","last_fullsized":"2024-09-21T07:54:04.964047+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:48:20.271595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038430+0000","last_change":"2024-09-21T07:54:03.956973+0000","last_active":"2024-09-21T07:54:06.038430+0000","last_peered":"2024-09-21T07:54:06.038430+0000","last_clean":"2024-09-21T07:54:06.038430+0000","last_became_active":"2024-09-21T07:54:03.956642+0000","last_became_peered":"2024-09-21T07:54:03.956642+0000","last_unstale":"2024-09-21T07:54:06.038430+0000","last_undegraded":"2024-09-21T07:54:06.038430+0000","last_fullsized":"2024-09-21T07:54:06.038430+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:46:40.718444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038835+0000","last_change":"2024-09-21T07:54:03.963106+0000","last_active":"2024-09-21T07:54:06.038835+0000","last_peered":"2024-09-21T07:54:06.038835+0000","last_clean":"2024-09-21T07:54:06.038835+0000","last_became_active":"2024-09-21T07:54:03.962213+0000","last_became_peered":"2024-09-21T07:54:03.962213+0000","last_unstale":"2024-09-21T07:54:06.038835+0000","last_undegraded":"2024-09-21T07:54:06.038835+0000","last_fullsized":"2024-09-21T07:54:06.038835+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.785305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900221+0000","last_change":"2024-09-21T07:54:03.978295+0000","last_active":"2024-09-21T07:54:05.900221+0000","last_peered":"2024-09-21T07:54:05.900221+0000","last_clean":"2024-09-21T07:54:05.900221+0000","last_became_active":"2024-09-21T07:54:03.977951+0000","last_became_peered":"2024-09-21T07:54:03.977951+0000","last_unstale":"2024-09-21T07:54:05.900221+0000","last_undegraded":"2024-09-21T07:54:05.900221+0000","last_fullsized":"2024-09-21T07:54:05.900221+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:59.863067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.367119+0000","last_change":"2024-09-21T07:54:03.973172+0000","last_active":"2024-09-21T07:54:06.367119+0000","last_peered":"2024-09-21T07:54:06.367119+0000","last_clean":"2024-09-21T07:54:06.367119+0000","last_became_active":"2024-09-21T07:54:03.972809+0000","last_became_peered":"2024-09-21T07:54:03.972809+0000","last_unstale":"2024-09-21T07:54:06.367119+0000","last_undegraded":"2024-09-21T07:54:06.367119+0000","last_fullsized":"2024-09-21T07:54:06.367119+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:35:33.139818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959559+0000","last_change":"2024-09-21T07:54:03.969375+0000","last_active":"2024-09-21T07:54:04.959559+0000","last_peered":"2024-09-21T07:54:04.959559+0000","last_clean":"2024-09-21T07:54:04.959559+0000","last_became_active":"2024-09-21T07:54:03.969005+0000","last_became_peered":"2024-09-21T07:54:03.969005+0000","last_unstale":"2024-09-21T07:54:04.959559+0000","last_undegraded":"2024-09-21T07:54:04.959559+0000","last_fullsized":"2024-09-21T07:54:04.959559+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:12:28.249397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900691+0000","last_change":"2024-09-21T07:54:03.964077+0000","last_active":"2024-09-21T07:54:05.900691+0000","last_peered":"2024-09-21T07:54:05.900691+0000","last_clean":"2024-09-21T07:54:05.900691+0000","last_became_active":"2024-09-21T07:54:03.961800+0000","last_became_peered":"2024-09-21T07:54:03.961800+0000","last_unstale":"2024-09-21T07:54:05.900691+0000","last_undegraded":"2024-09-21T07:54:05.900691+0000","last_fullsized":"2024-09-21T07:54:05.900691+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.270102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964124+0000","last_change":"2024-09-21T07:54:03.982366+0000","last_active":"2024-09-21T07:54:04.964124+0000","last_peered":"2024-09-21T07:54:04.964124+0000","last_clean":"2024-09-21T07:54:04.964124+0000","last_became_active":"2024-09-21T07:54:03.978306+0000","last_became_peered":"2024-09-21T07:54:03.978306+0000","last_unstale":"2024-09-21T07:54:04.964124+0000","last_undegraded":"2024-09-21T07:54:04.964124+0000","last_fullsized":"2024-09-21T07:54:04.964124+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:43:21.416321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366738+0000","last_change":"2024-09-21T07:54:03.979276+0000","last_active":"2024-09-21T07:54:06.366738+0000","last_peered":"2024-09-21T07:54:06.366738+0000","last_clean":"2024-09-21T07:54:06.366738+0000","last_became_active":"2024-09-21T07:54:03.978909+0000","last_became_peered":"2024-09-21T07:54:03.978909+0000","last_unstale":"2024-09-21T07:54:06.366738+0000","last_undegraded":"2024-09-21T07:54:06.366738+0000","last_fullsized":"2024-09-21T07:54:06.366738+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:47.739616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958712+0000","last_change":"2024-09-21T07:54:03.961010+0000","last_active":"2024-09-21T07:54:04.958712+0000","last_peered":"2024-09-21T07:54:04.958712+0000","last_clean":"2024-09-21T07:54:04.958712+0000","last_became_active":"2024-09-21T07:54:03.960727+0000","last_became_peered":"2024-09-21T07:54:03.960727+0000","last_unstale":"2024-09-21T07:54:04.958712+0000","last_undegraded":"2024-09-21T07:54:04.958712+0000","last_fullsized":"2024-09-21T07:54:04.958712+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:36:40.969184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959104+0000","last_change":"2024-09-21T07:54:03.972825+0000","last_active":"2024-09-21T07:54:04.959104+0000","last_peered":"2024-09-21T07:54:04.959104+0000","last_clean":"2024-09-21T07:54:04.959104+0000","last_became_active":"2024-09-21T07:54:03.972542+0000","last_became_peered":"2024-09-21T07:54:03.972542+0000","last_unstale":"2024-09-21T07:54:04.959104+0000","last_undegraded":"2024-09-21T07:54:04.959104+0000","last_fullsized":"2024-09-21T07:54:04.959104+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:37.766666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958576+0000","last_change":"2024-09-21T07:54:03.950116+0000","last_active":"2024-09-21T07:54:04.958576+0000","last_peered":"2024-09-21T07:54:04.958576+0000","last_clean":"2024-09-21T07:54:04.958576+0000","last_became_active":"2024-09-21T07:54:03.949816+0000","last_became_peered":"2024-09-21T07:54:03.949816+0000","last_unstale":"2024-09-21T07:54:04.958576+0000","last_undegraded":"2024-09-21T07:54:04.958576+0000","last_fullsized":"2024-09-21T07:54:04.958576+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:29:24.910790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958613+0000","last_change":"2024-09-21T07:54:03.952909+0000","last_active":"2024-09-21T07:54:04.958613+0000","last_peered":"2024-09-21T07:54:04.958613+0000","last_clean":"2024-09-21T07:54:04.958613+0000","last_became_active":"2024-09-21T07:54:03.952608+0000","last_became_peered":"2024-09-21T07:54:03.952608+0000","last_unstale":"2024-09-21T07:54:04.958613+0000","last_undegraded":"2024-09-21T07:54:04.958613+0000","last_fullsized":"2024-09-21T07:54:04.958613+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:28.510634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959642+0000","last_change":"2024-09-21T07:54:03.978147+0000","last_active":"2024-09-21T07:54:04.959642+0000","last_peered":"2024-09-21T07:54:04.959642+0000","last_clean":"2024-09-21T07:54:04.959642+0000","last_became_active":"2024-09-21T07:54:03.977835+0000","last_became_peered":"2024-09-21T07:54:03.977835+0000","last_unstale":"2024-09-21T07:54:04.959642+0000","last_undegraded":"2024-09-21T07:54:04.959642+0000","last_fullsized":"2024-09-21T07:54:04.959642+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:08.623383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959080+0000","last_change":"2024-09-21T07:54:03.961202+0000","last_active":"2024-09-21T07:54:04.959080+0000","last_peered":"2024-09-21T07:54:04.959080+0000","last_clean":"2024-09-21T07:54:04.959080+0000","last_became_active":"2024-09-21T07:54:03.960869+0000","last_became_peered":"2024-09-21T07:54:03.960869+0000","last_unstale":"2024-09-21T07:54:04.959080+0000","last_undegraded":"2024-09-21T07:54:04.959080+0000","last_fullsized":"2024-09-21T07:54:04.959080+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:17.680155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900585+0000","last_change":"2024-09-21T07:54:03.962961+0000","last_active":"2024-09-21T07:54:05.900585+0000","last_peered":"2024-09-21T07:54:05.900585+0000","last_clean":"2024-09-21T07:54:05.900585+0000","last_became_active":"2024-09-21T07:54:03.961993+0000","last_became_peered":"2024-09-21T07:54:03.961993+0000","last_unstale":"2024-09-21T07:54:05.900585+0000","last_undegraded":"2024-09-21T07:54:05.900585+0000","last_fullsized":"2024-09-21T07:54:05.900585+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:16:18.411941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968589+0000","last_change":"2024-09-21T07:54:03.978915+0000","last_active":"2024-09-21T07:54:04.968589+0000","last_peered":"2024-09-21T07:54:04.968589+0000","last_clean":"2024-09-21T07:54:04.968589+0000","last_became_active":"2024-09-21T07:54:03.978592+0000","last_became_peered":"2024-09-21T07:54:03.978592+0000","last_unstale":"2024-09-21T07:54:04.968589+0000","last_undegraded":"2024-09-21T07:54:04.968589+0000","last_fullsized":"2024-09-21T07:54:04.968589+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:02.296991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038310+0000","last_change":"2024-09-21T07:54:03.955664+0000","last_active":"2024-09-21T07:54:06.038310+0000","last_peered":"2024-09-21T07:54:06.038310+0000","last_clean":"2024-09-21T07:54:06.038310+0000","last_became_active":"2024-09-21T07:54:03.955362+0000","last_became_peered":"2024-09-21T07:54:03.955362+0000","last_unstale":"2024-09-21T07:54:06.038310+0000","last_undegraded":"2024-09-21T07:54:06.038310+0000","last_fullsized":"2024-09-21T07:54:06.038310+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:51.717790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968316+0000","last_change":"2024-09-21T07:54:03.964613+0000","last_active":"2024-09-21T07:54:04.968316+0000","last_peered":"2024-09-21T07:54:04.968316+0000","last_clean":"2024-09-21T07:54:04.968316+0000","last_became_active":"2024-09-21T07:54:03.964282+0000","last_became_peered":"2024-09-21T07:54:03.964282+0000","last_unstale":"2024-09-21T07:54:04.968316+0000","last_undegraded":"2024-09-21T07:54:04.968316+0000","last_fullsized":"2024-09-21T07:54:04.968316+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.785305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366577+0000","last_change":"2024-09-21T07:54:03.970963+0000","last_active":"2024-09-21T07:54:06.366577+0000","last_peered":"2024-09-21T07:54:06.366577+0000","last_clean":"2024-09-21T07:54:06.366577+0000","last_became_active":"2024-09-21T07:54:03.969094+0000","last_became_peered":"2024-09-21T07:54:03.969094+0000","last_unstale":"2024-09-21T07:54:06.366577+0000","last_undegraded":"2024-09-21T07:54:06.366577+0000","last_fullsized":"2024-09-21T07:54:06.366577+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:00:40.604013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959710+0000","last_change":"2024-09-21T07:54:03.969116+0000","last_active":"2024-09-21T07:54:04.959710+0000","last_peered":"2024-09-21T07:54:04.959710+0000","last_clean":"2024-09-21T07:54:04.959710+0000","last_became_active":"2024-09-21T07:54:03.968512+0000","last_became_peered":"2024-09-21T07:54:03.968512+0000","last_unstale":"2024-09-21T07:54:04.959710+0000","last_undegraded":"2024-09-21T07:54:04.959710+0000","last_fullsized":"2024-09-21T07:54:04.959710+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:21.288740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1f","version":"144'4","reported_seq":19,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.969098+0000","last_change":"2024-09-21T07:54:02.963513+0000","last_active":"2024-09-21T07:54:04.969098+0000","last_peered":"2024-09-21T07:54:04.969098+0000","last_clean":"2024-09-21T07:54:04.969098+0000","last_became_active":"2024-09-21T07:54:02.962145+0000","last_became_peered":"2024-09-21T07:54:02.962145+0000","last_unstale":"2024-09-21T07:54:04.969098+0000","last_undegraded":"2024-09-21T07:54:04.969098+0000","last_fullsized":"2024-09-21T07:54:04.969098+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:56.239396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.969182+0000","last_change":"2024-09-21T07:54:03.962723+0000","last_active":"2024-09-21T07:54:04.969182+0000","last_peered":"2024-09-21T07:54:04.969182+0000","last_clean":"2024-09-21T07:54:04.969182+0000","last_became_active":"2024-09-21T07:54:03.962425+0000","last_became_peered":"2024-09-21T07:54:03.962425+0000","last_unstale":"2024-09-21T07:54:04.969182+0000","last_undegraded":"2024-09-21T07:54:04.969182+0000","last_fullsized":"2024-09-21T07:54:04.969182+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:20:37.159884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1e","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038602+0000","last_change":"2024-09-21T07:54:02.957220+0000","last_active":"2024-09-21T07:54:06.038602+0000","last_peered":"2024-09-21T07:54:06.038602+0000","last_clean":"2024-09-21T07:54:06.038602+0000","last_became_active":"2024-09-21T07:54:02.956920+0000","last_became_peered":"2024-09-21T07:54:02.956920+0000","last_unstale":"2024-09-21T07:54:06.038602+0000","last_undegraded":"2024-09-21T07:54:06.038602+0000","last_fullsized":"2024-09-21T07:54:06.038602+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:58:18.227182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366973+0000","last_change":"2024-09-21T07:54:03.964431+0000","last_active":"2024-09-21T07:54:06.366973+0000","last_peered":"2024-09-21T07:54:06.366973+0000","last_clean":"2024-09-21T07:54:06.366973+0000","last_became_active":"2024-09-21T07:54:03.964101+0000","last_became_peered":"2024-09-21T07:54:03.964101+0000","last_unstale":"2024-09-21T07:54:06.366973+0000","last_undegraded":"2024-09-21T07:54:06.366973+0000","last_fullsized":"2024-09-21T07:54:06.366973+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:15:29.556167+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1d","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366656+0000","last_change":"2024-09-21T07:54:02.966445+0000","last_active":"2024-09-21T07:54:06.366656+0000","last_peered":"2024-09-21T07:54:06.366656+0000","last_clean":"2024-09-21T07:54:06.366656+0000","last_became_active":"2024-09-21T07:54:02.965438+0000","last_became_peered":"2024-09-21T07:54:02.965438+0000","last_unstale":"2024-09-21T07:54:06.366656+0000","last_undegraded":"2024-09-21T07:54:06.366656+0000","last_fullsized":"2024-09-21T07:54:06.366656+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:52:18.199582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038681+0000","last_change":"2024-09-21T07:54:03.962416+0000","last_active":"2024-09-21T07:54:06.038681+0000","last_peered":"2024-09-21T07:54:06.038681+0000","last_clean":"2024-09-21T07:54:06.038681+0000","last_became_active":"2024-09-21T07:54:03.962001+0000","last_became_peered":"2024-09-21T07:54:03.962001+0000","last_unstale":"2024-09-21T07:54:06.038681+0000","last_undegraded":"2024-09-21T07:54:06.038681+0000","last_fullsized":"2024-09-21T07:54:06.038681+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:37.766666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1c","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958891+0000","last_change":"2024-09-21T07:54:02.958593+0000","last_active":"2024-09-21T07:54:04.958891+0000","last_peered":"2024-09-21T07:54:04.958891+0000","last_clean":"2024-09-21T07:54:04.958891+0000","last_became_active":"2024-09-21T07:54:02.958211+0000","last_became_peered":"2024-09-21T07:54:02.958211+0000","last_unstale":"2024-09-21T07:54:04.958891+0000","last_undegraded":"2024-09-21T07:54:04.958891+0000","last_fullsized":"2024-09-21T07:54:04.958891+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:59:31.711026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958016+0000","last_change":"2024-09-21T07:54:03.967668+0000","last_active":"2024-09-21T07:54:04.958016+0000","last_peered":"2024-09-21T07:54:04.958016+0000","last_clean":"2024-09-21T07:54:04.958016+0000","last_became_active":"2024-09-21T07:54:03.967343+0000","last_became_peered":"2024-09-21T07:54:03.967343+0000","last_unstale":"2024-09-21T07:54:04.958016+0000","last_undegraded":"2024-09-21T07:54:04.958016+0000","last_fullsized":"2024-09-21T07:54:04.958016+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:07.942496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1b","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968550+0000","last_change":"2024-09-21T07:54:02.961138+0000","last_active":"2024-09-21T07:54:04.968550+0000","last_peered":"2024-09-21T07:54:04.968550+0000","last_clean":"2024-09-21T07:54:04.968550+0000","last_became_active":"2024-09-21T07:54:02.960815+0000","last_became_peered":"2024-09-21T07:54:02.960815+0000","last_unstale":"2024-09-21T07:54:04.968550+0000","last_undegraded":"2024-09-21T07:54:04.968550+0000","last_fullsized":"2024-09-21T07:54:04.968550+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:43.585849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366880+0000","last_change":"2024-09-21T07:54:03.970083+0000","last_active":"2024-09-21T07:54:06.366880+0000","last_peered":"2024-09-21T07:54:06.366880+0000","last_clean":"2024-09-21T07:54:06.366880+0000","last_became_active":"2024-09-21T07:54:03.969739+0000","last_became_peered":"2024-09-21T07:54:03.969739+0000","last_unstale":"2024-09-21T07:54:06.366880+0000","last_undegraded":"2024-09-21T07:54:06.366880+0000","last_fullsized":"2024-09-21T07:54:06.366880+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:07.992623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.1a","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958160+0000","last_change":"2024-09-21T07:54:02.962660+0000","last_active":"2024-09-21T07:54:04.958160+0000","last_peered":"2024-09-21T07:54:04.958160+0000","last_clean":"2024-09-21T07:54:04.958160+0000","last_became_active":"2024-09-21T07:54:02.962322+0000","last_became_peered":"2024-09-21T07:54:02.962322+0000","last_unstale":"2024-09-21T07:54:04.958160+0000","last_undegraded":"2024-09-21T07:54:04.958160+0000","last_fullsized":"2024-09-21T07:54:04.958160+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:55:34.944411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958237+0000","last_change":"2024-09-21T07:54:03.970486+0000","last_active":"2024-09-21T07:54:04.958237+0000","last_peered":"2024-09-21T07:54:04.958237+0000","last_clean":"2024-09-21T07:54:04.958237+0000","last_became_active":"2024-09-21T07:54:03.966867+0000","last_became_peered":"2024-09-21T07:54:03.966867+0000","last_unstale":"2024-09-21T07:54:04.958237+0000","last_undegraded":"2024-09-21T07:54:04.958237+0000","last_fullsized":"2024-09-21T07:54:04.958237+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:39:43.139253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968488+0000","last_change":"2024-09-21T07:54:02.962550+0000","last_active":"2024-09-21T07:54:04.968488+0000","last_peered":"2024-09-21T07:54:04.968488+0000","last_clean":"2024-09-21T07:54:04.968488+0000","last_became_active":"2024-09-21T07:54:02.961969+0000","last_became_peered":"2024-09-21T07:54:02.961969+0000","last_unstale":"2024-09-21T07:54:04.968488+0000","last_undegraded":"2024-09-21T07:54:04.968488+0000","last_fullsized":"2024-09-21T07:54:04.968488+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:02.980966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957714+0000","last_change":"2024-09-21T07:54:03.962509+0000","last_active":"2024-09-21T07:54:04.957714+0000","last_peered":"2024-09-21T07:54:04.957714+0000","last_clean":"2024-09-21T07:54:04.957714+0000","last_became_active":"2024-09-21T07:54:03.962188+0000","last_became_peered":"2024-09-21T07:54:03.962188+0000","last_unstale":"2024-09-21T07:54:04.957714+0000","last_undegraded":"2024-09-21T07:54:04.957714+0000","last_fullsized":"2024-09-21T07:54:04.957714+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:17:32.715935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957793+0000","last_change":"2024-09-21T07:54:02.962051+0000","last_active":"2024-09-21T07:54:04.957793+0000","last_peered":"2024-09-21T07:54:04.957793+0000","last_clean":"2024-09-21T07:54:04.957793+0000","last_became_active":"2024-09-21T07:54:02.961633+0000","last_became_peered":"2024-09-21T07:54:02.961633+0000","last_unstale":"2024-09-21T07:54:04.957793+0000","last_undegraded":"2024-09-21T07:54:04.957793+0000","last_fullsized":"2024-09-21T07:54:04.957793+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:25:38.462224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900477+0000","last_change":"2024-09-21T07:54:03.960976+0000","last_active":"2024-09-21T07:54:05.900477+0000","last_peered":"2024-09-21T07:54:05.900477+0000","last_clean":"2024-09-21T07:54:05.900477+0000","last_became_active":"2024-09-21T07:54:03.960025+0000","last_became_peered":"2024-09-21T07:54:03.960025+0000","last_unstale":"2024-09-21T07:54:05.900477+0000","last_undegraded":"2024-09-21T07:54:05.900477+0000","last_fullsized":"2024-09-21T07:54:05.900477+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:20:00.560494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.367135+0000","last_change":"2024-09-21T07:54:02.967133+0000","last_active":"2024-09-21T07:54:06.367135+0000","last_peered":"2024-09-21T07:54:06.367135+0000","last_clean":"2024-09-21T07:54:06.367135+0000","last_became_active":"2024-09-21T07:54:02.966787+0000","last_became_peered":"2024-09-21T07:54:02.966787+0000","last_unstale":"2024-09-21T07:54:06.367135+0000","last_undegraded":"2024-09-21T07:54:06.367135+0000","last_fullsized":"2024-09-21T07:54:06.367135+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:59:45.242747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900683+0000","last_change":"2024-09-21T07:54:03.961976+0000","last_active":"2024-09-21T07:54:05.900683+0000","last_peered":"2024-09-21T07:54:05.900683+0000","last_clean":"2024-09-21T07:54:05.900683+0000","last_became_active":"2024-09-21T07:54:03.959046+0000","last_became_peered":"2024-09-21T07:54:03.959046+0000","last_unstale":"2024-09-21T07:54:05.900683+0000","last_undegraded":"2024-09-21T07:54:05.900683+0000","last_fullsized":"2024-09-21T07:54:05.900683+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:44:51.717790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.15","version":"144'3","reported_seq":18,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959294+0000","last_change":"2024-09-21T07:54:02.962199+0000","last_active":"2024-09-21T07:54:04.959294+0000","last_peered":"2024-09-21T07:54:04.959294+0000","last_clean":"2024-09-21T07:54:04.959294+0000","last_became_active":"2024-09-21T07:54:02.961858+0000","last_became_peered":"2024-09-21T07:54:02.961858+0000","last_unstale":"2024-09-21T07:54:04.959294+0000","last_undegraded":"2024-09-21T07:54:04.959294+0000","last_fullsized":"2024-09-21T07:54:04.959294+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:08.430367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968833+0000","last_change":"2024-09-21T07:54:03.979293+0000","last_active":"2024-09-21T07:54:04.968833+0000","last_peered":"2024-09-21T07:54:04.968833+0000","last_clean":"2024-09-21T07:54:04.968833+0000","last_became_active":"2024-09-21T07:54:03.978572+0000","last_became_peered":"2024-09-21T07:54:03.978572+0000","last_unstale":"2024-09-21T07:54:04.968833+0000","last_undegraded":"2024-09-21T07:54:04.968833+0000","last_fullsized":"2024-09-21T07:54:04.968833+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:04.826345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.14","version":"144'6","reported_seq":21,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959146+0000","last_change":"2024-09-21T07:54:02.966382+0000","last_active":"2024-09-21T07:54:04.959146+0000","last_peered":"2024-09-21T07:54:04.959146+0000","last_clean":"2024-09-21T07:54:04.959146+0000","last_became_active":"2024-09-21T07:54:02.966060+0000","last_became_peered":"2024-09-21T07:54:02.966060+0000","last_unstale":"2024-09-21T07:54:04.959146+0000","last_undegraded":"2024-09-21T07:54:04.959146+0000","last_fullsized":"2024-09-21T07:54:04.959146+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:23:50.372027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900231+0000","last_change":"2024-09-21T07:54:03.976974+0000","last_active":"2024-09-21T07:54:05.900231+0000","last_peered":"2024-09-21T07:54:05.900231+0000","last_clean":"2024-09-21T07:54:05.900231+0000","last_became_active":"2024-09-21T07:54:03.976570+0000","last_became_peered":"2024-09-21T07:54:03.976570+0000","last_unstale":"2024-09-21T07:54:05.900231+0000","last_undegraded":"2024-09-21T07:54:05.900231+0000","last_fullsized":"2024-09-21T07:54:05.900231+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:25.013417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.13","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958438+0000","last_change":"2024-09-21T07:54:02.962536+0000","last_active":"2024-09-21T07:54:04.958438+0000","last_peered":"2024-09-21T07:54:04.958438+0000","last_clean":"2024-09-21T07:54:04.958438+0000","last_became_active":"2024-09-21T07:54:02.962092+0000","last_became_peered":"2024-09-21T07:54:02.962092+0000","last_unstale":"2024-09-21T07:54:04.958438+0000","last_undegraded":"2024-09-21T07:54:04.958438+0000","last_fullsized":"2024-09-21T07:54:04.958438+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:25.218982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958869+0000","last_change":"2024-09-21T07:54:03.977300+0000","last_active":"2024-09-21T07:54:04.958869+0000","last_peered":"2024-09-21T07:54:04.958869+0000","last_clean":"2024-09-21T07:54:04.958869+0000","last_became_active":"2024-09-21T07:54:03.976964+0000","last_became_peered":"2024-09-21T07:54:03.976964+0000","last_unstale":"2024-09-21T07:54:04.958869+0000","last_undegraded":"2024-09-21T07:54:04.958869+0000","last_fullsized":"2024-09-21T07:54:04.958869+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:03:56.835444+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.367039+0000","last_change":"2024-09-21T07:54:02.966964+0000","last_active":"2024-09-21T07:54:06.367039+0000","last_peered":"2024-09-21T07:54:06.367039+0000","last_clean":"2024-09-21T07:54:06.367039+0000","last_became_active":"2024-09-21T07:54:02.966459+0000","last_became_peered":"2024-09-21T07:54:02.966459+0000","last_unstale":"2024-09-21T07:54:06.367039+0000","last_undegraded":"2024-09-21T07:54:06.367039+0000","last_fullsized":"2024-09-21T07:54:06.367039+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:21:11.679312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366704+0000","last_change":"2024-09-21T07:54:03.972605+0000","last_active":"2024-09-21T07:54:06.366704+0000","last_peered":"2024-09-21T07:54:06.366704+0000","last_clean":"2024-09-21T07:54:06.366704+0000","last_became_active":"2024-09-21T07:54:03.971880+0000","last_became_peered":"2024-09-21T07:54:03.971880+0000","last_unstale":"2024-09-21T07:54:06.366704+0000","last_undegraded":"2024-09-21T07:54:06.366704+0000","last_fullsized":"2024-09-21T07:54:06.366704+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:44.780322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900460+0000","last_change":"2024-09-21T07:54:02.961634+0000","last_active":"2024-09-21T07:54:05.900460+0000","last_peered":"2024-09-21T07:54:05.900460+0000","last_clean":"2024-09-21T07:54:05.900460+0000","last_became_active":"2024-09-21T07:54:02.960524+0000","last_became_peered":"2024-09-21T07:54:02.960524+0000","last_unstale":"2024-09-21T07:54:05.900460+0000","last_undegraded":"2024-09-21T07:54:05.900460+0000","last_fullsized":"2024-09-21T07:54:05.900460+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:27:54.217436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038297+0000","last_change":"2024-09-21T07:54:03.962547+0000","last_active":"2024-09-21T07:54:06.038297+0000","last_peered":"2024-09-21T07:54:06.038297+0000","last_clean":"2024-09-21T07:54:06.038297+0000","last_became_active":"2024-09-21T07:54:03.961672+0000","last_became_peered":"2024-09-21T07:54:03.961672+0000","last_unstale":"2024-09-21T07:54:06.038297+0000","last_undegraded":"2024-09-21T07:54:06.038297+0000","last_fullsized":"2024-09-21T07:54:06.038297+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:37.270102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.10","version":"144'8","reported_seq":23,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958319+0000","last_change":"2024-09-21T07:54:02.967430+0000","last_active":"2024-09-21T07:54:04.958319+0000","last_peered":"2024-09-21T07:54:04.958319+0000","last_clean":"2024-09-21T07:54:04.958319+0000","last_became_active":"2024-09-21T07:54:02.966983+0000","last_became_peered":"2024-09-21T07:54:02.966983+0000","last_unstale":"2024-09-21T07:54:04.958319+0000","last_undegraded":"2024-09-21T07:54:04.958319+0000","last_fullsized":"2024-09-21T07:54:04.958319+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:17:08.179354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900854+0000","last_change":"2024-09-21T07:54:03.964969+0000","last_active":"2024-09-21T07:54:05.900854+0000","last_peered":"2024-09-21T07:54:05.900854+0000","last_clean":"2024-09-21T07:54:05.900854+0000","last_became_active":"2024-09-21T07:54:03.964136+0000","last_became_peered":"2024-09-21T07:54:03.964136+0000","last_unstale":"2024-09-21T07:54:05.900854+0000","last_undegraded":"2024-09-21T07:54:05.900854+0000","last_fullsized":"2024-09-21T07:54:05.900854+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:55.776930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.f","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959064+0000","last_change":"2024-09-21T07:54:02.965358+0000","last_active":"2024-09-21T07:54:04.959064+0000","last_peered":"2024-09-21T07:54:04.959064+0000","last_clean":"2024-09-21T07:54:04.959064+0000","last_became_active":"2024-09-21T07:54:02.965047+0000","last_became_peered":"2024-09-21T07:54:02.965047+0000","last_unstale":"2024-09-21T07:54:04.959064+0000","last_undegraded":"2024-09-21T07:54:04.959064+0000","last_fullsized":"2024-09-21T07:54:04.959064+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:20:31.297113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959968+0000","last_change":"2024-09-21T07:54:03.962307+0000","last_active":"2024-09-21T07:54:04.959968+0000","last_peered":"2024-09-21T07:54:04.959968+0000","last_clean":"2024-09-21T07:54:04.959968+0000","last_became_active":"2024-09-21T07:54:03.961867+0000","last_became_peered":"2024-09-21T07:54:03.961867+0000","last_unstale":"2024-09-21T07:54:04.959968+0000","last_undegraded":"2024-09-21T07:54:04.959968+0000","last_fullsized":"2024-09-21T07:54:04.959968+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:35:50.043563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.e","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.038656+0000","last_change":"2024-09-21T07:54:02.959157+0000","last_active":"2024-09-21T07:54:06.038656+0000","last_peered":"2024-09-21T07:54:06.038656+0000","last_clean":"2024-09-21T07:54:06.038656+0000","last_became_active":"2024-09-21T07:54:02.958842+0000","last_became_peered":"2024-09-21T07:54:02.958842+0000","last_unstale":"2024-09-21T07:54:06.038656+0000","last_undegraded":"2024-09-21T07:54:06.038656+0000","last_fullsized":"2024-09-21T07:54:06.038656+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:07:59.905148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958053+0000","last_change":"2024-09-21T07:54:03.965420+0000","last_active":"2024-09-21T07:54:04.958053+0000","last_peered":"2024-09-21T07:54:04.958053+0000","last_clean":"2024-09-21T07:54:04.958053+0000","last_became_active":"2024-09-21T07:54:03.964768+0000","last_became_peered":"2024-09-21T07:54:03.964768+0000","last_unstale":"2024-09-21T07:54:04.958053+0000","last_undegraded":"2024-09-21T07:54:04.958053+0000","last_fullsized":"2024-09-21T07:54:04.958053+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:05.973590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.d","version":"144'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958958+0000","last_change":"2024-09-21T07:54:02.961479+0000","last_active":"2024-09-21T07:54:04.958958+0000","last_peered":"2024-09-21T07:54:04.958958+0000","last_clean":"2024-09-21T07:54:04.958958+0000","last_became_active":"2024-09-21T07:54:02.961164+0000","last_became_peered":"2024-09-21T07:54:02.961164+0000","last_unstale":"2024-09-21T07:54:04.958958+0000","last_undegraded":"2024-09-21T07:54:04.958958+0000","last_fullsized":"2024-09-21T07:54:04.958958+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:53:44.093493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959666+0000","last_change":"2024-09-21T07:54:03.968709+0000","last_active":"2024-09-21T07:54:04.959666+0000","last_peered":"2024-09-21T07:54:04.959666+0000","last_clean":"2024-09-21T07:54:04.959666+0000","last_became_active":"2024-09-21T07:54:03.968406+0000","last_became_peered":"2024-09-21T07:54:03.968406+0000","last_unstale":"2024-09-21T07:54:04.959666+0000","last_undegraded":"2024-09-21T07:54:04.959666+0000","last_fullsized":"2024-09-21T07:54:04.959666+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:19:40.213473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.964224+0000","last_change":"2024-09-21T07:54:02.960646+0000","last_active":"2024-09-21T07:54:04.964224+0000","last_peered":"2024-09-21T07:54:04.964224+0000","last_clean":"2024-09-21T07:54:04.964224+0000","last_became_active":"2024-09-21T07:54:02.959784+0000","last_became_peered":"2024-09-21T07:54:02.959784+0000","last_unstale":"2024-09-21T07:54:04.964224+0000","last_undegraded":"2024-09-21T07:54:04.964224+0000","last_fullsized":"2024-09-21T07:54:04.964224+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:22:50.206378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968339+0000","last_change":"2024-09-21T07:54:03.955882+0000","last_active":"2024-09-21T07:54:04.968339+0000","last_peered":"2024-09-21T07:54:04.968339+0000","last_clean":"2024-09-21T07:54:04.968339+0000","last_became_active":"2024-09-21T07:54:03.955600+0000","last_became_peered":"2024-09-21T07:54:03.955600+0000","last_unstale":"2024-09-21T07:54:04.968339+0000","last_undegraded":"2024-09-21T07:54:04.968339+0000","last_fullsized":"2024-09-21T07:54:04.968339+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:10.424899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.b","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959121+0000","last_change":"2024-09-21T07:54:02.954640+0000","last_active":"2024-09-21T07:54:04.959121+0000","last_peered":"2024-09-21T07:54:04.959121+0000","last_clean":"2024-09-21T07:54:04.959121+0000","last_became_active":"2024-09-21T07:54:02.954337+0000","last_became_peered":"2024-09-21T07:54:02.954337+0000","last_unstale":"2024-09-21T07:54:04.959121+0000","last_undegraded":"2024-09-21T07:54:04.959121+0000","last_fullsized":"2024-09-21T07:54:04.959121+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:23.793120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959125+0000","last_change":"2024-09-21T07:54:03.968291+0000","last_active":"2024-09-21T07:54:04.959125+0000","last_peered":"2024-09-21T07:54:04.959125+0000","last_clean":"2024-09-21T07:54:04.959125+0000","last_became_active":"2024-09-21T07:54:03.967801+0000","last_became_peered":"2024-09-21T07:54:03.967801+0000","last_unstale":"2024-09-21T07:54:04.959125+0000","last_undegraded":"2024-09-21T07:54:04.959125+0000","last_fullsized":"2024-09-21T07:54:04.959125+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:22.785305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.963791+0000","last_change":"2024-09-21T07:54:02.959629+0000","last_active":"2024-09-21T07:54:04.963791+0000","last_peered":"2024-09-21T07:54:04.963791+0000","last_clean":"2024-09-21T07:54:04.963791+0000","last_became_active":"2024-09-21T07:54:02.958113+0000","last_became_peered":"2024-09-21T07:54:02.958113+0000","last_unstale":"2024-09-21T07:54:04.963791+0000","last_undegraded":"2024-09-21T07:54:04.963791+0000","last_fullsized":"2024-09-21T07:54:04.963791+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:48:28.662913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968244+0000","last_change":"2024-09-21T07:54:03.975168+0000","last_active":"2024-09-21T07:54:04.968244+0000","last_peered":"2024-09-21T07:54:04.968244+0000","last_clean":"2024-09-21T07:54:04.968244+0000","last_became_active":"2024-09-21T07:54:03.974786+0000","last_became_peered":"2024-09-21T07:54:03.974786+0000","last_unstale":"2024-09-21T07:54:04.968244+0000","last_undegraded":"2024-09-21T07:54:04.968244+0000","last_fullsized":"2024-09-21T07:54:04.968244+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:11:37.766666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.957927+0000","last_change":"2024-09-21T07:54:02.967784+0000","last_active":"2024-09-21T07:54:04.957927+0000","last_peered":"2024-09-21T07:54:04.957927+0000","last_clean":"2024-09-21T07:54:04.957927+0000","last_became_active":"2024-09-21T07:54:02.967465+0000","last_became_peered":"2024-09-21T07:54:02.967465+0000","last_unstale":"2024-09-21T07:54:04.957927+0000","last_undegraded":"2024-09-21T07:54:04.957927+0000","last_fullsized":"2024-09-21T07:54:04.957927+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:59:01.289564+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900331+0000","last_change":"2024-09-21T07:54:03.957594+0000","last_active":"2024-09-21T07:54:05.900331+0000","last_peered":"2024-09-21T07:54:05.900331+0000","last_clean":"2024-09-21T07:54:05.900331+0000","last_became_active":"2024-09-21T07:54:03.957233+0000","last_became_peered":"2024-09-21T07:54:03.957233+0000","last_unstale":"2024-09-21T07:54:05.900331+0000","last_undegraded":"2024-09-21T07:54:05.900331+0000","last_fullsized":"2024-09-21T07:54:05.900331+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:14.602178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.963771+0000","last_change":"2024-09-21T07:54:02.957009+0000","last_active":"2024-09-21T07:54:04.963771+0000","last_peered":"2024-09-21T07:54:04.963771+0000","last_clean":"2024-09-21T07:54:04.963771+0000","last_became_active":"2024-09-21T07:54:02.956077+0000","last_became_peered":"2024-09-21T07:54:02.956077+0000","last_unstale":"2024-09-21T07:54:04.963771+0000","last_undegraded":"2024-09-21T07:54:04.963771+0000","last_fullsized":"2024-09-21T07:54:04.963771+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:26.957303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959407+0000","last_change":"2024-09-21T07:54:03.979237+0000","last_active":"2024-09-21T07:54:04.959407+0000","last_peered":"2024-09-21T07:54:04.959407+0000","last_clean":"2024-09-21T07:54:04.959407+0000","last_became_active":"2024-09-21T07:54:03.977294+0000","last_became_peered":"2024-09-21T07:54:03.977294+0000","last_unstale":"2024-09-21T07:54:04.959407+0000","last_undegraded":"2024-09-21T07:54:04.959407+0000","last_fullsized":"2024-09-21T07:54:04.959407+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:14:10.424899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.7","version":"144'2","reported_seq":17,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366999+0000","last_change":"2024-09-21T07:54:02.965440+0000","last_active":"2024-09-21T07:54:06.366999+0000","last_peered":"2024-09-21T07:54:06.366999+0000","last_clean":"2024-09-21T07:54:06.366999+0000","last_became_active":"2024-09-21T07:54:02.965078+0000","last_became_peered":"2024-09-21T07:54:02.965078+0000","last_unstale":"2024-09-21T07:54:06.366999+0000","last_undegraded":"2024-09-21T07:54:06.366999+0000","last_fullsized":"2024-09-21T07:54:06.366999+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:10:43.073196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959964+0000","last_change":"2024-09-21T07:54:03.970694+0000","last_active":"2024-09-21T07:54:04.959964+0000","last_peered":"2024-09-21T07:54:04.959964+0000","last_clean":"2024-09-21T07:54:04.959964+0000","last_became_active":"2024-09-21T07:54:03.969861+0000","last_became_peered":"2024-09-21T07:54:03.969861+0000","last_unstale":"2024-09-21T07:54:04.959964+0000","last_undegraded":"2024-09-21T07:54:04.959964+0000","last_fullsized":"2024-09-21T07:54:04.959964+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:06:48.151079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.6","version":"145'1","reported_seq":16,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.965432+0000","last_change":"2024-09-21T07:54:02.955571+0000","last_active":"2024-09-21T07:54:04.965432+0000","last_peered":"2024-09-21T07:54:04.965432+0000","last_clean":"2024-09-21T07:54:04.965432+0000","last_became_active":"2024-09-21T07:54:02.955159+0000","last_became_peered":"2024-09-21T07:54:02.955159+0000","last_unstale":"2024-09-21T07:54:04.965432+0000","last_undegraded":"2024-09-21T07:54:04.965432+0000","last_fullsized":"2024-09-21T07:54:04.965432+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:04.619726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958160+0000","last_change":"2024-09-21T07:54:03.966124+0000","last_active":"2024-09-21T07:54:04.958160+0000","last_peered":"2024-09-21T07:54:04.958160+0000","last_clean":"2024-09-21T07:54:04.958160+0000","last_became_active":"2024-09-21T07:54:03.965809+0000","last_became_peered":"2024-09-21T07:54:03.965809+0000","last_unstale":"2024-09-21T07:54:04.958160+0000","last_undegraded":"2024-09-21T07:54:04.958160+0000","last_fullsized":"2024-09-21T07:54:04.958160+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:12.134848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968943+0000","last_change":"2024-09-21T07:54:02.963499+0000","last_active":"2024-09-21T07:54:04.968943+0000","last_peered":"2024-09-21T07:54:04.968943+0000","last_clean":"2024-09-21T07:54:04.968943+0000","last_became_active":"2024-09-21T07:54:02.963167+0000","last_became_peered":"2024-09-21T07:54:02.963167+0000","last_unstale":"2024-09-21T07:54:04.968943+0000","last_undegraded":"2024-09-21T07:54:04.968943+0000","last_fullsized":"2024-09-21T07:54:04.968943+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:17.455346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.959862+0000","last_change":"2024-09-21T07:54:03.955452+0000","last_active":"2024-09-21T07:54:04.959862+0000","last_peered":"2024-09-21T07:54:04.959862+0000","last_clean":"2024-09-21T07:54:04.959862+0000","last_became_active":"2024-09-21T07:54:03.955071+0000","last_became_peered":"2024-09-21T07:54:03.955071+0000","last_unstale":"2024-09-21T07:54:04.959862+0000","last_undegraded":"2024-09-21T07:54:04.959862+0000","last_fullsized":"2024-09-21T07:54:04.959862+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:13.921075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958963+0000","last_change":"2024-09-21T07:54:02.952969+0000","last_active":"2024-09-21T07:54:04.958963+0000","last_peered":"2024-09-21T07:54:04.958963+0000","last_clean":"2024-09-21T07:54:04.958963+0000","last_became_active":"2024-09-21T07:54:02.952626+0000","last_became_peered":"2024-09-21T07:54:02.952626+0000","last_unstale":"2024-09-21T07:54:04.958963+0000","last_undegraded":"2024-09-21T07:54:04.958963+0000","last_fullsized":"2024-09-21T07:54:04.958963+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:48:28.850088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:06.366774+0000","last_change":"2024-09-21T07:54:03.969012+0000","last_active":"2024-09-21T07:54:06.366774+0000","last_peered":"2024-09-21T07:54:06.366774+0000","last_clean":"2024-09-21T07:54:06.366774+0000","last_became_active":"2024-09-21T07:54:03.967345+0000","last_became_peered":"2024-09-21T07:54:03.967345+0000","last_unstale":"2024-09-21T07:54:06.366774+0000","last_undegraded":"2024-09-21T07:54:06.366774+0000","last_fullsized":"2024-09-21T07:54:06.366774+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:28.542602+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958594+0000","last_change":"2024-09-21T07:54:02.959907+0000","last_active":"2024-09-21T07:54:04.958594+0000","last_peered":"2024-09-21T07:54:04.958594+0000","last_clean":"2024-09-21T07:54:04.958594+0000","last_became_active":"2024-09-21T07:54:02.959583+0000","last_became_peered":"2024-09-21T07:54:02.959583+0000","last_unstale":"2024-09-21T07:54:04.958594+0000","last_undegraded":"2024-09-21T07:54:04.958594+0000","last_fullsized":"2024-09-21T07:54:04.958594+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:04:37.103572+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968410+0000","last_change":"2024-09-21T07:54:03.963537+0000","last_active":"2024-09-21T07:54:04.968410+0000","last_peered":"2024-09-21T07:54:04.968410+0000","last_clean":"2024-09-21T07:54:04.968410+0000","last_became_active":"2024-09-21T07:54:03.963239+0000","last_became_peered":"2024-09-21T07:54:03.963239+0000","last_unstale":"2024-09-21T07:54:04.968410+0000","last_undegraded":"2024-09-21T07:54:04.968410+0000","last_fullsized":"2024-09-21T07:54:04.968410+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:01:08.623383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900424+0000","last_change":"2024-09-21T07:54:02.957031+0000","last_active":"2024-09-21T07:54:05.900424+0000","last_peered":"2024-09-21T07:54:05.900424+0000","last_clean":"2024-09-21T07:54:05.900424+0000","last_became_active":"2024-09-21T07:54:02.955404+0000","last_became_peered":"2024-09-21T07:54:02.955404+0000","last_unstale":"2024-09-21T07:54:05.900424+0000","last_undegraded":"2024-09-21T07:54:05.900424+0000","last_fullsized":"2024-09-21T07:54:05.900424+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:51:45.509611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900143+0000","last_change":"2024-09-21T07:54:03.972749+0000","last_active":"2024-09-21T07:54:05.900143+0000","last_peered":"2024-09-21T07:54:05.900143+0000","last_clean":"2024-09-21T07:54:05.900143+0000","last_became_active":"2024-09-21T07:54:03.972418+0000","last_became_peered":"2024-09-21T07:54:03.972418+0000","last_unstale":"2024-09-21T07:54:05.900143+0000","last_undegraded":"2024-09-21T07:54:05.900143+0000","last_fullsized":"2024-09-21T07:54:05.900143+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:57.246823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968560+0000","last_change":"2024-09-21T07:54:02.966502+0000","last_active":"2024-09-21T07:54:04.968560+0000","last_peered":"2024-09-21T07:54:04.968560+0000","last_clean":"2024-09-21T07:54:04.968560+0000","last_became_active":"2024-09-21T07:54:02.966190+0000","last_became_peered":"2024-09-21T07:54:02.966190+0000","last_unstale":"2024-09-21T07:54:04.968560+0000","last_undegraded":"2024-09-21T07:54:04.968560+0000","last_fullsized":"2024-09-21T07:54:04.968560+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:42.665964+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968435+0000","last_change":"2024-09-21T07:54:03.960880+0000","last_active":"2024-09-21T07:54:04.968435+0000","last_peered":"2024-09-21T07:54:04.968435+0000","last_clean":"2024-09-21T07:54:04.968435+0000","last_became_active":"2024-09-21T07:54:03.959027+0000","last_became_peered":"2024-09-21T07:54:03.959027+0000","last_unstale":"2024-09-21T07:54:04.968435+0000","last_undegraded":"2024-09-21T07:54:04.968435+0000","last_fullsized":"2024-09-21T07:54:04.968435+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:08:26.226409+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.968139+0000","last_change":"2024-09-21T07:54:02.963984+0000","last_active":"2024-09-21T07:54:04.968139+0000","last_peered":"2024-09-21T07:54:04.968139+0000","last_clean":"2024-09-21T07:54:04.968139+0000","last_became_active":"2024-09-21T07:54:02.961988+0000","last_became_peered":"2024-09-21T07:54:02.961988+0000","last_unstale":"2024-09-21T07:54:04.968139+0000","last_undegraded":"2024-09-21T07:54:04.968139+0000","last_fullsized":"2024-09-21T07:54:04.968139+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:27:37.630887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":13,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:05.900760+0000","last_change":"2024-09-21T07:54:03.959690+0000","last_active":"2024-09-21T07:54:05.900760+0000","last_peered":"2024-09-21T07:54:05.900760+0000","last_clean":"2024-09-21T07:54:05.900760+0000","last_became_active":"2024-09-21T07:54:03.958796+0000","last_became_peered":"2024-09-21T07:54:03.958796+0000","last_unstale":"2024-09-21T07:54:05.900760+0000","last_undegraded":"2024-09-21T07:54:05.900760+0000","last_fullsized":"2024-09-21T07:54:05.900760+0000","mapping_epoch":143,"log_start":"0'0","ondisk_log_start":"0'0","created":143,"last_epoch_clean":144,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:02.929879+0000","last_clean_scrub_stamp":"2024-09-21T07:54:02.929879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:07.430818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":15,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958728+0000","last_change":"2024-09-21T07:54:02.963980+0000","last_active":"2024-09-21T07:54:04.958728+0000","last_peered":"2024-09-21T07:54:04.958728+0000","last_clean":"2024-09-21T07:54:04.958728+0000","last_became_active":"2024-09-21T07:54:02.963687+0000","last_became_peered":"2024-09-21T07:54:02.963687+0000","last_unstale":"2024-09-21T07:54:04.958728+0000","last_undegraded":"2024-09-21T07:54:04.958728+0000","last_fullsized":"2024-09-21T07:54:04.958728+0000","mapping_epoch":142,"log_start":"0'0","ondisk_log_start":"0'0","created":142,"last_epoch_clean":143,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:01.922452+0000","last_clean_scrub_stamp":"2024-09-21T07:54:01.922452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:26:27.748680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":372,"reported_epoch":145,"state":"active+clean","last_fresh":"2024-09-21T07:54:04.958394+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:54:04.958394+0000","last_peered":"2024-09-21T07:54:04.958394+0000","last_clean":"2024-09-21T07:54:04.958394+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:54:04.958394+0000","last_undegraded":"2024-09-21T07:54:04.958394+0000","last_fullsized":"2024-09-21T07:54:04.958394+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542246,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32944,"kb_used_data":1756,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338896,"statfs":{"total":96636764160,"available":96603029504,"internally_reserved":0,"allocated":1798144,"data_stored":794088,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509541,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":800450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509542,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33552,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338288,"statfs":{"total":96636764160,"available":96602406912,"internally_reserved":0,"allocated":2412544,"data_stored":1391456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509542,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32968,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338872,"statfs":{"total":96636764160,"available":96603004928,"internally_reserved":0,"allocated":1814528,"data_stored":800450,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476840,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32980,"kb_used_data":1784,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338860,"statfs":{"total":96636764160,"available":96602992640,"internally_reserved":0,"allocated":1826816,"data_stored":802050,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476840,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32964,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338876,"statfs":{"total":96636764160,"available":96603009024,"internally_reserved":0,"allocated":1814528,"data_stored":800998,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476837,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33568,"kb_used_data":2372,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338272,"statfs":{"total":96636764160,"available":96602390528,"internally_reserved":0,"allocated":2428928,"data_stored":1392578,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476837,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32960,"kb_used_data":1772,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338880,"statfs":{"total":96636764160,"available":96603013120,"internally_reserved":0,"allocated":1814528,"data_stored":800486,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1794,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:54:08.864 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:54:08.864 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:54:09.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 -- 172.21.15.88:0/278653369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06f960 msgr2=0x7f7dfc0747e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 --2- 172.21.15.88:0/278653369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06f960 0x7f7dfc0747e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f7dec005fd0 tx=0x7f7dec02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 -- 172.21.15.88:0/278653369 shutdown_connections 2024-09-21T07:54:09.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 --2- 172.21.15.88:0/278653369 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dfc06ddb0 0x7f7dfc06e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 --2- 172.21.15.88:0/278653369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06f960 0x7f7dfc0747e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 -- 172.21.15.88:0/278653369 >> 172.21.15.88:0/278653369 conn(0x7f7dfc073920 msgr2=0x7f7dfc073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:09.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.126+0000 7f7e02200640 1 -- 172.21.15.88:0/278653369 shutdown_connections 2024-09-21T07:54:09.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.127+0000 7f7e02200640 1 -- 172.21.15.88:0/278653369 wait complete. 2024-09-21T07:54:09.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.129+0000 7f7e02200640 1 Processor -- start 2024-09-21T07:54:09.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.129+0000 7f7e02200640 1 -- start start 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7e02200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 0x7f7dfc153f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7e02200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dfc06f960 0x7f7dfc136d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7e02200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dfc153950 0x7f7dfc1f5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7e02200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc15d510 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7e02200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7dfc15d390 con 0x7f7dfc06f960 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7e02200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7dfc15d690 con 0x7f7dfc153950 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.130+0000 7f7dfb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 0x7f7dfc153f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 0x7f7dfc153f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51446/0 (socket says 172.21.15.88:51446) 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 -- 172.21.15.88:0/756597016 learned_addr learned my addr 172.21.15.88:0/756597016 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfaa00640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dfc06f960 0x7f7dfc136d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfbe00640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dfc153950 0x7f7dfc1f5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 -- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dfc153950 msgr2=0x7f7dfc1f5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dfc153950 0x7f7dfc1f5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 -- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dfc06f960 msgr2=0x7f7dfc136d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dfc06f960 0x7f7dfc136d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 -- 172.21.15.88:0/756597016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7dfc14cf30 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.131+0000 7f7dfb400640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 0x7f7dfc153f10 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7dec006010 tx=0x7f7dec0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:09.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.132+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec03e030 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.132+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7dec0047a0 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.132+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7dfc14d210 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.132+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec035040 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.134 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.132+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7dfc14d7d0 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.133+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7dfc15dc30 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.134+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f7dec041030 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.135+0000 7f7df3400640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ddc079280 0x7f7ddc07b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.135+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f7dec0bc230 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.135+0000 7f7dfaa00640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ddc079280 0x7f7ddc07b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.136+0000 7f7dfaa00640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ddc079280 0x7f7ddc07b790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc14aeb0 tx=0x7f7de8009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:09.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.140+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7dfc06e700 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.394+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7f7dfc0741d0 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.398+0000 7f7df3400640 1 -- 172.21.15.88:0/756597016 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) ==== 190+0+147 (secure 0 0 0) 0x7f7dec083560 con 0x7f7dfc06ddb0 2024-09-21T07:54:09.400 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:54:09.400 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:54:09.400 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:54:09.400 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:54:09.400 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:09.400 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ddc079280 msgr2=0x7f7ddc07b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ddc079280 0x7f7ddc07b790 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f7dfc14aeb0 tx=0x7f7de8009060 comp rx=0 tx=0).stop 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 msgr2=0x7f7dfc153f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 0x7f7dfc153f10 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f7dec006010 tx=0x7f7dec0039e0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 shutdown_connections 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ddc079280 0x7f7ddc07b790 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7dfc153950 0x7f7dfc1f5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7dfc06f960 0x7f7dfc136d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 --2- 172.21.15.88:0/756597016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7dfc06ddb0 0x7f7dfc153f10 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 >> 172.21.15.88:0/756597016 conn(0x7f7dfc073920 msgr2=0x7f7dfc0b11d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:09.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.400+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 shutdown_connections 2024-09-21T07:54:09.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.401+0000 7f7e02200640 1 -- 172.21.15.88:0/756597016 wait complete. 2024-09-21T07:54:09.424 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 -- 172.21.15.88:0/2998405928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c0af170 msgr2=0x7f976c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 --2- 172.21.15.88:0/2998405928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c0af170 0x7f976c0b2f60 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f976c0cfe20 tx=0x7f975402a440 comp rx=0 tx=0).stop 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 -- 172.21.15.88:0/2998405928 shutdown_connections 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 --2- 172.21.15.88:0/2998405928 >> v2:172.21.15.175:3300/0 conn(0x7f976c15a0f0 0x7f976c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 --2- 172.21.15.88:0/2998405928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c0af170 0x7f976c0b2f60 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 -- 172.21.15.88:0/2998405928 >> 172.21.15.88:0/2998405928 conn(0x7f976c0b1e40 msgr2=0x7f976c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 -- 172.21.15.88:0/2998405928 shutdown_connections 2024-09-21T07:54:09.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.646+0000 7f9770c00640 1 -- 172.21.15.88:0/2998405928 wait complete. 2024-09-21T07:54:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.648+0000 7f9770c00640 1 Processor -- start 2024-09-21T07:54:09.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.648+0000 7f9770c00640 1 -- start start 2024-09-21T07:54:09.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f9770c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976c0ad9c0 0x7f976c0c5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f9770c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976c0af170 0x7f976c14cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f9770c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 0x7f976c09c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f9770c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f976c06f290 con 0x7f976c15a0f0 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f9770c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f976c070c80 con 0x7f976c0af170 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f9770c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f976c06f410 con 0x7f976c0ad9c0 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f976be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 0x7f976c09c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f976be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 0x7f976c09c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51474/0 (socket says 172.21.15.88:51474) 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.650+0000 7f976be00640 1 -- 172.21.15.88:0/66833205 learned_addr learned my addr 172.21.15.88:0/66833205 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9762a00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976c0af170 0x7f976c14cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976b400640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976c0ad9c0 0x7f976c0c5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976be00640 1 -- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976c0ad9c0 msgr2=0x7f976c0c5e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976be00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976c0ad9c0 0x7f976c0c5e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976be00640 1 -- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976c0af170 msgr2=0x7f976c14cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976be00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976c0af170 0x7f976c14cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976be00640 1 -- 172.21.15.88:0/66833205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f976c0c50b0 con 0x7f976c15a0f0 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9762a00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976c0af170 0x7f976c14cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f976be00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 0x7f976c09c920 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f976400e9c0 tx=0x7f976400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764004d20 con 0x7f976c15a0f0 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f976400cb20 con 0x7f976c15a0f0 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f976c0c5390 con 0x7f976c15a0f0 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9764005020 con 0x7f976c15a0f0 2024-09-21T07:54:09.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.651+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f976c0c5950 con 0x7f976c15a0f0 2024-09-21T07:54:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.652+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f976c1f8020 con 0x7f976c15a0f0 2024-09-21T07:54:09.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.654+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f97640051c0 con 0x7f976c15a0f0 2024-09-21T07:54:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.655+0000 7f9763e00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9748079250 0x7f974807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.655+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f976409c450 con 0x7f976c15a0f0 2024-09-21T07:54:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.655+0000 7f976b400640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9748079250 0x7f974807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:09.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.656+0000 7f976b400640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9748079250 0x7f974807b760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f975c006fd0 tx=0x7f975c008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:09.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.659+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9764016030 con 0x7f976c15a0f0 2024-09-21T07:54:09.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.880+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f976c06f930 con 0x7f976c15a0f0 2024-09-21T07:54:09.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.880+0000 7f9763e00640 1 -- 172.21.15.88:0/66833205 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 99 v99) ==== 94+0+5233 (secure 0 0 0) 0x7f9764063d00 con 0x7f976c15a0f0 2024-09-21T07:54:09.882 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:09.882 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":99,"btime":"2024-09-21T07:54:06:961559+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:03.440969+0000","modified":"2024-09-21T07:54:06.451584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3712798522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3712798522},{"type":"v1","addr":"172.21.15.88:6835","nonce":3712798522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":11}]} 2024-09-21T07:54:09.882 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 99 2024-09-21T07:54:09.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9748079250 msgr2=0x7f974807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9748079250 0x7f974807b760 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f975c006fd0 tx=0x7f975c008040 comp rx=0 tx=0).stop 2024-09-21T07:54:09.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 msgr2=0x7f976c09c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:09.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 0x7f976c09c920 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f976400e9c0 tx=0x7f976400ee80 comp rx=0 tx=0).stop 2024-09-21T07:54:09.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 shutdown_connections 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9748079250 0x7f974807b760 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f976c15a0f0 0x7f976c09c920 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f976c0af170 0x7f976c14cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 --2- 172.21.15.88:0/66833205 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f976c0ad9c0 0x7f976c0c5e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 >> 172.21.15.88:0/66833205 conn(0x7f976c0b1e40 msgr2=0x7f976c070530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 shutdown_connections 2024-09-21T07:54:09.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:09.883+0000 7f9770c00640 1 -- 172.21.15.88:0/66833205 wait complete. 2024-09-21T07:54:10.311 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 99, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:54:03.440969+0000', 'modified': '2024-09-21T07:54:06.451584+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8394': {'gid': 8394, 'name': 'a', 'rank': 0, 'incarnation': 95, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/3712798522', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 3712798522}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 3712798522}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8394, 'qdb_cluster': [8394]} 2024-09-21T07:54:10.311 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:54:10.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe3ca00640 1 -- 172.21.15.88:0/3938615580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c004800 msgr2=0x7efe1c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe3ca00640 1 --2- 172.21.15.88:0/3938615580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe1c004800 0x7efe1c004c10 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7efe24009b50 tx=0x7efe1c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe36800640 1 -- 172.21.15.88:0/3938615580 reap_dead start 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe3ca00640 1 -- 172.21.15.88:0/3938615580 shutdown_connections 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe3ca00640 1 -- 172.21.15.88:0/3938615580 >> 172.21.15.88:0/3938615580 conn(0x7efe3806d3d0 msgr2=0x7efe380b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe3ca00640 1 -- 172.21.15.88:0/3938615580 shutdown_connections 2024-09-21T07:54:10.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.533+0000 7efe3ca00640 1 -- 172.21.15.88:0/3938615580 wait complete. 2024-09-21T07:54:10.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.535+0000 7efe3ca00640 1 Processor -- start 2024-09-21T07:54:10.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.536+0000 7efe3ca00640 1 -- start start 2024-09-21T07:54:10.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe3ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe1c004800 0x7efe38155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe3ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe380adb10 0x7efe380adf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe3ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 0x7efe380b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe3ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe380cb950 con 0x7efe380b4850 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe3ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe380afab0 con 0x7efe380adb10 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe3ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe380cbad0 con 0x7efe1c004800 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 0x7efe380b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 0x7efe380b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51480/0 (socket says 172.21.15.88:51480) 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe36800640 1 -- 172.21.15.88:0/2183172945 learned_addr learned my addr 172.21.15.88:0/2183172945 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.537+0000 7efe35400640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe380adb10 0x7efe380adf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe35e00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe1c004800 0x7efe38155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe36800640 1 -- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe1c004800 msgr2=0x7efe38155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe36800640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe1c004800 0x7efe38155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe36800640 1 -- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe380adb10 msgr2=0x7efe380adf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe36800640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe380adb10 0x7efe380adf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe36800640 1 -- 172.21.15.88:0/2183172945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe380b51a0 con 0x7efe380b4850 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe36800640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 0x7efe380b4c60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efe30001520 tx=0x7efe30001a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe2e000640 1 -- 172.21.15.88:0/2183172945 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30011870 con 0x7efe380b4850 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe2e000640 1 -- 172.21.15.88:0/2183172945 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe30017070 con 0x7efe380b4850 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe2e000640 1 -- 172.21.15.88:0/2183172945 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30011de0 con 0x7efe380b4850 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe380b3280 con 0x7efe380b4850 2024-09-21T07:54:10.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.538+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe380b3840 con 0x7efe380b4850 2024-09-21T07:54:10.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.541+0000 7efe2e000640 1 -- 172.21.15.88:0/2183172945 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7efe3001c460 con 0x7efe380b4850 2024-09-21T07:54:10.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.542+0000 7efe2e000640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe180792b0 0x7efe1807b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.542+0000 7efe2e000640 1 -- 172.21.15.88:0/2183172945 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7efe3009c080 con 0x7efe380b4850 2024-09-21T07:54:10.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.542+0000 7efe35e00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe180792b0 0x7efe1807b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.542+0000 7efe43ac7740 1 Processor -- start 2024-09-21T07:54:10.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.543+0000 7efe43ac7740 1 -- start start 2024-09-21T07:54:10.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.543+0000 7efe35e00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe180792b0 0x7efe1807b7c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe38072000 tx=0x7efe28006720 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a5b5270 0x56342a985970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a985eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 0x56342a955f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe43ac7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56342a8df430 con 0x56342a594590 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56342a8cf5d0 con 0x56342a5b5270 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56342a8bea00 con 0x56342a58c3b0 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 0x56342a955f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 0x56342a955f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51488/0 (socket says 172.21.15.88:51488) 2024-09-21T07:54:10.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe36800640 1 -- 172.21.15.88:0/3454883389 learned_addr learned my addr 172.21.15.88:0/3454883389 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe35e00640 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a5b5270 0x56342a985970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.544+0000 7efe35400640 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a985eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe36800640 1 -- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 msgr2=0x56342a985eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe36800640 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a985eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe36800640 1 -- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a5b5270 msgr2=0x56342a985970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe36800640 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a5b5270 0x56342a985970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe36800640 1 -- 172.21.15.88:0/3454883389 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342a887520 con 0x56342a594590 2024-09-21T07:54:10.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe36800640 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 0x56342a955f60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efe3009c050 tx=0x7efe30062f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afdd01e0edfdcc9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe22600640 1 -- 172.21.15.88:0/3454883389 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30085dc0 con 0x56342a594590 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe22600640 1 -- 172.21.15.88:0/3454883389 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe30067f30 con 0x56342a594590 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.545+0000 7efe22600640 1 -- 172.21.15.88:0/3454883389 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3006a3c0 con 0x56342a594590 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 -- 172.21.15.88:0/3454883389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 msgr2=0x56342a955f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 0x56342a955f60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efe3009c050 tx=0x7efe30062f10 comp rx=0 tx=0).stop 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 -- 172.21.15.88:0/3454883389 shutdown_connections 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a985eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a594590 0x56342a955f60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3454883389 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a5b5270 0x56342a985970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 -- 172.21.15.88:0/3454883389 >> 172.21.15.88:0/3454883389 conn(0x56342a5a93c0 msgr2=0x56342a95b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 -- 172.21.15.88:0/3454883389 shutdown_connections 2024-09-21T07:54:10.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.546+0000 7efe43ac7740 1 -- 172.21.15.88:0/3454883389 wait complete. 2024-09-21T07:54:10.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.547+0000 7efe43ac7740 1 -- start start 2024-09-21T07:54:10.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.547+0000 7efe43ac7740 1 Processor -- start 2024-09-21T07:54:10.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 0x56342a9779a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a97fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a5b5270 0x56342a9800c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56342a865330 con 0x56342a58c3b0 2024-09-21T07:54:10.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56342a8c2810 con 0x56342a594590 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56342a62f620 con 0x56342a5b5270 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe43ac7740 10 client.0 _ll_register_callbacks cb 0x56342a86a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 0x56342a9779a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 0x56342a9779a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51496/0 (socket says 172.21.15.88:51496) 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35e00640 1 -- 172.21.15.88:0/2449035619 learned_addr learned my addr 172.21.15.88:0/2449035619 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe36800640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a5b5270 0x56342a9800c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35400640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a97fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35e00640 1 -- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a5b5270 msgr2=0x56342a9800c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35e00640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a5b5270 0x56342a9800c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.549+0000 7efe35e00640 1 -- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 msgr2=0x56342a97fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe35e00640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a97fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe35e00640 1 -- 172.21.15.88:0/2449035619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342a620850 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe35e00640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 0x56342a9779a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7efe28003de0 tx=0x7efe38070140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187d46ae2f6214b7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe16a00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe16a00640 1 -- 172.21.15.88:0/2449035619 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe28012bd0 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe16a00640 1 -- 172.21.15.88:0/2449035619 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7efe28012d70 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe30067f30 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe16a00640 1 -- 172.21.15.88:0/2449035619 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7efe2802b420 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe43ac7740 10 client.8529 fetch_fsmap learned FSMap version 99 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.550+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x56342a6976b0 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe16a00640 1 -- 172.21.15.88:0/2449035619 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe28024030 con 0x56342a58c3b0 2024-09-21T07:54:10.554 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe16a00640 1 -- 172.21.15.88:0/2449035619 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 99) ==== 3502+0+0 (secure 0 0 0) 0x7efe2802b5c0 con 0x56342a58c3b0 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe43ac7740 10 client.8529 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe43ac7740 10 client.8529 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe43ac7740 20 client.8529 populate_metadata read hostname 'smithi088' 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe43ac7740 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342a573e40 0x56342a97b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe43ac7740 4 client.8529 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 --> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56342a8d1520 con 0x56342a573e40 2024-09-21T07:54:10.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.551+0000 7efe35400640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342a573e40 0x56342a97b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.552+0000 7efe16a00640 10 client.8529 ms_handle_connect on v2:172.21.15.88:6832/3712798522 2024-09-21T07:54:10.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.552+0000 7efe35400640 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342a573e40 0x56342a97b700 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e283beabe4aa9a8e server_cookie=1a6092ba8b00361e in_seq=0 out_seq=0 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe16a00640 1 -- 172.21.15.88:0/2449035619 <== mds.0 v2:172.21.15.88:6832/3712798522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56342a8d1520 con 0x56342a573e40 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe16a00640 10 client.8529 handle_command_reply: tid=0 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 client.8529 shutdown 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 msgr2=0x56342a9779a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 0x56342a9779a0 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7efe28003de0 tx=0x7efe38070140 comp rx=0 tx=0).stop 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 shutdown_connections 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342a573e40 0x56342a97b700 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a5b5270 0x56342a9800c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a97fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 --2- 172.21.15.88:0/2449035619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a58c3b0 0x56342a9779a0 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe16a00640 0 client.8529 ms_handle_reset on v2:172.21.15.88:6832/3712798522 2024-09-21T07:54:10.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 >> 172.21.15.88:0/2449035619 conn(0x56342a5a93c0 msgr2=0x56342a9573f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 shutdown_connections 2024-09-21T07:54:10.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 1 -- 172.21.15.88:0/2449035619 wait complete. 2024-09-21T07:54:10.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.553+0000 7efe43ac7740 20 client.8529 trim_cache size 0 max 16384 2024-09-21T07:54:10.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.572+0000 7efe43ac7740 1 Processor -- start 2024-09-21T07:54:10.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.572+0000 7efe43ac7740 1 -- start start 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a58c3b0 0x56342a956e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a594590 0x56342a9573a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a985390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe43ac7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe38201570 con 0x56342a5b5270 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7efe38200900 con 0x56342a58c3b0 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7efe38200f50 con 0x56342a594590 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a985390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a985390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51504/0 (socket says 172.21.15.88:51504) 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 -- 172.21.15.88:0/425614496 learned_addr learned my addr 172.21.15.88:0/425614496 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe35400640 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a594590 0x56342a9573a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe35e00640 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a58c3b0 0x56342a956e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 -- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a594590 msgr2=0x56342a9573a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a594590 0x56342a9573a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 -- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a58c3b0 msgr2=0x56342a956e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a58c3b0 0x56342a956e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.574+0000 7efe36800640 1 -- 172.21.15.88:0/425614496 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56342a95c650 con 0x56342a5b5270 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.575+0000 7efe36800640 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a985390 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7efe30087040 tx=0x7efe30001490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22bf1829d49adf6f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.575+0000 7efe21c00640 1 -- 172.21.15.88:0/425614496 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30085dc0 con 0x56342a5b5270 2024-09-21T07:54:10.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.575+0000 7efe21c00640 1 -- 172.21.15.88:0/425614496 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe30063500 con 0x56342a5b5270 2024-09-21T07:54:10.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.575+0000 7efe21c00640 1 -- 172.21.15.88:0/425614496 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe3006a380 con 0x56342a5b5270 2024-09-21T07:54:10.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 -- 172.21.15.88:0/425614496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 msgr2=0x56342a985390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a985390 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7efe30087040 tx=0x7efe30001490 comp rx=0 tx=0).stop 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 -- 172.21.15.88:0/425614496 shutdown_connections 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a594590 0x56342a9573a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a985390 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 --2- 172.21.15.88:0/425614496 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a58c3b0 0x56342a956e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 -- 172.21.15.88:0/425614496 >> 172.21.15.88:0/425614496 conn(0x56342a5a93c0 msgr2=0x56342a9a1c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 -- 172.21.15.88:0/425614496 shutdown_connections 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.576+0000 7efe43ac7740 1 -- 172.21.15.88:0/425614496 wait complete. 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.577+0000 7efe43ac7740 1 -- start start 2024-09-21T07:54:10.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.577+0000 7efe43ac7740 1 Processor -- start 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a9c3230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a9a2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a9a2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56342a9b3370 con 0x56342a5b5270 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x56342a99be60 con 0x56342a594590 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x56342a9b3960 con 0x56342a58c3b0 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.578+0000 7efe43ac7740 10 client.0 _ll_register_callbacks cb 0x56342a86a420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:10.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a9a2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a9a2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51508/0 (socket says 172.21.15.88:51508) 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 -- 172.21.15.88:0/3490415478 learned_addr learned my addr 172.21.15.88:0/3490415478 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe35e00640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a9c3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe35400640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a9a2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 -- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 msgr2=0x56342a9c3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a9c3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 -- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 msgr2=0x56342a9a2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a9a2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 -- 172.21.15.88:0/3490415478 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56342a97f3a0 con 0x56342a5b5270 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe36800640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a9a2b70 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7efe30063440 tx=0x7efe30085df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0e897e408feaf6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe16a00640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe16a00640 1 -- 172.21.15.88:0/3490415478 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe30085e60 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.579+0000 7efe16a00640 1 -- 172.21.15.88:0/3490415478 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7efe3004e820 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe30063500 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe16a00640 1 -- 172.21.15.88:0/3490415478 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=99) ==== 24+0+0 (secure 0 0 0) 0x7efe3004ea20 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 10 client.8535 fetch_fsmap learned FSMap version 99 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=99,monmap=2+}) -- 0x56342a97f0c0 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe16a00640 1 -- 172.21.15.88:0/3490415478 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7efe30063500 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe16a00640 1 -- 172.21.15.88:0/3490415478 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 99) ==== 3502+0+0 (secure 0 0 0) 0x7efe30062870 con 0x56342a5b5270 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 10 client.8535 fetch_fsmap finished waiting for FSMap version 99 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 10 client.8535 resolve_mds: resolved name 'a' to daemon mds.a 2024-09-21T07:54:10.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 20 client.8535 populate_metadata read hostname 'smithi088' 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342aa216a0 0x56342aa21ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 4 client.8535 mds_command: new command op to 8394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 --> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] -- command(tid 0: {"prefix": "status"}) -- 0x56342a8ccfd0 con 0x56342aa216a0 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.580+0000 7efe35e00640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342aa216a0 0x56342aa21ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe35e00640 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342aa216a0 0x56342aa21ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88f3462513398db7 server_cookie=560d4c3caddc9f80 in_seq=0 out_seq=0 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe16a00640 10 client.8535 ms_handle_connect on v2:172.21.15.88:6832/3712798522 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe16a00640 1 -- 172.21.15.88:0/3490415478 <== mds.0 v2:172.21.15.88:6832/3712798522 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56342a8ccfd0 con 0x56342aa216a0 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe16a00640 10 client.8535 handle_command_reply: tid=0 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 client.8535 shutdown 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 msgr2=0x56342a9a2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a9a2b70 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7efe30063440 tx=0x7efe30085df0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 shutdown_connections 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:6832/3712798522,v1:172.21.15.88:6835/3712798522] conn(0x56342aa216a0 0x56342aa21ab0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56342a5b5270 0x56342a9a2b70 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x56342a594590 0x56342a9a2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe16a00640 0 client.8535 ms_handle_reset on v2:172.21.15.88:6832/3712798522 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 --2- 172.21.15.88:0/3490415478 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x56342a58c3b0 0x56342a9c3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.581+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 >> 172.21.15.88:0/3490415478 conn(0x56342a5a93c0 msgr2=0x56342a97bdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 shutdown_connections 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe43ac7740 1 -- 172.21.15.88:0/3490415478 wait complete. 2024-09-21T07:54:10.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe43ac7740 20 client.8535 trim_cache size 0 max 16384 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 8394, 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6329872930000002, 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 99, 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 145, 2024-09-21T07:54:10.588 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 145, 2024-09-21T07:54:10.589 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.647440308 2024-09-21T07:54:10.589 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:54:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe180792b0 msgr2=0x7efe1807b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe180792b0 0x7efe1807b7c0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7efe38072000 tx=0x7efe28006720 comp rx=0 tx=0).stop 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 msgr2=0x7efe380b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 0x7efe380b4c60 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efe30001520 tx=0x7efe30001a30 comp rx=0 tx=0).stop 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 shutdown_connections 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7efe180792b0 0x7efe1807b7c0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe380b4850 0x7efe380b4c60 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7efe380adb10 0x7efe380adf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 --2- 172.21.15.88:0/2183172945 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7efe1c004800 0x7efe38155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.582+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 >> 172.21.15.88:0/2183172945 conn(0x7efe3806d3d0 msgr2=0x7efe38073390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.583+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 shutdown_connections 2024-09-21T07:54:10.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.583+0000 7efe3ca00640 1 -- 172.21.15.88:0/2183172945 wait complete. 2024-09-21T07:54:10.606 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c1d200640 1 -- 172.21.15.88:0/1517196963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 msgr2=0x7f3bfc004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c1d200640 1 --2- 172.21.15.88:0/1517196963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3bfc004800 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f3c18072ff0 tx=0x7f3bfc005830 comp rx=0 tx=0).stop 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c16800640 1 -- 172.21.15.88:0/1517196963 reap_dead start 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c1d200640 1 -- 172.21.15.88:0/1517196963 shutdown_connections 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c1d200640 1 -- 172.21.15.88:0/1517196963 >> 172.21.15.88:0/1517196963 conn(0x7f3c1806d730 msgr2=0x7f3c180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c1d200640 1 -- 172.21.15.88:0/1517196963 shutdown_connections 2024-09-21T07:54:10.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.823+0000 7f3c1d200640 1 -- 172.21.15.88:0/1517196963 wait complete. 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.826+0000 7f3c1d200640 1 Processor -- start 2024-09-21T07:54:10.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.826+0000 7f3c1d200640 1 -- start start 2024-09-21T07:54:10.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c1d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3c181f5ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c1d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c181f8bd0 0x7f3c181f4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c1d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1809b320 0x7f3c1815b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c1d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c180b4da0 con 0x7f3bfc003fd0 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c1d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3c180b4c20 con 0x7f3c1809b320 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c1d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3c18160e30 con 0x7f3c181f8bd0 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c15e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3c181f5ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c15e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3c181f5ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51520/0 (socket says 172.21.15.88:51520) 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.827+0000 7f3c15e00640 1 -- 172.21.15.88:0/3852853976 learned_addr learned my addr 172.21.15.88:0/3852853976 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c16800640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1809b320 0x7f3c1815b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c0d400640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c181f8bd0 0x7f3c181f4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c15e00640 1 -- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c181f8bd0 msgr2=0x7f3c181f4e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c15e00640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c181f8bd0 0x7f3c181f4e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c15e00640 1 -- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1809b320 msgr2=0x7f3c1815b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:10.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c15e00640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1809b320 0x7f3c1815b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c15e00640 1 -- 172.21.15.88:0/3852853976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c181f6530 con 0x7f3bfc003fd0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c15e00640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3c181f5ff0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3c08011ae0 tx=0x7f3c08009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.828+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0801e070 con 0x7f3bfc003fd0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.829+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3c0800fe70 con 0x7f3bfc003fd0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.829+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08004290 con 0x7f3bfc003fd0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.829+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c181f9600 con 0x7f3bfc003fd0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.829+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c181f57f0 con 0x7f3bfc003fd0 2024-09-21T07:54:10.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.830+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3c180ad4f0 con 0x7f3bfc003fd0 2024-09-21T07:54:10.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.832+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f3c08021070 con 0x7f3bfc003fd0 2024-09-21T07:54:10.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.833+0000 7f3c0ea00640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3bf4079250 0x7f3bf407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:10.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.833+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f3c0809c490 con 0x7f3bfc003fd0 2024-09-21T07:54:10.834 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.833+0000 7f3c0d400640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3bf4079250 0x7f3bf407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:10.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.834+0000 7f3c0d400640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3bf4079250 0x7f3bf407b760 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c00034000 tx=0x7f3c000046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:10.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:10.838+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3c080637f0 con 0x7f3bfc003fd0 2024-09-21T07:54:11.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.073+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f3c1815e200 con 0x7f3bfc003fd0 2024-09-21T07:54:11.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.073+0000 7f3c0ea00640 1 -- 172.21.15.88:0/3852853976 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f3c08068220 con 0x7f3bfc003fd0 2024-09-21T07:54:11.075 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3bf4079250 msgr2=0x7f3bf407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3bf4079250 0x7f3bf407b760 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3c00034000 tx=0x7f3c000046c0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 msgr2=0x7f3c181f5ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:11.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3c181f5ff0 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7f3c08011ae0 tx=0x7f3c08009e90 comp rx=0 tx=0).stop 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 shutdown_connections 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3bf4079250 0x7f3bf407b760 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3c181f8bd0 0x7f3c181f4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3c1809b320 0x7f3c1815b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 --2- 172.21.15.88:0/3852853976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bfc003fd0 0x7f3c181f5ff0 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 >> 172.21.15.88:0/3852853976 conn(0x7f3c1806d730 msgr2=0x7f3c180b0470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 shutdown_connections 2024-09-21T07:54:11.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11.076+0000 7f3c1d200640 1 -- 172.21.15.88:0/3852853976 wait complete. 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:11.100 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:54:11.101 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:54:11.101 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:11.101 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.101 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:11.137 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42243sec preferred_lft 42243sec 2024-09-21T07:54:11.138 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.139 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:11.140 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:54:11.140 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:54:11.140 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:11.140 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.140 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:11.229 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:54:11.233 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.233 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:54:11.268 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:54:11.269 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:11.269 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:54:11.269 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:54:11.269 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:11.269 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:11.269 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:54:11.269 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:54:11.269 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:54:11.269 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:54:11.269 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:54:11.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:11.458 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:11 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:11.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.467 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:54:11.501 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:54:11.501 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:54:11.502 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:11.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.502 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:54:11.598 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:11.598 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:11.632 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:11.633 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:11.633 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:11.708 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:54:11.708 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.708 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:11.904 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:11.904 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:54:11.904 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:54:11.925 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:11.925 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:54:11.925 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:54:11.925 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:11.925 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:54:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:54:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:11.955 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:54:11.956 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:54:11.956 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:54:11.957 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:54:11.957 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:11.957 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:11.957 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:11.957 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:11.962 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:11.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:11.962 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:12.020 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:12.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:12.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:12.115 INFO:teuthology.orchestra.run.smithi088.stdout:client8544 2024-09-21T07:54:12.115 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:12.116 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:12.151 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2bea000640 1 -- 172.21.15.88:0/2355635939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 msgr2=0x7f2bc8003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2bea000640 1 --2- 172.21.15.88:0/2355635939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2bc8003ea0 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f2be40d1ca0 tx=0x7f2bc8005830 comp rx=0 tx=0).stop 2024-09-21T07:54:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2be9600640 1 -- 172.21.15.88:0/2355635939 reap_dead start 2024-09-21T07:54:12.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2bea000640 1 -- 172.21.15.88:0/2355635939 shutdown_connections 2024-09-21T07:54:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2bea000640 1 -- 172.21.15.88:0/2355635939 >> 172.21.15.88:0/2355635939 conn(0x7f2be40d44a0 msgr2=0x7f2be40d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2bea000640 1 -- 172.21.15.88:0/2355635939 shutdown_connections 2024-09-21T07:54:12.412 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.410+0000 7f2bea000640 1 -- 172.21.15.88:0/2355635939 wait complete. 2024-09-21T07:54:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.412+0000 7f2bea000640 1 Processor -- start 2024-09-21T07:54:12.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.412+0000 7f2bea000640 1 -- start start 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bea000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2be415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bea000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be415bcc0 0x7f2be415d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bea000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415dbc0 0x7f2be41f24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bea000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2be40d0600 con 0x7f2bc8003a30 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bea000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2be40d0480 con 0x7f2be415bcc0 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bea000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2be40d0d10 con 0x7f2be415dbc0 2024-09-21T07:54:12.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2be8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2be415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2be8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2be415b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51540/0 (socket says 172.21.15.88:51540) 2024-09-21T07:54:12.416 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2be8c00640 1 -- 172.21.15.88:0/180002353 learned_addr learned my addr 172.21.15.88:0/180002353 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2be9600640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415dbc0 0x7f2be41f24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.414+0000 7f2bdbe00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be415bcc0 0x7f2be415d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be8c00640 1 -- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415dbc0 msgr2=0x7f2be41f24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be8c00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415dbc0 0x7f2be41f24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be8c00640 1 -- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be415bcc0 msgr2=0x7f2be415d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be8c00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be415bcc0 0x7f2be415d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be8c00640 1 -- 172.21.15.88:0/180002353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2be415e930 con 0x7f2bc8003a30 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2bdbe00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be415bcc0 0x7f2be415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be9600640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415dbc0 0x7f2be41f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be8c00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2be415b780 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4034040 tx=0x7f2bd40062e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.417 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4031070 con 0x7f2bc8003a30 2024-09-21T07:54:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2bd4003900 con 0x7f2bc8003a30 2024-09-21T07:54:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2be415e370 con 0x7f2bc8003a30 2024-09-21T07:54:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bd4003d50 con 0x7f2bc8003a30 2024-09-21T07:54:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.415+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2be40cc390 con 0x7f2bc8003a30 2024-09-21T07:54:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.416+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2be408cc70 con 0x7f2bc8003a30 2024-09-21T07:54:12.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.418+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f2bd4041440 con 0x7f2bc8003a30 2024-09-21T07:54:12.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.419+0000 7f2be1600640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bc0079250 0x7f2bc007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:12.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.419+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f2bd40bc1d0 con 0x7f2bc8003a30 2024-09-21T07:54:12.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.419+0000 7f2bdbe00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bc0079250 0x7f2bc007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:12.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.419+0000 7f2bdbe00640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bc0079250 0x7f2bc007b760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000c00 tx=0x7f2bcc006c00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:12.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.423+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2be408cc70 con 0x7f2bc8003a30 2024-09-21T07:54:12.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.645+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f2be40d1200 con 0x7f2bc8003a30 2024-09-21T07:54:12.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.645+0000 7f2be1600640 1 -- 172.21.15.88:0/180002353 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v99) ==== 53+0+73 (secure 0 0 0) 0x7f2bd4046060 con 0x7f2bc8003a30 2024-09-21T07:54:12.647 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:12.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bc0079250 msgr2=0x7f2bc007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bc0079250 0x7f2bc007b760 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2bcc000c00 tx=0x7f2bcc006c00 comp rx=0 tx=0).stop 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 msgr2=0x7f2be415b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2be415b780 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f2bd4034040 tx=0x7f2bd40062e0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 shutdown_connections 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2bc0079250 0x7f2bc007b760 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2be415dbc0 0x7f2be41f24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2be415bcc0 0x7f2be415d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 --2- 172.21.15.88:0/180002353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc8003a30 0x7f2be415b780 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 >> 172.21.15.88:0/180002353 conn(0x7f2be40d44a0 msgr2=0x7f2be40b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 shutdown_connections 2024-09-21T07:54:12.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:12.648+0000 7f2bea000640 1 -- 172.21.15.88:0/180002353 wait complete. 2024-09-21T07:54:12.673 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:54:12.674 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:54:12.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:12.674 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:54:12.715 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:12.716 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42192sec preferred_lft 42192sec 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:12.717 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:12.718 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:12.718 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:12.814 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:54:12.817 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:12.817 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:54:12.855 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:54:12.855 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:12.855 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:54:12.855 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:54:12.856 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:12.856 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:12.856 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:54:12.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:54:12.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:54:12.856 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:54:12.856 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:54:12.974 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:12 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:13.045 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:13.053 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:13.054 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:54:13.089 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:54:13.089 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:54:13.089 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:54:13.089 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:13.090 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:54:13.185 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:13.186 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:13.221 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:54:13.221 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:13.221 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:13.297 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:54:13.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:13.297 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:13.491 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:13.491 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:54:13.491 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:54:13.511 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:13.511 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:54:13.511 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:54:13.511 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:13.511 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:54:13.540 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:54:13.542 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:54:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:13.542 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:13.545 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:13.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:13.545 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:13.601 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:13.679 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:13 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:13.682 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:13 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:13.689 INFO:teuthology.orchestra.run.smithi175.stdout:client7085 2024-09-21T07:54:13.690 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:13.690 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:13.725 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:54:13.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 -- 172.21.15.88:0/3144742050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc070470 msgr2=0x7fe1cc0ae080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/3144742050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc070470 0x7fe1cc0ae080 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fe1c4009830 tx=0x7fe1c402a530 comp rx=0 tx=0).stop 2024-09-21T07:54:13.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 -- 172.21.15.88:0/3144742050 shutdown_connections 2024-09-21T07:54:13.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/3144742050 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc070470 0x7fe1cc0ae080 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 -- 172.21.15.88:0/3144742050 >> 172.21.15.88:0/3144742050 conn(0x7fe1cc0b2220 msgr2=0x7fe1cc0d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:13.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 -- 172.21.15.88:0/3144742050 shutdown_connections 2024-09-21T07:54:13.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.956+0000 7fe1d0e00640 1 -- 172.21.15.88:0/3144742050 wait complete. 2024-09-21T07:54:13.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.958+0000 7fe1d0e00640 1 Processor -- start 2024-09-21T07:54:13.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.959+0000 7fe1d0e00640 1 -- start start 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 0x7fe1cc159f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc0b56e0 0x7fe1cc14b420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1d0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1cc09cc20 0x7fe1cc09d030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1d0e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0ae740 con 0x7fe1cc0aec80 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1d0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe1cc0ae5c0 con 0x7fe1cc0b56e0 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1d0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe1cc0ae8c0 con 0x7fe1cc09cc20 2024-09-21T07:54:13.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1cb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 0x7fe1cc159f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1cb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 0x7fe1cc159f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51556/0 (socket says 172.21.15.88:51556) 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1cb400640 1 -- 172.21.15.88:0/1810877240 learned_addr learned my addr 172.21.15.88:0/1810877240 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.960+0000 7fe1cbe00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1cc09cc20 0x7fe1cc09d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1c2a00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc0b56e0 0x7fe1cc14b420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1cb400640 1 -- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1cc09cc20 msgr2=0x7fe1cc09d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1cb400640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1cc09cc20 0x7fe1cc09d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1cb400640 1 -- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc0b56e0 msgr2=0x7fe1cc14b420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1cb400640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc0b56e0 0x7fe1cc14b420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1cb400640 1 -- 172.21.15.88:0/1810877240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1cc09d570 con 0x7fe1cc0aec80 2024-09-21T07:54:13.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1cb400640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 0x7fe1cc159f50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00c780 tx=0x7fe1bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:13.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc003b70 con 0x7fe1cc0aec80 2024-09-21T07:54:13.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe1bc003d10 con 0x7fe1cc0aec80 2024-09-21T07:54:13.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1bc00aab0 con 0x7fe1cc0aec80 2024-09-21T07:54:13.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.961+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1cc14b960 con 0x7fe1cc0aec80 2024-09-21T07:54:13.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.962+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1cc15aba0 con 0x7fe1cc0aec80 2024-09-21T07:54:13.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.963+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1cc0ad970 con 0x7fe1cc0aec80 2024-09-21T07:54:13.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.964+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fe1bc00ac50 con 0x7fe1cc0aec80 2024-09-21T07:54:13.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.965+0000 7fe1c3e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1a8079280 0x7fe1a807b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:13.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.965+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7fe1bc09b690 con 0x7fe1cc0aec80 2024-09-21T07:54:13.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.965+0000 7fe1c2a00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1a8079280 0x7fe1a807b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:13.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.966+0000 7fe1c2a00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1a8079280 0x7fe1a807b790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0ca5e0 tx=0x7fe1b4007cc0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:13.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:13.969+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe1cc09d570 con 0x7fe1cc0aec80 2024-09-21T07:54:14.192 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.191+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7fe1cc0b0300 con 0x7fe1cc0aec80 2024-09-21T07:54:14.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.991+0000 7fe1c3e00640 1 -- 172.21.15.88:0/1810877240 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v100) ==== 136+0+0 (secure 0 0 0) 0x7fe1cc0b2ef0 con 0x7fe1cc0aec80 2024-09-21T07:54:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1a8079280 msgr2=0x7fe1a807b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1a8079280 0x7fe1a807b790 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fe1cc0ca5e0 tx=0x7fe1b4007cc0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 msgr2=0x7fe1cc159f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 0x7fe1cc159f50 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc00c780 tx=0x7fe1bc00cc40 comp rx=0 tx=0).stop 2024-09-21T07:54:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 shutdown_connections 2024-09-21T07:54:14.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe1a8079280 0x7fe1a807b790 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe1cc09cc20 0x7fe1cc09d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe1cc0b56e0 0x7fe1cc14b420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 --2- 172.21.15.88:0/1810877240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1cc0aec80 0x7fe1cc159f50 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 >> 172.21.15.88:0/1810877240 conn(0x7fe1cc0b2220 msgr2=0x7fe1cc0b0820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 shutdown_connections 2024-09-21T07:54:14.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:14.995+0000 7fe1d0e00640 1 -- 172.21.15.88:0/1810877240 wait complete. 2024-09-21T07:54:15.022 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 -- 172.21.15.88:0/149857546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b015ce30 msgr2=0x7ff9b015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 --2- 172.21.15.88:0/149857546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b015ce30 0x7ff9b015d240 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7ff9b0071220 tx=0x7ff9a802a980 comp rx=0 tx=0).stop 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 -- 172.21.15.88:0/149857546 shutdown_connections 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 --2- 172.21.15.88:0/149857546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b015ce30 0x7ff9b015d240 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 --2- 172.21.15.88:0/149857546 >> v2:172.21.15.175:3300/0 conn(0x7ff9b00705b0 0x7ff9b00709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 --2- 172.21.15.88:0/149857546 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9b00cfc80 0x7ff9b0156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 -- 172.21.15.88:0/149857546 >> 172.21.15.88:0/149857546 conn(0x7ff9b006d3d0 msgr2=0x7ff9b00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 -- 172.21.15.88:0/149857546 shutdown_connections 2024-09-21T07:54:15.245 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.243+0000 7ff9b7200640 1 -- 172.21.15.88:0/149857546 wait complete. 2024-09-21T07:54:15.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.246+0000 7ff9b7200640 1 Processor -- start 2024-09-21T07:54:15.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.246+0000 7ff9b7200640 1 -- start start 2024-09-21T07:54:15.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9b7200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 0x7ff9b01f5350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9b7200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9b00cfc80 0x7ff9b0072f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9b7200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9b015ce30 0x7ff9b0073460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9b7200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00cce40 con 0x7ff9b00705b0 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9b7200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b00cccc0 con 0x7ff9b015ce30 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9b7200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9b00ccfc0 con 0x7ff9b00cfc80 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 0x7ff9b01f5350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 0x7ff9b01f5350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51580/0 (socket says 172.21.15.88:51580) 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.247+0000 7ff9afe00640 1 -- 172.21.15.88:0/2132197729 learned_addr learned my addr 172.21.15.88:0/2132197729 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9afe00640 1 -- 172.21.15.88:0/2132197729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9b00cfc80 msgr2=0x7ff9b0072f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9afe00640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9b00cfc80 0x7ff9b0072f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9afe00640 1 -- 172.21.15.88:0/2132197729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9b015ce30 msgr2=0x7ff9b0073460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9afe00640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9b015ce30 0x7ff9b0073460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9afe00640 1 -- 172.21.15.88:0/2132197729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b00c50b0 con 0x7ff9b00705b0 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9afe00640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 0x7ff9b01f5350 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000d750 tx=0x7ff9a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:15.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0017070 con 0x7ff9b00705b0 2024-09-21T07:54:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9a0009b20 con 0x7ff9b00705b0 2024-09-21T07:54:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b00c5390 con 0x7ff9b00705b0 2024-09-21T07:54:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a0010460 con 0x7ff9b00705b0 2024-09-21T07:54:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.248+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b00c5950 con 0x7ff9b00705b0 2024-09-21T07:54:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.249+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff9b015eec0 con 0x7ff9b00705b0 2024-09-21T07:54:15.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.251+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7ff9a001a070 con 0x7ff9b00705b0 2024-09-21T07:54:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.251+0000 7ff9a7e00640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff990079250 0x7ff99007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.252+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7ff9a009bce0 con 0x7ff9b00705b0 2024-09-21T07:54:15.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.252+0000 7ff9af400640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff990079250 0x7ff99007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.252+0000 7ff9af400640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff990079250 0x7ff99007b760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00af330 tx=0x7ff99c005c90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:15.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.256+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff9b015eec0 con 0x7ff9b00705b0 2024-09-21T07:54:15.466 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.464+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7ff9b00cd550 con 0x7ff990079250 2024-09-21T07:54:15.467 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.466+0000 7ff9a7e00640 1 -- 172.21.15.88:0/2132197729 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7ff9b00cd550 con 0x7ff990079250 2024-09-21T07:54:15.467 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:54:15.467 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:15.467 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:54:15.467 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:15.467 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff990079250 msgr2=0x7ff99007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff990079250 0x7ff99007b760 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ff9b00af330 tx=0x7ff99c005c90 comp rx=0 tx=0).stop 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 msgr2=0x7ff9b01f5350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 0x7ff9b01f5350 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7ff9a000d750 tx=0x7ff9a000dc10 comp rx=0 tx=0).stop 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 shutdown_connections 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff990079250 0x7ff99007b760 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9b00cfc80 0x7ff9b0072f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9b015ce30 0x7ff9b0073460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 --2- 172.21.15.88:0/2132197729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b00705b0 0x7ff9b01f5350 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.468+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 >> 172.21.15.88:0/2132197729 conn(0x7ff9b006d3d0 msgr2=0x7ff9b015f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.469+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 shutdown_connections 2024-09-21T07:54:15.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.469+0000 7ff9b7200640 1 -- 172.21.15.88:0/2132197729 wait complete. 2024-09-21T07:54:15.493 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 -- 172.21.15.88:0/2262037504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700cd1d0 msgr2=0x7f8c700af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 --2- 172.21.15.88:0/2262037504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700cd1d0 0x7f8c700af0d0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f8c68005fd0 tx=0x7f8c6802aaa0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 -- 172.21.15.88:0/2262037504 shutdown_connections 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 --2- 172.21.15.88:0/2262037504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700cd1d0 0x7f8c700af0d0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 --2- 172.21.15.88:0/2262037504 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700b3f30 0x7f8c70156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 -- 172.21.15.88:0/2262037504 >> 172.21.15.88:0/2262037504 conn(0x7f8c7006d620 msgr2=0x7f8c700b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 -- 172.21.15.88:0/2262037504 shutdown_connections 2024-09-21T07:54:15.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.712+0000 7f8c76a00640 1 -- 172.21.15.88:0/2262037504 wait complete. 2024-09-21T07:54:15.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.714+0000 7f8c76a00640 1 Processor -- start 2024-09-21T07:54:15.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.715+0000 7f8c76a00640 1 -- start start 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 0x7f8c701f7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700cd1d0 0x7f8c70153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c76a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7015d6e0 0x7f8c70145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c76a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8c700af9a0 con 0x7f8c700b3f30 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c76a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8c700af820 con 0x7f8c7015d6e0 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c76a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8c700afb20 con 0x7f8c700cd1d0 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c75600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 0x7f8c701f7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c75600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 0x7f8c701f7180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:51600/0 (socket says 172.21.15.88:51600) 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.716+0000 7f8c75600640 1 -- 172.21.15.88:0/3732179506 learned_addr learned my addr 172.21.15.88:0/3732179506 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c76000640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7015d6e0 0x7f8c70145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c75600640 1 -- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700cd1d0 msgr2=0x7f8c70153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c74c00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700cd1d0 0x7f8c70153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c75600640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700cd1d0 0x7f8c70153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c75600640 1 -- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7015d6e0 msgr2=0x7f8c70145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c75600640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7015d6e0 0x7f8c70145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c75600640 1 -- 172.21.15.88:0/3732179506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c701f76c0 con 0x7f8c700b3f30 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c76000640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7015d6e0 0x7f8c70145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c74c00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700cd1d0 0x7f8c70153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c75600640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 0x7f8c701f7180 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400e790 tx=0x7f8c6400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c6400eea0 con 0x7f8c700b3f30 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8c6400cdb0 con 0x7f8c700b3f30 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c64010690 con 0x7f8c700b3f30 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c701f7b40 con 0x7f8c700b3f30 2024-09-21T07:54:15.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.717+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c70152a90 con 0x7f8c700b3f30 2024-09-21T07:54:15.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.719+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c70156ca0 con 0x7f8c700b3f30 2024-09-21T07:54:15.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.720+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f8c64010830 con 0x7f8c700b3f30 2024-09-21T07:54:15.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.721+0000 7f8c6d600640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c50079280 0x7f8c5007b790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:15.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.721+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f8c6409c3d0 con 0x7f8c700b3f30 2024-09-21T07:54:15.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.721+0000 7f8c74c00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c50079280 0x7f8c5007b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:15.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.722+0000 7f8c74c00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c50079280 0x7f8c5007b790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c7009e240 tx=0x7f8c5c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:15.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.725+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f8c64063de0 con 0x7f8c700b3f30 2024-09-21T07:54:15.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.910+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f8c700707f0 con 0x7f8c700b3f30 2024-09-21T07:54:15.912 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.911+0000 7f8c6d600640 1 -- 172.21.15.88:0/3732179506 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v48)=0 v48) ==== 122+0+0 (secure 0 0 0) 0x7f8c640636e0 con 0x7f8c700b3f30 2024-09-21T07:54:15.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c50079280 msgr2=0x7f8c5007b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.914 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c50079280 0x7f8c5007b790 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f8c7009e240 tx=0x7f8c5c009040 comp rx=0 tx=0).stop 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 msgr2=0x7f8c701f7180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 0x7f8c701f7180 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f8c6400e790 tx=0x7f8c6400ec50 comp rx=0 tx=0).stop 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 shutdown_connections 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8c50079280 0x7f8c5007b790 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8c700cd1d0 0x7f8c70153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8c7015d6e0 0x7f8c70145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 --2- 172.21.15.88:0/3732179506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c700b3f30 0x7f8c701f7180 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 >> 172.21.15.88:0/3732179506 conn(0x7f8c7006d620 msgr2=0x7f8c7015f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 shutdown_connections 2024-09-21T07:54:15.915 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:15.913+0000 7f8c76a00640 1 -- 172.21.15.88:0/3732179506 wait complete. 2024-09-21T07:54:15.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:15.938 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/_nogroup/subvol3430) 2024-09-21T07:54:16.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:16.000 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp16960/subvol85081) 2024-09-21T07:54:16.058 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol3430 2024-09-21T07:54:16.280 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 -- 172.21.15.88:0/240108481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 msgr2=0x7f9238156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 --2- 172.21.15.88:0/240108481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238156ca0 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7f9228005fd0 tx=0x7f922802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 -- 172.21.15.88:0/240108481 shutdown_connections 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 --2- 172.21.15.88:0/240108481 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f923815d6e0 0x7f92380ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 --2- 172.21.15.88:0/240108481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238156ca0 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 -- 172.21.15.88:0/240108481 >> 172.21.15.88:0/240108481 conn(0x7f923806d620 msgr2=0x7f92380b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 -- 172.21.15.88:0/240108481 shutdown_connections 2024-09-21T07:54:16.281 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.279+0000 7f923d600640 1 -- 172.21.15.88:0/240108481 wait complete. 2024-09-21T07:54:16.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.281+0000 7f923d600640 1 Processor -- start 2024-09-21T07:54:16.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.281+0000 7f923d600640 1 -- start start 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238155190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92380cd000 0x7f92381e9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923815d6e0 0x7f92381e9890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f92380af960 con 0x7f92380b3f30 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f92380af7e0 con 0x7f923815d6e0 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f92380afae0 con 0x7f92380cd000 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f9237e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238155190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f9237e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238155190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43754/0 (socket says 172.21.15.88:43754) 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f9237e00640 1 -- 172.21.15.88:0/842156540 learned_addr learned my addr 172.21.15.88:0/842156540 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f922f400640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92380cd000 0x7f92381e9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.283+0000 7f923cc00640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923815d6e0 0x7f92381e9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9237e00640 1 -- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92380cd000 msgr2=0x7f92381e9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9237e00640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92380cd000 0x7f92381e9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9237e00640 1 -- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923815d6e0 msgr2=0x7f92381e9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9237e00640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923815d6e0 0x7f92381e9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9237e00640 1 -- 172.21.15.88:0/842156540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f923809dff0 con 0x7f92380b3f30 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f922f400640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92380cd000 0x7f92381e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9237e00640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238155190 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9228006010 tx=0x7f92280038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f922803e030 con 0x7f92380b3f30 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9228003ca0 con 0x7f92380b3f30 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9238145960 con 0x7f92380b3f30 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9228035040 con 0x7f92380b3f30 2024-09-21T07:54:16.287 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.284+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9238073030 con 0x7f92380b3f30 2024-09-21T07:54:16.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.285+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9238156ca0 con 0x7f92380b3f30 2024-09-21T07:54:16.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.286+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f9228041030 con 0x7f92380b3f30 2024-09-21T07:54:16.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.287+0000 7f9234c00640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9214079250 0x7f921407b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.288+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f92280bc110 con 0x7f92380b3f30 2024-09-21T07:54:16.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.288+0000 7f922f400640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9214079250 0x7f921407b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.288+0000 7f922f400640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9214079250 0x7f921407b760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9220005fd0 tx=0x7f9220005ea0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:16.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.292+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9228083ad0 con 0x7f92380b3f30 2024-09-21T07:54:16.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.504+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol3430", "target": ["mon-mgr", ""]}) -- 0x7f9238145e80 con 0x7f9214079250 2024-09-21T07:54:16.512 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:16.511+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:16.512 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:16.511+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:16.512 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:16.511+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:16.513 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:16.511+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:16.513 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:16.511+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:16.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.542+0000 7f9234c00640 1 -- 172.21.15.88:0/842156540 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+29 (secure 0 0 0) 0x7f922000a310 con 0x7f9214079250 2024-09-21T07:54:16.543 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol3430 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9214079250 msgr2=0x7f921407b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9214079250 0x7f921407b760 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f9220005fd0 tx=0x7f9220005ea0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 msgr2=0x7f9238155190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238155190 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7f9228006010 tx=0x7f92280038a0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 shutdown_connections 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9214079250 0x7f921407b760 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.544+0000 7f923d600640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f92380cd000 0x7f92381e9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.545+0000 7f923d600640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f923815d6e0 0x7f92381e9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.545+0000 7f923d600640 1 --2- 172.21.15.88:0/842156540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f92380b3f30 0x7f9238155190 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.545+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 >> 172.21.15.88:0/842156540 conn(0x7f923806d620 msgr2=0x7f92380cfef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:16.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.545+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 shutdown_connections 2024-09-21T07:54:16.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.545+0000 7f923d600640 1 -- 172.21.15.88:0/842156540 wait complete. 2024-09-21T07:54:16.567 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol85081 subvol_grp16960 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 -- 172.21.15.88:0/4228129578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4003a00 msgr2=0x7fa6d4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 --2- 172.21.15.88:0/4228129578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4003a00 0x7fa6d4003e30 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0006f60 tx=0x7fa6d4005830 comp rx=0 tx=0).stop 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 -- 172.21.15.88:0/4228129578 shutdown_connections 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 --2- 172.21.15.88:0/4228129578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6d4003a00 0x7fa6d4003e30 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 -- 172.21.15.88:0/4228129578 >> 172.21.15.88:0/4228129578 conn(0x7fa6f00b2540 msgr2=0x7fa6f00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 -- 172.21.15.88:0/4228129578 shutdown_connections 2024-09-21T07:54:16.791 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.789+0000 7fa6f5600640 1 -- 172.21.15.88:0/4228129578 wait complete. 2024-09-21T07:54:16.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.792+0000 7fa6f5600640 1 Processor -- start 2024-09-21T07:54:16.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.792+0000 7fa6f5600640 1 -- start start 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6f5600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d4003a00 0x7fa6f00990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6f5600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0099790 0x7fa6f0156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6f5600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 0x7fa6f0156fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6f5600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00b34b0 con 0x7fa6f015b180 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6f5600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa6f00b3330 con 0x7fa6d4003a00 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6f5600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa6f006cda0 con 0x7fa6f0099790 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.793+0000 7fa6ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 0x7fa6f0156fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 0x7fa6f0156fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43778/0 (socket says 172.21.15.88:43778) 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 -- 172.21.15.88:0/1622169059 learned_addr learned my addr 172.21.15.88:0/1622169059 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 -- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0099790 msgr2=0x7fa6f0156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6eea00640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d4003a00 0x7fa6f00990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0099790 0x7fa6f0156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 -- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d4003a00 msgr2=0x7fa6f00990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d4003a00 0x7fa6f00990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 -- 172.21.15.88:0/1622169059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6f00cae30 con 0x7fa6f015b180 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6eea00640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d4003a00 0x7fa6f00990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:16.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6ef400640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 0x7fa6f0156fd0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00cf310 tx=0x7fa6e800dd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e8003d80 con 0x7fa6f015b180 2024-09-21T07:54:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6e800b780 con 0x7fa6f015b180 2024-09-21T07:54:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6e800bbd0 con 0x7fa6f015b180 2024-09-21T07:54:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.794+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6f0157810 con 0x7fa6f015b180 2024-09-21T07:54:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.795+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6f0157dd0 con 0x7fa6f015b180 2024-09-21T07:54:16.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.796+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6f00cb110 con 0x7fa6f015b180 2024-09-21T07:54:16.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.797+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7fa6e8006070 con 0x7fa6f015b180 2024-09-21T07:54:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.798+0000 7fa6e6a00640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6d0079250 0x7fa6d007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.798+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7fa6e809be40 con 0x7fa6f015b180 2024-09-21T07:54:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.798+0000 7fa6eea00640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6d0079250 0x7fa6d007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:16.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.799+0000 7fa6eea00640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6d0079250 0x7fa6d007b760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0033000 tx=0x7fa6f00afba0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:16.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:16.802+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6e8063880 con 0x7fa6f015b180 2024-09-21T07:54:17.017 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.016+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol85081", "group_name": "subvol_grp16960", "target": ["mon-mgr", ""]}) -- 0x7fa6f01582f0 con 0x7fa6d0079250 2024-09-21T07:54:17.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.034+0000 7fa6e6a00640 1 -- 172.21.15.88:0/1622169059 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fa6f01582f0 con 0x7fa6d0079250 2024-09-21T07:54:17.036 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp16960/subvol85081 2024-09-21T07:54:17.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6d0079250 msgr2=0x7fa6d007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6d0079250 0x7fa6d007b760 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa6e0033000 tx=0x7fa6f00afba0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 msgr2=0x7fa6f0156fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 0x7fa6f0156fd0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fa6f00cf310 tx=0x7fa6e800dd00 comp rx=0 tx=0).stop 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 shutdown_connections 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6d0079250 0x7fa6d007b760 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0099790 0x7fa6f0156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f015b180 0x7fa6f0156fd0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 --2- 172.21.15.88:0/1622169059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa6d4003a00 0x7fa6f00990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 >> 172.21.15.88:0/1622169059 conn(0x7fa6f00b2540 msgr2=0x7fa6f00b3c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 shutdown_connections 2024-09-21T07:54:17.039 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.037+0000 7fa6f5600640 1 -- 172.21.15.88:0/1622169059 wait complete. 2024-09-21T07:54:17.064 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol3430 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.286+0000 7f9bad600640 1 -- 172.21.15.88:0/1351463841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b8c003a30 msgr2=0x7f9b8c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.286+0000 7f9bad600640 1 --2- 172.21.15.88:0/1351463841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b8c003a30 0x7f9b8c003ea0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f9b980016c0 tx=0x7f9b8c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.286+0000 7f9ba7400640 1 -- 172.21.15.88:0/1351463841 reap_dead start 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.286+0000 7f9bad600640 1 -- 172.21.15.88:0/1351463841 shutdown_connections 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.286+0000 7f9bad600640 1 -- 172.21.15.88:0/1351463841 >> 172.21.15.88:0/1351463841 conn(0x7f9ba806d1c0 msgr2=0x7f9ba806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.287+0000 7f9bad600640 1 -- 172.21.15.88:0/1351463841 shutdown_connections 2024-09-21T07:54:17.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.287+0000 7f9bad600640 1 -- 172.21.15.88:0/1351463841 wait complete. 2024-09-21T07:54:17.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.289+0000 7f9bad600640 1 Processor -- start 2024-09-21T07:54:17.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.289+0000 7f9bad600640 1 -- start start 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.290+0000 7f9bad600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c003a30 0x7f9ba80b4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.290+0000 7f9bad600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba8144560 0x7f9ba8144970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.290+0000 7f9bad600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 0x7f9ba815bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.290+0000 7f9bad600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba8165490 con 0x7f9ba815b900 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.290+0000 7f9bad600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9ba80ce8e0 con 0x7f9b8c003a30 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.290+0000 7f9bad600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9ba8165610 con 0x7f9ba8144560 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 0x7f9ba815bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 0x7f9ba815bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43802/0 (socket says 172.21.15.88:43802) 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 -- 172.21.15.88:0/1231083209 learned_addr learned my addr 172.21.15.88:0/1231083209 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:17.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba6a00640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c003a30 0x7f9ba80b4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba6000640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba8144560 0x7f9ba8144970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 -- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba8144560 msgr2=0x7f9ba8144970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba8144560 0x7f9ba8144970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 -- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c003a30 msgr2=0x7f9ba80b4f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c003a30 0x7f9ba80b4f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 -- 172.21.15.88:0/1231083209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ba81f5440 con 0x7f9ba815b900 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba6000640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba8144560 0x7f9ba8144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.291+0000 7f9ba7400640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 0x7f9ba815bd10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80730d0 tx=0x7f9ba0010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.292+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0004b20 con 0x7f9ba815b900 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.292+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9ba0015070 con 0x7f9ba815b900 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.292+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ba81f5940 con 0x7f9ba815b900 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.292+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ba0003970 con 0x7f9ba815b900 2024-09-21T07:54:17.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.292+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ba81f5f00 con 0x7f9ba815b900 2024-09-21T07:54:17.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.293+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ba81f56b0 con 0x7f9ba815b900 2024-09-21T07:54:17.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.294+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f9ba0003b10 con 0x7f9ba815b900 2024-09-21T07:54:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.295+0000 7f9b9ea00640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9b88079250 0x7f9b8807b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.295+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f9ba009c410 con 0x7f9ba815b900 2024-09-21T07:54:17.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.295+0000 7f9ba6a00640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9b88079250 0x7f9b8807b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.296+0000 7f9ba6a00640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9b88079250 0x7f9b8807b760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9b98032000 tx=0x7f9b98001190 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.300 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.299+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9ba0063740 con 0x7f9ba815b900 2024-09-21T07:54:17.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.511+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol3430", "target": ["mon-mgr", ""]}) -- 0x7f9ba80b1ab0 con 0x7f9b88079250 2024-09-21T07:54:17.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.516+0000 7f9b9ea00640 1 -- 172.21.15.88:0/1231083209 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+29 (secure 0 0 0) 0x7f9ba80b1ab0 con 0x7f9b88079250 2024-09-21T07:54:17.517 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol3430 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9b88079250 msgr2=0x7f9b8807b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9b88079250 0x7f9b8807b760 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f9b98032000 tx=0x7f9b98001190 comp rx=0 tx=0).stop 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 msgr2=0x7f9ba815bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 0x7f9ba815bd10 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f9ba80730d0 tx=0x7f9ba0010f40 comp rx=0 tx=0).stop 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 shutdown_connections 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9b88079250 0x7f9b8807b760 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9ba8144560 0x7f9ba8144970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ba815b900 0x7f9ba815bd10 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 --2- 172.21.15.88:0/1231083209 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9b8c003a30 0x7f9ba80b4f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 >> 172.21.15.88:0/1231083209 conn(0x7f9ba806d1c0 msgr2=0x7f9ba8141b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.518+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 shutdown_connections 2024-09-21T07:54:17.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.519+0000 7f9bad600640 1 -- 172.21.15.88:0/1231083209 wait complete. 2024-09-21T07:54:17.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:17.545 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/11803f02422ffb80fc3518da0006c5eb.meta) 2024-09-21T07:54:17.606 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:54:17.606 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:54:17.606 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:54:17.607 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol3430 2024-09-21T07:54:17.607 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:54:17.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:17.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol85081 subvol_grp16960 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 -- 172.21.15.88:0/1374048937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 msgr2=0x7f0d400af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 --2- 172.21.15.88:0/1374048937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 0x7f0d400af510 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f0d38005fd0 tx=0x7f0d3802a8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 -- 172.21.15.88:0/1374048937 shutdown_connections 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 --2- 172.21.15.88:0/1374048937 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 0x7f0d400af510 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 --2- 172.21.15.88:0/1374048937 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d4006d4b0 0x7f0d400b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 -- 172.21.15.88:0/1374048937 >> 172.21.15.88:0/1374048937 conn(0x7f0d400b2540 msgr2=0x7f0d400d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 -- 172.21.15.88:0/1374048937 shutdown_connections 2024-09-21T07:54:17.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.833+0000 7f0d47800640 1 -- 172.21.15.88:0/1374048937 wait complete. 2024-09-21T07:54:17.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.835+0000 7f0d47800640 1 Processor -- start 2024-09-21T07:54:17.836 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.836+0000 7f0d47800640 1 -- start start 2024-09-21T07:54:17.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d47800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 0x7f0d400cf1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d47800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d400cc830 0x7f0d400cf710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d47800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 0x7f0d400cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d47800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400b3620 con 0x7f0d401557f0 2024-09-21T07:54:17.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d47800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0d400b34a0 con 0x7f0d400af100 2024-09-21T07:54:17.839 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d47800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0d400b37a0 con 0x7f0d400cc830 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d45800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 0x7f0d400cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d45800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 0x7f0d400cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43816/0 (socket says 172.21.15.88:43816) 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d45800640 1 -- 172.21.15.88:0/3048456299 learned_addr learned my addr 172.21.15.88:0/3048456299 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d3fe00640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d400cc830 0x7f0d400cf710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.837+0000 7f0d44e00640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 0x7f0d400cf1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d45800640 1 -- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d400cc830 msgr2=0x7f0d400cf710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d45800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d400cc830 0x7f0d400cf710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d45800640 1 -- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 msgr2=0x7f0d400cf1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d45800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 0x7f0d400cf1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d45800640 1 -- 172.21.15.88:0/3048456299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4015b9e0 con 0x7f0d401557f0 2024-09-21T07:54:17.840 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d45800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 0x7f0d400cdfb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006010 tx=0x7f0d38031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.838+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d38031990 con 0x7f0d401557f0 2024-09-21T07:54:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.839+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0d38031c20 con 0x7f0d401557f0 2024-09-21T07:54:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.839+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4015bcc0 con 0x7f0d401557f0 2024-09-21T07:54:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.839+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d380330d0 con 0x7f0d401557f0 2024-09-21T07:54:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.839+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4015c280 con 0x7f0d401557f0 2024-09-21T07:54:17.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.840+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0d400b3cc0 con 0x7f0d401557f0 2024-09-21T07:54:17.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.841+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 24) ==== 102053+0+0 (secure 0 0 0) 0x7f0d38033270 con 0x7f0d401557f0 2024-09-21T07:54:17.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.842+0000 7f0d3cc00640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0d20079250 0x7f0d2007b760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:17.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.843+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f0d380bce50 con 0x7f0d401557f0 2024-09-21T07:54:17.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.843+0000 7f0d44e00640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0d20079250 0x7f0d2007b760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:17.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.843+0000 7f0d44e00640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0d20079250 0x7f0d2007b760 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d30000f60 tx=0x7f0d30009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:17.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:17.847+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0d3803f030 con 0x7f0d401557f0 2024-09-21T07:54:18.016 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.014+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f0d380803b0 con 0x7f0d401557f0 2024-09-21T07:54:18.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.066+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol85081", "group_name": "subvol_grp16960", "target": ["mon-mgr", ""]}) -- 0x7f0d40157280 con 0x7f0d20079250 2024-09-21T07:54:18.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.070+0000 7f0d3cc00640 1 -- 172.21.15.88:0/3048456299 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0d40157280 con 0x7f0d20079250 2024-09-21T07:54:18.071 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp16960/subvol85081 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.072+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0d20079250 msgr2=0x7f0d2007b760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.072+0000 7f0d47800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0d20079250 0x7f0d2007b760 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0d30000f60 tx=0x7f0d30009040 comp rx=0 tx=0).stop 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.072+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 msgr2=0x7f0d400cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.072+0000 7f0d47800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 0x7f0d400cdfb0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f0d38006010 tx=0x7f0d38031700 comp rx=0 tx=0).stop 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 shutdown_connections 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0d20079250 0x7f0d2007b760 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0d400cc830 0x7f0d400cf710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d401557f0 0x7f0d400cdfb0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 --2- 172.21.15.88:0/3048456299 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0d400af100 0x7f0d400cf1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 >> 172.21.15.88:0/3048456299 conn(0x7f0d400b2540 msgr2=0x7f0d400ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 shutdown_connections 2024-09-21T07:54:18.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.073+0000 7f0d47800640 1 -- 172.21.15.88:0/3048456299 wait complete. 2024-09-21T07:54:18.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:18.097 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/d506aee788b82ab453728ab2e64d68b8.meta) 2024-09-21T07:54:18.156 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:54:18.156 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:54:18.156 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:54:18.157 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/subvol_grp16960/subvol85081 2024-09-21T07:54:18.157 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:54:18.157 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:18.160 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol3430 2024-09-21T07:54:18.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 -- 172.21.15.88:0/3314732228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08145550 msgr2=0x7f2c0806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 --2- 172.21.15.88:0/3314732228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08145550 0x7f2c0806fb40 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc009830 tx=0x7f2bfc02a5a0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 -- 172.21.15.88:0/3314732228 shutdown_connections 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 --2- 172.21.15.88:0/3314732228 >> v2:172.21.15.175:3300/0 conn(0x7f2c08145a40 0x7f2c08073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 --2- 172.21.15.88:0/3314732228 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08145550 0x7f2c0806fb40 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 -- 172.21.15.88:0/3314732228 >> 172.21.15.88:0/3314732228 conn(0x7f2c0806d1c0 msgr2=0x7f2c0806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 -- 172.21.15.88:0/3314732228 shutdown_connections 2024-09-21T07:54:18.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.381+0000 7f2c0f800640 1 -- 172.21.15.88:0/3314732228 wait complete. 2024-09-21T07:54:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.383+0000 7f2c0f800640 1 Processor -- start 2024-09-21T07:54:18.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.384+0000 7f2c0f800640 1 -- start start 2024-09-21T07:54:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0f800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08073a80 0x7f2c0813ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0f800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c08145550 0x7f2c081f96c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0f800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 0x7f2c081400a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0f800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2c08160a30 con 0x7f2c08145a40 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0f800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c080ce940 con 0x7f2c08145550 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0f800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c08160bb0 con 0x7f2c08073a80 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 0x7f2c081400a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0d800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 0x7f2c081400a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43824/0 (socket says 172.21.15.88:43824) 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0d800640 1 -- 172.21.15.88:0/1801659507 learned_addr learned my addr 172.21.15.88:0/1801659507 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c0ce00640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08073a80 0x7f2c0813ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.385+0000 7f2c07e00640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c08145550 0x7f2c081f96c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0d800640 1 -- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08073a80 msgr2=0x7f2c0813ded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0d800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08073a80 0x7f2c0813ded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0d800640 1 -- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c08145550 msgr2=0x7f2c081f96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0d800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c08145550 0x7f2c081f96c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0d800640 1 -- 172.21.15.88:0/1801659507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c081fa360 con 0x7f2c08145a40 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0d800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 0x7f2c081400a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c700 tx=0x7f2c0000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:18.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c00003b70 con 0x7f2c08145a40 2024-09-21T07:54:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c0815e970 con 0x7f2c08145a40 2024-09-21T07:54:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c00003d10 con 0x7f2c08145a40 2024-09-21T07:54:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c0000aab0 con 0x7f2c08145a40 2024-09-21T07:54:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.386+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c0815ef30 con 0x7f2c08145a40 2024-09-21T07:54:18.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.387+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c081610f0 con 0x7f2c08145a40 2024-09-21T07:54:18.390 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.389+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f2c0000ac50 con 0x7f2c08145a40 2024-09-21T07:54:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.390+0000 7f2c04c00640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2be8079310 0x7f2be807b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.390+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f2c0009bc50 con 0x7f2c08145a40 2024-09-21T07:54:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.390+0000 7f2c0ce00640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2be8079310 0x7f2be807b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.391+0000 7f2c0ce00640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2be8079310 0x7f2be807b820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc034000 tx=0x7f2bfc004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:18.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.394+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c08073e90 con 0x7f2c08145a40 2024-09-21T07:54:18.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.604+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol3430", "target": ["mon-mgr", ""]}) -- 0x7f2c08160810 con 0x7f2be8079310 2024-09-21T07:54:18.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.633+0000 7f2c04c00640 1 -- 172.21.15.88:0/1801659507 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2c08160810 con 0x7f2be8079310 2024-09-21T07:54:18.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2be8079310 msgr2=0x7f2be807b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2be8079310 0x7f2be807b820 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f2bfc034000 tx=0x7f2bfc004730 comp rx=0 tx=0).stop 2024-09-21T07:54:18.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 msgr2=0x7f2c081400a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 0x7f2c081400a0 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f2c0000c700 tx=0x7f2c0000cbc0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 shutdown_connections 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2be8079310 0x7f2be807b820 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c08145a40 0x7f2c081400a0 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c08145550 0x7f2c081f96c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 --2- 172.21.15.88:0/1801659507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c08073a80 0x7f2c0813ded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 >> 172.21.15.88:0/1801659507 conn(0x7f2c0806d1c0 msgr2=0x7f2c081436b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 shutdown_connections 2024-09-21T07:54:18.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.636+0000 7f2c0f800640 1 -- 172.21.15.88:0/1801659507 wait complete. 2024-09-21T07:54:18.639 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.638+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.639 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.638+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.639 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.638+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.639 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.638+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.639 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.638+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.657 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.656+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.658 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.656+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.658 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.656+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.658 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.656+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.658 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:18.656+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:18.664 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol85081 subvol_grp16960 2024-09-21T07:54:18.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.887+0000 7f2994a00640 1 -- 172.21.15.88:0/3120634502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 msgr2=0x7f29900af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.887+0000 7f2994a00640 1 --2- 172.21.15.88:0/3120634502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29900af510 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f2988005fd0 tx=0x7f298802aa90 comp rx=0 tx=0).stop 2024-09-21T07:54:18.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.888+0000 7f2994a00640 1 -- 172.21.15.88:0/3120634502 shutdown_connections 2024-09-21T07:54:18.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.888+0000 7f2994a00640 1 --2- 172.21.15.88:0/3120634502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29900af510 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.888+0000 7f2994a00640 1 --2- 172.21.15.88:0/3120634502 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 0x7f29900b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.888+0000 7f2994a00640 1 -- 172.21.15.88:0/3120634502 >> 172.21.15.88:0/3120634502 conn(0x7f29900b2540 msgr2=0x7f29900d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:18.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.888+0000 7f2994a00640 1 -- 172.21.15.88:0/3120634502 shutdown_connections 2024-09-21T07:54:18.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.888+0000 7f2994a00640 1 -- 172.21.15.88:0/3120634502 wait complete. 2024-09-21T07:54:18.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.890+0000 7f2994a00640 1 Processor -- start 2024-09-21T07:54:18.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.890+0000 7f2994a00640 1 -- start start 2024-09-21T07:54:18.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.891+0000 7f2994a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299006cda0 0x7f2990156360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.891+0000 7f2994a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29901f6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.891+0000 7f2994a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 0x7f29900c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.891+0000 7f2994a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29900b36a0 con 0x7f29900af100 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.891+0000 7f2994a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f29900b3520 con 0x7f299006cda0 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.891+0000 7f2994a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f29900b3820 con 0x7f29900cc830 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29901f6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29901f6280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43862/0 (socket says 172.21.15.88:43862) 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 -- 172.21.15.88:0/2297444797 learned_addr learned my addr 172.21.15.88:0/2297444797 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f298de00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299006cda0 0x7f2990156360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f298e800640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 0x7f29900c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 -- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 msgr2=0x7f29900c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 0x7f29900c9c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 -- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299006cda0 msgr2=0x7f2990156360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299006cda0 0x7f2990156360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 -- 172.21.15.88:0/2297444797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29901552c0 con 0x7f29900af100 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f298e800640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 0x7f29900c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f298de00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299006cda0 0x7f2990156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2985400640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29901f6280 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f297800c3a0 tx=0x7f297800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.892+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978017070 con 0x7f29900af100 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.893+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f29780053c0 con 0x7f29900af100 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.893+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29901555a0 con 0x7f29900af100 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.893+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2978003850 con 0x7f29900af100 2024-09-21T07:54:18.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.893+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2990155b60 con 0x7f29900af100 2024-09-21T07:54:18.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.894+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2990155010 con 0x7f29900af100 2024-09-21T07:54:18.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.895+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f297801a070 con 0x7f29900af100 2024-09-21T07:54:18.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.896+0000 7f2986a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f296c079310 0x7f296c07b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:18.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.896+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f297809b3b0 con 0x7f29900af100 2024-09-21T07:54:18.897 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.896+0000 7f298de00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f296c079310 0x7f296c07b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:18.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.897+0000 7f298de00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f296c079310 0x7f296c07b820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2980005fd0 tx=0x7f298000f360 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:18.901 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:18.900+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2978062790 con 0x7f29900af100 2024-09-21T07:54:19.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.120+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol85081", "group_name": "subvol_grp16960", "target": ["mon-mgr", ""]}) -- 0x7f29900b3d40 con 0x7f296c079310 2024-09-21T07:54:19.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.147+0000 7f2986a00640 1 -- 172.21.15.88:0/2297444797 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f29900b3d40 con 0x7f296c079310 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.149+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f296c079310 msgr2=0x7f296c07b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.149+0000 7f2994a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f296c079310 0x7f296c07b820 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f2980005fd0 tx=0x7f298000f360 comp rx=0 tx=0).stop 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.149+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 msgr2=0x7f29901f6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.149+0000 7f2994a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29901f6280 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f297800c3a0 tx=0x7f297800c860 comp rx=0 tx=0).stop 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 shutdown_connections 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f296c079310 0x7f296c07b820 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f29900cc830 0x7f29900c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29900af100 0x7f29901f6280 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 --2- 172.21.15.88:0/2297444797 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f299006cda0 0x7f2990156360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 >> 172.21.15.88:0/2297444797 conn(0x7f29900b2540 msgr2=0x7f299015fc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 shutdown_connections 2024-09-21T07:54:19.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.150+0000 7f2994a00640 1 -- 172.21.15.88:0/2297444797 wait complete. 2024-09-21T07:54:19.175 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:54:19.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:19.256 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:54:19.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:19.264 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp16960 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.489+0000 7f5ec2200640 1 -- 172.21.15.88:0/1100463715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc0cd000 msgr2=0x7f5ebc0af0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.489+0000 7f5ec2200640 1 --2- 172.21.15.88:0/1100463715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc0cd000 0x7f5ebc0af0d0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f5ebc15de20 tx=0x7f5eb402a9e0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.489+0000 7f5ec2200640 1 -- 172.21.15.88:0/1100463715 shutdown_connections 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.489+0000 7f5ec2200640 1 --2- 172.21.15.88:0/1100463715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc0cd000 0x7f5ebc0af0d0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.489+0000 7f5ec2200640 1 --2- 172.21.15.88:0/1100463715 >> v2:172.21.15.175:3300/0 conn(0x7f5ebc156ca0 0x7f5ebc0ccac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.489+0000 7f5ec2200640 1 -- 172.21.15.88:0/1100463715 >> 172.21.15.88:0/1100463715 conn(0x7f5ebc06d620 msgr2=0x7f5ebc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.490+0000 7f5ec2200640 1 -- 172.21.15.88:0/1100463715 shutdown_connections 2024-09-21T07:54:19.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.490+0000 7f5ec2200640 1 -- 172.21.15.88:0/1100463715 wait complete. 2024-09-21T07:54:19.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.492+0000 7f5ec2200640 1 Processor -- start 2024-09-21T07:54:19.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.492+0000 7f5ec2200640 1 -- start start 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ebc0b3f30 0x7f5ebc09a230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ebc0cd000 0x7f5ebc098660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 0x7f5ebc098ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0af9a0 con 0x7f5ebc156ca0 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ebc0af820 con 0x7f5ebc0b3f30 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ebc0afb20 con 0x7f5ebc0cd000 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.493+0000 7f5ec1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 0x7f5ebc098ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 0x7f5ebc098ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43888/0 (socket says 172.21.15.88:43888) 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 -- 172.21.15.88:0/2916376672 learned_addr learned my addr 172.21.15.88:0/2916376672 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:19.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5eb3e00640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ebc0cd000 0x7f5ebc098660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec0e00640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ebc0b3f30 0x7f5ebc09a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 -- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ebc0cd000 msgr2=0x7f5ebc098660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ebc0cd000 0x7f5ebc098660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 -- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ebc0b3f30 msgr2=0x7f5ebc09a230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ebc0b3f30 0x7f5ebc09a230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 -- 172.21.15.88:0/2916376672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ebc09a770 con 0x7f5ebc156ca0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.494+0000 7f5ec1800640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 0x7f5ebc098ba0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4005fd0 tx=0x7f5eb4003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.495+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4033050 con 0x7f5ebc156ca0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.495+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5eb4004450 con 0x7f5ebc156ca0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.495+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ebc1f7c00 con 0x7f5ebc156ca0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.495+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5eb4038450 con 0x7f5ebc156ca0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.495+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ebc1f7640 con 0x7f5ebc156ca0 2024-09-21T07:54:19.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.496+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ebc1f70b0 con 0x7f5ebc156ca0 2024-09-21T07:54:19.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.497+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) ==== 102104+0+0 (secure 0 0 0) 0x7f5eb4036070 con 0x7f5ebc156ca0 2024-09-21T07:54:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.498+0000 7f5eb9600640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e980792e0 0x7f5e9807b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.498+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f5eb40bceb0 con 0x7f5ebc156ca0 2024-09-21T07:54:19.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.498+0000 7f5ec0e00640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e980792e0 0x7f5e9807b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:19.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.499+0000 7f5ec0e00640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e980792e0 0x7f5e9807b7f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5eac006fd0 tx=0x7f5eac009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:19.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.503+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5eb40840d0 con 0x7f5ebc156ca0 2024-09-21T07:54:19.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.711+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp16960", "target": ["mon-mgr", ""]}) -- 0x7f5ebc0af0d0 con 0x7f5e980792e0 2024-09-21T07:54:19.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.727+0000 7f5eb9600640 1 -- 172.21.15.88:0/2916376672 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5ebc0af0d0 con 0x7f5e980792e0 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e980792e0 msgr2=0x7f5e9807b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e980792e0 0x7f5e9807b7f0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f5eac006fd0 tx=0x7f5eac009060 comp rx=0 tx=0).stop 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 msgr2=0x7f5ebc098ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 0x7f5ebc098ba0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f5eb4005fd0 tx=0x7f5eb4003f10 comp rx=0 tx=0).stop 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 shutdown_connections 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e980792e0 0x7f5e9807b7f0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ebc0cd000 0x7f5ebc098660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ebc156ca0 0x7f5ebc098ba0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 --2- 172.21.15.88:0/2916376672 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ebc0b3f30 0x7f5ebc09a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:19.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.729+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 >> 172.21.15.88:0/2916376672 conn(0x7f5ebc06d620 msgr2=0x7f5ebc0ae4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:19.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.730+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 shutdown_connections 2024-09-21T07:54:19.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:19.730+0000 7f5ec2200640 1 -- 172.21.15.88:0/2916376672 wait complete. 2024-09-21T07:54:19.757 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:54:19.757 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:19.850 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:19.940 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:54:19.940 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:19.940 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:19.940 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:19.978 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:19.978 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:20.058 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee2de00640 1 -- 172.21.15.88:0/2484034413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee0c004800 msgr2=0x7fee0c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee2de00640 1 --2- 172.21.15.88:0/2484034413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee0c004800 0x7fee0c004c10 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fee10009b50 tx=0x7fee0c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee27400640 1 -- 172.21.15.88:0/2484034413 reap_dead start 2024-09-21T07:54:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee2de00640 1 -- 172.21.15.88:0/2484034413 shutdown_connections 2024-09-21T07:54:20.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee2de00640 1 -- 172.21.15.88:0/2484034413 >> 172.21.15.88:0/2484034413 conn(0x7fee2806d3d0 msgr2=0x7fee280b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:20.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee2de00640 1 -- 172.21.15.88:0/2484034413 shutdown_connections 2024-09-21T07:54:20.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.318+0000 7fee2de00640 1 -- 172.21.15.88:0/2484034413 wait complete. 2024-09-21T07:54:20.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.321+0000 7fee2de00640 1 Processor -- start 2024-09-21T07:54:20.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.321+0000 7fee2de00640 1 -- start start 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.322+0000 7fee2de00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c004800 0x7fee2814ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.322+0000 7fee2de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 0x7fee280c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.322+0000 7fee2de00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee28074230 0x7fee28074640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.322+0000 7fee2de00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee280cb950 con 0x7fee281f1060 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.322+0000 7fee2de00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fee280afab0 con 0x7fee0c004800 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.322+0000 7fee2de00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fee280cbad0 con 0x7fee28074230 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 0x7fee280c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 0x7fee280c9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43914/0 (socket says 172.21.15.88:43914) 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 -- 172.21.15.88:0/1528147799 learned_addr learned my addr 172.21.15.88:0/1528147799 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee26a00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c004800 0x7fee2814ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee27400640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee28074230 0x7fee28074640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 -- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee28074230 msgr2=0x7fee28074640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee28074230 0x7fee28074640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 -- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c004800 msgr2=0x7fee2814ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c004800 0x7fee2814ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 -- 172.21.15.88:0/1528147799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee28074b80 con 0x7fee281f1060 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.323+0000 7fee1fe00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 0x7fee280c9d60 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fee10036040 tx=0x7fee10033670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.324+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10038040 con 0x7fee281f1060 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.324+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fee10004020 con 0x7fee281f1060 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.324+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee10004470 con 0x7fee281f1060 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.324+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281f5480 con 0x7fee281f1060 2024-09-21T07:54:20.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.324+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee280c7800 con 0x7fee281f1060 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.325+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee280c7a40 con 0x7fee281f1060 2024-09-21T07:54:20.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.326+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fee10045020 con 0x7fee281f1060 2024-09-21T07:54:20.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.327+0000 7fee1f400640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee04079400 0x7fee0407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:20.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.327+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7fee100bc9f0 con 0x7fee281f1060 2024-09-21T07:54:20.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.327+0000 7fee26a00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee04079400 0x7fee0407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:20.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.328+0000 7fee26a00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee04079400 0x7fee0407b910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee28072000 tx=0x7fee180067a0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:20.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.332+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fee280c7a40 con 0x7fee281f1060 2024-09-21T07:54:20.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.528+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fee280ad440 con 0x7fee281f1060 2024-09-21T07:54:20.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.528+0000 7fee1f400640 1 -- 172.21.15.88:0/1528147799 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7fee28074b80 con 0x7fee281f1060 2024-09-21T07:54:20.529 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:54:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee04079400 msgr2=0x7fee0407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee04079400 0x7fee0407b910 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fee28072000 tx=0x7fee180067a0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 msgr2=0x7fee280c9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 0x7fee280c9d60 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fee10036040 tx=0x7fee10033670 comp rx=0 tx=0).stop 2024-09-21T07:54:20.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 shutdown_connections 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fee04079400 0x7fee0407b910 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fee28074230 0x7fee28074640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee281f1060 0x7fee280c9d60 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 --2- 172.21.15.88:0/1528147799 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fee0c004800 0x7fee2814ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 >> 172.21.15.88:0/1528147799 conn(0x7fee2806d3d0 msgr2=0x7fee280adb10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 shutdown_connections 2024-09-21T07:54:20.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.531+0000 7fee2de00640 1 -- 172.21.15.88:0/1528147799 wait complete. 2024-09-21T07:54:20.555 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:54:20.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:20.556 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:20.618 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:54:20.618 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:20.618 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:20.653 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:20.653 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:20.653 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:20.693 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:20.694 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:20.772 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553e00640 1 -- 172.21.15.88:0/4119202153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 msgr2=0x7f1538005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553e00640 1 --2- 172.21.15.88:0/4119202153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f1538005150 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x7f154c005fd0 tx=0x7f1538005830 comp rx=0 tx=0).stop 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553400640 1 -- 172.21.15.88:0/4119202153 reap_dead start 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553e00640 1 -- 172.21.15.88:0/4119202153 shutdown_connections 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553e00640 1 -- 172.21.15.88:0/4119202153 >> 172.21.15.88:0/4119202153 conn(0x7f15540b1e40 msgr2=0x7f15540b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553e00640 1 -- 172.21.15.88:0/4119202153 shutdown_connections 2024-09-21T07:54:20.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.995+0000 7f1553e00640 1 -- 172.21.15.88:0/4119202153 wait complete. 2024-09-21T07:54:20.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.998+0000 7f1553e00640 1 Processor -- start 2024-09-21T07:54:20.999 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.998+0000 7f1553e00640 1 -- start start 2024-09-21T07:54:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1553e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f15541f67f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1553e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1554149060 0x7f15540cdc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1553e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1d0 0x7f15541f88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1553e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f155415d7f0 con 0x7f1538004d40 2024-09-21T07:54:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1553e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f155415d670 con 0x7f15540ce1d0 2024-09-21T07:54:21.002 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1553e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f155415d970 con 0x7f1554149060 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1552a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f15541f67f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1552a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f15541f67f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43934/0 (socket says 172.21.15.88:43934) 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:20.999+0000 7f1552a00640 1 -- 172.21.15.88:0/6147729 learned_addr learned my addr 172.21.15.88:0/6147729 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552000640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1554149060 0x7f15540cdc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1553400640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1d0 0x7f15541f88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552a00640 1 -- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1554149060 msgr2=0x7f15540cdc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552a00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1554149060 0x7f15540cdc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552a00640 1 -- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1d0 msgr2=0x7f15541f88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552a00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1d0 0x7f15541f88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552a00640 1 -- 172.21.15.88:0/6147729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15540cf8b0 con 0x7f1538004d40 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1552a00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f15541f67f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1544006640 tx=0x7f1544003a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154400f780 con 0x7f1538004d40 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1544016020 con 0x7f1538004d40 2024-09-21T07:54:21.003 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15540ccbf0 con 0x7f1538004d40 2024-09-21T07:54:21.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.000+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f154400fbc0 con 0x7f1538004d40 2024-09-21T07:54:21.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.001+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15540cd1b0 con 0x7f1538004d40 2024-09-21T07:54:21.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.001+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f155406a610 con 0x7f1538004d40 2024-09-21T07:54:21.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.003+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f1544011430 con 0x7f1538004d40 2024-09-21T07:54:21.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.004+0000 7f154aa00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1534079400 0x7f153407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.004+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f154409cbb0 con 0x7f1538004d40 2024-09-21T07:54:21.005 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.004+0000 7f1552000640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1534079400 0x7f153407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.006 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.005+0000 7f1552000640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1534079400 0x7f153407b910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f15540d1080 tx=0x7f1540005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.009 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.008+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f155406a610 con 0x7f1538004d40 2024-09-21T07:54:21.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.205+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f155406fdc0 con 0x7f1538004d40 2024-09-21T07:54:21.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.206+0000 7f154aa00640 1 -- 172.21.15.88:0/6147729 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v145) ==== 80+0+0 (secure 0 0 0) 0x7f155406fdc0 con 0x7f1538004d40 2024-09-21T07:54:21.207 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:54:21.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1534079400 msgr2=0x7f153407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1534079400 0x7f153407b910 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f15540d1080 tx=0x7f1540005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:21.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 msgr2=0x7f15541f67f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f15541f67f0 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7f1544006640 tx=0x7f1544003a80 comp rx=0 tx=0).stop 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 shutdown_connections 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1534079400 0x7f153407b910 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1554149060 0x7f15540cdc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f15540ce1d0 0x7f15541f88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 --2- 172.21.15.88:0/6147729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1538004d40 0x7f15541f67f0 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 >> 172.21.15.88:0/6147729 conn(0x7f15540b1e40 msgr2=0x7f15540b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 shutdown_connections 2024-09-21T07:54:21.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.208+0000 7f1553e00640 1 -- 172.21.15.88:0/6147729 wait complete. 2024-09-21T07:54:21.231 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:54:21.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:21.232 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:21.301 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:54:21.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:21.301 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:54:21.339 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:21.339 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:21.339 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:21.374 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:21.374 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:21.375 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:21.375 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:21.416 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:21.416 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 -- 172.21.15.88:0/1717203318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f6006f9a0 msgr2=0x7f0f600b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 --2- 172.21.15.88:0/1717203318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f6006f9a0 0x7f0f600b2f60 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005fd0 tx=0x7f0f5802a970 comp rx=0 tx=0).stop 2024-09-21T07:54:21.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 -- 172.21.15.88:0/1717203318 shutdown_connections 2024-09-21T07:54:21.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 --2- 172.21.15.88:0/1717203318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f6006f9a0 0x7f0f600b2f60 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 -- 172.21.15.88:0/1717203318 >> 172.21.15.88:0/1717203318 conn(0x7f0f600b1e40 msgr2=0x7f0f600b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 -- 172.21.15.88:0/1717203318 shutdown_connections 2024-09-21T07:54:21.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.634+0000 7f0f65400640 1 -- 172.21.15.88:0/1717203318 wait complete. 2024-09-21T07:54:21.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.636+0000 7f0f65400640 1 Processor -- start 2024-09-21T07:54:21.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.636+0000 7f0f65400640 1 -- start start 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f65400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f600ad9c0 0x7f0f600cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f65400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f600af170 0x7f0f60155180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f65400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 0x7f0f601556c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f65400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6015a4c0 con 0x7f0f601509f0 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f65400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0f6015a340 con 0x7f0f600af170 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f65400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0f6015d670 con 0x7f0f600ad9c0 2024-09-21T07:54:21.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f64a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 0x7f0f601556c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f64a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 0x7f0f601556c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43948/0 (socket says 172.21.15.88:43948) 2024-09-21T07:54:21.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f64a00640 1 -- 172.21.15.88:0/1703589322 learned_addr learned my addr 172.21.15.88:0/1703589322 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f64a00640 1 -- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f600ad9c0 msgr2=0x7f0f600cb3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f64a00640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f600ad9c0 0x7f0f600cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f64a00640 1 -- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f600af170 msgr2=0x7f0f60155180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.638+0000 7f0f5f400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f600af170 0x7f0f60155180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f64a00640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f600af170 0x7f0f60155180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f64a00640 1 -- 172.21.15.88:0/1703589322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f600c94c0 con 0x7f0f601509f0 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f5f400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f600af170 0x7f0f60155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f64a00640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 0x7f0f601556c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0f58033040 tx=0x7f0f58003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58034040 con 0x7f0f601509f0 2024-09-21T07:54:21.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0f58004640 con 0x7f0f601509f0 2024-09-21T07:54:21.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f600c97a0 con 0x7f0f601509f0 2024-09-21T07:54:21.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f58045480 con 0x7f0f601509f0 2024-09-21T07:54:21.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.639+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f600ce240 con 0x7f0f601509f0 2024-09-21T07:54:21.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.640+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f600cbcc0 con 0x7f0f601509f0 2024-09-21T07:54:21.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.641+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f0f58037020 con 0x7f0f601509f0 2024-09-21T07:54:21.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.642+0000 7f0f57e00640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0f40079400 0x7f0f4007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:21.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.643+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f0f580bafd0 con 0x7f0f601509f0 2024-09-21T07:54:21.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.643+0000 7f0f5fe00640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0f40079400 0x7f0f4007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:21.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.643+0000 7f0f5fe00640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0f40079400 0x7f0f4007b910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50006bc0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:21.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.647+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0f58084ea0 con 0x7f0f601509f0 2024-09-21T07:54:21.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.871+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0f600ccac0 con 0x7f0f601509f0 2024-09-21T07:54:21.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.872+0000 7f0f57e00640 1 -- 172.21.15.88:0/1703589322 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5234 (secure 0 0 0) 0x7f0f600ccac0 con 0x7f0f601509f0 2024-09-21T07:54:21.873 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:21.873 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":100,"btime":"2024-09-21T07:54:14:989822+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:03.440969+0000","modified":"2024-09-21T07:54:06.451584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3712798522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3712798522},{"type":"v1","addr":"172.21.15.88:6835","nonce":3712798522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":11}]} 2024-09-21T07:54:21.873 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 100 2024-09-21T07:54:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0f40079400 msgr2=0x7f0f4007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0f40079400 0x7f0f4007b910 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f0f50006fd0 tx=0x7f0f50006bc0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 msgr2=0x7f0f601556c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:21.875 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 0x7f0f601556c0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7f0f58033040 tx=0x7f0f58003f10 comp rx=0 tx=0).stop 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 shutdown_connections 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0f40079400 0x7f0f4007b910 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0f601509f0 0x7f0f601556c0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0f600af170 0x7f0f60155180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 --2- 172.21.15.88:0/1703589322 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0f600ad9c0 0x7f0f600cb3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 >> 172.21.15.88:0/1703589322 conn(0x7f0f600b1e40 msgr2=0x7f0f600b05e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 shutdown_connections 2024-09-21T07:54:21.876 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:21.874+0000 7f0f65400640 1 -- 172.21.15.88:0/1703589322 wait complete. 2024-09-21T07:54:21.899 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:22.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f09a00640 1 -- 172.21.15.88:0/2225834374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ee8004d40 msgr2=0x7f7ee8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f09a00640 1 --2- 172.21.15.88:0/2225834374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ee8004d40 0x7f7ee8005150 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7f7f040aea60 tx=0x7f7ee8005830 comp rx=0 tx=0).stop 2024-09-21T07:54:22.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f03400640 1 -- 172.21.15.88:0/2225834374 reap_dead start 2024-09-21T07:54:22.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f09a00640 1 -- 172.21.15.88:0/2225834374 shutdown_connections 2024-09-21T07:54:22.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f09a00640 1 -- 172.21.15.88:0/2225834374 >> 172.21.15.88:0/2225834374 conn(0x7f7f0406d730 msgr2=0x7f7f040b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f09a00640 1 -- 172.21.15.88:0/2225834374 shutdown_connections 2024-09-21T07:54:22.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.121+0000 7f7f09a00640 1 -- 172.21.15.88:0/2225834374 wait complete. 2024-09-21T07:54:22.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.123+0000 7f7f09a00640 1 Processor -- start 2024-09-21T07:54:22.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.124+0000 7f7f09a00640 1 -- start start 2024-09-21T07:54:22.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7f09a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee8004d40 0x7f7f0409dae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7f09a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 0x7f7f04156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7f09a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f04151b90 0x7f7f04156b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7f09a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f040b3ea0 con 0x7f7f04151780 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7f09a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7f040b4e30 con 0x7f7ee8004d40 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7f09a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7f040b4020 con 0x7f7f04151b90 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7efa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 0x7f7f04156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7efa000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 0x7f7f04156620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43974/0 (socket says 172.21.15.88:43974) 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.125+0000 7f7efa000640 1 -- 172.21.15.88:0/3882305652 learned_addr learned my addr 172.21.15.88:0/3882305652 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7f02a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee8004d40 0x7f7f0409dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7f03400640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f04151b90 0x7f7f04156b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efa000640 1 -- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f04151b90 msgr2=0x7f7f04156b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efa000640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f04151b90 0x7f7f04156b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efa000640 1 -- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee8004d40 msgr2=0x7f7f0409dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efa000640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee8004d40 0x7f7f0409dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efa000640 1 -- 172.21.15.88:0/3882305652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f0409a2d0 con 0x7f7f04151780 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7f02a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee8004d40 0x7f7f0409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efa000640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 0x7f7f04156620 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00e6d0 tx=0x7f7eec00eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec004230 con 0x7f7f04151780 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.126+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7eec016070 con 0x7f7f04151780 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.127+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f041f59f0 con 0x7f7f04151780 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.127+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eec00caa0 con 0x7f7f04151780 2024-09-21T07:54:22.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.127+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f041f5fb0 con 0x7f7f04151780 2024-09-21T07:54:22.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.128+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f040b3960 con 0x7f7f04151780 2024-09-21T07:54:22.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.129+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f7eec00cc40 con 0x7f7f04151780 2024-09-21T07:54:22.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.130+0000 7f7efb400640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ee0079400 0x7f7ee007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.131+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f7eec099a00 con 0x7f7f04151780 2024-09-21T07:54:22.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.131+0000 7f7f02a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ee0079400 0x7f7ee007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.131+0000 7f7f02a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ee0079400 0x7f7ee007b910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009c50 tx=0x7f7ef4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.135+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7eec063980 con 0x7f7f04151780 2024-09-21T07:54:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.358+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f0415e500 con 0x7f7f04151780 2024-09-21T07:54:22.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.358+0000 7f7efb400640 1 -- 172.21.15.88:0/3882305652 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5234 (secure 0 0 0) 0x7f7eec063280 con 0x7f7f04151780 2024-09-21T07:54:22.360 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:22.360 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":100,"btime":"2024-09-21T07:54:14:989822+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:03.440969+0000","modified":"2024-09-21T07:54:06.451584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3712798522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3712798522},{"type":"v1","addr":"172.21.15.88:6835","nonce":3712798522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":11}]} 2024-09-21T07:54:22.360 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 100 2024-09-21T07:54:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ee0079400 msgr2=0x7f7ee007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ee0079400 0x7f7ee007b910 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f7ef4009c50 tx=0x7f7ef4005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 msgr2=0x7f7f04156620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 0x7f7f04156620 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x7f7eec00e6d0 tx=0x7f7eec00eb90 comp rx=0 tx=0).stop 2024-09-21T07:54:22.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 shutdown_connections 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7ee0079400 0x7f7ee007b910 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7f04151b90 0x7f7f04156b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f04151780 0x7f7f04156620 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 --2- 172.21.15.88:0/3882305652 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7ee8004d40 0x7f7f0409dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 >> 172.21.15.88:0/3882305652 conn(0x7f7f0406d730 msgr2=0x7f7f0415ea80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 shutdown_connections 2024-09-21T07:54:22.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.361+0000 7f7f09a00640 1 -- 172.21.15.88:0/3882305652 wait complete. 2024-09-21T07:54:22.386 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 -- 172.21.15.88:0/4167846932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b48003a00 msgr2=0x7f6b48003e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 --2- 172.21.15.88:0/4167846932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b48003a00 0x7f6b48003e50 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c4a0 tx=0x7f6b48005830 comp rx=0 tx=0).stop 2024-09-21T07:54:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 -- 172.21.15.88:0/4167846932 shutdown_connections 2024-09-21T07:54:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 --2- 172.21.15.88:0/4167846932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b48003a00 0x7f6b48003e50 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 -- 172.21.15.88:0/4167846932 >> 172.21.15.88:0/4167846932 conn(0x7f6b6006d730 msgr2=0x7f6b600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 -- 172.21.15.88:0/4167846932 shutdown_connections 2024-09-21T07:54:22.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.606+0000 7f6b68200640 1 -- 172.21.15.88:0/4167846932 wait complete. 2024-09-21T07:54:22.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.608+0000 7f6b68200640 1 Processor -- start 2024-09-21T07:54:22.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.608+0000 7f6b68200640 1 -- start start 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b68200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b48003a00 0x7f6b6009dde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b68200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 0x7f6b601f5670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b68200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b60149720 0x7f6b6009bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b68200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b6006f260 con 0x7f6b601591d0 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b68200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b600b4e30 con 0x7f6b60149720 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b68200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b600ad4f0 con 0x7f6b48003a00 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b64e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 0x7f6b601f5670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b64e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 0x7f6b601f5670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43994/0 (socket says 172.21.15.88:43994) 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.610+0000 7f6b64e00640 1 -- 172.21.15.88:0/1865300366 learned_addr learned my addr 172.21.15.88:0/1865300366 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b64e00640 1 -- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b48003a00 msgr2=0x7f6b6009dde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b65800640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b48003a00 0x7f6b6009dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b66200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b60149720 0x7f6b6009bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b64e00640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b48003a00 0x7f6b6009dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b64e00640 1 -- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b60149720 msgr2=0x7f6b6009bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b64e00640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b60149720 0x7f6b6009bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b64e00640 1 -- 172.21.15.88:0/1865300366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b6009a370 con 0x7f6b601591d0 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b66200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b60149720 0x7f6b6009bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:22.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b64e00640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 0x7f6b601f5670 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00f780 tx=0x7f6b4c00fc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c015070 con 0x7f6b601591d0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b4c00fe90 con 0x7f6b601591d0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b601f5f60 con 0x7f6b601591d0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b4c00bb00 con 0x7f6b601591d0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.611+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b601f6520 con 0x7f6b601591d0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.612+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b601535c0 con 0x7f6b601591d0 2024-09-21T07:54:22.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.613+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f6b4c00bca0 con 0x7f6b601591d0 2024-09-21T07:54:22.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.614+0000 7f6b5d600640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b40079400 0x7f6b4007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:22.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.615+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f6b4c09c870 con 0x7f6b601591d0 2024-09-21T07:54:22.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.615+0000 7f6b65800640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b40079400 0x7f6b4007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:22.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.615+0000 7f6b65800640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b40079400 0x7f6b4007b910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c470 tx=0x7f6b54001c00 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:22.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.619+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b4c064100 con 0x7f6b601591d0 2024-09-21T07:54:22.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.805+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6b600d08d0 con 0x7f6b601591d0 2024-09-21T07:54:22.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.806+0000 7f6b5d600640 1 -- 172.21.15.88:0/1865300366 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16795 (secure 0 0 0) 0x7f6b4c063a00 con 0x7f6b601591d0 2024-09-21T07:54:22.807 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:22.807 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":145,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:04.947773+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:01.356225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:02.401381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:22.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b40079400 msgr2=0x7f6b4007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b40079400 0x7f6b4007b910 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400c470 tx=0x7f6b54001c00 comp rx=0 tx=0).stop 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 msgr2=0x7f6b601f5670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 0x7f6b601f5670 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f6b4c00f780 tx=0x7f6b4c00fc40 comp rx=0 tx=0).stop 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 shutdown_connections 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6b40079400 0x7f6b4007b910 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b60149720 0x7f6b6009bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b601591d0 0x7f6b601f5670 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 --2- 172.21.15.88:0/1865300366 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b48003a00 0x7f6b6009dde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 >> 172.21.15.88:0/1865300366 conn(0x7f6b6006d730 msgr2=0x7f6b600cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 shutdown_connections 2024-09-21T07:54:22.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:22.808+0000 7f6b68200640 1 -- 172.21.15.88:0/1865300366 wait complete. 2024-09-21T07:54:22.831 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:54:22.832 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 -- 172.21.15.88:0/436242501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00b3720 msgr2=0x7f93c00d1090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 --2- 172.21.15.88:0/436242501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00b3720 0x7f93c00d1090 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f93a8005fd0 tx=0x7f93a802a540 comp rx=0 tx=0).stop 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 -- 172.21.15.88:0/436242501 shutdown_connections 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 --2- 172.21.15.88:0/436242501 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93c00d15d0 0x7f93c00cf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 --2- 172.21.15.88:0/436242501 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00b3720 0x7f93c00d1090 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 -- 172.21.15.88:0/436242501 >> 172.21.15.88:0/436242501 conn(0x7f93c00d4680 msgr2=0x7f93c00d4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 -- 172.21.15.88:0/436242501 shutdown_connections 2024-09-21T07:54:23.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.051+0000 7f93c6e00640 1 -- 172.21.15.88:0/436242501 wait complete. 2024-09-21T07:54:23.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.054+0000 7f93c6e00640 1 Processor -- start 2024-09-21T07:54:23.055 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.054+0000 7f93c6e00640 1 -- start start 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.055+0000 7f93c6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 0x7f93c00cdf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.055+0000 7f93c6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93c00b9780 0x7f93c015a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.055+0000 7f93c6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00d15d0 0x7f93c015a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.055+0000 7f93c6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cfd40 con 0x7f93c00b3720 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.055+0000 7f93c6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f93c00cfbc0 con 0x7f93c00d15d0 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.055+0000 7f93c6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f93c00cfec0 con 0x7f93c00b9780 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 0x7f93c00cdf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 0x7f93c00cdf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44004/0 (socket says 172.21.15.88:44004) 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 -- 172.21.15.88:0/25901794 learned_addr learned my addr 172.21.15.88:0/25901794 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93b7400640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93c00b9780 0x7f93c015a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93c4e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00d15d0 0x7f93c015a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 -- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93c00b9780 msgr2=0x7f93c015a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93c00b9780 0x7f93c015a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 -- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00d15d0 msgr2=0x7f93c015a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00d15d0 0x7f93c015a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 -- 172.21.15.88:0/25901794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93c0144700 con 0x7f93c00b3720 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93c4e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00d15d0 0x7f93c015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.056+0000 7f93bfe00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 0x7f93c00cdf70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c730 tx=0x7f93b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.057+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b00050d0 con 0x7f93c00b3720 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.057+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f93b0005290 con 0x7f93c00b3720 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.057+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93b0003640 con 0x7f93c00b3720 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.057+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93c01449e0 con 0x7f93c00b3720 2024-09-21T07:54:23.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.057+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93c0144fa0 con 0x7f93c00b3720 2024-09-21T07:54:23.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.058+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f93c008d100 con 0x7f93c00b3720 2024-09-21T07:54:23.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.059+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f93b00039f0 con 0x7f93c00b3720 2024-09-21T07:54:23.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.060+0000 7f93bcc00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f939c079430 0x7f939c07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.060+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f93b009bc00 con 0x7f93c00b3720 2024-09-21T07:54:23.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.060+0000 7f93b7400640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f939c079430 0x7f939c07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.061+0000 7f93b7400640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f939c079430 0x7f939c07b940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f93a8033000 tx=0x7f93a8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.065+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f93c008d100 con 0x7f93c00b3720 2024-09-21T07:54:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.287+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f93c00d1090 con 0x7f93c00b3720 2024-09-21T07:54:23.288 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.287+0000 7f93bcc00640 1 -- 172.21.15.88:0/25901794 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5234 (secure 0 0 0) 0x7f93b009de20 con 0x7f93c00b3720 2024-09-21T07:54:23.289 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:23.289 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":100,"btime":"2024-09-21T07:54:14:989822+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:03.440969+0000","modified":"2024-09-21T07:54:06.451584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3712798522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3712798522},{"type":"v1","addr":"172.21.15.88:6835","nonce":3712798522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":11}]} 2024-09-21T07:54:23.289 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 100 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f939c079430 msgr2=0x7f939c07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f939c079430 0x7f939c07b940 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f93a8033000 tx=0x7f93a8004730 comp rx=0 tx=0).stop 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 msgr2=0x7f93c00cdf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 0x7f93c00cdf70 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c730 tx=0x7f93b000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 shutdown_connections 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f939c079430 0x7f939c07b940 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f93c00b9780 0x7f93c015a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f93c00d15d0 0x7f93c015a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 --2- 172.21.15.88:0/25901794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93c00b3720 0x7f93c00cdf70 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 >> 172.21.15.88:0/25901794 conn(0x7f93c00d4680 msgr2=0x7f93c006edb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 shutdown_connections 2024-09-21T07:54:23.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.290+0000 7f93c6e00640 1 -- 172.21.15.88:0/25901794 wait complete. 2024-09-21T07:54:23.313 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:23.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.531+0000 7f03f5200640 1 -- 172.21.15.88:0/317852754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4004800 msgr2=0x7f03d4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.531+0000 7f03f5200640 1 --2- 172.21.15.88:0/317852754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4004800 0x7f03d4004c10 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f03d800c4a0 tx=0x7f03d4005830 comp rx=0 tx=0).stop 2024-09-21T07:54:23.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.532+0000 7f03f5200640 1 -- 172.21.15.88:0/317852754 shutdown_connections 2024-09-21T07:54:23.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.532+0000 7f03f5200640 1 --2- 172.21.15.88:0/317852754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03d4004800 0x7f03d4004c10 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.532+0000 7f03f5200640 1 -- 172.21.15.88:0/317852754 >> 172.21.15.88:0/317852754 conn(0x7f03f009d8a0 msgr2=0x7f03f008bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.532+0000 7f03f5200640 1 -- 172.21.15.88:0/317852754 shutdown_connections 2024-09-21T07:54:23.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.532+0000 7f03f5200640 1 -- 172.21.15.88:0/317852754 wait complete. 2024-09-21T07:54:23.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.534+0000 7f03f5200640 1 Processor -- start 2024-09-21T07:54:23.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.534+0000 7f03f5200640 1 -- start start 2024-09-21T07:54:23.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.535+0000 7f03f5200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d4004800 0x7f03f01f6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.535+0000 7f03f5200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f01501b0 0x7f03f014f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.535+0000 7f03f5200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 0x7f03f00cda10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.535+0000 7f03f5200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00b3ef0 con 0x7f03f00cd600 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.535+0000 7f03f5200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03f00b4740 con 0x7f03f01501b0 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.535+0000 7f03f5200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03f00b4070 con 0x7f03d4004800 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 0x7f03f00cda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 0x7f03f00cda10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44036/0 (socket says 172.21.15.88:44036) 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 -- 172.21.15.88:0/2283025738 learned_addr learned my addr 172.21.15.88:0/2283025738 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03ef400640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d4004800 0x7f03f01f6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.539 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03e6a00640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f01501b0 0x7f03f014f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 -- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d4004800 msgr2=0x7f03f01f6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d4004800 0x7f03f01f6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 -- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f01501b0 msgr2=0x7f03f014f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f01501b0 0x7f03f014f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 -- 172.21.15.88:0/2283025738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03f00cdf50 con 0x7f03f00cd600 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03e6a00640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f01501b0 0x7f03f014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.536+0000 7f03efe00640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 0x7f03f00cda10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f03f00b9890 tx=0x7f03e8001a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.537+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8011870 con 0x7f03f00cd600 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.537+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03e8017070 con 0x7f03f00cd600 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.537+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03e8011da0 con 0x7f03f00cd600 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.537+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03f01f9690 con 0x7f03f00cd600 2024-09-21T07:54:23.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.537+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03f01f9c50 con 0x7f03f00cd600 2024-09-21T07:54:23.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.538+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03f00b6d40 con 0x7f03f00cd600 2024-09-21T07:54:23.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.539+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f03e801c460 con 0x7f03f00cd600 2024-09-21T07:54:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.540+0000 7f03e7e00640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03cc079400 0x7f03cc07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.540+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f03e809c460 con 0x7f03f00cd600 2024-09-21T07:54:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.541+0000 7f03ef400640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03cc079400 0x7f03cc07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:23.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.541+0000 7f03ef400640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03cc079400 0x7f03cc07b910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f03f0052860 tx=0x7f03e0005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:23.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.545+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03f00cdf50 con 0x7f03f00cd600 2024-09-21T07:54:23.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.767+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f03f00cb010 con 0x7f03f00cd600 2024-09-21T07:54:23.768 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.767+0000 7f03e7e00640 1 -- 172.21.15.88:0/2283025738 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 100 v100) ==== 95+0+5234 (secure 0 0 0) 0x7f03e8063cf0 con 0x7f03f00cd600 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":100,"btime":"2024-09-21T07:54:14:989822+0000","default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93}],"filesystems":[{"mdsmap":{"epoch":99,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:03.440969+0000","modified":"2024-09-21T07:54:06.451584+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8394":{"gid":8394,"name":"a","rank":0,"incarnation":95,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/3712798522","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":3712798522},{"type":"v1","addr":"172.21.15.88:6835","nonce":3712798522}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8394,"qdb_cluster":[8394]},"id":11}]} 2024-09-21T07:54:23.769 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 100 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03cc079400 msgr2=0x7f03cc07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03cc079400 0x7f03cc07b910 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f03f0052860 tx=0x7f03e0005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 msgr2=0x7f03f00cda10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 0x7f03f00cda10 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f03f00b9890 tx=0x7f03e8001a70 comp rx=0 tx=0).stop 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 shutdown_connections 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03cc079400 0x7f03cc07b910 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03f00cd600 0x7f03f00cda10 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03f01501b0 0x7f03f014f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 --2- 172.21.15.88:0/2283025738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03d4004800 0x7f03f01f6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:23.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 >> 172.21.15.88:0/2283025738 conn(0x7f03f009d8a0 msgr2=0x7f03f00d2270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:23.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 shutdown_connections 2024-09-21T07:54:23.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:23.770+0000 7f03f5200640 1 -- 172.21.15.88:0/2283025738 wait complete. 2024-09-21T07:54:23.793 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:24.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.017+0000 7f4a7ae00640 1 -- 172.21.15.88:0/2907633059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 msgr2=0x7f4a5c00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.017+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/2907633059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a5c00b9c0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f4a6c009ed0 tx=0x7f4a5c005830 comp rx=0 tx=0).stop 2024-09-21T07:54:24.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.018+0000 7f4a7ae00640 1 -- 172.21.15.88:0/2907633059 shutdown_connections 2024-09-21T07:54:24.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.018+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/2907633059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a5c00b9c0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.018+0000 7f4a7ae00640 1 -- 172.21.15.88:0/2907633059 >> 172.21.15.88:0/2907633059 conn(0x7f4a7409aa50 msgr2=0x7f4a740b6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.018+0000 7f4a7ae00640 1 -- 172.21.15.88:0/2907633059 shutdown_connections 2024-09-21T07:54:24.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.018+0000 7f4a7ae00640 1 -- 172.21.15.88:0/2907633059 wait complete. 2024-09-21T07:54:24.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.020+0000 7f4a7ae00640 1 Processor -- start 2024-09-21T07:54:24.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.020+0000 7f4a7ae00640 1 -- start start 2024-09-21T07:54:24.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a7415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a7415b730 0x7f4a7415f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7415fa80 0x7f4a741f8ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7406cda0 con 0x7f4a5c005350 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4a7415ca80 con 0x7f4a7415b730 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4a7406cf20 con 0x7f4a7415fa80 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a7415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a7415b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44044/0 (socket says 172.21.15.88:44044) 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 -- 172.21.15.88:0/78480285 learned_addr learned my addr 172.21.15.88:0/78480285 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79000640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a7415b730 0x7f4a7415f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a7a400640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7415fa80 0x7f4a741f8ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 -- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7415fa80 msgr2=0x7f4a741f8ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7415fa80 0x7f4a741f8ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 -- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a7415b730 msgr2=0x7f4a7415f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a7415b730 0x7f4a7415f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79a00640 1 -- 172.21.15.88:0/78480285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a7415bb40 con 0x7f4a5c005350 2024-09-21T07:54:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.022+0000 7f4a79000640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a7415b730 0x7f4a7415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.023+0000 7f4a79a00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a7415b1f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004ed0 tx=0x7f4a68001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.023+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68015020 con 0x7f4a5c005350 2024-09-21T07:54:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.023+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4a68011960 con 0x7f4a5c005350 2024-09-21T07:54:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.023+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a740b1060 con 0x7f4a5c005350 2024-09-21T07:54:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.023+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a68011d90 con 0x7f4a5c005350 2024-09-21T07:54:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.023+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a7415d360 con 0x7f4a5c005350 2024-09-21T07:54:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.024+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a740b1580 con 0x7f4a5c005350 2024-09-21T07:54:24.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.025+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f4a68018020 con 0x7f4a5c005350 2024-09-21T07:54:24.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.026+0000 7f4a71600640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4a54079400 0x7f4a5407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.026+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f4a6809cb20 con 0x7f4a5c005350 2024-09-21T07:54:24.027 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.026+0000 7f4a79000640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4a54079400 0x7f4a5407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.027+0000 7f4a79000640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4a54079400 0x7f4a5407b910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4a60001860 tx=0x7f4a60001110 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.031+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4a740b1580 con 0x7f4a5c005350 2024-09-21T07:54:24.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.218+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4a74077140 con 0x7f4a5c005350 2024-09-21T07:54:24.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.219+0000 7f4a71600640 1 -- 172.21.15.88:0/78480285 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) ==== 74+0+16795 (secure 0 0 0) 0x7f4a74077140 con 0x7f4a5c005350 2024-09-21T07:54:24.221 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:24.221 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":145,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:04.947773+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:01.356225+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"144","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:02.401381+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4a54079400 msgr2=0x7f4a5407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4a54079400 0x7f4a5407b910 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f4a60001860 tx=0x7f4a60001110 comp rx=0 tx=0).stop 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 msgr2=0x7f4a7415b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a7415b1f0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f4a68004ed0 tx=0x7f4a68001630 comp rx=0 tx=0).stop 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 shutdown_connections 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4a54079400 0x7f4a5407b910 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4a7415fa80 0x7f4a741f8ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4a7415b730 0x7f4a7415f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 --2- 172.21.15.88:0/78480285 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a5c005350 0x7f4a7415b1f0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 >> 172.21.15.88:0/78480285 conn(0x7f4a7409aa50 msgr2=0x7f4a740b4610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 shutdown_connections 2024-09-21T07:54:24.224 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.222+0000 7f4a7ae00640 1 -- 172.21.15.88:0/78480285 wait complete. 2024-09-21T07:54:24.248 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 -- 172.21.15.88:0/1112691095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80af6a0 msgr2=0x7f40b80ad1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 --2- 172.21.15.88:0/1112691095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80af6a0 0x7f40b80ad1c0 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f40a4005fd0 tx=0x7f40a402a440 comp rx=0 tx=0).stop 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 -- 172.21.15.88:0/1112691095 shutdown_connections 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 --2- 172.21.15.88:0/1112691095 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b80ad700 0x7f40b80adb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 --2- 172.21.15.88:0/1112691095 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80af6a0 0x7f40b80ad1c0 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 -- 172.21.15.88:0/1112691095 >> 172.21.15.88:0/1112691095 conn(0x7f40b806d3d0 msgr2=0x7f40b80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 -- 172.21.15.88:0/1112691095 shutdown_connections 2024-09-21T07:54:24.469 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.467+0000 7f40bfe00640 1 -- 172.21.15.88:0/1112691095 wait complete. 2024-09-21T07:54:24.470 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.470+0000 7f40bfe00640 1 Processor -- start 2024-09-21T07:54:24.471 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.470+0000 7f40bfe00640 1 -- start start 2024-09-21T07:54:24.473 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b8070660 0x7f40b8073c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 0x7f40b80c9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bfe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b80af6a0 0x7f40b80c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bfe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80cb950 con 0x7f40b80ad700 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bfe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f40b80afab0 con 0x7f40b8070660 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bfe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f40b80cbad0 con 0x7f40b80af6a0 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 0x7f40b80c9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 0x7f40b80c9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44052/0 (socket says 172.21.15.88:44052) 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.471+0000 7f40bca00640 1 -- 172.21.15.88:0/3308351954 learned_addr learned my addr 172.21.15.88:0/3308351954 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bd400640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b8070660 0x7f40b8073c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bde00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b80af6a0 0x7f40b80c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bca00640 1 -- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b80af6a0 msgr2=0x7f40b80c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.474 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bca00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b80af6a0 0x7f40b80c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bca00640 1 -- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b8070660 msgr2=0x7f40b8073c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bca00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b8070660 0x7f40b8073c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bca00640 1 -- 172.21.15.88:0/3308351954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b80c7740 con 0x7f40b80ad700 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bd400640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b8070660 0x7f40b8073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40bca00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 0x7f40b80c9920 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f40a40099f0 tx=0x7f40a40039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.472+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a403e030 con 0x7f40b80ad700 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.473+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f40a40047a0 con 0x7f40b80ad700 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.473+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a4035040 con 0x7f40b80ad700 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.473+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b80cd560 con 0x7f40b80ad700 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.473+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b8164a30 con 0x7f40b80ad700 2024-09-21T07:54:24.475 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.474+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b80c74a0 con 0x7f40b80ad700 2024-09-21T07:54:24.476 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.475+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f40a4041030 con 0x7f40b80ad700 2024-09-21T07:54:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.476+0000 7f40b5600640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4098079460 0x7f409807b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.476+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(145..145 src has 1..145) ==== 5737+0+0 (secure 0 0 0) 0x7f40a40bc6a0 con 0x7f40b80ad700 2024-09-21T07:54:24.477 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.476+0000 7f40bd400640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4098079460 0x7f409807b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.478 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.477+0000 7f40bd400640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4098079460 0x7f409807b970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006fd0 tx=0x7f40ac008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.480+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f40b80c74a0 con 0x7f40b80ad700 2024-09-21T07:54:24.705 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.704+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f40b80adb10 con 0x7f40b80ad700 2024-09-21T07:54:24.706 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:54:24.705+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:54:24.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.707+0000 7f40b5600640 1 -- 172.21.15.88:0/3308351954 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v101) ==== 161+0+0 (secure 0 0 0) 0x7f40a40835a0 con 0x7f40b80ad700 2024-09-21T07:54:24.708 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:54:24.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.709+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4098079460 msgr2=0x7f409807b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.709+0000 7f40bfe00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4098079460 0x7f409807b970 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f40ac006fd0 tx=0x7f40ac008040 comp rx=0 tx=0).stop 2024-09-21T07:54:24.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.709+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 msgr2=0x7f40b80c9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.709+0000 7f40bfe00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 0x7f40b80c9920 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f40a40099f0 tx=0x7f40a40039e0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 shutdown_connections 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4098079460 0x7f409807b970 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f40b80af6a0 0x7f40b80c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b80ad700 0x7f40b80c9920 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 --2- 172.21.15.88:0/3308351954 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f40b8070660 0x7f40b8073c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 >> 172.21.15.88:0/3308351954 conn(0x7f40b806d3d0 msgr2=0x7f40b8159520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 shutdown_connections 2024-09-21T07:54:24.712 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.710+0000 7f40bfe00640 1 -- 172.21.15.88:0/3308351954 wait complete. 2024-09-21T07:54:24.736 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:54:24.852 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 -- 172.21.15.88:0/4232322345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145a40 msgr2=0x7f34b4073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 --2- 172.21.15.88:0/4232322345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145a40 0x7f34b4073540 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7f34b4072bd0 tx=0x7f34a002a680 comp rx=0 tx=0).stop 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 -- 172.21.15.88:0/4232322345 shutdown_connections 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 --2- 172.21.15.88:0/4232322345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145a40 0x7f34b4073540 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 --2- 172.21.15.88:0/4232322345 >> v2:172.21.15.175:3300/0 conn(0x7f34b4145550 0x7f34b406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 -- 172.21.15.88:0/4232322345 >> 172.21.15.88:0/4232322345 conn(0x7f34b406d1c0 msgr2=0x7f34b406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:24.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 -- 172.21.15.88:0/4232322345 shutdown_connections 2024-09-21T07:54:24.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.962+0000 7f34bc200640 1 -- 172.21.15.88:0/4232322345 wait complete. 2024-09-21T07:54:24.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.965+0000 7f34bc200640 1 Processor -- start 2024-09-21T07:54:24.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.965+0000 7f34bc200640 1 -- start start 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.966+0000 7f34bc200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b4073a80 0x7f34b415bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.966+0000 7f34bc200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 0x7f34b41365c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.966+0000 7f34bc200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b4145a40 0x7f34b4136b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.966+0000 7f34bc200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34b4160a30 con 0x7f34b4145550 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.966+0000 7f34bc200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f34b40ce870 con 0x7f34b4145a40 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.966+0000 7f34bc200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f34b4160bb0 con 0x7f34b4073a80 2024-09-21T07:54:24.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 0x7f34b41365c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 0x7f34b41365c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44062/0 (socket says 172.21.15.88:44062) 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 -- 172.21.15.88:0/224502856 learned_addr learned my addr 172.21.15.88:0/224502856 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b9800640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b4073a80 0x7f34b415bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34ba200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b4145a40 0x7f34b4136b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 -- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b4073a80 msgr2=0x7f34b415bfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b4073a80 0x7f34b415bfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 -- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b4145a40 msgr2=0x7f34b4136b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b4145a40 0x7f34b4136b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:24.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.967+0000 7f34b8e00640 1 -- 172.21.15.88:0/224502856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f34b4138ca0 con 0x7f34b4145550 2024-09-21T07:54:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.968+0000 7f34b8e00640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 0x7f34b41365c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034040 tx=0x7f34a0003910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.968+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a003e030 con 0x7f34b4145550 2024-09-21T07:54:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.968+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f34a0003e00 con 0x7f34b4145550 2024-09-21T07:54:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.968+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34a0035040 con 0x7f34b4145550 2024-09-21T07:54:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.968+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f34b4160310 con 0x7f34b4145550 2024-09-21T07:54:24.970 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.968+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34b4138990 con 0x7f34b4145550 2024-09-21T07:54:24.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.969+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f34b41610d0 con 0x7f34b4145550 2024-09-21T07:54:24.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.970+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f34a0041030 con 0x7f34b4145550 2024-09-21T07:54:24.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.971+0000 7f34b1600640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3494079400 0x7f349407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:24.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.971+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5823+0+0 (secure 0 0 0) 0x7f34a00bc710 con 0x7f34b4145550 2024-09-21T07:54:24.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.972+0000 7f34b9800640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3494079400 0x7f349407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:24.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.972+0000 7f34b9800640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3494079400 0x7f349407b910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f34a8004630 tx=0x7f34a800b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:24.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:24.976+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f34a00835a0 con 0x7f34b4145550 2024-09-21T07:54:25.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.204+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f34b4143e80 con 0x7f34b4145550 2024-09-21T07:54:25.710 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.709+0000 7f34b1600640 1 -- 172.21.15.88:0/224502856 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v102) ==== 104+0+0 (secure 0 0 0) 0x7f34a0087fd0 con 0x7f34b4145550 2024-09-21T07:54:25.713 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3494079400 msgr2=0x7f349407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3494079400 0x7f349407b910 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f34a8004630 tx=0x7f34a800b2e0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 msgr2=0x7f34b41365c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 0x7f34b41365c0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7f34a0034040 tx=0x7f34a0003910 comp rx=0 tx=0).stop 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 shutdown_connections 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3494079400 0x7f349407b910 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f34b4145a40 0x7f34b4136b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f34b4145550 0x7f34b41365c0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 --2- 172.21.15.88:0/224502856 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f34b4073a80 0x7f34b415bfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 >> 172.21.15.88:0/224502856 conn(0x7f34b406d1c0 msgr2=0x7f34b4141aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.712+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 shutdown_connections 2024-09-21T07:54:25.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.713+0000 7f34bc200640 1 -- 172.21.15.88:0/224502856 wait complete. 2024-09-21T07:54:25.741 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:54:25.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.964+0000 7f669ea00640 1 -- 172.21.15.88:0/417117965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 msgr2=0x7f669806d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.964+0000 7f669ea00640 1 --2- 172.21.15.88:0/417117965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f669806d1b0 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f6684005fd0 tx=0x7f668402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.964+0000 7f669ea00640 1 -- 172.21.15.88:0/417117965 shutdown_connections 2024-09-21T07:54:25.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.964+0000 7f669ea00640 1 --2- 172.21.15.88:0/417117965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f669806d1b0 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.964+0000 7f669ea00640 1 --2- 172.21.15.88:0/417117965 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66980cc830 0x7f66980b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.967 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.964+0000 7f669ea00640 1 -- 172.21.15.88:0/417117965 >> 172.21.15.88:0/417117965 conn(0x7f66980b2540 msgr2=0x7f66980d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:25.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.965+0000 7f669ea00640 1 -- 172.21.15.88:0/417117965 shutdown_connections 2024-09-21T07:54:25.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.965+0000 7f669ea00640 1 -- 172.21.15.88:0/417117965 wait complete. 2024-09-21T07:54:25.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.967+0000 7f669ea00640 1 Processor -- start 2024-09-21T07:54:25.969 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.967+0000 7f669ea00640 1 -- start start 2024-09-21T07:54:25.971 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.968+0000 7f669ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f66981f57a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.968+0000 7f669ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66980af100 0x7f66981e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.968+0000 7f669ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66980cc830 0x7f66981e9a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.968+0000 7f669ea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66980b3600 con 0x7f669806cda0 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.968+0000 7f669ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f66980b3480 con 0x7f66980af100 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.968+0000 7f669ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f66980b3780 con 0x7f66980cc830 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f66981f57a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f66981f57a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40756/0 (socket says 172.21.15.88:40756) 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 -- 172.21.15.88:0/1248882583 learned_addr learned my addr 172.21.15.88:0/1248882583 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697400640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66980af100 0x7f66981e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f669ca00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66980cc830 0x7f66981e9a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 -- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66980cc830 msgr2=0x7f66981e9a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66980cc830 0x7f66981e9a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 -- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66980af100 msgr2=0x7f66981e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66980af100 0x7f66981e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 -- 172.21.15.88:0/1248882583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66981fa400 con 0x7f669806cda0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f6697e00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f66981f57a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f668800ea20 tx=0x7f668800eee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.969+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f668800cde0 con 0x7f669806cda0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.970+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6688011560 con 0x7f669806cda0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.970+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6688011990 con 0x7f669806cda0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.970+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66981fa120 con 0x7f669806cda0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.970+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6698150210 con 0x7f669806cda0 2024-09-21T07:54:25.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.971+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66981f9ea0 con 0x7f669806cda0 2024-09-21T07:54:25.974 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.972+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f6688011b30 con 0x7f669806cda0 2024-09-21T07:54:25.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.974+0000 7f668fe00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6678079400 0x7f667807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:25.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.974+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(146..146 src has 1..146) ==== 5823+0+0 (secure 0 0 0) 0x7f668809cc90 con 0x7f669806cda0 2024-09-21T07:54:25.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.974+0000 7f6697400640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6678079400 0x7f667807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:25.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.974+0000 7f6697400640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6678079400 0x7f667807b910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6684033000 tx=0x7f6684004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:25.980 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:25.978+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f66981f9ea0 con 0x7f669806cda0 2024-09-21T07:54:26.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:26.167+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f66980b3ca0 con 0x7f669806cda0 2024-09-21T07:54:27.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.060+0000 7f668fe00640 1 -- 172.21.15.88:0/1248882583 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v147) ==== 181+0+0 (secure 0 0 0) 0x7f6688016030 con 0x7f669806cda0 2024-09-21T07:54:27.063 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:54:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6678079400 msgr2=0x7f667807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6678079400 0x7f667807b910 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f6684033000 tx=0x7f6684004730 comp rx=0 tx=0).stop 2024-09-21T07:54:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 msgr2=0x7f66981f57a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f66981f57a0 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f668800ea20 tx=0x7f668800eee0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 shutdown_connections 2024-09-21T07:54:27.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6678079400 0x7f667807b910 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f66980cc830 0x7f66981e9a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f66980af100 0x7f66981e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 --2- 172.21.15.88:0/1248882583 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f669806cda0 0x7f66981f57a0 secure :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f668800ea20 tx=0x7f668800eee0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.064+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 >> 172.21.15.88:0/1248882583 conn(0x7f66980b2540 msgr2=0x7f6698073860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:27.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.065+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 shutdown_connections 2024-09-21T07:54:27.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.066+0000 7f669ea00640 1 -- 172.21.15.88:0/1248882583 wait complete. 2024-09-21T07:54:27.098 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:54:27.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdaf400640 1 -- 172.21.15.88:0/3513648090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd90005150 msgr2=0x7fbd9000b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdaf400640 1 --2- 172.21.15.88:0/3513648090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd90005150 0x7fbd9000b9c0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7fbda0006fd0 tx=0x7fbd90005830 comp rx=0 tx=0).stop 2024-09-21T07:54:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdad400640 1 -- 172.21.15.88:0/3513648090 reap_dead start 2024-09-21T07:54:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdaf400640 1 -- 172.21.15.88:0/3513648090 shutdown_connections 2024-09-21T07:54:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdaf400640 1 -- 172.21.15.88:0/3513648090 >> 172.21.15.88:0/3513648090 conn(0x7fbda80b2540 msgr2=0x7fbda80d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdaf400640 1 -- 172.21.15.88:0/3513648090 shutdown_connections 2024-09-21T07:54:27.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.320+0000 7fbdaf400640 1 -- 172.21.15.88:0/3513648090 wait complete. 2024-09-21T07:54:27.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.323+0000 7fbdaf400640 1 Processor -- start 2024-09-21T07:54:27.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.323+0000 7fbdaf400640 1 -- start start 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbdaf400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90005150 0x7fbda81f2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbdaf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 0x7fbda809ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbdaf400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda809bd80 0x7fbda809c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbdaf400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbda80b3120 con 0x7fbda8153f90 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbdaf400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbda806d500 con 0x7fbd90005150 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbdaf400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbda80b32a0 con 0x7fbda809bd80 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbda7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 0x7fbda809ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbda7e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 0x7fbda809ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40776/0 (socket says 172.21.15.88:40776) 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.324+0000 7fbda7e00640 1 -- 172.21.15.88:0/2366509413 learned_addr learned my addr 172.21.15.88:0/2366509413 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbdad400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda809bd80 0x7fbda809c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbdaca00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90005150 0x7fbda81f2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda7e00640 1 -- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda809bd80 msgr2=0x7fbda809c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda7e00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda809bd80 0x7fbda809c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda7e00640 1 -- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90005150 msgr2=0x7fbda81f2b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda7e00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90005150 0x7fbda81f2b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda7e00640 1 -- 172.21.15.88:0/2366509413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbda81f61b0 con 0x7fbda8153f90 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbdad400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda809bd80 0x7fbda809c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda7e00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 0x7fbda809ad20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005fd0 tx=0x7fbd9400fd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd9400ba50 con 0x7fbda8153f90 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbd9400bce0 con 0x7fbda8153f90 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbda81f6490 con 0x7fbda8153f90 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.325+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd94011680 con 0x7fbda8153f90 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.326+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbda809a800 con 0x7fbda8153f90 2024-09-21T07:54:27.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.327+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbda806a700 con 0x7fbda8153f90 2024-09-21T07:54:27.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.328+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fbd94019020 con 0x7fbda8153f90 2024-09-21T07:54:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.329+0000 7fbda4c00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd88079400 0x7fbd8807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.329+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(147..147 src has 1..147) ==== 5385+0+0 (secure 0 0 0) 0x7fbd9409c3c0 con 0x7fbda8153f90 2024-09-21T07:54:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.329+0000 7fbdaca00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd88079400 0x7fbd8807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:27.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.330+0000 7fbdaca00640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd88079400 0x7fbd8807b910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009c50 tx=0x7fbd9c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:27.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.333+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbda806a700 con 0x7fbda8153f90 2024-09-21T07:54:27.509 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:27.514 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:27.531 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:27.529+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fbda80d1030 con 0x7fbda8153f90 2024-09-21T07:54:28.069 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.066+0000 7fbda4c00640 1 -- 172.21.15.88:0/2366509413 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v148) ==== 169+0+0 (secure 0 0 0) 0x7fbd94063d20 con 0x7fbda8153f90 2024-09-21T07:54:28.069 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:54:28.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.070+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd88079400 msgr2=0x7fbd8807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.070+0000 7fbdaf400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd88079400 0x7fbd8807b910 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fbd9c009c50 tx=0x7fbd9c005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:28.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.070+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 msgr2=0x7fbda809ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.070+0000 7fbdaf400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 0x7fbda809ad20 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7fbd94005fd0 tx=0x7fbd9400fd50 comp rx=0 tx=0).stop 2024-09-21T07:54:28.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 shutdown_connections 2024-09-21T07:54:28.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbd88079400 0x7fbd8807b910 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbda809bd80 0x7fbda809c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbda8153f90 0x7fbda809ad20 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 --2- 172.21.15.88:0/2366509413 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbd90005150 0x7fbda81f2b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 >> 172.21.15.88:0/2366509413 conn(0x7fbda80b2540 msgr2=0x7fbda80ae1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:28.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 shutdown_connections 2024-09-21T07:54:28.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.071+0000 7fbdaf400640 1 -- 172.21.15.88:0/2366509413 wait complete. 2024-09-21T07:54:28.100 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:54:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 -- 172.21.15.88:0/2697757519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc0cfc80 msgr2=0x7faafc156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 --2- 172.21.15.88:0/2697757519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc0cfc80 0x7faafc156e40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7faafc0b4fa0 tx=0x7faaf002a480 comp rx=0 tx=0).stop 2024-09-21T07:54:28.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 -- 172.21.15.88:0/2697757519 shutdown_connections 2024-09-21T07:54:28.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 --2- 172.21.15.88:0/2697757519 >> v2:172.21.15.175:3300/0 conn(0x7faafc15ce30 0x7faafc15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 --2- 172.21.15.88:0/2697757519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc0cfc80 0x7faafc156e40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 -- 172.21.15.88:0/2697757519 >> 172.21.15.88:0/2697757519 conn(0x7faafc06d3d0 msgr2=0x7faafc0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:28.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 -- 172.21.15.88:0/2697757519 shutdown_connections 2024-09-21T07:54:28.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.320+0000 7fab04800640 1 -- 172.21.15.88:0/2697757519 wait complete. 2024-09-21T07:54:28.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.322+0000 7fab04800640 1 Processor -- start 2024-09-21T07:54:28.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.322+0000 7fab04800640 1 -- start start 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab04800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 0x7faafc1f5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab04800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faafc0cfc80 0x7faafc074b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab04800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc15ce30 0x7faafc0ca0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab04800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faafc0cd050 con 0x7faafc0705b0 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab04800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faafc0cced0 con 0x7faafc0cfc80 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab04800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faafc0cd1d0 con 0x7faafc15ce30 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab01e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 0x7faafc1f5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab01e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 0x7faafc1f5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40788/0 (socket says 172.21.15.88:40788) 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab01e00640 1 -- 172.21.15.88:0/3342695467 learned_addr learned my addr 172.21.15.88:0/3342695467 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab02800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc15ce30 0x7faafc0ca0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.324+0000 7fab01400640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faafc0cfc80 0x7faafc074b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab01e00640 1 -- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc15ce30 msgr2=0x7faafc0ca0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab01e00640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc15ce30 0x7faafc0ca0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab01e00640 1 -- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faafc0cfc80 msgr2=0x7faafc074b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab01e00640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faafc0cfc80 0x7faafc074b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab01e00640 1 -- 172.21.15.88:0/3342695467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faafc0cc010 con 0x7faafc0705b0 2024-09-21T07:54:28.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab01e00640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 0x7faafc1f5880 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009920 tx=0x7faaf0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0031d60 con 0x7faafc0705b0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faaf0033e10 con 0x7faafc0705b0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf0005260 con 0x7faafc0705b0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faafc0cc2f0 con 0x7faafc0705b0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.325+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faafc1f5dc0 con 0x7faafc0705b0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.326+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faafc0cd6f0 con 0x7faafc0705b0 2024-09-21T07:54:28.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.328+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7faaf0005400 con 0x7faafc0705b0 2024-09-21T07:54:28.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.329+0000 7faaf6000640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faae0079400 0x7faae007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.329+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7faaf00bcd60 con 0x7faafc0705b0 2024-09-21T07:54:28.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.329+0000 7fab01400640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faae0079400 0x7faae007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.330+0000 7fab01400640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faae0079400 0x7faae007b910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faafc15c4c0 tx=0x7faaec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:28.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.334+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faaf003f030 con 0x7faafc0705b0 2024-09-21T07:54:28.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.522+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7faafc0afa70 con 0x7faafc0705b0 2024-09-21T07:54:28.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.527+0000 7faaf6000640 1 -- 172.21.15.88:0/3342695467 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v49) ==== 71+0+0 (secure 0 0 0) 0x7faaf0083e90 con 0x7faafc0705b0 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.529+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faae0079400 msgr2=0x7faae007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.529+0000 7fab04800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faae0079400 0x7faae007b910 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7faafc15c4c0 tx=0x7faaec009040 comp rx=0 tx=0).stop 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.529+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 msgr2=0x7faafc1f5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.529+0000 7fab04800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 0x7faafc1f5880 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7faaf0009920 tx=0x7faaf0033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 shutdown_connections 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faae0079400 0x7faae007b910 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faafc15ce30 0x7faafc0ca0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faafc0cfc80 0x7faafc074b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 --2- 172.21.15.88:0/3342695467 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faafc0705b0 0x7faafc1f5880 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 >> 172.21.15.88:0/3342695467 conn(0x7faafc06d3d0 msgr2=0x7faafc0c6ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:28.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 shutdown_connections 2024-09-21T07:54:28.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.530+0000 7fab04800640 1 -- 172.21.15.88:0/3342695467 wait complete. 2024-09-21T07:54:28.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1' 2024-09-21T07:54:28.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 -- 172.21.15.88:0/2948288242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f557015d670 msgr2=0x7f5570070530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 --2- 172.21.15.88:0/2948288242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f557015d670 0x7f5570070530 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f5570051590 tx=0x7f556802a980 comp rx=0 tx=0).stop 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 -- 172.21.15.88:0/2948288242 shutdown_connections 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 --2- 172.21.15.88:0/2948288242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f557015d670 0x7f5570070530 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 --2- 172.21.15.88:0/2948288242 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f55700b2f60 0x7f557015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 --2- 172.21.15.88:0/2948288242 >> v2:172.21.15.175:3300/0 conn(0x7f55700ad9c0 0x7f55700af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 -- 172.21.15.88:0/2948288242 >> 172.21.15.88:0/2948288242 conn(0x7f55700b1e40 msgr2=0x7f55700b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 -- 172.21.15.88:0/2948288242 shutdown_connections 2024-09-21T07:54:28.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.775+0000 7f5575000640 1 -- 172.21.15.88:0/2948288242 wait complete. 2024-09-21T07:54:28.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.777+0000 7f5575000640 1 Processor -- start 2024-09-21T07:54:28.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.778+0000 7f5575000640 1 -- start start 2024-09-21T07:54:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5575000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55700ad9c0 0x7f557014f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5575000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 0x7f55700cb900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5575000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015d670 0x7f55700cec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5575000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f557006f290 con 0x7f55700b2f60 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5575000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5570070c80 con 0x7f55700ad9c0 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5575000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f557006f410 con 0x7f557015d670 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5566a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 0x7f55700cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5566a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 0x7f55700cb900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40802/0 (socket says 172.21.15.88:40802) 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.779+0000 7f5566a00640 1 -- 172.21.15.88:0/1704269781 learned_addr learned my addr 172.21.15.88:0/1704269781 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f556f400640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55700ad9c0 0x7f557014f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5566a00640 1 -- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015d670 msgr2=0x7f55700cec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f556fe00640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015d670 0x7f55700cec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5566a00640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015d670 0x7f55700cec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5566a00640 1 -- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55700ad9c0 msgr2=0x7f557014f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5566a00640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55700ad9c0 0x7f557014f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5566a00640 1 -- 172.21.15.88:0/1704269781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557014c3e0 con 0x7f55700b2f60 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f556f400640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55700ad9c0 0x7f557014f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:28.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f556fe00640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015d670 0x7f55700cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5566a00640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 0x7f55700cb900 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f555800d700 tx=0x7f555800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558017070 con 0x7f55700b2f60 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5558009b10 con 0x7f55700b2f60 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55701f9580 con 0x7f55700b2f60 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558010460 con 0x7f55700b2f60 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.780+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5570158070 con 0x7f55700b2f60 2024-09-21T07:54:28.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.781+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f55701f92f0 con 0x7f55700b2f60 2024-09-21T07:54:28.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.783+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f555801a070 con 0x7f55700b2f60 2024-09-21T07:54:28.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.784+0000 7f5567e00640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f554c079430 0x7f554c07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:28.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.784+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7f555809bd60 con 0x7f55700b2f60 2024-09-21T07:54:28.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.784+0000 7f556f400640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f554c079430 0x7f554c07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:28.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.785+0000 7f556f400640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f554c079430 0x7f554c07b940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ab0 tx=0x7f5560005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:28.790 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:28.788+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f55580634e0 con 0x7f55700b2f60 2024-09-21T07:54:29.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.046+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]} v 0) -- 0x7f557006f930 con 0x7f55700b2f60 2024-09-21T07:54:29.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.068+0000 7f5567e00640 1 -- 172.21.15.88:0/1704269781 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_legacy_to_v1"]}]=0 v473) ==== 149+0+0 (secure 0 0 0) 0x7f5558062de0 con 0x7f55700b2f60 2024-09-21T07:54:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f554c079430 msgr2=0x7f554c07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f554c079430 0x7f554c07b940 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f5560009ab0 tx=0x7f5560005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:29.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 msgr2=0x7f55700cb900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 0x7f55700cb900 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f555800d700 tx=0x7f555800dbc0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 shutdown_connections 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f554c079430 0x7f554c07b940 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f557015d670 0x7f55700cec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f55700b2f60 0x7f55700cb900 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 --2- 172.21.15.88:0/1704269781 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f55700ad9c0 0x7f557014f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 >> 172.21.15.88:0/1704269781 conn(0x7f55700b1e40 msgr2=0x7f557015c0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 shutdown_connections 2024-09-21T07:54:29.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.071+0000 7f5575000640 1 -- 172.21.15.88:0/1704269781 wait complete. 2024-09-21T07:54:29.095 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- rather than going through a full upgrade cycle, ... ok 2024-09-21T07:54:29.095 INFO:tasks.cephfs_test_runner:Starting test: test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) poor man's upgrade test -- theme continues... 2024-09-21T07:54:29.096 INFO:tasks.cephfs_test_runner:test_subvolume_upgrade_v1_to_v2 (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:54:29.096 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2813a00640 1 -- 172.21.15.88:0/4282623559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 msgr2=0x7f27f4009880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2813a00640 1 --2- 172.21.15.88:0/4282623559 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f27f4009880 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f27f800bb20 tx=0x7f27f4003810 comp rx=0 tx=0).stop 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2811a00640 1 -- 172.21.15.88:0/4282623559 reap_dead start 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2813a00640 1 -- 172.21.15.88:0/4282623559 shutdown_connections 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2813a00640 1 -- 172.21.15.88:0/4282623559 >> 172.21.15.88:0/4282623559 conn(0x7f280c06d730 msgr2=0x7f280c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2813a00640 1 -- 172.21.15.88:0/4282623559 shutdown_connections 2024-09-21T07:54:29.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.319+0000 7f2813a00640 1 -- 172.21.15.88:0/4282623559 wait complete. 2024-09-21T07:54:29.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.322+0000 7f2813a00640 1 Processor -- start 2024-09-21T07:54:29.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.322+0000 7f2813a00640 1 -- start start 2024-09-21T07:54:29.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2813a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f280c152a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2813a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c1511a0 0x7f280c158150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2813a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f280c150980 0x7f280c158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2813a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f280c0712b0 con 0x7f27f40073a0 2024-09-21T07:54:29.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2813a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f280c071130 con 0x7f280c150980 2024-09-21T07:54:29.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2813a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f280c0ad7d0 con 0x7f280c1511a0 2024-09-21T07:54:29.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2811000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f280c152a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2811000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f280c152a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40822/0 (socket says 172.21.15.88:40822) 2024-09-21T07:54:29.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.323+0000 7f2811000640 1 -- 172.21.15.88:0/2396191479 learned_addr learned my addr 172.21.15.88:0/2396191479 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:29.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811000640 1 -- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c1511a0 msgr2=0x7f280c158150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f280c150980 0x7f280c158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f280be00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c1511a0 0x7f280c158150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811000640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c1511a0 0x7f280c158150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811000640 1 -- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f280c150980 msgr2=0x7f280c158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811000640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f280c150980 0x7f280c158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811000640 1 -- 172.21.15.88:0/2396191479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f280c155a70 con 0x7f27f40073a0 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f280c150980 0x7f280c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f280be00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c1511a0 0x7f280c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2811000640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f280c152a60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2800004650 tx=0x7f280000bac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2800010b10 con 0x7f27f40073a0 2024-09-21T07:54:29.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2800015070 con 0x7f27f40073a0 2024-09-21T07:54:29.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f280c155d50 con 0x7f27f40073a0 2024-09-21T07:54:29.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f280001a470 con 0x7f27f40073a0 2024-09-21T07:54:29.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.324+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f280c157950 con 0x7f27f40073a0 2024-09-21T07:54:29.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.325+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f280c152860 con 0x7f27f40073a0 2024-09-21T07:54:29.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.327+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f280001a610 con 0x7f27f40073a0 2024-09-21T07:54:29.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.328+0000 7f2808c00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079400 0x7f27ec07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.328+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7f280009cac0 con 0x7f27f40073a0 2024-09-21T07:54:29.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.328+0000 7f280be00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079400 0x7f27ec07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.329+0000 7f280be00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079400 0x7f27ec07b910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f27f8036000 tx=0x7f27f8035040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.332+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f280c152860 con 0x7f27f40073a0 2024-09-21T07:54:29.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.522+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f280c15a900 con 0x7f27f40073a0 2024-09-21T07:54:29.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.523+0000 7f2808c00640 1 -- 172.21.15.88:0/2396191479 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v148) ==== 74+0+13098 (secure 0 0 0) 0x7f280c15a900 con 0x7f27f40073a0 2024-09-21T07:54:29.525 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:29.525 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":148,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:28.062447+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":143,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/3712798522":"2024-09-22T07:54:24.705429+0000","172.21.15.88:6832/3712798522":"2024-09-22T07:54:24.705429+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:29.527 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079400 msgr2=0x7f27ec07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079400 0x7f27ec07b910 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f27f8036000 tx=0x7f27f8035040 comp rx=0 tx=0).stop 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 msgr2=0x7f280c152a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f280c152a60 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2800004650 tx=0x7f280000bac0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 shutdown_connections 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f27ec079400 0x7f27ec07b910 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f280c1511a0 0x7f280c158150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f280c150980 0x7f280c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 --2- 172.21.15.88:0/2396191479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27f40073a0 0x7f280c152a60 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 >> 172.21.15.88:0/2396191479 conn(0x7f280c06d730 msgr2=0x7f280c152150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 shutdown_connections 2024-09-21T07:54:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.525+0000 7f2813a00640 1 -- 172.21.15.88:0/2396191479 wait complete. 2024-09-21T07:54:29.552 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:54:29.552 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.773+0000 7f27b6e00640 1 -- 172.21.15.88:0/1207252924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 msgr2=0x7f27b00ad7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.773+0000 7f27b6e00640 1 --2- 172.21.15.88:0/1207252924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b00ad7e0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7f27b00753f0 tx=0x7f27a802aa30 comp rx=0 tx=0).stop 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 -- 172.21.15.88:0/1207252924 shutdown_connections 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 --2- 172.21.15.88:0/1207252924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b00ad7e0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 --2- 172.21.15.88:0/1207252924 >> v2:172.21.15.175:3300/0 conn(0x7f27b0160490 0x7f27b015d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 --2- 172.21.15.88:0/1207252924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b00b4f00 0x7f27b00704a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 -- 172.21.15.88:0/1207252924 >> 172.21.15.88:0/1207252924 conn(0x7f27b006d710 msgr2=0x7f27b00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 -- 172.21.15.88:0/1207252924 shutdown_connections 2024-09-21T07:54:29.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.774+0000 7f27b6e00640 1 -- 172.21.15.88:0/1207252924 wait complete. 2024-09-21T07:54:29.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.776+0000 7f27b6e00640 1 Processor -- start 2024-09-21T07:54:29.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.776+0000 7f27b6e00640 1 -- start start 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.777+0000 7f27b6e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b0153150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.777+0000 7f27b6e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b00b4f00 0x7f27b0145730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.777+0000 7f27b6e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b0160490 0x7f27b014f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.777+0000 7f27b6e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27b0160a20 con 0x7f27b00ad3d0 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.777+0000 7f27b6e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27b01608a0 con 0x7f27b00b4f00 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.777+0000 7f27b6e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27b006ef70 con 0x7f27b0160490 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b0153150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b0153150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40832/0 (socket says 172.21.15.88:40832) 2024-09-21T07:54:29.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 -- 172.21.15.88:0/3933952440 learned_addr learned my addr 172.21.15.88:0/3933952440 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27a7e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b00b4f00 0x7f27b0145730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27b4e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b0160490 0x7f27b014f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 -- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b0160490 msgr2=0x7f27b014f930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b0160490 0x7f27b014f930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 -- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b00b4f00 msgr2=0x7f27b0145730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b00b4f00 0x7f27b0145730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 -- 172.21.15.88:0/3933952440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27b01f59e0 con 0x7f27b00ad3d0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27afe00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b0153150 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f27a000d660 tx=0x7f27a000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.778+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a0017070 con 0x7f27b00ad3d0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.779+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f27a0005310 con 0x7f27b00ad3d0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.779+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27a00097c0 con 0x7f27b00ad3d0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.779+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27b01f5cc0 con 0x7f27b00ad3d0 2024-09-21T07:54:29.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.779+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27b01f6280 con 0x7f27b00ad3d0 2024-09-21T07:54:29.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.780+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27b00704a0 con 0x7f27b00ad3d0 2024-09-21T07:54:29.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.781+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f27a0009960 con 0x7f27b00ad3d0 2024-09-21T07:54:29.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.782+0000 7f27acc00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f278c079400 0x7f278c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:29.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.782+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7f27a009bd10 con 0x7f27b00ad3d0 2024-09-21T07:54:29.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.782+0000 7f27a7e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f278c079400 0x7f278c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:29.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.783+0000 7f27a7e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f278c079400 0x7f278c07b910 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2798005fd0 tx=0x7f2798005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:29.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.786+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f27a0063490 con 0x7f27b00ad3d0 2024-09-21T07:54:29.972 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.970+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f27b00adae0 con 0x7f27b00ad3d0 2024-09-21T07:54:29.973 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.971+0000 7f27acc00640 1 -- 172.21.15.88:0/3933952440 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v148) ==== 93+0+10 (secure 0 0 0) 0x7f27a0062d90 con 0x7f27b00ad3d0 2024-09-21T07:54:29.973 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f278c079400 msgr2=0x7f278c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f278c079400 0x7f278c07b910 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2798005fd0 tx=0x7f2798005850 comp rx=0 tx=0).stop 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 msgr2=0x7f27b0153150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b0153150 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f27a000d660 tx=0x7f27a000db20 comp rx=0 tx=0).stop 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 shutdown_connections 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f278c079400 0x7f278c07b910 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27b0160490 0x7f27b014f930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27b00b4f00 0x7f27b0145730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 --2- 172.21.15.88:0/3933952440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27b00ad3d0 0x7f27b0153150 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.973+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 >> 172.21.15.88:0/3933952440 conn(0x7f27b006d710 msgr2=0x7f27b0156780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:29.976 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.974+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 shutdown_connections 2024-09-21T07:54:29.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:29.974+0000 7f27b6e00640 1 -- 172.21.15.88:0/3933952440 wait complete. 2024-09-21T07:54:29.999 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.220+0000 7f65a2e00640 1 -- 172.21.15.88:0/4178952207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6580003a00 msgr2=0x7f6580003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.220+0000 7f65a2e00640 1 --2- 172.21.15.88:0/4178952207 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6580003a00 0x7f6580003e70 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c4a0 tx=0x7f6580005830 comp rx=0 tx=0).stop 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.220+0000 7f65a0e00640 1 -- 172.21.15.88:0/4178952207 reap_dead start 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.220+0000 7f65a2e00640 1 -- 172.21.15.88:0/4178952207 shutdown_connections 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.220+0000 7f65a2e00640 1 -- 172.21.15.88:0/4178952207 >> 172.21.15.88:0/4178952207 conn(0x7f659c06d730 msgr2=0x7f659c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.221+0000 7f65a2e00640 1 -- 172.21.15.88:0/4178952207 shutdown_connections 2024-09-21T07:54:30.223 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.221+0000 7f65a2e00640 1 -- 172.21.15.88:0/4178952207 wait complete. 2024-09-21T07:54:30.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.223+0000 7f65a2e00640 1 Processor -- start 2024-09-21T07:54:30.225 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.223+0000 7f65a2e00640 1 -- start start 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.224+0000 7f65a2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6580003a00 0x7f659c159fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.224+0000 7f65a2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f659c074190 0x7f659c06f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.224+0000 7f65a2e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 0x7f659c142e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.224+0000 7f65a2e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0b4da0 con 0x7f659c06f960 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.224+0000 7f65a2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f659c0b4c20 con 0x7f6580003a00 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.224+0000 7f65a2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f659c0b3960 con 0x7f659c074190 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 0x7f659c142e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 0x7f659c142e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40856/0 (socket says 172.21.15.88:40856) 2024-09-21T07:54:30.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 -- 172.21.15.88:0/3834243802 learned_addr learned my addr 172.21.15.88:0/3834243802 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f659be00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6580003a00 0x7f659c159fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f659b400640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f659c074190 0x7f659c06f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 -- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f659c074190 msgr2=0x7f659c06f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f659c074190 0x7f659c06f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 -- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6580003a00 msgr2=0x7f659c159fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6580003a00 0x7f659c159fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 -- 172.21.15.88:0/3834243802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f659c15a4e0 con 0x7f659c06f960 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f659be00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6580003a00 0x7f659c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.225+0000 7f65a0e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 0x7f659c142e60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6594005fd0 tx=0x7f6594010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.226+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594003970 con 0x7f659c06f960 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.226+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6594015070 con 0x7f659c06f960 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.226+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6594004030 con 0x7f659c06f960 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.226+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f659c06fd70 con 0x7f659c06f960 2024-09-21T07:54:30.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.226+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f659c1f9480 con 0x7f659c06f960 2024-09-21T07:54:30.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.227+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f659c0ad4f0 con 0x7f659c06f960 2024-09-21T07:54:30.230 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.228+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f65940041d0 con 0x7f659c06f960 2024-09-21T07:54:30.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.230+0000 7f6593e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f657c079400 0x7f657c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:30.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.230+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7f659409c550 con 0x7f659c06f960 2024-09-21T07:54:30.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.230+0000 7f659be00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f657c079400 0x7f657c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:30.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.231+0000 7f659be00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f657c079400 0x7f657c07b910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c4a0 tx=0x7f658c038040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:30.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.234+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f659c06fd70 con 0x7f659c06f960 2024-09-21T07:54:30.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:30.491+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f659c0b2210 con 0x7f659c06f960 2024-09-21T07:54:31.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.078+0000 7f6593e00640 1 -- 172.21.15.88:0/3834243802 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v475) ==== 148+0+0 (secure 0 0 0) 0x7f6594064050 con 0x7f659c06f960 2024-09-21T07:54:31.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f657c079400 msgr2=0x7f657c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f657c079400 0x7f657c07b910 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f658c00c4a0 tx=0x7f658c038040 comp rx=0 tx=0).stop 2024-09-21T07:54:31.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 msgr2=0x7f659c142e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 0x7f659c142e60 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7f6594005fd0 tx=0x7f6594010f40 comp rx=0 tx=0).stop 2024-09-21T07:54:31.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 shutdown_connections 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f657c079400 0x7f657c07b910 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f659c074190 0x7f659c06f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f659c06f960 0x7f659c142e60 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 --2- 172.21.15.88:0/3834243802 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6580003a00 0x7f659c159fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 >> 172.21.15.88:0/3834243802 conn(0x7f659c06d730 msgr2=0x7f659c15ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 shutdown_connections 2024-09-21T07:54:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.081+0000 7f65a2e00640 1 -- 172.21.15.88:0/3834243802 wait complete. 2024-09-21T07:54:31.111 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:54:31.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 -- 172.21.15.88:0/2702391086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feca4004800 msgr2=0x7feca4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 --2- 172.21.15.88:0/2702391086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feca4004800 0x7feca4004c10 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7feca80016c0 tx=0x7feca4005830 comp rx=0 tx=0).stop 2024-09-21T07:54:31.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 -- 172.21.15.88:0/2702391086 shutdown_connections 2024-09-21T07:54:31.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 --2- 172.21.15.88:0/2702391086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feca4004800 0x7feca4004c10 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 -- 172.21.15.88:0/2702391086 >> 172.21.15.88:0/2702391086 conn(0x7fecc008bfe0 msgr2=0x7fecc009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 -- 172.21.15.88:0/2702391086 shutdown_connections 2024-09-21T07:54:31.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.330+0000 7fecc5a00640 1 -- 172.21.15.88:0/2702391086 wait complete. 2024-09-21T07:54:31.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.332+0000 7fecc5a00640 1 Processor -- start 2024-09-21T07:54:31.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.332+0000 7fecc5a00640 1 -- start start 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca4004800 0x7fecc00cde30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc00c8ae0 0x7fecc00c9ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 0x7fecc00a0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0162bd0 con 0x7fecc00ca0e0 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fecc0162a50 con 0x7feca4004800 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fecc0162d50 con 0x7fecc00c8ae0 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 0x7fecc00a0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 0x7fecc00a0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40872/0 (socket says 172.21.15.88:40872) 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecc5000640 1 -- 172.21.15.88:0/1455913449 learned_addr learned my addr 172.21.15.88:0/1455913449 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecbfe00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca4004800 0x7fecc00cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.334+0000 7fecb7400640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc00c8ae0 0x7fecc00c9ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5000640 1 -- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc00c8ae0 msgr2=0x7fecc00c9ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5000640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc00c8ae0 0x7fecc00c9ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5000640 1 -- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca4004800 msgr2=0x7fecc00cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5000640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca4004800 0x7fecc00cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5000640 1 -- 172.21.15.88:0/1455913449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecc00ce370 con 0x7fecc00ca0e0 2024-09-21T07:54:31.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5000640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 0x7fecc00a0180 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fecc00ba2a0 tx=0x7fecb800ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb800ca10 con 0x7fecc00ca0e0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fecb8018070 con 0x7fecc00ca0e0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecc00ca4f0 con 0x7fecc00ca0e0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8011460 con 0x7fecc00ca0e0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.335+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecc009f6a0 con 0x7fecc00ca0e0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.336+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecc00d24a0 con 0x7fecc00ca0e0 2024-09-21T07:54:31.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.338+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fecb8011600 con 0x7fecc00ca0e0 2024-09-21T07:54:31.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.339+0000 7fecbcc00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fec9c079400 0x7fec9c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.339+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7fecb8099870 con 0x7fecc00ca0e0 2024-09-21T07:54:31.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.339+0000 7fecbfe00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fec9c079400 0x7fec9c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.340+0000 7fecbfe00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fec9c079400 0x7fec9c07b910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009cd0 tx=0x7fecb0005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.344+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fecc00ca4f0 con 0x7fecc00ca0e0 2024-09-21T07:54:31.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.530+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fecc00b4620 con 0x7fecc00ca0e0 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.531+0000 7fecbcc00640 1 -- 172.21.15.88:0/1455913449 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v49) ==== 59+0+2294 (secure 0 0 0) 0x7fecb806ae20 con 0x7fecc00ca0e0 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:54:31.533 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:54:31.534 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:54:31.535 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:54:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fec9c079400 msgr2=0x7fec9c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fec9c079400 0x7fec9c07b910 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fecb0009cd0 tx=0x7fecb0005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 msgr2=0x7fecc00a0180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 0x7fecc00a0180 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fecc00ba2a0 tx=0x7fecb800ec50 comp rx=0 tx=0).stop 2024-09-21T07:54:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 shutdown_connections 2024-09-21T07:54:31.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fec9c079400 0x7fec9c07b910 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fecc00c8ae0 0x7fecc00c9ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecc00ca0e0 0x7fecc00a0180 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 --2- 172.21.15.88:0/1455913449 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7feca4004800 0x7fecc00cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 >> 172.21.15.88:0/1455913449 conn(0x7fecc008bfe0 msgr2=0x7fecc00b10d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 shutdown_connections 2024-09-21T07:54:31.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.533+0000 7fecc5a00640 1 -- 172.21.15.88:0/1455913449 wait complete. 2024-09-21T07:54:31.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 -- 172.21.15.88:0/1437868921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145a40 msgr2=0x7f2b64073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 --2- 172.21.15.88:0/1437868921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145a40 0x7f2b64073540 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f2b64072bd0 tx=0x7f2b4c02a6c0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 -- 172.21.15.88:0/1437868921 shutdown_connections 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 --2- 172.21.15.88:0/1437868921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145a40 0x7f2b64073540 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 --2- 172.21.15.88:0/1437868921 >> v2:172.21.15.175:3300/0 conn(0x7f2b64145550 0x7f2b6406fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 -- 172.21.15.88:0/1437868921 >> 172.21.15.88:0/1437868921 conn(0x7f2b6406d1c0 msgr2=0x7f2b6406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 -- 172.21.15.88:0/1437868921 shutdown_connections 2024-09-21T07:54:31.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.778+0000 7f2b6b400640 1 -- 172.21.15.88:0/1437868921 wait complete. 2024-09-21T07:54:31.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.781+0000 7f2b6b400640 1 Processor -- start 2024-09-21T07:54:31.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.781+0000 7f2b6b400640 1 -- start start 2024-09-21T07:54:31.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b64073a80 0x7f2b6415bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 0x7f2b6413d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b6b400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b64145a40 0x7f2b6413dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b6b400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64160a30 con 0x7f2b64145550 2024-09-21T07:54:31.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b6b400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2b640ce870 con 0x7f2b64073a80 2024-09-21T07:54:31.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b6b400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2b64160bb0 con 0x7f2b64145a40 2024-09-21T07:54:31.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 0x7f2b6413d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b5be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 0x7f2b6413d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40892/0 (socket says 172.21.15.88:40892) 2024-09-21T07:54:31.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.782+0000 7f2b5be00640 1 -- 172.21.15.88:0/3373319834 learned_addr learned my addr 172.21.15.88:0/3373319834 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b69400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b64145a40 0x7f2b6413dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b68a00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b64073a80 0x7f2b6415bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b5be00640 1 -- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b64145a40 msgr2=0x7f2b6413dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b5be00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b64145a40 0x7f2b6413dbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b5be00640 1 -- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b64073a80 msgr2=0x7f2b6415bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b5be00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b64073a80 0x7f2b6415bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b5be00640 1 -- 172.21.15.88:0/3373319834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b641fa050 con 0x7f2b64145550 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b69400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b64145a40 0x7f2b6413dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b68a00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b64073a80 0x7f2b6415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b5be00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 0x7f2b6413d670 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0099c0 tx=0x7f2b4c0039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c03e030 con 0x7f2b64145550 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2b4c0047a0 con 0x7f2b64145550 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2b4c035040 con 0x7f2b64145550 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.783+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b641601a0 con 0x7f2b64145550 2024-09-21T07:54:31.787 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.784+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b640b0190 con 0x7f2b64145550 2024-09-21T07:54:31.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.785+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b641f9dc0 con 0x7f2b64145550 2024-09-21T07:54:31.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.786+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f2b4c041030 con 0x7f2b64145550 2024-09-21T07:54:31.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.787+0000 7f2b61600640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b40079400 0x7f2b4007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:31.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.787+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7f2b4c0bc570 con 0x7f2b64145550 2024-09-21T07:54:31.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.787+0000 7f2b68a00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b40079400 0x7f2b4007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:31.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.788+0000 7f2b68a00640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b40079400 0x7f2b4007b910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400ba60 tx=0x7f2b54005cf0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:31.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.791+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2b4c083cf0 con 0x7f2b64145550 2024-09-21T07:54:31.977 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.975+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f2b641610f0 con 0x7f2b64145550 2024-09-21T07:54:31.982 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.980+0000 7f2b61600640 1 -- 172.21.15.88:0/3373319834 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7f2b4c0835f0 con 0x7f2b64145550 2024-09-21T07:54:31.984 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b40079400 msgr2=0x7f2b4007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b40079400 0x7f2b4007b910 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f2b5400ba60 tx=0x7f2b54005cf0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 msgr2=0x7f2b6413d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 0x7f2b6413d670 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7f2b4c0099c0 tx=0x7f2b4c0039e0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 shutdown_connections 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2b40079400 0x7f2b4007b910 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2b64145a40 0x7f2b6413dbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2b64145550 0x7f2b6413d670 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 --2- 172.21.15.88:0/3373319834 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2b64073a80 0x7f2b6415bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 >> 172.21.15.88:0/3373319834 conn(0x7f2b6406d1c0 msgr2=0x7f2b64144380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 shutdown_connections 2024-09-21T07:54:31.985 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:31.982+0000 7f2b6b400640 1 -- 172.21.15.88:0/3373319834 wait complete. 2024-09-21T07:54:32.006 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:54:32.006 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:32.088 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:32.179 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:54:32.179 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:32.179 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:32.215 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:32.215 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:32.215 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:32.251 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:32.251 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:32.251 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:32.290 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:32.290 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:32.290 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:32.327 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:32.327 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:54:32.327 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.548+0000 7f04cd600640 1 -- 172.21.15.88:0/2094945990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c80d2220 msgr2=0x7f04c80d2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.548+0000 7f04cd600640 1 --2- 172.21.15.88:0/2094945990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c80d2220 0x7f04c80d2630 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f04b0009860 tx=0x7f04b002a540 comp rx=0 tx=0).stop 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.548+0000 7f04cd600640 1 -- 172.21.15.88:0/2094945990 shutdown_connections 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.548+0000 7f04cd600640 1 --2- 172.21.15.88:0/2094945990 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c80d2220 0x7f04c80d2630 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.548+0000 7f04cd600640 1 --2- 172.21.15.88:0/2094945990 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04c8159210 0x7f04c80d08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.549+0000 7f04cd600640 1 -- 172.21.15.88:0/2094945990 >> 172.21.15.88:0/2094945990 conn(0x7f04c806d400 msgr2=0x7f04c80710f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.549+0000 7f04cd600640 1 -- 172.21.15.88:0/2094945990 shutdown_connections 2024-09-21T07:54:32.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.549+0000 7f04cd600640 1 -- 172.21.15.88:0/2094945990 wait complete. 2024-09-21T07:54:32.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.551+0000 7f04cd600640 1 Processor -- start 2024-09-21T07:54:32.553 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.551+0000 7f04cd600640 1 -- start start 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.552+0000 7f04cd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04c80ceca0 0x7f04c80c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.552+0000 7f04cd600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 0x7f04c8151540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.552+0000 7f04cd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c8159210 0x7f04c8151a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.552+0000 7f04cd600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f04c806f1f0 con 0x7f04c80d2220 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.552+0000 7f04cd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f04c806f070 con 0x7f04c8159210 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.552+0000 7f04cd600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f04c806f370 con 0x7f04c80ceca0 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 0x7f04c8151540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 0x7f04c8151540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40902/0 (socket says 172.21.15.88:40902) 2024-09-21T07:54:32.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 -- 172.21.15.88:0/2575025398 learned_addr learned my addr 172.21.15.88:0/2575025398 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04ccc00640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c8159210 0x7f04c8151a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04c7e00640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04c80ceca0 0x7f04c80c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 -- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04c80ceca0 msgr2=0x7f04c80c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04c80ceca0 0x7f04c80c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 -- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c8159210 msgr2=0x7f04c8151a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c8159210 0x7f04c8151a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 -- 172.21.15.88:0/2575025398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04c80b3370 con 0x7f04c80d2220 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.553+0000 7f04bf400640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 0x7f04c8151540 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f04b002ab30 tx=0x7f04b0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.554+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0031d60 con 0x7f04c80d2220 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.554+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f04b0033e10 con 0x7f04c80d2220 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.554+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04c80b3650 con 0x7f04c80d2220 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.554+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f04b0005260 con 0x7f04c80d2220 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.554+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04c80ca950 con 0x7f04c80d2220 2024-09-21T07:54:32.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.555+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04c80cc3f0 con 0x7f04c80d2220 2024-09-21T07:54:32.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.556+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f04b0005400 con 0x7f04c80d2220 2024-09-21T07:54:32.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.557+0000 7f04c4c00640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f04a4079430 0x7f04a407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:32.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.557+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(148..148 src has 1..148) ==== 4971+0+0 (secure 0 0 0) 0x7f04b00bcdb0 con 0x7f04c80d2220 2024-09-21T07:54:32.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.557+0000 7f04c7e00640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f04a4079430 0x7f04a407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:32.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.558+0000 7f04c7e00640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f04a4079430 0x7f04a407b940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009530 tx=0x7f04b8009260 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:32.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.561+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f04b003f030 con 0x7f04c80d2220 2024-09-21T07:54:32.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.786+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f04c80d2630 con 0x7f04c80d2220 2024-09-21T07:54:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.787+0000 7f04c4c00640 1 -- 172.21.15.88:0/2575025398 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) ==== 95+0+3916 (secure 0 0 0) 0x7f04b0083e50 con 0x7f04c80d2220 2024-09-21T07:54:32.789 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:32.789 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":102,"btime":"2024-09-21T07:54:25:707602+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7816,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6801/2995099149","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2995099149},{"type":"v1","addr":"172.21.15.175:6801","nonce":2995099149}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":7819,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/4276188743","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6834","nonce":4276188743},{"type":"v1","addr":"172.21.15.175:6835","nonce":4276188743}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":94},{"gid":8391,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/3237606276","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":3237606276},{"type":"v1","addr":"172.21.15.88:6834","nonce":3237606276}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":93},{"gid":8685,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/738718728","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":738718728},{"type":"v1","addr":"172.21.15.88:6835","nonce":738718728}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":102}],"filesystems":[]} 2024-09-21T07:54:32.789 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 102 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.789+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f04a4079430 msgr2=0x7f04a407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.789+0000 7f04cd600640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f04a4079430 0x7f04a407b940 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f04b8009530 tx=0x7f04b8009260 comp rx=0 tx=0).stop 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.789+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 msgr2=0x7f04c8151540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.789+0000 7f04cd600640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 0x7f04c8151540 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7f04b002ab30 tx=0x7f04b0033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.789+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 shutdown_connections 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f04a4079430 0x7f04a407b940 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f04c8159210 0x7f04c8151a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f04c80d2220 0x7f04c8151540 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 --2- 172.21.15.88:0/2575025398 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f04c80ceca0 0x7f04c80c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:32.792 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 >> 172.21.15.88:0/2575025398 conn(0x7f04c806d400 msgr2=0x7f04c80ce330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 shutdown_connections 2024-09-21T07:54:32.793 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:32.790+0000 7f04cd600640 1 -- 172.21.15.88:0/2575025398 wait complete. 2024-09-21T07:54:32.813 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:54:32.814 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:54:32.814 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:54:32.814 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:32.814 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:54:32.814 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:54:32.814 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:54:32.814 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:32.815 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:54:32.815 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:54:32.815 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:54:32.815 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:32.815 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:54:32.815 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:54:32.815 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:54:32.816 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:54:32.836 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:54:32.837 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:54:32.839 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:54:32.839 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:54:32.840 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:54:32.840 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:54:32.841 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:54:32.841 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:54:32.883 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:54:32.883 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:32.885 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:54:32.885 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:32.886 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:54:32.886 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:32.887 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:54:32.888 DEBUG:teuthology.parallel:result is None 2024-09-21T07:54:32.888 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:54:33.079 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:54:33.079 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:54:33.082 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:54:33.082 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:54:33.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 -- 172.21.15.88:0/2506319814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 msgr2=0x7f17401518d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 --2- 172.21.15.88:0/2506319814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17401518d0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f1730005fd0 tx=0x7f173002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 -- 172.21.15.88:0/2506319814 shutdown_connections 2024-09-21T07:54:33.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 --2- 172.21.15.88:0/2506319814 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17400b3520 0x7f17400b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 --2- 172.21.15.88:0/2506319814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17401518d0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 -- 172.21.15.88:0/2506319814 >> 172.21.15.88:0/2506319814 conn(0x7f17400b1e40 msgr2=0x7f17400b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:33.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 -- 172.21.15.88:0/2506319814 shutdown_connections 2024-09-21T07:54:33.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.120+0000 7f173fe00640 1 -- 172.21.15.88:0/2506319814 wait complete. 2024-09-21T07:54:33.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.122+0000 7f173fe00640 1 Processor -- start 2024-09-21T07:54:33.124 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.122+0000 7f173fe00640 1 -- start start 2024-09-21T07:54:33.126 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17400b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17400735c0 0x7f174014b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17400b3520 0x7f1740159820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1740070060 con 0x7f1740070f50 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f174006fee0 con 0x7f17400735c0 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f17400b3e70 con 0x7f17400b3520 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17400b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17400b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40948/0 (socket says 172.21.15.88:40948) 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 -- 172.21.15.88:0/667067115 learned_addr learned my addr 172.21.15.88:0/667067115 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173f400640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17400b3520 0x7f1740159820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173e000640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17400735c0 0x7f174014b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 -- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17400b3520 msgr2=0x7f1740159820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17400b3520 0x7f1740159820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.127 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 -- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17400735c0 msgr2=0x7f174014b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:33.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17400735c0 0x7f174014b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:33.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173ea00640 1 -- 172.21.15.88:0/667067115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17401f8da0 con 0x7f1740070f50 2024-09-21T07:54:33.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.124+0000 7f173e000640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17400735c0 0x7f174014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:33.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.125+0000 7f173ea00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17400b13b0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1730006010 tx=0x7f17300038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:33.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.125+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f173003e030 con 0x7f1740070f50 2024-09-21T07:54:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.125+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1730003ca0 con 0x7f1740070f50 2024-09-21T07:54:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.125+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1730035040 con 0x7f1740070f50 2024-09-21T07:54:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.125+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17401f9040 con 0x7f1740070f50 2024-09-21T07:54:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.125+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17401f9600 con 0x7f1740070f50 2024-09-21T07:54:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.126+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1740152e50 con 0x7f1740070f50 2024-09-21T07:54:33.129 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.127+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f1730041030 con 0x7f1740070f50 2024-09-21T07:54:33.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.128+0000 7f1737400640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1720079430 0x7f172007b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:33.130 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.128+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(149..149 src has 1..149) ==== 4971+0+0 (secure 0 0 0) 0x7f17300bc320 con 0x7f1740070f50 2024-09-21T07:54:33.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.129+0000 7f173e000640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1720079430 0x7f172007b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:33.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.129+0000 7f173e000640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1720079430 0x7f172007b940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1740151fe0 tx=0x7f172c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:33.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.133+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1740152e50 con 0x7f1740070f50 2024-09-21T07:54:33.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:33.333+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f17401518d0 con 0x7f1740070f50 2024-09-21T07:54:34.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.092+0000 7f1737400640 1 -- 172.21.15.88:0/667067115 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v151) ==== 97+0+0 (secure 0 0 0) 0x7f17300833c0 con 0x7f1740070f50 2024-09-21T07:54:34.095 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.095+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1720079430 msgr2=0x7f172007b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.095+0000 7f173fe00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1720079430 0x7f172007b940 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f1740151fe0 tx=0x7f172c009040 comp rx=0 tx=0).stop 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.095+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 msgr2=0x7f17400b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.095+0000 7f173fe00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17400b13b0 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f1730006010 tx=0x7f17300038a0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 shutdown_connections 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1720079430 0x7f172007b940 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f17400b3520 0x7f1740159820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f17400735c0 0x7f174014b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.098 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 --2- 172.21.15.88:0/667067115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1740070f50 0x7f17400b13b0 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 >> 172.21.15.88:0/667067115 conn(0x7f17400b1e40 msgr2=0x7f1740154be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:34.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 shutdown_connections 2024-09-21T07:54:34.099 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.096+0000 7f173fe00640 1 -- 172.21.15.88:0/667067115 wait complete. 2024-09-21T07:54:34.122 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.338+0000 7f8028c00640 1 -- 172.21.15.88:0/3186846488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 msgr2=0x7f80240cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 --2- 172.21.15.88:0/3186846488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80240cfd30 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7f80240b5b30 tx=0x7f801402a560 comp rx=0 tx=0).stop 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 -- 172.21.15.88:0/3186846488 shutdown_connections 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 --2- 172.21.15.88:0/3186846488 >> v2:172.21.15.175:3300/0 conn(0x7f80241620e0 0x7f80240d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 --2- 172.21.15.88:0/3186846488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80240cfd30 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 -- 172.21.15.88:0/3186846488 >> 172.21.15.88:0/3186846488 conn(0x7f802409d820 msgr2=0x7f802408bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:34.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 -- 172.21.15.88:0/3186846488 shutdown_connections 2024-09-21T07:54:34.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.339+0000 7f8028c00640 1 -- 172.21.15.88:0/3186846488 wait complete. 2024-09-21T07:54:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.341+0000 7f8028c00640 1 Processor -- start 2024-09-21T07:54:34.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.341+0000 7f8028c00640 1 -- start start 2024-09-21T07:54:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.342+0000 7f8028c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80240b9610 0x7f80240b3490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.342+0000 7f8028c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80241f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.342+0000 7f8028c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80241620e0 0x7f802415c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.342+0000 7f8028c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80240b4360 con 0x7f80240b9b80 2024-09-21T07:54:34.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.342+0000 7f8028c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f80240b41e0 con 0x7f80241620e0 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.342+0000 7f8028c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f80240b44e0 con 0x7f80240b9610 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80241f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80241f5f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40962/0 (socket says 172.21.15.88:40962) 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 -- 172.21.15.88:0/85236961 learned_addr learned my addr 172.21.15.88:0/85236961 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f8023e00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80241620e0 0x7f802415c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f8023400640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80240b9610 0x7f80240b3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 -- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80240b9610 msgr2=0x7f80240b3490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80240b9610 0x7f80240b3490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 -- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80241620e0 msgr2=0x7f802415c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80241620e0 0x7f802415c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.346 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.343+0000 7f801aa00640 1 -- 172.21.15.88:0/85236961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f802415cca0 con 0x7f80240b9b80 2024-09-21T07:54:34.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.344+0000 7f801aa00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80241f5f70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c970 tx=0x7f800c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:34.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.344+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c007bb0 con 0x7f80240b9b80 2024-09-21T07:54:34.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.344+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f800c007e40 con 0x7f80240b9b80 2024-09-21T07:54:34.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.344+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80240cacf0 con 0x7f80240b9b80 2024-09-21T07:54:34.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.344+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f800c005290 con 0x7f80240b9b80 2024-09-21T07:54:34.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.344+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80240b2c90 con 0x7f80240b9b80 2024-09-21T07:54:34.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.345+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f80240caa60 con 0x7f80240b9b80 2024-09-21T07:54:34.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.346+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f800c005430 con 0x7f80240b9b80 2024-09-21T07:54:34.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.347+0000 7f801be00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8000079400 0x7f800007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.347+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4885+0+0 (secure 0 0 0) 0x7f800c09c1b0 con 0x7f80240b9b80 2024-09-21T07:54:34.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.348+0000 7f8023400640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8000079400 0x7f800007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.348+0000 7f8023400640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8000079400 0x7f800007b910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8014033000 tx=0x7f8014004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:34.354 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.352+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f802415cca0 con 0x7f80240b9b80 2024-09-21T07:54:34.606 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.604+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f80240d1660 con 0x7f80240b9b80 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.605+0000 7f801be00640 1 -- 172.21.15.88:0/85236961 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) ==== 80+0+5304 (secure 0 0 0) 0x7f800c014070 con 0x7f80240b9b80 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.607 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.608 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:54:34.609 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.610 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:54:34.611 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.612 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.613 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.614 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.615 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.616 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:54:34.617 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.618 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:54:34.619 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:54:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:34.620 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:54:34.620 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8000079400 msgr2=0x7f800007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8000079400 0x7f800007b910 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f8014033000 tx=0x7f8014004730 comp rx=0 tx=0).stop 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 msgr2=0x7f80241f5f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80241f5f70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7f800c00c970 tx=0x7f800c00ce30 comp rx=0 tx=0).stop 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 shutdown_connections 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8000079400 0x7f800007b910 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f80241620e0 0x7f802415c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80240b9b80 0x7f80241f5f70 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 --2- 172.21.15.88:0/85236961 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f80240b9610 0x7f80240b3490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 >> 172.21.15.88:0/85236961 conn(0x7f802409d820 msgr2=0x7f80240b5330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 shutdown_connections 2024-09-21T07:54:34.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.607+0000 7f8028c00640 1 -- 172.21.15.88:0/85236961 wait complete. 2024-09-21T07:54:34.633 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:54:34.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 -- 172.21.15.88:0/1226380358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad40b3550 msgr2=0x7f5ad40b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/1226380358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad40b3550 0x7f5ad40b3960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5acc005fd0 tx=0x7f5acc02a540 comp rx=0 tx=0).stop 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 -- 172.21.15.88:0/1226380358 shutdown_connections 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/1226380358 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad40b3550 0x7f5ad40b3960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/1226380358 >> v2:172.21.15.175:3300/0 conn(0x7f5ad415d760 0x7f5ad41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 -- 172.21.15.88:0/1226380358 >> 172.21.15.88:0/1226380358 conn(0x7f5ad406d730 msgr2=0x7f5ad40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 -- 172.21.15.88:0/1226380358 shutdown_connections 2024-09-21T07:54:34.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.850+0000 7f5ad9a00640 1 -- 172.21.15.88:0/1226380358 wait complete. 2024-09-21T07:54:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.852+0000 7f5ad9a00640 1 Processor -- start 2024-09-21T07:54:34.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.853+0000 7f5ad9a00640 1 -- start start 2024-09-21T07:54:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad9a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ad40ad0e0 0x7f5ad41f6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad9a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 0x7f5ad41ef5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad9a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad415d760 0x7f5ad41546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad9a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad406f240 con 0x7f5ad40b3550 2024-09-21T07:54:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad9a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ad40b4e30 con 0x7f5ad40ad0e0 2024-09-21T07:54:34.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad9a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ad40b3ea0 con 0x7f5ad415d760 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 0x7f5ad41ef5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 0x7f5ad41ef5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40968/0 (socket says 172.21.15.88:40968) 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.854+0000 7f5ad2000640 1 -- 172.21.15.88:0/2209845617 learned_addr learned my addr 172.21.15.88:0/2209845617 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ad40ad0e0 0x7f5ad41f6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad3400640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad415d760 0x7f5ad41546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2000640 1 -- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad415d760 msgr2=0x7f5ad41546a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2000640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad415d760 0x7f5ad41546a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2000640 1 -- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ad40ad0e0 msgr2=0x7f5ad41f6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2000640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ad40ad0e0 0x7f5ad41f6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2000640 1 -- 172.21.15.88:0/2209845617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ad4156830 con 0x7f5ad40b3550 2024-09-21T07:54:34.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5ad2000640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 0x7f5ad41ef5f0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c390 tx=0x7f5ac000c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.855+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0017070 con 0x7f5ad40b3550 2024-09-21T07:54:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.856+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5ac00035f0 con 0x7f5ad40b3550 2024-09-21T07:54:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.856+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ac0003a20 con 0x7f5ad40b3550 2024-09-21T07:54:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.856+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ad4156b10 con 0x7f5ad40b3550 2024-09-21T07:54:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.856+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ad4156df0 con 0x7f5ad40b3550 2024-09-21T07:54:34.859 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.857+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ad41565a0 con 0x7f5ad40b3550 2024-09-21T07:54:34.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.858+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f5ac001a070 con 0x7f5ad40b3550 2024-09-21T07:54:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.859+0000 7f5acaa00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ab4079400 0x7f5ab407b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.859+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4885+0+0 (secure 0 0 0) 0x7f5ac009af70 con 0x7f5ad40b3550 2024-09-21T07:54:34.861 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.859+0000 7f5ad2a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ab4079400 0x7f5ab407b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:34.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.860+0000 7f5ad2a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ab4079400 0x7f5ab407b910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005fd0 tx=0x7f5ac4009500 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:34.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:34.864+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5ac0062060 con 0x7f5ad40b3550 2024-09-21T07:54:35.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.051+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f5ad40b42f0 con 0x7f5ad40b3550 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.051+0000 7f5acaa00640 1 -- 172.21.15.88:0/2209845617 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v152) ==== 84+0+65 (secure 0 0 0) 0x7f5ac0066a90 con 0x7f5ad40b3550 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:35.054 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:54:35.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ab4079400 msgr2=0x7f5ab407b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ab4079400 0x7f5ab407b910 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5ac4005fd0 tx=0x7f5ac4009500 comp rx=0 tx=0).stop 2024-09-21T07:54:35.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 msgr2=0x7f5ad41ef5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 0x7f5ad41ef5f0 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7f5ac000c390 tx=0x7f5ac000c850 comp rx=0 tx=0).stop 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 shutdown_connections 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5ab4079400 0x7f5ab407b910 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ad415d760 0x7f5ad41546a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ad40b3550 0x7f5ad41ef5f0 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 --2- 172.21.15.88:0/2209845617 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ad40ad0e0 0x7f5ad41f6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 >> 172.21.15.88:0/2209845617 conn(0x7f5ad406d730 msgr2=0x7f5ad415e8a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 shutdown_connections 2024-09-21T07:54:35.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.054+0000 7f5ad9a00640 1 -- 172.21.15.88:0/2209845617 wait complete. 2024-09-21T07:54:35.079 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:54:35.079 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:54:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.301+0000 7f1b64000640 1 -- 172.21.15.88:0/3283192557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c06cda0 msgr2=0x7f1b5c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.301+0000 7f1b64000640 1 --2- 172.21.15.88:0/3283192557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c06cda0 0x7f1b5c06d1b0 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f1b5c06d8a0 tx=0x7f1b4802a580 comp rx=0 tx=0).stop 2024-09-21T07:54:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 -- 172.21.15.88:0/3283192557 shutdown_connections 2024-09-21T07:54:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 --2- 172.21.15.88:0/3283192557 >> v2:172.21.15.175:3300/0 conn(0x7f1b5c0af100 0x7f1b5c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 --2- 172.21.15.88:0/3283192557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c06cda0 0x7f1b5c06d1b0 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 --2- 172.21.15.88:0/3283192557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0cc830 0x7f1b5c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 -- 172.21.15.88:0/3283192557 >> 172.21.15.88:0/3283192557 conn(0x7f1b5c0b2540 msgr2=0x7f1b5c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 -- 172.21.15.88:0/3283192557 shutdown_connections 2024-09-21T07:54:35.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.302+0000 7f1b64000640 1 -- 172.21.15.88:0/3283192557 wait complete. 2024-09-21T07:54:35.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.304+0000 7f1b64000640 1 Processor -- start 2024-09-21T07:54:35.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.304+0000 7f1b64000640 1 -- start start 2024-09-21T07:54:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.305+0000 7f1b64000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c06cda0 0x7f1b5c0ca080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.305+0000 7f1b64000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0af100 0x7f1b5c1f3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.305+0000 7f1b64000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 0x7f1b5c1f3b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.305+0000 7f1b64000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b34b0 con 0x7f1b5c0cc830 2024-09-21T07:54:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.305+0000 7f1b64000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b3330 con 0x7f1b5c06cda0 2024-09-21T07:54:35.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.305+0000 7f1b64000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1b5c0b3630 con 0x7f1b5c0af100 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 0x7f1b5c1f3b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 0x7f1b5c1f3b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40982/0 (socket says 172.21.15.88:40982) 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 -- 172.21.15.88:0/3850558716 learned_addr learned my addr 172.21.15.88:0/3850558716 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b61600640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c06cda0 0x7f1b5c0ca080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b60c00640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0af100 0x7f1b5c1f3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 -- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0af100 msgr2=0x7f1b5c1f3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0af100 0x7f1b5c1f3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 -- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c06cda0 msgr2=0x7f1b5c0ca080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c06cda0 0x7f1b5c0ca080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 -- 172.21.15.88:0/3850558716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b5c09ce50 con 0x7f1b5c0cc830 2024-09-21T07:54:35.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.306+0000 7f1b62000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 0x7f1b5c1f3b40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1b540058c0 tx=0x7f1b5400da70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.307+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b54017070 con 0x7f1b5c0cc830 2024-09-21T07:54:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.307+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1b54005310 con 0x7f1b5c0cc830 2024-09-21T07:54:35.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.307+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b5c09d130 con 0x7f1b5c0cc830 2024-09-21T07:54:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.307+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b540047a0 con 0x7f1b5c0cc830 2024-09-21T07:54:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.307+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b5c09d6f0 con 0x7f1b5c0cc830 2024-09-21T07:54:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.308+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b5c0af510 con 0x7f1b5c0cc830 2024-09-21T07:54:35.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.309+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f1b54004940 con 0x7f1b5c0cc830 2024-09-21T07:54:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.310+0000 7f1b59600640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c079430 0x7f1b3c07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.310+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(152..152 src has 1..152) ==== 4885+0+0 (secure 0 0 0) 0x7f1b5409bbc0 con 0x7f1b5c0cc830 2024-09-21T07:54:35.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.310+0000 7f1b61600640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c079430 0x7f1b3c07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:35.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.311+0000 7f1b61600640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c079430 0x7f1b3c07b940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005fd0 tx=0x7f1b500095e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:35.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.315+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1b54062d40 con 0x7f1b5c0cc830 2024-09-21T07:54:35.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:35.497+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f1b5c09cbe0 con 0x7f1b5c0cc830 2024-09-21T07:54:36.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.101+0000 7f1b59600640 1 -- 172.21.15.88:0/3850558716 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v153) ==== 138+0+0 (secure 0 0 0) 0x7f1b54067770 con 0x7f1b5c0cc830 2024-09-21T07:54:36.103 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:54:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.104+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c079430 msgr2=0x7f1b3c07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.104+0000 7f1b64000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c079430 0x7f1b3c07b940 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f1b50005fd0 tx=0x7f1b500095e0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.104+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 msgr2=0x7f1b5c1f3b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.104+0000 7f1b64000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 0x7f1b5c1f3b40 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f1b540058c0 tx=0x7f1b5400da70 comp rx=0 tx=0).stop 2024-09-21T07:54:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 shutdown_connections 2024-09-21T07:54:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1b3c079430 0x7f1b3c07b940 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1b5c0af100 0x7f1b5c1f3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b5c0cc830 0x7f1b5c1f3b40 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 --2- 172.21.15.88:0/3850558716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1b5c06cda0 0x7f1b5c0ca080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 >> 172.21.15.88:0/3850558716 conn(0x7f1b5c0b2540 msgr2=0x7f1b5c0b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:36.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 shutdown_connections 2024-09-21T07:54:36.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.105+0000 7f1b64000640 1 -- 172.21.15.88:0/3850558716 wait complete. 2024-09-21T07:54:36.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:54:36.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 -- 172.21.15.88:0/1930896023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990073c50 msgr2=0x7f69900cd610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 --2- 172.21.15.88:0/1930896023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990073c50 0x7f69900cd610 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f6988005fd0 tx=0x7f698802ab20 comp rx=0 tx=0).stop 2024-09-21T07:54:36.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 -- 172.21.15.88:0/1930896023 shutdown_connections 2024-09-21T07:54:36.361 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 --2- 172.21.15.88:0/1930896023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990073c50 0x7f69900cd610 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 --2- 172.21.15.88:0/1930896023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6990145550 0x7f699006fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 -- 172.21.15.88:0/1930896023 >> 172.21.15.88:0/1930896023 conn(0x7f699006d1c0 msgr2=0x7f699006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 -- 172.21.15.88:0/1930896023 shutdown_connections 2024-09-21T07:54:36.362 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.359+0000 7f6996400640 1 -- 172.21.15.88:0/1930896023 wait complete. 2024-09-21T07:54:36.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.361+0000 7f6996400640 1 Processor -- start 2024-09-21T07:54:36.363 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.361+0000 7f6996400640 1 -- start start 2024-09-21T07:54:36.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f6996400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 0x7f699013aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f6996400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6990145a40 0x7f69901f9810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.365 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f6996400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69900aefb0 0x7f6990143d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f6996400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69900ce150 con 0x7f6990145550 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f6996400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f69900cdfd0 con 0x7f6990145a40 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f6996400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f69900ce2d0 con 0x7f69900aefb0 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 0x7f699013aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 0x7f699013aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37020/0 (socket says 172.21.15.88:37020) 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 -- 172.21.15.88:0/11915924 learned_addr learned my addr 172.21.15.88:0/11915924 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698ea00640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6990145a40 0x7f69901f9810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698fe00640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69900aefb0 0x7f6990143d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 -- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69900aefb0 msgr2=0x7f6990143d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69900aefb0 0x7f6990143d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 -- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6990145a40 msgr2=0x7f69901f9810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:36.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6990145a40 0x7f69901f9810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:36.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.363+0000 7f698f400640 1 -- 172.21.15.88:0/11915924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f699013cd00 con 0x7f6990145550 2024-09-21T07:54:36.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.364+0000 7f698f400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 0x7f699013aa20 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f698000d6d0 tx=0x7f698000db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:36.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.364+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980017070 con 0x7f6990145550 2024-09-21T07:54:36.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.364+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6980005330 con 0x7f6990145550 2024-09-21T07:54:36.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.364+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6980009820 con 0x7f6990145550 2024-09-21T07:54:36.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.364+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f699013cfe0 con 0x7f6990145550 2024-09-21T07:54:36.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.364+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f699013d5a0 con 0x7f6990145550 2024-09-21T07:54:36.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.365+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69900ce850 con 0x7f6990145550 2024-09-21T07:54:36.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.367+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f6980009bd0 con 0x7f6990145550 2024-09-21T07:54:36.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.368+0000 7f6987400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6970079430 0x7f697007b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:36.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.368+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(153..153 src has 1..153) ==== 5255+0+0 (secure 0 0 0) 0x7f698009c150 con 0x7f6990145550 2024-09-21T07:54:36.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.368+0000 7f698ea00640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6970079430 0x7f697007b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:36.370 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.368+0000 7f698ea00640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6970079430 0x7f697007b940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6990144d20 tx=0x7f697c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:36.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.371+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6980063560 con 0x7f6990145550 2024-09-21T07:54:36.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:36.557+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f69900b5040 con 0x7f6990145550 2024-09-21T07:54:37.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.104+0000 7f6987400640 1 -- 172.21.15.88:0/11915924 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v154) ==== 170+0+0 (secure 0 0 0) 0x7f6980067f90 con 0x7f6990145550 2024-09-21T07:54:37.106 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6970079430 msgr2=0x7f697007b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6970079430 0x7f697007b940 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f6990144d20 tx=0x7f697c009040 comp rx=0 tx=0).stop 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 msgr2=0x7f699013aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 0x7f699013aa20 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f698000d6d0 tx=0x7f698000db90 comp rx=0 tx=0).stop 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 shutdown_connections 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6970079430 0x7f697007b940 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f69900aefb0 0x7f6990143d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6990145a40 0x7f69901f9810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 --2- 172.21.15.88:0/11915924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6990145550 0x7f699013aa20 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.107+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 >> 172.21.15.88:0/11915924 conn(0x7f699006d1c0 msgr2=0x7f69900693d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:37.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.108+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 shutdown_connections 2024-09-21T07:54:37.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.108+0000 7f6996400640 1 -- 172.21.15.88:0/11915924 wait complete. 2024-09-21T07:54:37.150 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.372+0000 7f0b09a00640 1 -- 172.21.15.88:0/63590868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ae8003a00 msgr2=0x7f0ae8003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.372+0000 7f0b09a00640 1 --2- 172.21.15.88:0/63590868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ae8003a00 0x7f0ae8003e70 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009b50 tx=0x7f0ae8005830 comp rx=0 tx=0).stop 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.372+0000 7f0b09000640 1 -- 172.21.15.88:0/63590868 reap_dead start 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.372+0000 7f0b09a00640 1 -- 172.21.15.88:0/63590868 shutdown_connections 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.372+0000 7f0b09a00640 1 -- 172.21.15.88:0/63590868 >> 172.21.15.88:0/63590868 conn(0x7f0b0406d620 msgr2=0x7f0b040b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.373+0000 7f0b09a00640 1 -- 172.21.15.88:0/63590868 shutdown_connections 2024-09-21T07:54:37.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.373+0000 7f0b09a00640 1 -- 172.21.15.88:0/63590868 wait complete. 2024-09-21T07:54:37.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.375+0000 7f0b09a00640 1 Processor -- start 2024-09-21T07:54:37.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.375+0000 7f0b09a00640 1 -- start start 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.376+0000 7f0b09a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae8003a00 0x7f0b041449e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.376+0000 7f0b09a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b0415f510 0x7f0b04144f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 0x7f0b04153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04157030 con 0x7f0b041537a0 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0b04156eb0 con 0x7f0ae8003a00 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0b040af610 con 0x7f0b0415f510 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 0x7f0b04153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 0x7f0b04153bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37038/0 (socket says 172.21.15.88:37038) 2024-09-21T07:54:37.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 -- 172.21.15.88:0/1151555203 learned_addr learned my addr 172.21.15.88:0/1151555203 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b03e00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae8003a00 0x7f0b041449e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0afbe00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b0415f510 0x7f0b04144f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 -- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b0415f510 msgr2=0x7f0b04144f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b0415f510 0x7f0b04144f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 -- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae8003a00 msgr2=0x7f0b041449e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae8003a00 0x7f0b041449e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.377+0000 7f0b09000640 1 -- 172.21.15.88:0/1151555203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0415b8d0 con 0x7f0b041537a0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.378+0000 7f0b09000640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 0x7f0b04153bb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b5310 tx=0x7f0afc00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.378+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc004b20 con 0x7f0b041537a0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.378+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0afc004cc0 con 0x7f0b041537a0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.378+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0415bbb0 con 0x7f0b041537a0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.378+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0afc003bd0 con 0x7f0b041537a0 2024-09-21T07:54:37.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.378+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0415c170 con 0x7f0b041537a0 2024-09-21T07:54:37.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.379+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0b040cc650 con 0x7f0b041537a0 2024-09-21T07:54:37.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.381+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f0afc004030 con 0x7f0b041537a0 2024-09-21T07:54:37.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.382+0000 7f0b00c00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ae0079400 0x7f0ae007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:37.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.382+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(154..154 src has 1..154) ==== 5637+0+0 (secure 0 0 0) 0x7f0afc09c2d0 con 0x7f0b041537a0 2024-09-21T07:54:37.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.382+0000 7f0b03e00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ae0079400 0x7f0ae007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:37.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.383+0000 7f0b03e00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ae0079400 0x7f0ae007b910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009b50 tx=0x7f0af402d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:37.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.386+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0b040cc650 con 0x7f0b041537a0 2024-09-21T07:54:37.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:37.613+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f0b0415b660 con 0x7f0b041537a0 2024-09-21T07:54:38.135 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.133+0000 7f0b00c00640 1 -- 172.21.15.88:0/1151555203 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v105) ==== 174+0+0 (secure 0 0 0) 0x7f0afc06a070 con 0x7f0b041537a0 2024-09-21T07:54:38.141 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 24 and data pool 25 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ae0079400 msgr2=0x7f0ae007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ae0079400 0x7f0ae007b910 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f0af4009b50 tx=0x7f0af402d040 comp rx=0 tx=0).stop 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 msgr2=0x7f0b04153bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 0x7f0b04153bb0 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f0b040b5310 tx=0x7f0afc00f3c0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 shutdown_connections 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0ae0079400 0x7f0ae007b910 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0b0415f510 0x7f0b04144f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b041537a0 0x7f0b04153bb0 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 --2- 172.21.15.88:0/1151555203 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ae8003a00 0x7f0b041449e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.143+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 >> 172.21.15.88:0/1151555203 conn(0x7f0b0406d620 msgr2=0x7f0b0415ed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.144+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 shutdown_connections 2024-09-21T07:54:38.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.144+0000 7f0b09a00640 1 -- 172.21.15.88:0/1151555203 wait complete. 2024-09-21T07:54:38.180 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 -- 172.21.15.88:0/716267908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f611815d760 msgr2=0x7f61181535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 --2- 172.21.15.88:0/716267908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f611815d760 0x7f61181535c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f6118072d60 tx=0x7f610c02a560 comp rx=0 tx=0).stop 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 -- 172.21.15.88:0/716267908 shutdown_connections 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 --2- 172.21.15.88:0/716267908 >> v2:172.21.15.175:3300/0 conn(0x7f61180ad0e0 0x7f61180ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 --2- 172.21.15.88:0/716267908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f611815d760 0x7f61181535c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 -- 172.21.15.88:0/716267908 >> 172.21.15.88:0/716267908 conn(0x7f611806d730 msgr2=0x7f61180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 -- 172.21.15.88:0/716267908 shutdown_connections 2024-09-21T07:54:38.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.398+0000 7f6120600640 1 -- 172.21.15.88:0/716267908 wait complete. 2024-09-21T07:54:38.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.401+0000 7f6120600640 1 Processor -- start 2024-09-21T07:54:38.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.401+0000 7f6120600640 1 -- start start 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f6120600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 0x7f611814a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f6120600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f611815d760 0x7f611814ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f6120600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118160310 0x7f6118156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f6120600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f61180b4c20 con 0x7f61180ad0e0 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f6120600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6118153c70 con 0x7f6118160310 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f6120600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f61180b4da0 con 0x7f611815d760 2024-09-21T07:54:38.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f611dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 0x7f611814a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.402+0000 7f611dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 0x7f611814a670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37062/0 (socket says 172.21.15.88:37062) 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 -- 172.21.15.88:0/4061088669 learned_addr learned my addr 172.21.15.88:0/4061088669 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611e600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118160310 0x7f6118156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611d200640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f611815d760 0x7f611814ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 -- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f611815d760 msgr2=0x7f611814ccc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f611815d760 0x7f611814ccc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 -- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118160310 msgr2=0x7f6118156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118160310 0x7f6118156570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 -- 172.21.15.88:0/4061088669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f611814bc20 con 0x7f61180ad0e0 2024-09-21T07:54:38.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.403+0000 7f611dc00640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 0x7f611814a670 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f610c034040 tx=0x7f610c003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:38.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.404+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c03e030 con 0x7f61180ad0e0 2024-09-21T07:54:38.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.404+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f610c003d10 con 0x7f61180ad0e0 2024-09-21T07:54:38.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.404+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f611814bf00 con 0x7f61180ad0e0 2024-09-21T07:54:38.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.404+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f610c035040 con 0x7f61180ad0e0 2024-09-21T07:54:38.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.404+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f611814c4c0 con 0x7f61180ad0e0 2024-09-21T07:54:38.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.405+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6118070350 con 0x7f61180ad0e0 2024-09-21T07:54:38.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.406+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f610c041030 con 0x7f61180ad0e0 2024-09-21T07:54:38.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.407+0000 7f6115600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f60f8079400 0x7f60f807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.408+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5737+0+0 (secure 0 0 0) 0x7f610c0bc5c0 con 0x7f61180ad0e0 2024-09-21T07:54:38.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.408+0000 7f611d200640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f60f8079400 0x7f60f807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.408+0000 7f611d200640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f60f8079400 0x7f60f807b910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6118155430 tx=0x7f6104005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:38.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.412+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6118070350 con 0x7f61180ad0e0 2024-09-21T07:54:38.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.599+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6118160ff0 con 0x7f61180ad0e0 2024-09-21T07:54:38.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.600+0000 7f6115600640 1 -- 172.21.15.88:0/4061088669 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16752 (secure 0 0 0) 0x7f610c083480 con 0x7f61180ad0e0 2024-09-21T07:54:38.602 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:38.602 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":155,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:37.615613+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:35.511240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:36.571979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:38.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f60f8079400 msgr2=0x7f60f807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f60f8079400 0x7f60f807b910 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f6118155430 tx=0x7f6104005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 msgr2=0x7f611814a670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 0x7f611814a670 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f610c034040 tx=0x7f610c003820 comp rx=0 tx=0).stop 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 shutdown_connections 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f60f8079400 0x7f60f807b910 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f611815d760 0x7f611814ccc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6118160310 0x7f6118156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 --2- 172.21.15.88:0/4061088669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f61180ad0e0 0x7f611814a670 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 >> 172.21.15.88:0/4061088669 conn(0x7f611806d730 msgr2=0x7f61180cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 shutdown_connections 2024-09-21T07:54:38.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.602+0000 7f6120600640 1 -- 172.21.15.88:0/4061088669 wait complete. 2024-09-21T07:54:38.627 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:38.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 -- 172.21.15.88:0/2042703161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2240ad0e0 msgr2=0x7fc2240ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 --2- 172.21.15.88:0/2042703161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2240ad0e0 0x7fc2240ad4f0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7fc224075250 tx=0x7fc20c02a560 comp rx=0 tx=0).stop 2024-09-21T07:54:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 -- 172.21.15.88:0/2042703161 shutdown_connections 2024-09-21T07:54:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 --2- 172.21.15.88:0/2042703161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2240ad0e0 0x7fc2240ad4f0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 --2- 172.21.15.88:0/2042703161 >> v2:172.21.15.175:3300/0 conn(0x7fc22415d760 0x7fc2241535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 -- 172.21.15.88:0/2042703161 >> 172.21.15.88:0/2042703161 conn(0x7fc22406d730 msgr2=0x7fc2240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 -- 172.21.15.88:0/2042703161 shutdown_connections 2024-09-21T07:54:38.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.844+0000 7fc228c00640 1 -- 172.21.15.88:0/2042703161 wait complete. 2024-09-21T07:54:38.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.846+0000 7fc228c00640 1 Processor -- start 2024-09-21T07:54:38.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.847+0000 7fc228c00640 1 -- start start 2024-09-21T07:54:38.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc228c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2240ad0e0 0x7fc224156020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc228c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22415d760 0x7fc2241f8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc228c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 0x7fc224153e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc228c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2240b4c20 con 0x7fc224160310 2024-09-21T07:54:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc228c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc224153c70 con 0x7fc22415d760 2024-09-21T07:54:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc228c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc2240b4da0 con 0x7fc2240ad0e0 2024-09-21T07:54:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc222800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 0x7fc224153e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc222800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 0x7fc224153e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37086/0 (socket says 172.21.15.88:37086) 2024-09-21T07:54:38.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc222800640 1 -- 172.21.15.88:0/1824076625 learned_addr learned my addr 172.21.15.88:0/1824076625 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc219400640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22415d760 0x7fc2241f8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.848+0000 7fc221e00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2240ad0e0 0x7fc224156020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc222800640 1 -- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2240ad0e0 msgr2=0x7fc224156020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc222800640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2240ad0e0 0x7fc224156020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc222800640 1 -- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22415d760 msgr2=0x7fc2241f8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:38.855 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc222800640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22415d760 0x7fc2241f8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc222800640 1 -- 172.21.15.88:0/1824076625 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc224156560 con 0x7fc224160310 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc222800640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 0x7fc224153e40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c730 tx=0x7fc21c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c003b70 con 0x7fc224160310 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc21c003d30 con 0x7fc224160310 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2240731e0 con 0x7fc224160310 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.849+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21c0052d0 con 0x7fc224160310 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.850+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2240737a0 con 0x7fc224160310 2024-09-21T07:54:38.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.851+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22406a7e0 con 0x7fc224160310 2024-09-21T07:54:38.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.852+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fc21c005680 con 0x7fc224160310 2024-09-21T07:54:38.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.853+0000 7fc21aa00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc200079430 0x7fc20007b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:38.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.853+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(155..155 src has 1..155) ==== 5737+0+0 (secure 0 0 0) 0x7fc21c09bdd0 con 0x7fc224160310 2024-09-21T07:54:38.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.854+0000 7fc221e00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc200079430 0x7fc20007b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:38.858 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.854+0000 7fc221e00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc200079430 0x7fc20007b940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc214009b30 tx=0x7fc214005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:38.860 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:38.858+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc22406a7e0 con 0x7fc224160310 2024-09-21T07:54:39.048 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.046+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc22415e2a0 con 0x7fc224160310 2024-09-21T07:54:39.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.047+0000 7fc21aa00640 1 -- 172.21.15.88:0/1824076625 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) ==== 74+0+16752 (secure 0 0 0) 0x7fc22415e2a0 con 0x7fc224160310 2024-09-21T07:54:39.050 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:39.050 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":155,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:37.615613+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:35.511240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:36.571979+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.049+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc200079430 msgr2=0x7fc20007b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.049+0000 7fc228c00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc200079430 0x7fc20007b940 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fc214009b30 tx=0x7fc214005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 msgr2=0x7fc224153e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 0x7fc224153e40 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00c730 tx=0x7fc21c00cbf0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 shutdown_connections 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc200079430 0x7fc20007b940 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc224160310 0x7fc224153e40 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22415d760 0x7fc2241f8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 --2- 172.21.15.88:0/1824076625 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc2240ad0e0 0x7fc224156020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 >> 172.21.15.88:0/1824076625 conn(0x7fc22406d730 msgr2=0x7fc22415e4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:39.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 shutdown_connections 2024-09-21T07:54:39.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.050+0000 7fc228c00640 1 -- 172.21.15.88:0/1824076625 wait complete. 2024-09-21T07:54:39.078 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:54:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 -- 172.21.15.88:0/2207211593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff0145550 msgr2=0x7fcff006fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 --2- 172.21.15.88:0/2207211593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff0145550 0x7fcff006fb40 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4005fd0 tx=0x7fcfe402ab20 comp rx=0 tx=0).stop 2024-09-21T07:54:39.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 -- 172.21.15.88:0/2207211593 shutdown_connections 2024-09-21T07:54:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 --2- 172.21.15.88:0/2207211593 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff0073c50 0x7fcff00cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 --2- 172.21.15.88:0/2207211593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff0145550 0x7fcff006fb40 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 -- 172.21.15.88:0/2207211593 >> 172.21.15.88:0/2207211593 conn(0x7fcff006d1c0 msgr2=0x7fcff006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 -- 172.21.15.88:0/2207211593 shutdown_connections 2024-09-21T07:54:39.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.302+0000 7fcff8400640 1 -- 172.21.15.88:0/2207211593 wait complete. 2024-09-21T07:54:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.304+0000 7fcff8400640 1 Processor -- start 2024-09-21T07:54:39.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.304+0000 7fcff8400640 1 -- start start 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff8400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff0145550 0x7fcff01f65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff8400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff0145a40 0x7fcff015bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff8400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 0x7fcff01f9450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff8400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00ce100 con 0x7fcff01f9040 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff8400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcff00cdf80 con 0x7fcff0145a40 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff8400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcff00ce280 con 0x7fcff0145550 2024-09-21T07:54:39.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 0x7fcff01f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 0x7fcff01f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37106/0 (socket says 172.21.15.88:37106) 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 -- 172.21.15.88:0/794927381 learned_addr learned my addr 172.21.15.88:0/794927381 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff5a00640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff0145550 0x7fcff01f65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff5000640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff0145a40 0x7fcff015bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 -- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff0145550 msgr2=0x7fcff01f65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff0145550 0x7fcff01f65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 -- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff0145a40 msgr2=0x7fcff015bf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:39.310 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff0145a40 0x7fcff015bf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff6400640 1 -- 172.21.15.88:0/794927381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcff01f9ac0 con 0x7fcff01f9040 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.306+0000 7fcff5000640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff0145a40 0x7fcff015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.307+0000 7fcff6400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 0x7fcff01f9450 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800d5f0 tx=0x7fcfe800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.307+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe8017070 con 0x7fcff01f9040 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.307+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcfe8005330 con 0x7fcff01f9040 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.307+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcff01f9da0 con 0x7fcff01f9040 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.307+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcfe80047a0 con 0x7fcff01f9040 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.307+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcff01f6080 con 0x7fcff01f9040 2024-09-21T07:54:39.311 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.308+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcff00ce7a0 con 0x7fcff01f9040 2024-09-21T07:54:39.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.309+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fcfe8004940 con 0x7fcff01f9040 2024-09-21T07:54:39.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.310+0000 7fcfed600640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcfd0079400 0x7fcfd007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:39.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.310+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fcfe809c0d0 con 0x7fcff01f9040 2024-09-21T07:54:39.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.311+0000 7fcff5a00640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcfd0079400 0x7fcfd007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:39.313 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.311+0000 7fcff5a00640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcfd0079400 0x7fcfd007b910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4035000 tx=0x7fcfe4004ab0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:39.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.315+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcfe80634c0 con 0x7fcff01f9040 2024-09-21T07:54:39.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:39.539+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fcff00aeb30 con 0x7fcff01f9040 2024-09-21T07:54:40.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.143+0000 7fcfed600640 1 -- 172.21.15.88:0/794927381 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v107) ==== 118+0+0 (secure 0 0 0) 0x7fcfe8062dc0 con 0x7fcff01f9040 2024-09-21T07:54:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.145+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcfd0079400 msgr2=0x7fcfd007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcfd0079400 0x7fcfd007b910 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcfe4035000 tx=0x7fcfe4004ab0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 msgr2=0x7fcff01f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 0x7fcff01f9450 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800d5f0 tx=0x7fcfe800dab0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 shutdown_connections 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcfd0079400 0x7fcfd007b910 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcff01f9040 0x7fcff01f9450 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcff0145a40 0x7fcff015bf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 --2- 172.21.15.88:0/794927381 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcff0145550 0x7fcff01f65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 >> 172.21.15.88:0/794927381 conn(0x7fcff006d1c0 msgr2=0x7fcff00af090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 shutdown_connections 2024-09-21T07:54:40.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.146+0000 7fcff8400640 1 -- 172.21.15.88:0/794927381 wait complete. 2024-09-21T07:54:40.175 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:54:40.175 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 -- 172.21.15.88:0/2740920089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8145a40 msgr2=0x7fdcc8073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 --2- 172.21.15.88:0/2740920089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8145a40 0x7fdcc8073540 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fdcc81464e0 tx=0x7fdcb002a5f0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 -- 172.21.15.88:0/2740920089 shutdown_connections 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 --2- 172.21.15.88:0/2740920089 >> v2:172.21.15.175:3300/0 conn(0x7fdcc8073a80 0x7fdcc80cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 --2- 172.21.15.88:0/2740920089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8145a40 0x7fdcc8073540 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 -- 172.21.15.88:0/2740920089 >> 172.21.15.88:0/2740920089 conn(0x7fdcc806d1c0 msgr2=0x7fdcc806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:40.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 -- 172.21.15.88:0/2740920089 shutdown_connections 2024-09-21T07:54:40.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.398+0000 7fdccd600640 1 -- 172.21.15.88:0/2740920089 wait complete. 2024-09-21T07:54:40.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.401+0000 7fdccd600640 1 Processor -- start 2024-09-21T07:54:40.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.401+0000 7fdccd600640 1 -- start start 2024-09-21T07:54:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.402+0000 7fdccd600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 0x7fdcc81f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.402+0000 7fdccd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcc8145550 0x7fdcc81409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.402+0000 7fdccd600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcc8145a40 0x7fdcc80aeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.402+0000 7fdccd600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc80ce0e0 con 0x7fdcc8073a80 2024-09-21T07:54:40.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.402+0000 7fdccd600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdcc80cdf60 con 0x7fdcc8145a40 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.402+0000 7fdccd600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdcc80ce260 con 0x7fdcc8145550 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 0x7fdcc81f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 0x7fdcc81f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37130/0 (socket says 172.21.15.88:37130) 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 -- 172.21.15.88:0/3587257396 learned_addr learned my addr 172.21.15.88:0/3587257396 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcbe000640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcc8145550 0x7fdcc81409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc7400640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcc8145a40 0x7fdcc80aeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 -- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcc8145550 msgr2=0x7fdcc81409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcc8145550 0x7fdcc81409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 -- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcc8145a40 msgr2=0x7fdcc80aeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcc8145a40 0x7fdcc80aeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:40.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 -- 172.21.15.88:0/3587257396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdcc81fa020 con 0x7fdcc8073a80 2024-09-21T07:54:40.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.403+0000 7fdcc6a00640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 0x7fdcc81f9160 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800b3f0 tx=0x7fdcb800b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:40.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.404+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8015020 con 0x7fdcc8073a80 2024-09-21T07:54:40.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.404+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdcb8011920 con 0x7fdcc8073a80 2024-09-21T07:54:40.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.404+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcb8011d70 con 0x7fdcc8073a80 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.404+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdcc81f9d40 con 0x7fdcc8073a80 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.404+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdcc81e7e70 con 0x7fdcc8073a80 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.405+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdcc81e82b0 con 0x7fdcc8073a80 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.406+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fdcb8018020 con 0x7fdcc8073a80 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.408+0000 7fdcbf400640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca4079430 0x7fdca407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.408+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fdcb809bb50 con 0x7fdcc8073a80 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.408+0000 7fdcbe000640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca4079430 0x7fdca407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:40.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.409+0000 7fdcbe000640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca4079430 0x7fdca407b940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0033000 tx=0x7fdcb0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:40.414 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.412+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdcb801f030 con 0x7fdcc8073a80 2024-09-21T07:54:40.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:40.635+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fdcc80ce7a0 con 0x7fdcc8073a80 2024-09-21T07:54:41.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.145+0000 7fdcbf400640 1 -- 172.21.15.88:0/3587257396 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v108) ==== 122+0+0 (secure 0 0 0) 0x7fdcb8062980 con 0x7fdcc8073a80 2024-09-21T07:54:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca4079430 msgr2=0x7fdca407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca4079430 0x7fdca407b940 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0033000 tx=0x7fdcb0004730 comp rx=0 tx=0).stop 2024-09-21T07:54:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 msgr2=0x7fdcc81f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 0x7fdcc81f9160 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fdcb800b3f0 tx=0x7fdcb800b8b0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 shutdown_connections 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdca4079430 0x7fdca407b940 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdcc8145550 0x7fdcc81409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdcc8145a40 0x7fdcc80aeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 --2- 172.21.15.88:0/3587257396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdcc8073a80 0x7fdcc81f9160 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 >> 172.21.15.88:0/3587257396 conn(0x7fdcc806d1c0 msgr2=0x7fdcc8144100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 shutdown_connections 2024-09-21T07:54:41.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.148+0000 7fdccd600640 1 -- 172.21.15.88:0/3587257396 wait complete. 2024-09-21T07:54:41.178 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:54:41.178 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 -- 172.21.15.88:0/1990895420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 msgr2=0x7f09500ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 --2- 172.21.15.88:0/1990895420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09500ad4f0 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f0950072d90 tx=0x7f093c02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 -- 172.21.15.88:0/1990895420 shutdown_connections 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 --2- 172.21.15.88:0/1990895420 >> v2:172.21.15.175:3300/0 conn(0x7f0950160310 0x7f0950160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 --2- 172.21.15.88:0/1990895420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09500ad4f0 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 --2- 172.21.15.88:0/1990895420 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f095015d760 0x7f09501535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 -- 172.21.15.88:0/1990895420 >> 172.21.15.88:0/1990895420 conn(0x7f095006d730 msgr2=0x7f09500b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:41.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 -- 172.21.15.88:0/1990895420 shutdown_connections 2024-09-21T07:54:41.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.395+0000 7f0957600640 1 -- 172.21.15.88:0/1990895420 wait complete. 2024-09-21T07:54:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.398+0000 7f0957600640 1 Processor -- start 2024-09-21T07:54:41.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.398+0000 7f0957600640 1 -- start start 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0957600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09501518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0957600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f095015d760 0x7f095015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0957600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0950160310 0x7f09500b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0957600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f09500b4da0 con 0x7f09500ad0e0 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0957600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09500b4c20 con 0x7f095015d760 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0957600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0950160c60 con 0x7f0950160310 2024-09-21T07:54:41.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.399+0000 7f0954c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09501518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09501518f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37142/0 (socket says 172.21.15.88:37142) 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 -- 172.21.15.88:0/3027110976 learned_addr learned my addr 172.21.15.88:0/3027110976 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0955600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0950160310 0x7f09500b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f094fe00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f095015d760 0x7f095015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 -- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0950160310 msgr2=0x7f09500b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0950160310 0x7f09500b34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 -- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f095015d760 msgr2=0x7f095015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f095015d760 0x7f095015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 -- 172.21.15.88:0/3027110976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09501f5bb0 con 0x7f09500ad0e0 2024-09-21T07:54:41.403 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f0954c00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09501518f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f094400d6d0 tx=0x7f094400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.400+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944017070 con 0x7f09500ad0e0 2024-09-21T07:54:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.401+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0944005330 con 0x7f09500ad0e0 2024-09-21T07:54:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.401+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09501f5e90 con 0x7f09500ad0e0 2024-09-21T07:54:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.401+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0944009820 con 0x7f09500ad0e0 2024-09-21T07:54:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.401+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09501f6450 con 0x7f09500ad0e0 2024-09-21T07:54:41.404 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.402+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09501535c0 con 0x7f09500ad0e0 2024-09-21T07:54:41.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.403+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f0944009bd0 con 0x7f09500ad0e0 2024-09-21T07:54:41.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.404+0000 7f094cc00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0930079400 0x7f093007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.404+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f094409c400 con 0x7f09500ad0e0 2024-09-21T07:54:41.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.404+0000 7f094fe00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0930079400 0x7f093007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.405+0000 7f094fe00640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0930079400 0x7f093007b910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f093c033000 tx=0x7f093c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:41.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.409+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f09440637f0 con 0x7f09500ad0e0 2024-09-21T07:54:41.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.630+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f095015fcb0 con 0x7f09500ad0e0 2024-09-21T07:54:41.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.631+0000 7f094cc00640 1 -- 172.21.15.88:0/3027110976 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5239 (secure 0 0 0) 0x7f09440630f0 con 0x7f09500ad0e0 2024-09-21T07:54:41.633 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:41.634 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":108,"btime":"2024-09-21T07:54:41:144560+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:37.615557+0000","modified":"2024-09-21T07:54:40.637260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8772":{"gid":8772,"name":"b","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2687096264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2687096264},{"type":"v1","addr":"172.21.15.175:6835","nonce":2687096264}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8772,"qdb_cluster":[8772]},"id":12}]} 2024-09-21T07:54:41.634 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 108 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0930079400 msgr2=0x7f093007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0930079400 0x7f093007b910 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f093c033000 tx=0x7f093c004730 comp rx=0 tx=0).stop 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 msgr2=0x7f09501518f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09501518f0 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f094400d6d0 tx=0x7f094400db90 comp rx=0 tx=0).stop 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 shutdown_connections 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0930079400 0x7f093007b910 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0950160310 0x7f09500b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f095015d760 0x7f095015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 --2- 172.21.15.88:0/3027110976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f09500ad0e0 0x7f09501518f0 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 >> 172.21.15.88:0/3027110976 conn(0x7f095006d730 msgr2=0x7f095015a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 shutdown_connections 2024-09-21T07:54:41.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.634+0000 7f0957600640 1 -- 172.21.15.88:0/3027110976 wait complete. 2024-09-21T07:54:41.658 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:54:41.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 -- 172.21.15.88:0/243137718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c015ee30 msgr2=0x7f21c00b2840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 --2- 172.21.15.88:0/243137718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c015ee30 0x7f21c00b2840 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f21b0009830 tx=0x7f21b002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 -- 172.21.15.88:0/243137718 shutdown_connections 2024-09-21T07:54:41.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 --2- 172.21.15.88:0/243137718 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c00b5410 0x7f21c00b5820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 --2- 172.21.15.88:0/243137718 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c015ee30 0x7f21c00b2840 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 -- 172.21.15.88:0/243137718 >> 172.21.15.88:0/243137718 conn(0x7f21c00b2fd0 msgr2=0x7f21c00b33e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:41.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 -- 172.21.15.88:0/243137718 shutdown_connections 2024-09-21T07:54:41.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.877+0000 7f21c6400640 1 -- 172.21.15.88:0/243137718 wait complete. 2024-09-21T07:54:41.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.879+0000 7f21c6400640 1 Processor -- start 2024-09-21T07:54:41.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.879+0000 7f21c6400640 1 -- start start 2024-09-21T07:54:41.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21c6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c00b5410 0x7f21c01f6140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21c6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c00cf7c0 0x7f21c01f36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21c6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 0x7f21c01f3c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21c6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21c00b6150 con 0x7f21c015ee30 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21c6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21c00b5fd0 con 0x7f21c00cf7c0 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21c6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21c00b62d0 con 0x7f21c00b5410 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 0x7f21c01f3c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21bfe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 0x7f21c01f3c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37152/0 (socket says 172.21.15.88:37152) 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21bfe00640 1 -- 172.21.15.88:0/4051611 learned_addr learned my addr 172.21.15.88:0/4051611 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21bf400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c00b5410 0x7f21c01f6140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.881+0000 7f21bea00640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c00cf7c0 0x7f21c01f36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21bfe00640 1 -- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c00b5410 msgr2=0x7f21c01f6140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21bfe00640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c00b5410 0x7f21c01f6140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21bfe00640 1 -- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c00cf7c0 msgr2=0x7f21c01f36f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21bfe00640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c00cf7c0 0x7f21c01f36f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:41.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21bfe00640 1 -- 172.21.15.88:0/4051611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21c009d440 con 0x7f21c015ee30 2024-09-21T07:54:41.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21bfe00640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 0x7f21c01f3c30 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f21b800ea70 tx=0x7f21b800ef30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:41.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b800cde0 con 0x7f21c015ee30 2024-09-21T07:54:41.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f21b8004c10 con 0x7f21c015ee30 2024-09-21T07:54:41.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21c009d700 con 0x7f21c015ee30 2024-09-21T07:54:41.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21b8005020 con 0x7f21c015ee30 2024-09-21T07:54:41.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.882+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21c009dcc0 con 0x7f21c015ee30 2024-09-21T07:54:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.883+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21c00b67e0 con 0x7f21c015ee30 2024-09-21T07:54:41.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.885+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f21b80051c0 con 0x7f21c015ee30 2024-09-21T07:54:41.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.885+0000 7f21b7400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f21a0079400 0x7f21a007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:41.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.886+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f21b809cd00 con 0x7f21c015ee30 2024-09-21T07:54:41.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.886+0000 7f21bf400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f21a0079400 0x7f21a007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:41.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.886+0000 7f21bf400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f21a0079400 0x7f21a007b910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f21b0034000 tx=0x7f21b0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:41.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:41.890+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21b8063a20 con 0x7f21c015ee30 2024-09-21T07:54:42.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.076+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f21c01f41d0 con 0x7f21c015ee30 2024-09-21T07:54:42.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.078+0000 7f21b7400640 1 -- 172.21.15.88:0/4051611 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16739 (secure 0 0 0) 0x7f21b8068450 con 0x7f21c015ee30 2024-09-21T07:54:42.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:42.080 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":156,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:39.123067+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:35.511240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:36.571979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:54:42.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f21a0079400 msgr2=0x7f21a007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f21a0079400 0x7f21a007b910 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f21b0034000 tx=0x7f21b0004730 comp rx=0 tx=0).stop 2024-09-21T07:54:42.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 msgr2=0x7f21c01f3c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 0x7f21c01f3c30 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f21b800ea70 tx=0x7f21b800ef30 comp rx=0 tx=0).stop 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 shutdown_connections 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f21a0079400 0x7f21a007b910 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21c015ee30 0x7f21c01f3c30 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f21c00cf7c0 0x7f21c01f36f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 --2- 172.21.15.88:0/4051611 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21c00b5410 0x7f21c01f6140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 >> 172.21.15.88:0/4051611 conn(0x7f21c00b2fd0 msgr2=0x7f21c00ce000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 shutdown_connections 2024-09-21T07:54:42.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.080+0000 7f21c6400640 1 -- 172.21.15.88:0/4051611 wait complete. 2024-09-21T07:54:42.109 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:54:42.109 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:54:42.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.330+0000 7f272d600640 1 -- 172.21.15.88:0/3974890492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 msgr2=0x7f27280b5fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.330+0000 7f272d600640 1 --2- 172.21.15.88:0/3974890492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f27280b5fa0 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f27280d2010 tx=0x7f271402a560 comp rx=0 tx=0).stop 2024-09-21T07:54:42.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.330+0000 7f272d600640 1 -- 172.21.15.88:0/3974890492 shutdown_connections 2024-09-21T07:54:42.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.330+0000 7f272d600640 1 --2- 172.21.15.88:0/3974890492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f27280b5fa0 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.330+0000 7f272d600640 1 --2- 172.21.15.88:0/3974890492 >> v2:172.21.15.175:3300/0 conn(0x7f27280b3820 0x7f27280cf1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.330+0000 7f272d600640 1 -- 172.21.15.88:0/3974890492 >> 172.21.15.88:0/3974890492 conn(0x7f272809d8a0 msgr2=0x7f272808bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.331+0000 7f272d600640 1 -- 172.21.15.88:0/3974890492 shutdown_connections 2024-09-21T07:54:42.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.331+0000 7f272d600640 1 -- 172.21.15.88:0/3974890492 wait complete. 2024-09-21T07:54:42.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.333+0000 7f272d600640 1 Processor -- start 2024-09-21T07:54:42.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.333+0000 7f272d600640 1 -- start start 2024-09-21T07:54:42.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.334+0000 7f272d600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27280b3820 0x7f27280a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.334+0000 7f272d600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27280b64e0 0x7f2728161f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.334+0000 7f272d600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f2728158b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.334+0000 7f272d600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f27280b3ef0 con 0x7f27280cf6f0 2024-09-21T07:54:42.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.334+0000 7f272d600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f27280b4740 con 0x7f27280b64e0 2024-09-21T07:54:42.337 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.334+0000 7f272d600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f27280b4070 con 0x7f27280b3820 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f2728158b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f2728158b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37176/0 (socket says 172.21.15.88:37176) 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 -- 172.21.15.88:0/3952307092 learned_addr learned my addr 172.21.15.88:0/3952307092 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f2727e00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27280b3820 0x7f27280a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f2727400640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27280b64e0 0x7f2728161f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 -- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27280b3820 msgr2=0x7f27280a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27280b3820 0x7f27280a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 -- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27280b64e0 msgr2=0x7f2728161f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27280b64e0 0x7f2728161f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 -- 172.21.15.88:0/3952307092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27280a1b50 con 0x7f27280cf6f0 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f2727e00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27280b3820 0x7f27280a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:42.338 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f272cc00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f2728158b80 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f272000b440 tx=0x7f272000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:42.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720011830 con 0x7f27280cf6f0 2024-09-21T07:54:42.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2720016070 con 0x7f27280cf6f0 2024-09-21T07:54:42.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.335+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2720011d60 con 0x7f27280cf6f0 2024-09-21T07:54:42.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.336+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27280b2670 con 0x7f27280cf6f0 2024-09-21T07:54:42.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.336+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27280a20f0 con 0x7f27280cf6f0 2024-09-21T07:54:42.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.337+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27280cf1b0 con 0x7f27280cf6f0 2024-09-21T07:54:42.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.338+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f272001b460 con 0x7f27280cf6f0 2024-09-21T07:54:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.339+0000 7f271fe00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2708079400 0x7f270807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.339+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f2720098e60 con 0x7f27280cf6f0 2024-09-21T07:54:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.339+0000 7f2727e00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2708079400 0x7f270807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.340+0000 7f2727e00640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2708079400 0x7f270807b910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2718005fd0 tx=0x7f2718009480 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:42.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.343+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2720020020 con 0x7f27280cf6f0 2024-09-21T07:54:42.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.526+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f27280b23c0 con 0x7f2708079400 2024-09-21T07:54:42.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.530+0000 7f271fe00640 1 -- 172.21.15.88:0/3952307092 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7f27280b23c0 con 0x7f2708079400 2024-09-21T07:54:42.535 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:42.536 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:54:42.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2708079400 msgr2=0x7f270807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2708079400 0x7f270807b910 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f2718005fd0 tx=0x7f2718009480 comp rx=0 tx=0).stop 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 msgr2=0x7f2728158b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f2728158b80 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f272000b440 tx=0x7f272000b900 comp rx=0 tx=0).stop 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 shutdown_connections 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2708079400 0x7f270807b910 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f27280cf6f0 0x7f2728158b80 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f27280b64e0 0x7f2728161f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 --2- 172.21.15.88:0/3952307092 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f27280b3820 0x7f27280a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 >> 172.21.15.88:0/3952307092 conn(0x7f272809d8a0 msgr2=0x7f272806ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 shutdown_connections 2024-09-21T07:54:42.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.534+0000 7f272d600640 1 -- 172.21.15.88:0/3952307092 wait complete. 2024-09-21T07:54:42.562 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-21T07:54:41.748627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265996,"kb_used_data":16488,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708724,"statfs":{"total":773094113280,"available":772821733376,"internally_reserved":0,"allocated":16883712,"data_stored":8097636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.645433"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306312+0000","last_change":"2024-09-21T07:54:38.153070+0000","last_active":"2024-09-21T07:54:40.306312+0000","last_peered":"2024-09-21T07:54:40.306312+0000","last_clean":"2024-09-21T07:54:40.306312+0000","last_became_active":"2024-09-21T07:54:38.152727+0000","last_became_peered":"2024-09-21T07:54:38.152727+0000","last_unstale":"2024-09-21T07:54:40.306312+0000","last_undegraded":"2024-09-21T07:54:40.306312+0000","last_fullsized":"2024-09-21T07:54:40.306312+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:22.881250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131911+0000","last_change":"2024-09-21T07:54:38.144114+0000","last_active":"2024-09-21T07:54:39.131911+0000","last_peered":"2024-09-21T07:54:39.131911+0000","last_clean":"2024-09-21T07:54:39.131911+0000","last_became_active":"2024-09-21T07:54:38.143708+0000","last_became_peered":"2024-09-21T07:54:38.143708+0000","last_unstale":"2024-09-21T07:54:39.131911+0000","last_undegraded":"2024-09-21T07:54:39.131911+0000","last_fullsized":"2024-09-21T07:54:39.131911+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:42.560345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131738+0000","last_change":"2024-09-21T07:54:38.149178+0000","last_active":"2024-09-21T07:54:39.131738+0000","last_peered":"2024-09-21T07:54:39.131738+0000","last_clean":"2024-09-21T07:54:39.131738+0000","last_became_active":"2024-09-21T07:54:38.148866+0000","last_became_peered":"2024-09-21T07:54:38.148866+0000","last_unstale":"2024-09-21T07:54:39.131738+0000","last_undegraded":"2024-09-21T07:54:39.131738+0000","last_fullsized":"2024-09-21T07:54:39.131738+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:38.209311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131413+0000","last_change":"2024-09-21T07:54:38.131670+0000","last_active":"2024-09-21T07:54:39.131413+0000","last_peered":"2024-09-21T07:54:39.131413+0000","last_clean":"2024-09-21T07:54:39.131413+0000","last_became_active":"2024-09-21T07:54:38.131324+0000","last_became_peered":"2024-09-21T07:54:38.131324+0000","last_unstale":"2024-09-21T07:54:39.131413+0000","last_undegraded":"2024-09-21T07:54:39.131413+0000","last_fullsized":"2024-09-21T07:54:39.131413+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:18.765934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306585+0000","last_change":"2024-09-21T07:54:38.150970+0000","last_active":"2024-09-21T07:54:40.306585+0000","last_peered":"2024-09-21T07:54:40.306585+0000","last_clean":"2024-09-21T07:54:40.306585+0000","last_became_active":"2024-09-21T07:54:38.149199+0000","last_became_peered":"2024-09-21T07:54:38.149199+0000","last_unstale":"2024-09-21T07:54:40.306585+0000","last_undegraded":"2024-09-21T07:54:40.306585+0000","last_fullsized":"2024-09-21T07:54:40.306585+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:53.385435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130497+0000","last_change":"2024-09-21T07:54:38.153020+0000","last_active":"2024-09-21T07:54:39.130497+0000","last_peered":"2024-09-21T07:54:39.130497+0000","last_clean":"2024-09-21T07:54:39.130497+0000","last_became_active":"2024-09-21T07:54:38.152667+0000","last_became_peered":"2024-09-21T07:54:38.152667+0000","last_unstale":"2024-09-21T07:54:39.130497+0000","last_undegraded":"2024-09-21T07:54:39.130497+0000","last_fullsized":"2024-09-21T07:54:39.130497+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:42.230433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668499+0000","last_change":"2024-09-21T07:54:38.160117+0000","last_active":"2024-09-21T07:54:39.668499+0000","last_peered":"2024-09-21T07:54:39.668499+0000","last_clean":"2024-09-21T07:54:39.668499+0000","last_became_active":"2024-09-21T07:54:38.158818+0000","last_became_peered":"2024-09-21T07:54:38.158818+0000","last_unstale":"2024-09-21T07:54:39.668499+0000","last_undegraded":"2024-09-21T07:54:39.668499+0000","last_fullsized":"2024-09-21T07:54:39.668499+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:24.216221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306142+0000","last_change":"2024-09-21T07:54:38.149754+0000","last_active":"2024-09-21T07:54:40.306142+0000","last_peered":"2024-09-21T07:54:40.306142+0000","last_clean":"2024-09-21T07:54:40.306142+0000","last_became_active":"2024-09-21T07:54:38.149418+0000","last_became_peered":"2024-09-21T07:54:38.149418+0000","last_unstale":"2024-09-21T07:54:40.306142+0000","last_undegraded":"2024-09-21T07:54:40.306142+0000","last_fullsized":"2024-09-21T07:54:40.306142+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:28.647134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131894+0000","last_change":"2024-09-21T07:54:38.139255+0000","last_active":"2024-09-21T07:54:39.131894+0000","last_peered":"2024-09-21T07:54:39.131894+0000","last_clean":"2024-09-21T07:54:39.131894+0000","last_became_active":"2024-09-21T07:54:38.138941+0000","last_became_peered":"2024-09-21T07:54:38.138941+0000","last_unstale":"2024-09-21T07:54:39.131894+0000","last_undegraded":"2024-09-21T07:54:39.131894+0000","last_fullsized":"2024-09-21T07:54:39.131894+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:59.186075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151079+0000","last_change":"2024-09-21T07:54:38.147966+0000","last_active":"2024-09-21T07:54:39.151079+0000","last_peered":"2024-09-21T07:54:39.151079+0000","last_clean":"2024-09-21T07:54:39.151079+0000","last_became_active":"2024-09-21T07:54:38.147677+0000","last_became_peered":"2024-09-21T07:54:38.147677+0000","last_unstale":"2024-09-21T07:54:39.151079+0000","last_undegraded":"2024-09-21T07:54:39.151079+0000","last_fullsized":"2024-09-21T07:54:39.151079+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:40.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305718+0000","last_change":"2024-09-21T07:54:38.139613+0000","last_active":"2024-09-21T07:54:40.305718+0000","last_peered":"2024-09-21T07:54:40.305718+0000","last_clean":"2024-09-21T07:54:40.305718+0000","last_became_active":"2024-09-21T07:54:38.139299+0000","last_became_peered":"2024-09-21T07:54:38.139299+0000","last_unstale":"2024-09-21T07:54:40.305718+0000","last_undegraded":"2024-09-21T07:54:40.305718+0000","last_fullsized":"2024-09-21T07:54:40.305718+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:39.915497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306633+0000","last_change":"2024-09-21T07:54:38.146080+0000","last_active":"2024-09-21T07:54:40.306633+0000","last_peered":"2024-09-21T07:54:40.306633+0000","last_clean":"2024-09-21T07:54:40.306633+0000","last_became_active":"2024-09-21T07:54:38.145679+0000","last_became_peered":"2024-09-21T07:54:38.145679+0000","last_unstale":"2024-09-21T07:54:40.306633+0000","last_undegraded":"2024-09-21T07:54:40.306633+0000","last_fullsized":"2024-09-21T07:54:40.306633+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:22.323738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305749+0000","last_change":"2024-09-21T07:54:38.163477+0000","last_active":"2024-09-21T07:54:40.305749+0000","last_peered":"2024-09-21T07:54:40.305749+0000","last_clean":"2024-09-21T07:54:40.305749+0000","last_became_active":"2024-09-21T07:54:38.162282+0000","last_became_peered":"2024-09-21T07:54:38.162282+0000","last_unstale":"2024-09-21T07:54:40.305749+0000","last_undegraded":"2024-09-21T07:54:40.305749+0000","last_fullsized":"2024-09-21T07:54:40.305749+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:12.146623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130921+0000","last_change":"2024-09-21T07:54:38.162130+0000","last_active":"2024-09-21T07:54:39.130921+0000","last_peered":"2024-09-21T07:54:39.130921+0000","last_clean":"2024-09-21T07:54:39.130921+0000","last_became_active":"2024-09-21T07:54:38.161793+0000","last_became_peered":"2024-09-21T07:54:38.161793+0000","last_unstale":"2024-09-21T07:54:39.130921+0000","last_undegraded":"2024-09-21T07:54:39.130921+0000","last_fullsized":"2024-09-21T07:54:39.130921+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:38.161052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305706+0000","last_change":"2024-09-21T07:54:38.142040+0000","last_active":"2024-09-21T07:54:40.305706+0000","last_peered":"2024-09-21T07:54:40.305706+0000","last_clean":"2024-09-21T07:54:40.305706+0000","last_became_active":"2024-09-21T07:54:38.141726+0000","last_became_peered":"2024-09-21T07:54:38.141726+0000","last_unstale":"2024-09-21T07:54:40.305706+0000","last_undegraded":"2024-09-21T07:54:40.305706+0000","last_fullsized":"2024-09-21T07:54:40.305706+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:28.438022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130470+0000","last_change":"2024-09-21T07:54:38.149280+0000","last_active":"2024-09-21T07:54:39.130470+0000","last_peered":"2024-09-21T07:54:39.130470+0000","last_clean":"2024-09-21T07:54:39.130470+0000","last_became_active":"2024-09-21T07:54:38.148914+0000","last_became_peered":"2024-09-21T07:54:38.148914+0000","last_unstale":"2024-09-21T07:54:39.130470+0000","last_undegraded":"2024-09-21T07:54:39.130470+0000","last_fullsized":"2024-09-21T07:54:39.130470+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:22.323738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131491+0000","last_change":"2024-09-21T07:54:38.152328+0000","last_active":"2024-09-21T07:54:39.131491+0000","last_peered":"2024-09-21T07:54:39.131491+0000","last_clean":"2024-09-21T07:54:39.131491+0000","last_became_active":"2024-09-21T07:54:38.151987+0000","last_became_peered":"2024-09-21T07:54:38.151987+0000","last_unstale":"2024-09-21T07:54:39.131491+0000","last_undegraded":"2024-09-21T07:54:39.131491+0000","last_fullsized":"2024-09-21T07:54:39.131491+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:38.642387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668877+0000","last_change":"2024-09-21T07:54:38.157545+0000","last_active":"2024-09-21T07:54:39.668877+0000","last_peered":"2024-09-21T07:54:39.668877+0000","last_clean":"2024-09-21T07:54:39.668877+0000","last_became_active":"2024-09-21T07:54:38.156766+0000","last_became_peered":"2024-09-21T07:54:38.156766+0000","last_unstale":"2024-09-21T07:54:39.668877+0000","last_undegraded":"2024-09-21T07:54:39.668877+0000","last_fullsized":"2024-09-21T07:54:39.668877+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:25.386463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131727+0000","last_change":"2024-09-21T07:54:38.141136+0000","last_active":"2024-09-21T07:54:39.131727+0000","last_peered":"2024-09-21T07:54:39.131727+0000","last_clean":"2024-09-21T07:54:39.131727+0000","last_became_active":"2024-09-21T07:54:38.140534+0000","last_became_peered":"2024-09-21T07:54:38.140534+0000","last_unstale":"2024-09-21T07:54:39.131727+0000","last_undegraded":"2024-09-21T07:54:39.131727+0000","last_fullsized":"2024-09-21T07:54:39.131727+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:02.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.307126+0000","last_change":"2024-09-21T07:54:38.151914+0000","last_active":"2024-09-21T07:54:40.307126+0000","last_peered":"2024-09-21T07:54:40.307126+0000","last_clean":"2024-09-21T07:54:40.307126+0000","last_became_active":"2024-09-21T07:54:38.150867+0000","last_became_peered":"2024-09-21T07:54:38.150867+0000","last_unstale":"2024-09-21T07:54:40.307126+0000","last_undegraded":"2024-09-21T07:54:40.307126+0000","last_fullsized":"2024-09-21T07:54:40.307126+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:06.914429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.132027+0000","last_change":"2024-09-21T07:54:38.139731+0000","last_active":"2024-09-21T07:54:39.132027+0000","last_peered":"2024-09-21T07:54:39.132027+0000","last_clean":"2024-09-21T07:54:39.132027+0000","last_became_active":"2024-09-21T07:54:38.139251+0000","last_became_peered":"2024-09-21T07:54:38.139251+0000","last_unstale":"2024-09-21T07:54:39.132027+0000","last_undegraded":"2024-09-21T07:54:39.132027+0000","last_fullsized":"2024-09-21T07:54:39.132027+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:26.715211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306751+0000","last_change":"2024-09-21T07:54:38.150981+0000","last_active":"2024-09-21T07:54:40.306751+0000","last_peered":"2024-09-21T07:54:40.306751+0000","last_clean":"2024-09-21T07:54:40.306751+0000","last_became_active":"2024-09-21T07:54:38.150575+0000","last_became_peered":"2024-09-21T07:54:38.150575+0000","last_unstale":"2024-09-21T07:54:40.306751+0000","last_undegraded":"2024-09-21T07:54:40.306751+0000","last_fullsized":"2024-09-21T07:54:40.306751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:53.965649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150601+0000","last_change":"2024-09-21T07:54:38.154970+0000","last_active":"2024-09-21T07:54:39.150601+0000","last_peered":"2024-09-21T07:54:39.150601+0000","last_clean":"2024-09-21T07:54:39.150601+0000","last_became_active":"2024-09-21T07:54:38.154489+0000","last_became_peered":"2024-09-21T07:54:38.154489+0000","last_unstale":"2024-09-21T07:54:39.150601+0000","last_undegraded":"2024-09-21T07:54:39.150601+0000","last_fullsized":"2024-09-21T07:54:39.150601+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.336560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306378+0000","last_change":"2024-09-21T07:54:38.157797+0000","last_active":"2024-09-21T07:54:40.306378+0000","last_peered":"2024-09-21T07:54:40.306378+0000","last_clean":"2024-09-21T07:54:40.306378+0000","last_became_active":"2024-09-21T07:54:38.157479+0000","last_became_peered":"2024-09-21T07:54:38.157479+0000","last_unstale":"2024-09-21T07:54:40.306378+0000","last_undegraded":"2024-09-21T07:54:40.306378+0000","last_fullsized":"2024-09-21T07:54:40.306378+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:29.297002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131940+0000","last_change":"2024-09-21T07:54:38.144342+0000","last_active":"2024-09-21T07:54:39.131940+0000","last_peered":"2024-09-21T07:54:39.131940+0000","last_clean":"2024-09-21T07:54:39.131940+0000","last_became_active":"2024-09-21T07:54:38.143997+0000","last_became_peered":"2024-09-21T07:54:38.143997+0000","last_unstale":"2024-09-21T07:54:39.131940+0000","last_undegraded":"2024-09-21T07:54:39.131940+0000","last_fullsized":"2024-09-21T07:54:39.131940+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:48.774837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306983+0000","last_change":"2024-09-21T07:54:38.152045+0000","last_active":"2024-09-21T07:54:40.306983+0000","last_peered":"2024-09-21T07:54:40.306983+0000","last_clean":"2024-09-21T07:54:40.306983+0000","last_became_active":"2024-09-21T07:54:38.151298+0000","last_became_peered":"2024-09-21T07:54:38.151298+0000","last_unstale":"2024-09-21T07:54:40.306983+0000","last_undegraded":"2024-09-21T07:54:40.306983+0000","last_fullsized":"2024-09-21T07:54:40.306983+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:08.959152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.132137+0000","last_change":"2024-09-21T07:54:38.140829+0000","last_active":"2024-09-21T07:54:39.132137+0000","last_peered":"2024-09-21T07:54:39.132137+0000","last_clean":"2024-09-21T07:54:39.132137+0000","last_became_active":"2024-09-21T07:54:38.140053+0000","last_became_peered":"2024-09-21T07:54:38.140053+0000","last_unstale":"2024-09-21T07:54:39.132137+0000","last_undegraded":"2024-09-21T07:54:39.132137+0000","last_fullsized":"2024-09-21T07:54:39.132137+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:36.608157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150822+0000","last_change":"2024-09-21T07:54:38.159089+0000","last_active":"2024-09-21T07:54:39.150822+0000","last_peered":"2024-09-21T07:54:39.150822+0000","last_clean":"2024-09-21T07:54:39.150822+0000","last_became_active":"2024-09-21T07:54:38.158787+0000","last_became_peered":"2024-09-21T07:54:38.158787+0000","last_unstale":"2024-09-21T07:54:39.150822+0000","last_undegraded":"2024-09-21T07:54:39.150822+0000","last_fullsized":"2024-09-21T07:54:39.150822+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:42.115154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306335+0000","last_change":"2024-09-21T07:54:38.164509+0000","last_active":"2024-09-21T07:54:40.306335+0000","last_peered":"2024-09-21T07:54:40.306335+0000","last_clean":"2024-09-21T07:54:40.306335+0000","last_became_active":"2024-09-21T07:54:38.164209+0000","last_became_peered":"2024-09-21T07:54:38.164209+0000","last_unstale":"2024-09-21T07:54:40.306335+0000","last_undegraded":"2024-09-21T07:54:40.306335+0000","last_fullsized":"2024-09-21T07:54:40.306335+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:53.236650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306018+0000","last_change":"2024-09-21T07:54:38.150799+0000","last_active":"2024-09-21T07:54:40.306018+0000","last_peered":"2024-09-21T07:54:40.306018+0000","last_clean":"2024-09-21T07:54:40.306018+0000","last_became_active":"2024-09-21T07:54:38.150482+0000","last_became_peered":"2024-09-21T07:54:38.150482+0000","last_unstale":"2024-09-21T07:54:40.306018+0000","last_undegraded":"2024-09-21T07:54:40.306018+0000","last_fullsized":"2024-09-21T07:54:40.306018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.992461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306280+0000","last_change":"2024-09-21T07:54:38.153097+0000","last_active":"2024-09-21T07:54:40.306280+0000","last_peered":"2024-09-21T07:54:40.306280+0000","last_clean":"2024-09-21T07:54:40.306280+0000","last_became_active":"2024-09-21T07:54:38.150127+0000","last_became_peered":"2024-09-21T07:54:38.150127+0000","last_unstale":"2024-09-21T07:54:40.306280+0000","last_undegraded":"2024-09-21T07:54:40.306280+0000","last_fullsized":"2024-09-21T07:54:40.306280+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:20.059867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306757+0000","last_change":"2024-09-21T07:54:38.148967+0000","last_active":"2024-09-21T07:54:40.306757+0000","last_peered":"2024-09-21T07:54:40.306757+0000","last_clean":"2024-09-21T07:54:40.306757+0000","last_became_active":"2024-09-21T07:54:38.148433+0000","last_became_peered":"2024-09-21T07:54:38.148433+0000","last_unstale":"2024-09-21T07:54:40.306757+0000","last_undegraded":"2024-09-21T07:54:40.306757+0000","last_fullsized":"2024-09-21T07:54:40.306757+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:41.967021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306867+0000","last_change":"2024-09-21T07:54:38.152379+0000","last_active":"2024-09-21T07:54:40.306867+0000","last_peered":"2024-09-21T07:54:40.306867+0000","last_clean":"2024-09-21T07:54:40.306867+0000","last_became_active":"2024-09-21T07:54:38.151619+0000","last_became_peered":"2024-09-21T07:54:38.151619+0000","last_unstale":"2024-09-21T07:54:40.306867+0000","last_undegraded":"2024-09-21T07:54:40.306867+0000","last_fullsized":"2024-09-21T07:54:40.306867+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:55.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"155'4","reported_seq":19,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131816+0000","last_change":"2024-09-21T07:54:37.131993+0000","last_active":"2024-09-21T07:54:39.131816+0000","last_peered":"2024-09-21T07:54:39.131816+0000","last_clean":"2024-09-21T07:54:39.131816+0000","last_became_active":"2024-09-21T07:54:37.131638+0000","last_became_peered":"2024-09-21T07:54:37.131638+0000","last_unstale":"2024-09-21T07:54:39.131816+0000","last_undegraded":"2024-09-21T07:54:39.131816+0000","last_fullsized":"2024-09-21T07:54:39.131816+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:45.561827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151202+0000","last_change":"2024-09-21T07:54:38.155149+0000","last_active":"2024-09-21T07:54:39.151202+0000","last_peered":"2024-09-21T07:54:39.151202+0000","last_clean":"2024-09-21T07:54:39.151202+0000","last_became_active":"2024-09-21T07:54:38.154694+0000","last_became_peered":"2024-09-21T07:54:38.154694+0000","last_unstale":"2024-09-21T07:54:39.151202+0000","last_undegraded":"2024-09-21T07:54:39.151202+0000","last_fullsized":"2024-09-21T07:54:39.151202+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:55.461398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306860+0000","last_change":"2024-09-21T07:54:37.135270+0000","last_active":"2024-09-21T07:54:40.306860+0000","last_peered":"2024-09-21T07:54:40.306860+0000","last_clean":"2024-09-21T07:54:40.306860+0000","last_became_active":"2024-09-21T07:54:37.134872+0000","last_became_peered":"2024-09-21T07:54:37.134872+0000","last_unstale":"2024-09-21T07:54:40.306860+0000","last_undegraded":"2024-09-21T07:54:40.306860+0000","last_fullsized":"2024-09-21T07:54:40.306860+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:22.659485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131479+0000","last_change":"2024-09-21T07:54:38.137428+0000","last_active":"2024-09-21T07:54:39.131479+0000","last_peered":"2024-09-21T07:54:39.131479+0000","last_clean":"2024-09-21T07:54:39.131479+0000","last_became_active":"2024-09-21T07:54:38.137122+0000","last_became_peered":"2024-09-21T07:54:38.137122+0000","last_unstale":"2024-09-21T07:54:39.131479+0000","last_undegraded":"2024-09-21T07:54:39.131479+0000","last_fullsized":"2024-09-21T07:54:39.131479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.635432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306986+0000","last_change":"2024-09-21T07:54:37.134668+0000","last_active":"2024-09-21T07:54:40.306986+0000","last_peered":"2024-09-21T07:54:40.306986+0000","last_clean":"2024-09-21T07:54:40.306986+0000","last_became_active":"2024-09-21T07:54:37.134021+0000","last_became_peered":"2024-09-21T07:54:37.134021+0000","last_unstale":"2024-09-21T07:54:40.306986+0000","last_undegraded":"2024-09-21T07:54:40.306986+0000","last_fullsized":"2024-09-21T07:54:40.306986+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:58.511339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131596+0000","last_change":"2024-09-21T07:54:38.149306+0000","last_active":"2024-09-21T07:54:39.131596+0000","last_peered":"2024-09-21T07:54:39.131596+0000","last_clean":"2024-09-21T07:54:39.131596+0000","last_became_active":"2024-09-21T07:54:38.149009+0000","last_became_peered":"2024-09-21T07:54:38.149009+0000","last_unstale":"2024-09-21T07:54:39.131596+0000","last_undegraded":"2024-09-21T07:54:39.131596+0000","last_fullsized":"2024-09-21T07:54:39.131596+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:34.035726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306706+0000","last_change":"2024-09-21T07:54:37.135243+0000","last_active":"2024-09-21T07:54:40.306706+0000","last_peered":"2024-09-21T07:54:40.306706+0000","last_clean":"2024-09-21T07:54:40.306706+0000","last_became_active":"2024-09-21T07:54:37.134873+0000","last_became_peered":"2024-09-21T07:54:37.134873+0000","last_unstale":"2024-09-21T07:54:40.306706+0000","last_undegraded":"2024-09-21T07:54:40.306706+0000","last_fullsized":"2024-09-21T07:54:40.306706+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:17.615584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130717+0000","last_change":"2024-09-21T07:54:38.147610+0000","last_active":"2024-09-21T07:54:39.130717+0000","last_peered":"2024-09-21T07:54:39.130717+0000","last_clean":"2024-09-21T07:54:39.130717+0000","last_became_active":"2024-09-21T07:54:38.147304+0000","last_became_peered":"2024-09-21T07:54:38.147304+0000","last_unstale":"2024-09-21T07:54:39.130717+0000","last_undegraded":"2024-09-21T07:54:39.130717+0000","last_fullsized":"2024-09-21T07:54:39.130717+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:46.051760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306675+0000","last_change":"2024-09-21T07:54:37.135154+0000","last_active":"2024-09-21T07:54:40.306675+0000","last_peered":"2024-09-21T07:54:40.306675+0000","last_clean":"2024-09-21T07:54:40.306675+0000","last_became_active":"2024-09-21T07:54:37.134823+0000","last_became_peered":"2024-09-21T07:54:37.134823+0000","last_unstale":"2024-09-21T07:54:40.306675+0000","last_undegraded":"2024-09-21T07:54:40.306675+0000","last_fullsized":"2024-09-21T07:54:40.306675+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:56.547024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306551+0000","last_change":"2024-09-21T07:54:38.153790+0000","last_active":"2024-09-21T07:54:40.306551+0000","last_peered":"2024-09-21T07:54:40.306551+0000","last_clean":"2024-09-21T07:54:40.306551+0000","last_became_active":"2024-09-21T07:54:38.153395+0000","last_became_peered":"2024-09-21T07:54:38.153395+0000","last_unstale":"2024-09-21T07:54:40.306551+0000","last_undegraded":"2024-09-21T07:54:40.306551+0000","last_fullsized":"2024-09-21T07:54:40.306551+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:01.727107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306849+0000","last_change":"2024-09-21T07:54:37.133071+0000","last_active":"2024-09-21T07:54:40.306849+0000","last_peered":"2024-09-21T07:54:40.306849+0000","last_clean":"2024-09-21T07:54:40.306849+0000","last_became_active":"2024-09-21T07:54:37.132745+0000","last_became_peered":"2024-09-21T07:54:37.132745+0000","last_unstale":"2024-09-21T07:54:40.306849+0000","last_undegraded":"2024-09-21T07:54:40.306849+0000","last_fullsized":"2024-09-21T07:54:40.306849+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:35.793778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305916+0000","last_change":"2024-09-21T07:54:38.153390+0000","last_active":"2024-09-21T07:54:40.305916+0000","last_peered":"2024-09-21T07:54:40.305916+0000","last_clean":"2024-09-21T07:54:40.305916+0000","last_became_active":"2024-09-21T07:54:38.153073+0000","last_became_peered":"2024-09-21T07:54:38.153073+0000","last_unstale":"2024-09-21T07:54:40.305916+0000","last_undegraded":"2024-09-21T07:54:40.305916+0000","last_fullsized":"2024-09-21T07:54:40.305916+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:44:42.708593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668289+0000","last_change":"2024-09-21T07:54:37.122721+0000","last_active":"2024-09-21T07:54:39.668289+0000","last_peered":"2024-09-21T07:54:39.668289+0000","last_clean":"2024-09-21T07:54:39.668289+0000","last_became_active":"2024-09-21T07:54:37.122435+0000","last_became_peered":"2024-09-21T07:54:37.122435+0000","last_unstale":"2024-09-21T07:54:39.668289+0000","last_undegraded":"2024-09-21T07:54:39.668289+0000","last_fullsized":"2024-09-21T07:54:39.668289+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:11.575862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131657+0000","last_change":"2024-09-21T07:54:38.133751+0000","last_active":"2024-09-21T07:54:39.131657+0000","last_peered":"2024-09-21T07:54:39.131657+0000","last_clean":"2024-09-21T07:54:39.131657+0000","last_became_active":"2024-09-21T07:54:38.133274+0000","last_became_peered":"2024-09-21T07:54:38.133274+0000","last_unstale":"2024-09-21T07:54:39.131657+0000","last_undegraded":"2024-09-21T07:54:39.131657+0000","last_fullsized":"2024-09-21T07:54:39.131657+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:30.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306136+0000","last_change":"2024-09-21T07:54:37.121640+0000","last_active":"2024-09-21T07:54:40.306136+0000","last_peered":"2024-09-21T07:54:40.306136+0000","last_clean":"2024-09-21T07:54:40.306136+0000","last_became_active":"2024-09-21T07:54:37.121329+0000","last_became_peered":"2024-09-21T07:54:37.121329+0000","last_unstale":"2024-09-21T07:54:40.306136+0000","last_undegraded":"2024-09-21T07:54:40.306136+0000","last_fullsized":"2024-09-21T07:54:40.306136+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:23.948484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151298+0000","last_change":"2024-09-21T07:54:38.149137+0000","last_active":"2024-09-21T07:54:39.151298+0000","last_peered":"2024-09-21T07:54:39.151298+0000","last_clean":"2024-09-21T07:54:39.151298+0000","last_became_active":"2024-09-21T07:54:38.148823+0000","last_became_peered":"2024-09-21T07:54:38.148823+0000","last_unstale":"2024-09-21T07:54:39.151298+0000","last_undegraded":"2024-09-21T07:54:39.151298+0000","last_fullsized":"2024-09-21T07:54:39.151298+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:48.093733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151136+0000","last_change":"2024-09-21T07:54:37.133534+0000","last_active":"2024-09-21T07:54:39.151136+0000","last_peered":"2024-09-21T07:54:39.151136+0000","last_clean":"2024-09-21T07:54:39.151136+0000","last_became_active":"2024-09-21T07:54:37.133170+0000","last_became_peered":"2024-09-21T07:54:37.133170+0000","last_unstale":"2024-09-21T07:54:39.151136+0000","last_undegraded":"2024-09-21T07:54:39.151136+0000","last_fullsized":"2024-09-21T07:54:39.151136+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:37.992768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306711+0000","last_change":"2024-09-21T07:54:38.150645+0000","last_active":"2024-09-21T07:54:40.306711+0000","last_peered":"2024-09-21T07:54:40.306711+0000","last_clean":"2024-09-21T07:54:40.306711+0000","last_became_active":"2024-09-21T07:54:38.150309+0000","last_became_peered":"2024-09-21T07:54:38.150309+0000","last_unstale":"2024-09-21T07:54:40.306711+0000","last_undegraded":"2024-09-21T07:54:40.306711+0000","last_fullsized":"2024-09-21T07:54:40.306711+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:35.907070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"155'3","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.307090+0000","last_change":"2024-09-21T07:54:37.132095+0000","last_active":"2024-09-21T07:54:40.307090+0000","last_peered":"2024-09-21T07:54:40.307090+0000","last_clean":"2024-09-21T07:54:40.307090+0000","last_became_active":"2024-09-21T07:54:37.129665+0000","last_became_peered":"2024-09-21T07:54:37.129665+0000","last_unstale":"2024-09-21T07:54:40.307090+0000","last_undegraded":"2024-09-21T07:54:40.307090+0000","last_fullsized":"2024-09-21T07:54:40.307090+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:14.742504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151139+0000","last_change":"2024-09-21T07:54:38.153345+0000","last_active":"2024-09-21T07:54:39.151139+0000","last_peered":"2024-09-21T07:54:39.151139+0000","last_clean":"2024-09-21T07:54:39.151139+0000","last_became_active":"2024-09-21T07:54:38.151739+0000","last_became_peered":"2024-09-21T07:54:38.151739+0000","last_unstale":"2024-09-21T07:54:39.151139+0000","last_undegraded":"2024-09-21T07:54:39.151139+0000","last_fullsized":"2024-09-21T07:54:39.151139+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:46.307506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"155'6","reported_seq":21,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306824+0000","last_change":"2024-09-21T07:54:37.128556+0000","last_active":"2024-09-21T07:54:40.306824+0000","last_peered":"2024-09-21T07:54:40.306824+0000","last_clean":"2024-09-21T07:54:40.306824+0000","last_became_active":"2024-09-21T07:54:37.128229+0000","last_became_peered":"2024-09-21T07:54:37.128229+0000","last_unstale":"2024-09-21T07:54:40.306824+0000","last_undegraded":"2024-09-21T07:54:40.306824+0000","last_fullsized":"2024-09-21T07:54:40.306824+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:34.080920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150670+0000","last_change":"2024-09-21T07:54:38.161941+0000","last_active":"2024-09-21T07:54:39.150670+0000","last_peered":"2024-09-21T07:54:39.150670+0000","last_clean":"2024-09-21T07:54:39.150670+0000","last_became_active":"2024-09-21T07:54:38.161602+0000","last_became_peered":"2024-09-21T07:54:38.161602+0000","last_unstale":"2024-09-21T07:54:39.150670+0000","last_undegraded":"2024-09-21T07:54:39.150670+0000","last_fullsized":"2024-09-21T07:54:39.150670+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:32.786830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306254+0000","last_change":"2024-09-21T07:54:37.134012+0000","last_active":"2024-09-21T07:54:40.306254+0000","last_peered":"2024-09-21T07:54:40.306254+0000","last_clean":"2024-09-21T07:54:40.306254+0000","last_became_active":"2024-09-21T07:54:37.133019+0000","last_became_peered":"2024-09-21T07:54:37.133019+0000","last_unstale":"2024-09-21T07:54:40.306254+0000","last_undegraded":"2024-09-21T07:54:40.306254+0000","last_fullsized":"2024-09-21T07:54:40.306254+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:25.575716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668697+0000","last_change":"2024-09-21T07:54:38.157653+0000","last_active":"2024-09-21T07:54:39.668697+0000","last_peered":"2024-09-21T07:54:39.668697+0000","last_clean":"2024-09-21T07:54:39.668697+0000","last_became_active":"2024-09-21T07:54:38.157326+0000","last_became_peered":"2024-09-21T07:54:38.157326+0000","last_unstale":"2024-09-21T07:54:39.668697+0000","last_undegraded":"2024-09-21T07:54:39.668697+0000","last_fullsized":"2024-09-21T07:54:39.668697+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.842999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150668+0000","last_change":"2024-09-21T07:54:37.131349+0000","last_active":"2024-09-21T07:54:39.150668+0000","last_peered":"2024-09-21T07:54:39.150668+0000","last_clean":"2024-09-21T07:54:39.150668+0000","last_became_active":"2024-09-21T07:54:37.130898+0000","last_became_peered":"2024-09-21T07:54:37.130898+0000","last_unstale":"2024-09-21T07:54:39.150668+0000","last_undegraded":"2024-09-21T07:54:39.150668+0000","last_fullsized":"2024-09-21T07:54:39.150668+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:09.118262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305828+0000","last_change":"2024-09-21T07:54:38.149498+0000","last_active":"2024-09-21T07:54:40.305828+0000","last_peered":"2024-09-21T07:54:40.305828+0000","last_clean":"2024-09-21T07:54:40.305828+0000","last_became_active":"2024-09-21T07:54:38.148830+0000","last_became_peered":"2024-09-21T07:54:38.148830+0000","last_unstale":"2024-09-21T07:54:40.305828+0000","last_undegraded":"2024-09-21T07:54:40.305828+0000","last_fullsized":"2024-09-21T07:54:40.305828+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:48.203972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306617+0000","last_change":"2024-09-21T07:54:37.125794+0000","last_active":"2024-09-21T07:54:40.306617+0000","last_peered":"2024-09-21T07:54:40.306617+0000","last_clean":"2024-09-21T07:54:40.306617+0000","last_became_active":"2024-09-21T07:54:37.125448+0000","last_became_peered":"2024-09-21T07:54:37.125448+0000","last_unstale":"2024-09-21T07:54:40.306617+0000","last_undegraded":"2024-09-21T07:54:40.306617+0000","last_fullsized":"2024-09-21T07:54:40.306617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:25.229013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306353+0000","last_change":"2024-09-21T07:54:38.153327+0000","last_active":"2024-09-21T07:54:40.306353+0000","last_peered":"2024-09-21T07:54:40.306353+0000","last_clean":"2024-09-21T07:54:40.306353+0000","last_became_active":"2024-09-21T07:54:38.150147+0000","last_became_peered":"2024-09-21T07:54:38.150147+0000","last_unstale":"2024-09-21T07:54:40.306353+0000","last_undegraded":"2024-09-21T07:54:40.306353+0000","last_fullsized":"2024-09-21T07:54:40.306353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:41.142553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"155'8","reported_seq":23,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668367+0000","last_change":"2024-09-21T07:54:37.126507+0000","last_active":"2024-09-21T07:54:39.668367+0000","last_peered":"2024-09-21T07:54:39.668367+0000","last_clean":"2024-09-21T07:54:39.668367+0000","last_became_active":"2024-09-21T07:54:37.126176+0000","last_became_peered":"2024-09-21T07:54:37.126176+0000","last_unstale":"2024-09-21T07:54:39.668367+0000","last_undegraded":"2024-09-21T07:54:39.668367+0000","last_fullsized":"2024-09-21T07:54:39.668367+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:55.077145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151201+0000","last_change":"2024-09-21T07:54:38.154969+0000","last_active":"2024-09-21T07:54:39.151201+0000","last_peered":"2024-09-21T07:54:39.151201+0000","last_clean":"2024-09-21T07:54:39.151201+0000","last_became_active":"2024-09-21T07:54:38.154601+0000","last_became_peered":"2024-09-21T07:54:38.154601+0000","last_unstale":"2024-09-21T07:54:39.151201+0000","last_undegraded":"2024-09-21T07:54:39.151201+0000","last_fullsized":"2024-09-21T07:54:39.151201+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:02.422056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306967+0000","last_change":"2024-09-21T07:54:37.136015+0000","last_active":"2024-09-21T07:54:40.306967+0000","last_peered":"2024-09-21T07:54:40.306967+0000","last_clean":"2024-09-21T07:54:40.306967+0000","last_became_active":"2024-09-21T07:54:37.134841+0000","last_became_peered":"2024-09-21T07:54:37.134841+0000","last_unstale":"2024-09-21T07:54:40.306967+0000","last_undegraded":"2024-09-21T07:54:40.306967+0000","last_fullsized":"2024-09-21T07:54:40.306967+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:02.465129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306846+0000","last_change":"2024-09-21T07:54:38.154353+0000","last_active":"2024-09-21T07:54:40.306846+0000","last_peered":"2024-09-21T07:54:40.306846+0000","last_clean":"2024-09-21T07:54:40.306846+0000","last_became_active":"2024-09-21T07:54:38.153572+0000","last_became_peered":"2024-09-21T07:54:38.153572+0000","last_unstale":"2024-09-21T07:54:40.306846+0000","last_undegraded":"2024-09-21T07:54:40.306846+0000","last_fullsized":"2024-09-21T07:54:40.306846+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:38.209311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131245+0000","last_change":"2024-09-21T07:54:37.124563+0000","last_active":"2024-09-21T07:54:39.131245+0000","last_peered":"2024-09-21T07:54:39.131245+0000","last_clean":"2024-09-21T07:54:39.131245+0000","last_became_active":"2024-09-21T07:54:37.124222+0000","last_became_peered":"2024-09-21T07:54:37.124222+0000","last_unstale":"2024-09-21T07:54:39.131245+0000","last_undegraded":"2024-09-21T07:54:39.131245+0000","last_fullsized":"2024-09-21T07:54:39.131245+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:10.737349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131583+0000","last_change":"2024-09-21T07:54:38.140483+0000","last_active":"2024-09-21T07:54:39.131583+0000","last_peered":"2024-09-21T07:54:39.131583+0000","last_clean":"2024-09-21T07:54:39.131583+0000","last_became_active":"2024-09-21T07:54:38.138949+0000","last_became_peered":"2024-09-21T07:54:38.138949+0000","last_unstale":"2024-09-21T07:54:39.131583+0000","last_undegraded":"2024-09-21T07:54:39.131583+0000","last_fullsized":"2024-09-21T07:54:39.131583+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:31.419481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305799+0000","last_change":"2024-09-21T07:54:37.126665+0000","last_active":"2024-09-21T07:54:40.305799+0000","last_peered":"2024-09-21T07:54:40.305799+0000","last_clean":"2024-09-21T07:54:40.305799+0000","last_became_active":"2024-09-21T07:54:37.126361+0000","last_became_peered":"2024-09-21T07:54:37.126361+0000","last_unstale":"2024-09-21T07:54:40.305799+0000","last_undegraded":"2024-09-21T07:54:40.305799+0000","last_fullsized":"2024-09-21T07:54:40.305799+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:45.045525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306751+0000","last_change":"2024-09-21T07:54:38.145421+0000","last_active":"2024-09-21T07:54:40.306751+0000","last_peered":"2024-09-21T07:54:40.306751+0000","last_clean":"2024-09-21T07:54:40.306751+0000","last_became_active":"2024-09-21T07:54:38.145102+0000","last_became_peered":"2024-09-21T07:54:38.145102+0000","last_unstale":"2024-09-21T07:54:40.306751+0000","last_undegraded":"2024-09-21T07:54:40.306751+0000","last_fullsized":"2024-09-21T07:54:40.306751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:30.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306120+0000","last_change":"2024-09-21T07:54:37.124101+0000","last_active":"2024-09-21T07:54:40.306120+0000","last_peered":"2024-09-21T07:54:40.306120+0000","last_clean":"2024-09-21T07:54:40.306120+0000","last_became_active":"2024-09-21T07:54:37.123761+0000","last_became_peered":"2024-09-21T07:54:37.123761+0000","last_unstale":"2024-09-21T07:54:40.306120+0000","last_undegraded":"2024-09-21T07:54:40.306120+0000","last_fullsized":"2024-09-21T07:54:40.306120+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:15.281819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130943+0000","last_change":"2024-09-21T07:54:38.137283+0000","last_active":"2024-09-21T07:54:39.130943+0000","last_peered":"2024-09-21T07:54:39.130943+0000","last_clean":"2024-09-21T07:54:39.130943+0000","last_became_active":"2024-09-21T07:54:38.136700+0000","last_became_peered":"2024-09-21T07:54:38.136700+0000","last_unstale":"2024-09-21T07:54:39.130943+0000","last_undegraded":"2024-09-21T07:54:39.130943+0000","last_fullsized":"2024-09-21T07:54:39.130943+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:45.557399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306731+0000","last_change":"2024-09-21T07:54:37.130376+0000","last_active":"2024-09-21T07:54:40.306731+0000","last_peered":"2024-09-21T07:54:40.306731+0000","last_clean":"2024-09-21T07:54:40.306731+0000","last_became_active":"2024-09-21T07:54:37.129106+0000","last_became_peered":"2024-09-21T07:54:37.129106+0000","last_unstale":"2024-09-21T07:54:40.306731+0000","last_undegraded":"2024-09-21T07:54:40.306731+0000","last_fullsized":"2024-09-21T07:54:40.306731+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:27.509055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668611+0000","last_change":"2024-09-21T07:54:38.163017+0000","last_active":"2024-09-21T07:54:39.668611+0000","last_peered":"2024-09-21T07:54:39.668611+0000","last_clean":"2024-09-21T07:54:39.668611+0000","last_became_active":"2024-09-21T07:54:38.162690+0000","last_became_peered":"2024-09-21T07:54:38.162690+0000","last_unstale":"2024-09-21T07:54:39.668611+0000","last_undegraded":"2024-09-21T07:54:39.668611+0000","last_fullsized":"2024-09-21T07:54:39.668611+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:39.799812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130889+0000","last_change":"2024-09-21T07:54:37.128878+0000","last_active":"2024-09-21T07:54:39.130889+0000","last_peered":"2024-09-21T07:54:39.130889+0000","last_clean":"2024-09-21T07:54:39.130889+0000","last_became_active":"2024-09-21T07:54:37.128563+0000","last_became_peered":"2024-09-21T07:54:37.128563+0000","last_unstale":"2024-09-21T07:54:39.130889+0000","last_undegraded":"2024-09-21T07:54:39.130889+0000","last_fullsized":"2024-09-21T07:54:39.130889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:12.039826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306188+0000","last_change":"2024-09-21T07:54:38.136244+0000","last_active":"2024-09-21T07:54:40.306188+0000","last_peered":"2024-09-21T07:54:40.306188+0000","last_clean":"2024-09-21T07:54:40.306188+0000","last_became_active":"2024-09-21T07:54:38.135922+0000","last_became_peered":"2024-09-21T07:54:38.135922+0000","last_unstale":"2024-09-21T07:54:40.306188+0000","last_undegraded":"2024-09-21T07:54:40.306188+0000","last_fullsized":"2024-09-21T07:54:40.306188+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:13.175919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130471+0000","last_change":"2024-09-21T07:54:37.132479+0000","last_active":"2024-09-21T07:54:39.130471+0000","last_peered":"2024-09-21T07:54:39.130471+0000","last_clean":"2024-09-21T07:54:39.130471+0000","last_became_active":"2024-09-21T07:54:37.132145+0000","last_became_peered":"2024-09-21T07:54:37.132145+0000","last_unstale":"2024-09-21T07:54:39.130471+0000","last_undegraded":"2024-09-21T07:54:39.130471+0000","last_fullsized":"2024-09-21T07:54:39.130471+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:02.830384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150621+0000","last_change":"2024-09-21T07:54:38.143894+0000","last_active":"2024-09-21T07:54:39.150621+0000","last_peered":"2024-09-21T07:54:39.150621+0000","last_clean":"2024-09-21T07:54:39.150621+0000","last_became_active":"2024-09-21T07:54:38.143567+0000","last_became_peered":"2024-09-21T07:54:38.143567+0000","last_unstale":"2024-09-21T07:54:39.150621+0000","last_undegraded":"2024-09-21T07:54:39.150621+0000","last_fullsized":"2024-09-21T07:54:39.150621+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:06.423489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305489+0000","last_change":"2024-09-21T07:54:37.117418+0000","last_active":"2024-09-21T07:54:40.305489+0000","last_peered":"2024-09-21T07:54:40.305489+0000","last_clean":"2024-09-21T07:54:40.305489+0000","last_became_active":"2024-09-21T07:54:37.117096+0000","last_became_peered":"2024-09-21T07:54:37.117096+0000","last_unstale":"2024-09-21T07:54:40.305489+0000","last_undegraded":"2024-09-21T07:54:40.305489+0000","last_fullsized":"2024-09-21T07:54:40.305489+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:08.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131977+0000","last_change":"2024-09-21T07:54:38.142681+0000","last_active":"2024-09-21T07:54:39.131977+0000","last_peered":"2024-09-21T07:54:39.131977+0000","last_clean":"2024-09-21T07:54:39.131977+0000","last_became_active":"2024-09-21T07:54:38.142398+0000","last_became_peered":"2024-09-21T07:54:38.142398+0000","last_unstale":"2024-09-21T07:54:39.131977+0000","last_undegraded":"2024-09-21T07:54:39.131977+0000","last_fullsized":"2024-09-21T07:54:39.131977+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:11.602913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306245+0000","last_change":"2024-09-21T07:54:37.123390+0000","last_active":"2024-09-21T07:54:40.306245+0000","last_peered":"2024-09-21T07:54:40.306245+0000","last_clean":"2024-09-21T07:54:40.306245+0000","last_became_active":"2024-09-21T07:54:37.123079+0000","last_became_peered":"2024-09-21T07:54:37.123079+0000","last_unstale":"2024-09-21T07:54:40.306245+0000","last_undegraded":"2024-09-21T07:54:40.306245+0000","last_fullsized":"2024-09-21T07:54:40.306245+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:03.037587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.132051+0000","last_change":"2024-09-21T07:54:38.134768+0000","last_active":"2024-09-21T07:54:39.132051+0000","last_peered":"2024-09-21T07:54:39.132051+0000","last_clean":"2024-09-21T07:54:39.132051+0000","last_became_active":"2024-09-21T07:54:38.134376+0000","last_became_peered":"2024-09-21T07:54:38.134376+0000","last_unstale":"2024-09-21T07:54:39.132051+0000","last_undegraded":"2024-09-21T07:54:39.132051+0000","last_fullsized":"2024-09-21T07:54:39.132051+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:57.154703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"156'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131883+0000","last_change":"2024-09-21T07:54:37.136890+0000","last_active":"2024-09-21T07:54:39.131883+0000","last_peered":"2024-09-21T07:54:39.131883+0000","last_clean":"2024-09-21T07:54:39.131883+0000","last_became_active":"2024-09-21T07:54:37.136565+0000","last_became_peered":"2024-09-21T07:54:37.136565+0000","last_unstale":"2024-09-21T07:54:39.131883+0000","last_undegraded":"2024-09-21T07:54:39.131883+0000","last_fullsized":"2024-09-21T07:54:39.131883+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:10.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131496+0000","last_change":"2024-09-21T07:54:38.156378+0000","last_active":"2024-09-21T07:54:39.131496+0000","last_peered":"2024-09-21T07:54:39.131496+0000","last_clean":"2024-09-21T07:54:39.131496+0000","last_became_active":"2024-09-21T07:54:38.154094+0000","last_became_peered":"2024-09-21T07:54:38.154094+0000","last_unstale":"2024-09-21T07:54:39.131496+0000","last_undegraded":"2024-09-21T07:54:39.131496+0000","last_fullsized":"2024-09-21T07:54:39.131496+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:35.907070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131974+0000","last_change":"2024-09-21T07:54:37.133682+0000","last_active":"2024-09-21T07:54:39.131974+0000","last_peered":"2024-09-21T07:54:39.131974+0000","last_clean":"2024-09-21T07:54:39.131974+0000","last_became_active":"2024-09-21T07:54:37.133379+0000","last_became_peered":"2024-09-21T07:54:37.133379+0000","last_unstale":"2024-09-21T07:54:39.131974+0000","last_undegraded":"2024-09-21T07:54:39.131974+0000","last_fullsized":"2024-09-21T07:54:39.131974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:19.669011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.307003+0000","last_change":"2024-09-21T07:54:38.157914+0000","last_active":"2024-09-21T07:54:40.307003+0000","last_peered":"2024-09-21T07:54:40.307003+0000","last_clean":"2024-09-21T07:54:40.307003+0000","last_became_active":"2024-09-21T07:54:38.155018+0000","last_became_peered":"2024-09-21T07:54:38.155018+0000","last_unstale":"2024-09-21T07:54:40.307003+0000","last_undegraded":"2024-09-21T07:54:40.307003+0000","last_fullsized":"2024-09-21T07:54:40.307003+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:53.770659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306981+0000","last_change":"2024-09-21T07:54:37.135924+0000","last_active":"2024-09-21T07:54:40.306981+0000","last_peered":"2024-09-21T07:54:40.306981+0000","last_clean":"2024-09-21T07:54:40.306981+0000","last_became_active":"2024-09-21T07:54:37.135615+0000","last_became_peered":"2024-09-21T07:54:37.135615+0000","last_unstale":"2024-09-21T07:54:40.306981+0000","last_undegraded":"2024-09-21T07:54:40.306981+0000","last_fullsized":"2024-09-21T07:54:40.306981+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:30.233548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130989+0000","last_change":"2024-09-21T07:54:38.153321+0000","last_active":"2024-09-21T07:54:39.130989+0000","last_peered":"2024-09-21T07:54:39.130989+0000","last_clean":"2024-09-21T07:54:39.130989+0000","last_became_active":"2024-09-21T07:54:38.153003+0000","last_became_peered":"2024-09-21T07:54:38.153003+0000","last_unstale":"2024-09-21T07:54:39.130989+0000","last_undegraded":"2024-09-21T07:54:39.130989+0000","last_fullsized":"2024-09-21T07:54:39.130989+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:17.846050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131186+0000","last_change":"2024-09-21T07:54:37.129295+0000","last_active":"2024-09-21T07:54:39.131186+0000","last_peered":"2024-09-21T07:54:39.131186+0000","last_clean":"2024-09-21T07:54:39.131186+0000","last_became_active":"2024-09-21T07:54:37.129006+0000","last_became_peered":"2024-09-21T07:54:37.129006+0000","last_unstale":"2024-09-21T07:54:39.131186+0000","last_undegraded":"2024-09-21T07:54:39.131186+0000","last_fullsized":"2024-09-21T07:54:39.131186+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:31.892583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305964+0000","last_change":"2024-09-21T07:54:38.150500+0000","last_active":"2024-09-21T07:54:40.305964+0000","last_peered":"2024-09-21T07:54:40.305964+0000","last_clean":"2024-09-21T07:54:40.305964+0000","last_became_active":"2024-09-21T07:54:38.150157+0000","last_became_peered":"2024-09-21T07:54:38.150157+0000","last_unstale":"2024-09-21T07:54:40.305964+0000","last_undegraded":"2024-09-21T07:54:40.305964+0000","last_fullsized":"2024-09-21T07:54:40.305964+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:02.433626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130996+0000","last_change":"2024-09-21T07:54:37.129999+0000","last_active":"2024-09-21T07:54:39.130996+0000","last_peered":"2024-09-21T07:54:39.130996+0000","last_clean":"2024-09-21T07:54:39.130996+0000","last_became_active":"2024-09-21T07:54:37.129610+0000","last_became_peered":"2024-09-21T07:54:37.129610+0000","last_unstale":"2024-09-21T07:54:39.130996+0000","last_undegraded":"2024-09-21T07:54:39.130996+0000","last_fullsized":"2024-09-21T07:54:39.130996+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:00.940821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130881+0000","last_change":"2024-09-21T07:54:38.144090+0000","last_active":"2024-09-21T07:54:39.130881+0000","last_peered":"2024-09-21T07:54:39.130881+0000","last_clean":"2024-09-21T07:54:39.130881+0000","last_became_active":"2024-09-21T07:54:38.143757+0000","last_became_peered":"2024-09-21T07:54:38.143757+0000","last_unstale":"2024-09-21T07:54:39.130881+0000","last_undegraded":"2024-09-21T07:54:39.130881+0000","last_fullsized":"2024-09-21T07:54:39.130881+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:12.810972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130732+0000","last_change":"2024-09-21T07:54:37.129918+0000","last_active":"2024-09-21T07:54:39.130732+0000","last_peered":"2024-09-21T07:54:39.130732+0000","last_clean":"2024-09-21T07:54:39.130732+0000","last_became_active":"2024-09-21T07:54:37.129621+0000","last_became_peered":"2024-09-21T07:54:37.129621+0000","last_unstale":"2024-09-21T07:54:39.130732+0000","last_undegraded":"2024-09-21T07:54:39.130732+0000","last_fullsized":"2024-09-21T07:54:39.130732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:33.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306886+0000","last_change":"2024-09-21T07:54:38.153942+0000","last_active":"2024-09-21T07:54:40.306886+0000","last_peered":"2024-09-21T07:54:40.306886+0000","last_clean":"2024-09-21T07:54:40.306886+0000","last_became_active":"2024-09-21T07:54:38.153570+0000","last_became_peered":"2024-09-21T07:54:38.153570+0000","last_unstale":"2024-09-21T07:54:40.306886+0000","last_undegraded":"2024-09-21T07:54:40.306886+0000","last_fullsized":"2024-09-21T07:54:40.306886+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:45.515792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130454+0000","last_change":"2024-09-21T07:54:37.128531+0000","last_active":"2024-09-21T07:54:39.130454+0000","last_peered":"2024-09-21T07:54:39.130454+0000","last_clean":"2024-09-21T07:54:39.130454+0000","last_became_active":"2024-09-21T07:54:37.127695+0000","last_became_peered":"2024-09-21T07:54:37.127695+0000","last_unstale":"2024-09-21T07:54:39.130454+0000","last_undegraded":"2024-09-21T07:54:39.130454+0000","last_fullsized":"2024-09-21T07:54:39.130454+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:14.889147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130711+0000","last_change":"2024-09-21T07:54:38.150376+0000","last_active":"2024-09-21T07:54:39.130711+0000","last_peered":"2024-09-21T07:54:39.130711+0000","last_clean":"2024-09-21T07:54:39.130711+0000","last_became_active":"2024-09-21T07:54:38.150069+0000","last_became_peered":"2024-09-21T07:54:38.150069+0000","last_unstale":"2024-09-21T07:54:39.130711+0000","last_undegraded":"2024-09-21T07:54:39.130711+0000","last_fullsized":"2024-09-21T07:54:39.130711+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:58.946337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305721+0000","last_change":"2024-09-21T07:54:37.130492+0000","last_active":"2024-09-21T07:54:40.305721+0000","last_peered":"2024-09-21T07:54:40.305721+0000","last_clean":"2024-09-21T07:54:40.305721+0000","last_became_active":"2024-09-21T07:54:37.130184+0000","last_became_peered":"2024-09-21T07:54:37.130184+0000","last_unstale":"2024-09-21T07:54:40.305721+0000","last_undegraded":"2024-09-21T07:54:40.305721+0000","last_fullsized":"2024-09-21T07:54:40.305721+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:07.369777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":392,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130717+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:54:39.130717+0000","last_peered":"2024-09-21T07:54:39.130717+0000","last_clean":"2024-09-21T07:54:39.130717+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:54:39.130717+0000","last_undegraded":"2024-09-21T07:54:39.130717+0000","last_fullsized":"2024-09-21T07:54:39.130717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542253,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1974272,"data_stored":865734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509548,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":864100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509549,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33684,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338156,"statfs":{"total":96636764160,"available":96602271744,"internally_reserved":0,"allocated":2555904,"data_stored":1454982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509549,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":864044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476847,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1970176,"data_stored":865692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476847,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":864080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476844,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33676,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338164,"statfs":{"total":96636764160,"available":96602279936,"internally_reserved":0,"allocated":2547712,"data_stored":1454378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476844,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1961984,"data_stored":864626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:54:42.566 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:54:42.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212f200640 1 -- 172.21.15.88:0/588730987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 msgr2=0x7f2114004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212f200640 1 --2- 172.21.15.88:0/588730987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f2114004c10 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f211c00c310 tx=0x7f2114005830 comp rx=0 tx=0).stop 2024-09-21T07:54:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212e800640 1 -- 172.21.15.88:0/588730987 reap_dead start 2024-09-21T07:54:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212f200640 1 -- 172.21.15.88:0/588730987 shutdown_connections 2024-09-21T07:54:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212f200640 1 -- 172.21.15.88:0/588730987 >> 172.21.15.88:0/588730987 conn(0x7f21300b1e40 msgr2=0x7f21300b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212f200640 1 -- 172.21.15.88:0/588730987 shutdown_connections 2024-09-21T07:54:42.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.821+0000 7f212f200640 1 -- 172.21.15.88:0/588730987 wait complete. 2024-09-21T07:54:42.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.823+0000 7f212f200640 1 Processor -- start 2024-09-21T07:54:42.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.823+0000 7f212f200640 1 -- start start 2024-09-21T07:54:42.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.824+0000 7f212f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f21301f9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.824+0000 7f212f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21300b1590 0x7f21301477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.824+0000 7f212f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213014d5f0 0x7f21300af960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.824+0000 7f212f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2130164540 con 0x7f2114004800 2024-09-21T07:54:42.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.824+0000 7f212f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f21301643c0 con 0x7f213014d5f0 2024-09-21T07:54:42.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.824+0000 7f212f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f21301646c0 con 0x7f21300b1590 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f21301f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f21301f9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37204/0 (socket says 172.21.15.88:37204) 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 -- 172.21.15.88:0/3306825054 learned_addr learned my addr 172.21.15.88:0/3306825054 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212d400640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21300b1590 0x7f21301477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212e800640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213014d5f0 0x7f21300af960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 -- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21300b1590 msgr2=0x7f21301477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.830 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21300b1590 0x7f21301477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 -- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213014d5f0 msgr2=0x7f21300af960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213014d5f0 0x7f21300af960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 -- 172.21.15.88:0/3306825054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21300b0e60 con 0x7f2114004800 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.825+0000 7f212de00640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f21301f9430 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21200018c0 tx=0x7f2120001d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.826+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2120015070 con 0x7f2114004800 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.826+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f212000ebb0 con 0x7f2114004800 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.826+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21200020c0 con 0x7f2114004800 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.826+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21300b1140 con 0x7f2114004800 2024-09-21T07:54:42.831 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.826+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21300b02e0 con 0x7f2114004800 2024-09-21T07:54:42.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.827+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2130070000 con 0x7f2114004800 2024-09-21T07:54:42.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.828+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f2120018070 con 0x7f2114004800 2024-09-21T07:54:42.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.829+0000 7f2126000640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2110079400 0x7f211007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:42.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.829+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f212009c930 con 0x7f2114004800 2024-09-21T07:54:42.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.829+0000 7f212d400640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2110079400 0x7f211007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:42.833 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.830+0000 7f212d400640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2110079400 0x7f211007b910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f211c033000 tx=0x7f211c038040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:42.835 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:42.834+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f21200636d0 con 0x7f2114004800 2024-09-21T07:54:43.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.017+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f21300b0580 con 0x7f2110079400 2024-09-21T07:54:43.023 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.021+0000 7f2126000640 1 -- 172.21.15.88:0/3306825054 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252478 (secure 0 0 0) 0x7f21300b0580 con 0x7f2110079400 2024-09-21T07:54:43.026 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:43.027 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:54:43.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2110079400 msgr2=0x7f211007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2110079400 0x7f211007b910 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f211c033000 tx=0x7f211c038040 comp rx=0 tx=0).stop 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 msgr2=0x7f21301f9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f21301f9430 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f21200018c0 tx=0x7f2120001d80 comp rx=0 tx=0).stop 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 shutdown_connections 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2110079400 0x7f211007b910 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f21300b1590 0x7f21301477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f213014d5f0 0x7f21300af960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 --2- 172.21.15.88:0/3306825054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2114004800 0x7f21301f9430 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.025+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 >> 172.21.15.88:0/3306825054 conn(0x7f21300b1e40 msgr2=0x7f213014de20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.026+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 shutdown_connections 2024-09-21T07:54:43.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.026+0000 7f212f200640 1 -- 172.21.15.88:0/3306825054 wait complete. 2024-09-21T07:54:43.051 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":209,"stamp":"2024-09-21T07:54:41.748627+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":126,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":265996,"kb_used_data":16488,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754708724,"statfs":{"total":773094113280,"available":772821733376,"internally_reserved":0,"allocated":16883712,"data_stored":8097636,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.645433"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306312+0000","last_change":"2024-09-21T07:54:38.153070+0000","last_active":"2024-09-21T07:54:40.306312+0000","last_peered":"2024-09-21T07:54:40.306312+0000","last_clean":"2024-09-21T07:54:40.306312+0000","last_became_active":"2024-09-21T07:54:38.152727+0000","last_became_peered":"2024-09-21T07:54:38.152727+0000","last_unstale":"2024-09-21T07:54:40.306312+0000","last_undegraded":"2024-09-21T07:54:40.306312+0000","last_fullsized":"2024-09-21T07:54:40.306312+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:48:22.881250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131911+0000","last_change":"2024-09-21T07:54:38.144114+0000","last_active":"2024-09-21T07:54:39.131911+0000","last_peered":"2024-09-21T07:54:39.131911+0000","last_clean":"2024-09-21T07:54:39.131911+0000","last_became_active":"2024-09-21T07:54:38.143708+0000","last_became_peered":"2024-09-21T07:54:38.143708+0000","last_unstale":"2024-09-21T07:54:39.131911+0000","last_undegraded":"2024-09-21T07:54:39.131911+0000","last_fullsized":"2024-09-21T07:54:39.131911+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:20:42.560345+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131738+0000","last_change":"2024-09-21T07:54:38.149178+0000","last_active":"2024-09-21T07:54:39.131738+0000","last_peered":"2024-09-21T07:54:39.131738+0000","last_clean":"2024-09-21T07:54:39.131738+0000","last_became_active":"2024-09-21T07:54:38.148866+0000","last_became_peered":"2024-09-21T07:54:38.148866+0000","last_unstale":"2024-09-21T07:54:39.131738+0000","last_undegraded":"2024-09-21T07:54:39.131738+0000","last_fullsized":"2024-09-21T07:54:39.131738+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:38.209311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131413+0000","last_change":"2024-09-21T07:54:38.131670+0000","last_active":"2024-09-21T07:54:39.131413+0000","last_peered":"2024-09-21T07:54:39.131413+0000","last_clean":"2024-09-21T07:54:39.131413+0000","last_became_active":"2024-09-21T07:54:38.131324+0000","last_became_peered":"2024-09-21T07:54:38.131324+0000","last_unstale":"2024-09-21T07:54:39.131413+0000","last_undegraded":"2024-09-21T07:54:39.131413+0000","last_fullsized":"2024-09-21T07:54:39.131413+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:02:18.765934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306585+0000","last_change":"2024-09-21T07:54:38.150970+0000","last_active":"2024-09-21T07:54:40.306585+0000","last_peered":"2024-09-21T07:54:40.306585+0000","last_clean":"2024-09-21T07:54:40.306585+0000","last_became_active":"2024-09-21T07:54:38.149199+0000","last_became_peered":"2024-09-21T07:54:38.149199+0000","last_unstale":"2024-09-21T07:54:40.306585+0000","last_undegraded":"2024-09-21T07:54:40.306585+0000","last_fullsized":"2024-09-21T07:54:40.306585+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:55:53.385435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130497+0000","last_change":"2024-09-21T07:54:38.153020+0000","last_active":"2024-09-21T07:54:39.130497+0000","last_peered":"2024-09-21T07:54:39.130497+0000","last_clean":"2024-09-21T07:54:39.130497+0000","last_became_active":"2024-09-21T07:54:38.152667+0000","last_became_peered":"2024-09-21T07:54:38.152667+0000","last_unstale":"2024-09-21T07:54:39.130497+0000","last_undegraded":"2024-09-21T07:54:39.130497+0000","last_fullsized":"2024-09-21T07:54:39.130497+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:38:42.230433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668499+0000","last_change":"2024-09-21T07:54:38.160117+0000","last_active":"2024-09-21T07:54:39.668499+0000","last_peered":"2024-09-21T07:54:39.668499+0000","last_clean":"2024-09-21T07:54:39.668499+0000","last_became_active":"2024-09-21T07:54:38.158818+0000","last_became_peered":"2024-09-21T07:54:38.158818+0000","last_unstale":"2024-09-21T07:54:39.668499+0000","last_undegraded":"2024-09-21T07:54:39.668499+0000","last_fullsized":"2024-09-21T07:54:39.668499+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:36:24.216221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306142+0000","last_change":"2024-09-21T07:54:38.149754+0000","last_active":"2024-09-21T07:54:40.306142+0000","last_peered":"2024-09-21T07:54:40.306142+0000","last_clean":"2024-09-21T07:54:40.306142+0000","last_became_active":"2024-09-21T07:54:38.149418+0000","last_became_peered":"2024-09-21T07:54:38.149418+0000","last_unstale":"2024-09-21T07:54:40.306142+0000","last_undegraded":"2024-09-21T07:54:40.306142+0000","last_fullsized":"2024-09-21T07:54:40.306142+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:40:28.647134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131894+0000","last_change":"2024-09-21T07:54:38.139255+0000","last_active":"2024-09-21T07:54:39.131894+0000","last_peered":"2024-09-21T07:54:39.131894+0000","last_clean":"2024-09-21T07:54:39.131894+0000","last_became_active":"2024-09-21T07:54:38.138941+0000","last_became_peered":"2024-09-21T07:54:38.138941+0000","last_unstale":"2024-09-21T07:54:39.131894+0000","last_undegraded":"2024-09-21T07:54:39.131894+0000","last_fullsized":"2024-09-21T07:54:39.131894+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:33:59.186075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151079+0000","last_change":"2024-09-21T07:54:38.147966+0000","last_active":"2024-09-21T07:54:39.151079+0000","last_peered":"2024-09-21T07:54:39.151079+0000","last_clean":"2024-09-21T07:54:39.151079+0000","last_became_active":"2024-09-21T07:54:38.147677+0000","last_became_peered":"2024-09-21T07:54:38.147677+0000","last_unstale":"2024-09-21T07:54:39.151079+0000","last_undegraded":"2024-09-21T07:54:39.151079+0000","last_fullsized":"2024-09-21T07:54:39.151079+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:00:40.146248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305718+0000","last_change":"2024-09-21T07:54:38.139613+0000","last_active":"2024-09-21T07:54:40.305718+0000","last_peered":"2024-09-21T07:54:40.305718+0000","last_clean":"2024-09-21T07:54:40.305718+0000","last_became_active":"2024-09-21T07:54:38.139299+0000","last_became_peered":"2024-09-21T07:54:38.139299+0000","last_unstale":"2024-09-21T07:54:40.305718+0000","last_undegraded":"2024-09-21T07:54:40.305718+0000","last_fullsized":"2024-09-21T07:54:40.305718+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:39.915497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306633+0000","last_change":"2024-09-21T07:54:38.146080+0000","last_active":"2024-09-21T07:54:40.306633+0000","last_peered":"2024-09-21T07:54:40.306633+0000","last_clean":"2024-09-21T07:54:40.306633+0000","last_became_active":"2024-09-21T07:54:38.145679+0000","last_became_peered":"2024-09-21T07:54:38.145679+0000","last_unstale":"2024-09-21T07:54:40.306633+0000","last_undegraded":"2024-09-21T07:54:40.306633+0000","last_fullsized":"2024-09-21T07:54:40.306633+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:22.323738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305749+0000","last_change":"2024-09-21T07:54:38.163477+0000","last_active":"2024-09-21T07:54:40.305749+0000","last_peered":"2024-09-21T07:54:40.305749+0000","last_clean":"2024-09-21T07:54:40.305749+0000","last_became_active":"2024-09-21T07:54:38.162282+0000","last_became_peered":"2024-09-21T07:54:38.162282+0000","last_unstale":"2024-09-21T07:54:40.305749+0000","last_undegraded":"2024-09-21T07:54:40.305749+0000","last_fullsized":"2024-09-21T07:54:40.305749+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:12.146623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130921+0000","last_change":"2024-09-21T07:54:38.162130+0000","last_active":"2024-09-21T07:54:39.130921+0000","last_peered":"2024-09-21T07:54:39.130921+0000","last_clean":"2024-09-21T07:54:39.130921+0000","last_became_active":"2024-09-21T07:54:38.161793+0000","last_became_peered":"2024-09-21T07:54:38.161793+0000","last_unstale":"2024-09-21T07:54:39.130921+0000","last_undegraded":"2024-09-21T07:54:39.130921+0000","last_fullsized":"2024-09-21T07:54:39.130921+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:16:38.161052+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305706+0000","last_change":"2024-09-21T07:54:38.142040+0000","last_active":"2024-09-21T07:54:40.305706+0000","last_peered":"2024-09-21T07:54:40.305706+0000","last_clean":"2024-09-21T07:54:40.305706+0000","last_became_active":"2024-09-21T07:54:38.141726+0000","last_became_peered":"2024-09-21T07:54:38.141726+0000","last_unstale":"2024-09-21T07:54:40.305706+0000","last_undegraded":"2024-09-21T07:54:40.305706+0000","last_fullsized":"2024-09-21T07:54:40.305706+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:41:28.438022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130470+0000","last_change":"2024-09-21T07:54:38.149280+0000","last_active":"2024-09-21T07:54:39.130470+0000","last_peered":"2024-09-21T07:54:39.130470+0000","last_clean":"2024-09-21T07:54:39.130470+0000","last_became_active":"2024-09-21T07:54:38.148914+0000","last_became_peered":"2024-09-21T07:54:38.148914+0000","last_unstale":"2024-09-21T07:54:39.130470+0000","last_undegraded":"2024-09-21T07:54:39.130470+0000","last_fullsized":"2024-09-21T07:54:39.130470+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:22.323738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131491+0000","last_change":"2024-09-21T07:54:38.152328+0000","last_active":"2024-09-21T07:54:39.131491+0000","last_peered":"2024-09-21T07:54:39.131491+0000","last_clean":"2024-09-21T07:54:39.131491+0000","last_became_active":"2024-09-21T07:54:38.151987+0000","last_became_peered":"2024-09-21T07:54:38.151987+0000","last_unstale":"2024-09-21T07:54:39.131491+0000","last_undegraded":"2024-09-21T07:54:39.131491+0000","last_fullsized":"2024-09-21T07:54:39.131491+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:21:38.642387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668877+0000","last_change":"2024-09-21T07:54:38.157545+0000","last_active":"2024-09-21T07:54:39.668877+0000","last_peered":"2024-09-21T07:54:39.668877+0000","last_clean":"2024-09-21T07:54:39.668877+0000","last_became_active":"2024-09-21T07:54:38.156766+0000","last_became_peered":"2024-09-21T07:54:38.156766+0000","last_unstale":"2024-09-21T07:54:39.668877+0000","last_undegraded":"2024-09-21T07:54:39.668877+0000","last_fullsized":"2024-09-21T07:54:39.668877+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:25.386463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131727+0000","last_change":"2024-09-21T07:54:38.141136+0000","last_active":"2024-09-21T07:54:39.131727+0000","last_peered":"2024-09-21T07:54:39.131727+0000","last_clean":"2024-09-21T07:54:39.131727+0000","last_became_active":"2024-09-21T07:54:38.140534+0000","last_became_peered":"2024-09-21T07:54:38.140534+0000","last_unstale":"2024-09-21T07:54:39.131727+0000","last_undegraded":"2024-09-21T07:54:39.131727+0000","last_fullsized":"2024-09-21T07:54:39.131727+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:02.137388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.307126+0000","last_change":"2024-09-21T07:54:38.151914+0000","last_active":"2024-09-21T07:54:40.307126+0000","last_peered":"2024-09-21T07:54:40.307126+0000","last_clean":"2024-09-21T07:54:40.307126+0000","last_became_active":"2024-09-21T07:54:38.150867+0000","last_became_peered":"2024-09-21T07:54:38.150867+0000","last_unstale":"2024-09-21T07:54:40.307126+0000","last_undegraded":"2024-09-21T07:54:40.307126+0000","last_fullsized":"2024-09-21T07:54:40.307126+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:10:06.914429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.132027+0000","last_change":"2024-09-21T07:54:38.139731+0000","last_active":"2024-09-21T07:54:39.132027+0000","last_peered":"2024-09-21T07:54:39.132027+0000","last_clean":"2024-09-21T07:54:39.132027+0000","last_became_active":"2024-09-21T07:54:38.139251+0000","last_became_peered":"2024-09-21T07:54:38.139251+0000","last_unstale":"2024-09-21T07:54:39.132027+0000","last_undegraded":"2024-09-21T07:54:39.132027+0000","last_fullsized":"2024-09-21T07:54:39.132027+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:26.715211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306751+0000","last_change":"2024-09-21T07:54:38.150981+0000","last_active":"2024-09-21T07:54:40.306751+0000","last_peered":"2024-09-21T07:54:40.306751+0000","last_clean":"2024-09-21T07:54:40.306751+0000","last_became_active":"2024-09-21T07:54:38.150575+0000","last_became_peered":"2024-09-21T07:54:38.150575+0000","last_unstale":"2024-09-21T07:54:40.306751+0000","last_undegraded":"2024-09-21T07:54:40.306751+0000","last_fullsized":"2024-09-21T07:54:40.306751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:46:53.965649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150601+0000","last_change":"2024-09-21T07:54:38.154970+0000","last_active":"2024-09-21T07:54:39.150601+0000","last_peered":"2024-09-21T07:54:39.150601+0000","last_clean":"2024-09-21T07:54:39.150601+0000","last_became_active":"2024-09-21T07:54:38.154489+0000","last_became_peered":"2024-09-21T07:54:38.154489+0000","last_unstale":"2024-09-21T07:54:39.150601+0000","last_undegraded":"2024-09-21T07:54:39.150601+0000","last_fullsized":"2024-09-21T07:54:39.150601+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:57.336560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306378+0000","last_change":"2024-09-21T07:54:38.157797+0000","last_active":"2024-09-21T07:54:40.306378+0000","last_peered":"2024-09-21T07:54:40.306378+0000","last_clean":"2024-09-21T07:54:40.306378+0000","last_became_active":"2024-09-21T07:54:38.157479+0000","last_became_peered":"2024-09-21T07:54:38.157479+0000","last_unstale":"2024-09-21T07:54:40.306378+0000","last_undegraded":"2024-09-21T07:54:40.306378+0000","last_fullsized":"2024-09-21T07:54:40.306378+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:54:29.297002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131940+0000","last_change":"2024-09-21T07:54:38.144342+0000","last_active":"2024-09-21T07:54:39.131940+0000","last_peered":"2024-09-21T07:54:39.131940+0000","last_clean":"2024-09-21T07:54:39.131940+0000","last_became_active":"2024-09-21T07:54:38.143997+0000","last_became_peered":"2024-09-21T07:54:38.143997+0000","last_unstale":"2024-09-21T07:54:39.131940+0000","last_undegraded":"2024-09-21T07:54:39.131940+0000","last_fullsized":"2024-09-21T07:54:39.131940+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:39:48.774837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306983+0000","last_change":"2024-09-21T07:54:38.152045+0000","last_active":"2024-09-21T07:54:40.306983+0000","last_peered":"2024-09-21T07:54:40.306983+0000","last_clean":"2024-09-21T07:54:40.306983+0000","last_became_active":"2024-09-21T07:54:38.151298+0000","last_became_peered":"2024-09-21T07:54:38.151298+0000","last_unstale":"2024-09-21T07:54:40.306983+0000","last_undegraded":"2024-09-21T07:54:40.306983+0000","last_fullsized":"2024-09-21T07:54:40.306983+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:08.959152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.132137+0000","last_change":"2024-09-21T07:54:38.140829+0000","last_active":"2024-09-21T07:54:39.132137+0000","last_peered":"2024-09-21T07:54:39.132137+0000","last_clean":"2024-09-21T07:54:39.132137+0000","last_became_active":"2024-09-21T07:54:38.140053+0000","last_became_peered":"2024-09-21T07:54:38.140053+0000","last_unstale":"2024-09-21T07:54:39.132137+0000","last_undegraded":"2024-09-21T07:54:39.132137+0000","last_fullsized":"2024-09-21T07:54:39.132137+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:15:36.608157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150822+0000","last_change":"2024-09-21T07:54:38.159089+0000","last_active":"2024-09-21T07:54:39.150822+0000","last_peered":"2024-09-21T07:54:39.150822+0000","last_clean":"2024-09-21T07:54:39.150822+0000","last_became_active":"2024-09-21T07:54:38.158787+0000","last_became_peered":"2024-09-21T07:54:38.158787+0000","last_unstale":"2024-09-21T07:54:39.150822+0000","last_undegraded":"2024-09-21T07:54:39.150822+0000","last_fullsized":"2024-09-21T07:54:39.150822+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:25:42.115154+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306335+0000","last_change":"2024-09-21T07:54:38.164509+0000","last_active":"2024-09-21T07:54:40.306335+0000","last_peered":"2024-09-21T07:54:40.306335+0000","last_clean":"2024-09-21T07:54:40.306335+0000","last_became_active":"2024-09-21T07:54:38.164209+0000","last_became_peered":"2024-09-21T07:54:38.164209+0000","last_unstale":"2024-09-21T07:54:40.306335+0000","last_undegraded":"2024-09-21T07:54:40.306335+0000","last_fullsized":"2024-09-21T07:54:40.306335+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:53.236650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306018+0000","last_change":"2024-09-21T07:54:38.150799+0000","last_active":"2024-09-21T07:54:40.306018+0000","last_peered":"2024-09-21T07:54:40.306018+0000","last_clean":"2024-09-21T07:54:40.306018+0000","last_became_active":"2024-09-21T07:54:38.150482+0000","last_became_peered":"2024-09-21T07:54:38.150482+0000","last_unstale":"2024-09-21T07:54:40.306018+0000","last_undegraded":"2024-09-21T07:54:40.306018+0000","last_fullsized":"2024-09-21T07:54:40.306018+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:21:20.992461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306280+0000","last_change":"2024-09-21T07:54:38.153097+0000","last_active":"2024-09-21T07:54:40.306280+0000","last_peered":"2024-09-21T07:54:40.306280+0000","last_clean":"2024-09-21T07:54:40.306280+0000","last_became_active":"2024-09-21T07:54:38.150127+0000","last_became_peered":"2024-09-21T07:54:38.150127+0000","last_unstale":"2024-09-21T07:54:40.306280+0000","last_undegraded":"2024-09-21T07:54:40.306280+0000","last_fullsized":"2024-09-21T07:54:40.306280+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:53:20.059867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306757+0000","last_change":"2024-09-21T07:54:38.148967+0000","last_active":"2024-09-21T07:54:40.306757+0000","last_peered":"2024-09-21T07:54:40.306757+0000","last_clean":"2024-09-21T07:54:40.306757+0000","last_became_active":"2024-09-21T07:54:38.148433+0000","last_became_peered":"2024-09-21T07:54:38.148433+0000","last_unstale":"2024-09-21T07:54:40.306757+0000","last_undegraded":"2024-09-21T07:54:40.306757+0000","last_fullsized":"2024-09-21T07:54:40.306757+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:40:41.967021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306867+0000","last_change":"2024-09-21T07:54:38.152379+0000","last_active":"2024-09-21T07:54:40.306867+0000","last_peered":"2024-09-21T07:54:40.306867+0000","last_clean":"2024-09-21T07:54:40.306867+0000","last_became_active":"2024-09-21T07:54:38.151619+0000","last_became_peered":"2024-09-21T07:54:38.151619+0000","last_unstale":"2024-09-21T07:54:40.306867+0000","last_undegraded":"2024-09-21T07:54:40.306867+0000","last_fullsized":"2024-09-21T07:54:40.306867+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:43:55.799119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1f","version":"155'4","reported_seq":19,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131816+0000","last_change":"2024-09-21T07:54:37.131993+0000","last_active":"2024-09-21T07:54:39.131816+0000","last_peered":"2024-09-21T07:54:39.131816+0000","last_clean":"2024-09-21T07:54:39.131816+0000","last_became_active":"2024-09-21T07:54:37.131638+0000","last_became_peered":"2024-09-21T07:54:37.131638+0000","last_unstale":"2024-09-21T07:54:39.131816+0000","last_undegraded":"2024-09-21T07:54:39.131816+0000","last_fullsized":"2024-09-21T07:54:39.131816+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:50:45.561827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151202+0000","last_change":"2024-09-21T07:54:38.155149+0000","last_active":"2024-09-21T07:54:39.151202+0000","last_peered":"2024-09-21T07:54:39.151202+0000","last_clean":"2024-09-21T07:54:39.151202+0000","last_became_active":"2024-09-21T07:54:38.154694+0000","last_became_peered":"2024-09-21T07:54:38.154694+0000","last_unstale":"2024-09-21T07:54:39.151202+0000","last_undegraded":"2024-09-21T07:54:39.151202+0000","last_fullsized":"2024-09-21T07:54:39.151202+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:47:55.461398+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1e","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306860+0000","last_change":"2024-09-21T07:54:37.135270+0000","last_active":"2024-09-21T07:54:40.306860+0000","last_peered":"2024-09-21T07:54:40.306860+0000","last_clean":"2024-09-21T07:54:40.306860+0000","last_became_active":"2024-09-21T07:54:37.134872+0000","last_became_peered":"2024-09-21T07:54:37.134872+0000","last_unstale":"2024-09-21T07:54:40.306860+0000","last_undegraded":"2024-09-21T07:54:40.306860+0000","last_fullsized":"2024-09-21T07:54:40.306860+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:53:22.659485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131479+0000","last_change":"2024-09-21T07:54:38.137428+0000","last_active":"2024-09-21T07:54:39.131479+0000","last_peered":"2024-09-21T07:54:39.131479+0000","last_clean":"2024-09-21T07:54:39.131479+0000","last_became_active":"2024-09-21T07:54:38.137122+0000","last_became_peered":"2024-09-21T07:54:38.137122+0000","last_unstale":"2024-09-21T07:54:39.131479+0000","last_undegraded":"2024-09-21T07:54:39.131479+0000","last_fullsized":"2024-09-21T07:54:39.131479+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:08:52.635432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1d","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306986+0000","last_change":"2024-09-21T07:54:37.134668+0000","last_active":"2024-09-21T07:54:40.306986+0000","last_peered":"2024-09-21T07:54:40.306986+0000","last_clean":"2024-09-21T07:54:40.306986+0000","last_became_active":"2024-09-21T07:54:37.134021+0000","last_became_peered":"2024-09-21T07:54:37.134021+0000","last_unstale":"2024-09-21T07:54:40.306986+0000","last_undegraded":"2024-09-21T07:54:40.306986+0000","last_fullsized":"2024-09-21T07:54:40.306986+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:34:58.511339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131596+0000","last_change":"2024-09-21T07:54:38.149306+0000","last_active":"2024-09-21T07:54:39.131596+0000","last_peered":"2024-09-21T07:54:39.131596+0000","last_clean":"2024-09-21T07:54:39.131596+0000","last_became_active":"2024-09-21T07:54:38.149009+0000","last_became_peered":"2024-09-21T07:54:38.149009+0000","last_unstale":"2024-09-21T07:54:39.131596+0000","last_undegraded":"2024-09-21T07:54:39.131596+0000","last_fullsized":"2024-09-21T07:54:39.131596+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:34.035726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1c","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306706+0000","last_change":"2024-09-21T07:54:37.135243+0000","last_active":"2024-09-21T07:54:40.306706+0000","last_peered":"2024-09-21T07:54:40.306706+0000","last_clean":"2024-09-21T07:54:40.306706+0000","last_became_active":"2024-09-21T07:54:37.134873+0000","last_became_peered":"2024-09-21T07:54:37.134873+0000","last_unstale":"2024-09-21T07:54:40.306706+0000","last_undegraded":"2024-09-21T07:54:40.306706+0000","last_fullsized":"2024-09-21T07:54:40.306706+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:36:17.615584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130717+0000","last_change":"2024-09-21T07:54:38.147610+0000","last_active":"2024-09-21T07:54:39.130717+0000","last_peered":"2024-09-21T07:54:39.130717+0000","last_clean":"2024-09-21T07:54:39.130717+0000","last_became_active":"2024-09-21T07:54:38.147304+0000","last_became_peered":"2024-09-21T07:54:38.147304+0000","last_unstale":"2024-09-21T07:54:39.130717+0000","last_undegraded":"2024-09-21T07:54:39.130717+0000","last_fullsized":"2024-09-21T07:54:39.130717+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:46.051760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1b","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306675+0000","last_change":"2024-09-21T07:54:37.135154+0000","last_active":"2024-09-21T07:54:40.306675+0000","last_peered":"2024-09-21T07:54:40.306675+0000","last_clean":"2024-09-21T07:54:40.306675+0000","last_became_active":"2024-09-21T07:54:37.134823+0000","last_became_peered":"2024-09-21T07:54:37.134823+0000","last_unstale":"2024-09-21T07:54:40.306675+0000","last_undegraded":"2024-09-21T07:54:40.306675+0000","last_fullsized":"2024-09-21T07:54:40.306675+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:40:56.547024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306551+0000","last_change":"2024-09-21T07:54:38.153790+0000","last_active":"2024-09-21T07:54:40.306551+0000","last_peered":"2024-09-21T07:54:40.306551+0000","last_clean":"2024-09-21T07:54:40.306551+0000","last_became_active":"2024-09-21T07:54:38.153395+0000","last_became_peered":"2024-09-21T07:54:38.153395+0000","last_unstale":"2024-09-21T07:54:40.306551+0000","last_undegraded":"2024-09-21T07:54:40.306551+0000","last_fullsized":"2024-09-21T07:54:40.306551+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:01.727107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306849+0000","last_change":"2024-09-21T07:54:37.133071+0000","last_active":"2024-09-21T07:54:40.306849+0000","last_peered":"2024-09-21T07:54:40.306849+0000","last_clean":"2024-09-21T07:54:40.306849+0000","last_became_active":"2024-09-21T07:54:37.132745+0000","last_became_peered":"2024-09-21T07:54:37.132745+0000","last_unstale":"2024-09-21T07:54:40.306849+0000","last_undegraded":"2024-09-21T07:54:40.306849+0000","last_fullsized":"2024-09-21T07:54:40.306849+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:54:35.793778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305916+0000","last_change":"2024-09-21T07:54:38.153390+0000","last_active":"2024-09-21T07:54:40.305916+0000","last_peered":"2024-09-21T07:54:40.305916+0000","last_clean":"2024-09-21T07:54:40.305916+0000","last_became_active":"2024-09-21T07:54:38.153073+0000","last_became_peered":"2024-09-21T07:54:38.153073+0000","last_unstale":"2024-09-21T07:54:40.305916+0000","last_undegraded":"2024-09-21T07:54:40.305916+0000","last_fullsized":"2024-09-21T07:54:40.305916+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:44:42.708593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668289+0000","last_change":"2024-09-21T07:54:37.122721+0000","last_active":"2024-09-21T07:54:39.668289+0000","last_peered":"2024-09-21T07:54:39.668289+0000","last_clean":"2024-09-21T07:54:39.668289+0000","last_became_active":"2024-09-21T07:54:37.122435+0000","last_became_peered":"2024-09-21T07:54:37.122435+0000","last_unstale":"2024-09-21T07:54:39.668289+0000","last_undegraded":"2024-09-21T07:54:39.668289+0000","last_fullsized":"2024-09-21T07:54:39.668289+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:11.575862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131657+0000","last_change":"2024-09-21T07:54:38.133751+0000","last_active":"2024-09-21T07:54:39.131657+0000","last_peered":"2024-09-21T07:54:39.131657+0000","last_clean":"2024-09-21T07:54:39.131657+0000","last_became_active":"2024-09-21T07:54:38.133274+0000","last_became_peered":"2024-09-21T07:54:38.133274+0000","last_unstale":"2024-09-21T07:54:39.131657+0000","last_undegraded":"2024-09-21T07:54:39.131657+0000","last_fullsized":"2024-09-21T07:54:39.131657+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:30.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306136+0000","last_change":"2024-09-21T07:54:37.121640+0000","last_active":"2024-09-21T07:54:40.306136+0000","last_peered":"2024-09-21T07:54:40.306136+0000","last_clean":"2024-09-21T07:54:40.306136+0000","last_became_active":"2024-09-21T07:54:37.121329+0000","last_became_peered":"2024-09-21T07:54:37.121329+0000","last_unstale":"2024-09-21T07:54:40.306136+0000","last_undegraded":"2024-09-21T07:54:40.306136+0000","last_fullsized":"2024-09-21T07:54:40.306136+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:23.948484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151298+0000","last_change":"2024-09-21T07:54:38.149137+0000","last_active":"2024-09-21T07:54:39.151298+0000","last_peered":"2024-09-21T07:54:39.151298+0000","last_clean":"2024-09-21T07:54:39.151298+0000","last_became_active":"2024-09-21T07:54:38.148823+0000","last_became_peered":"2024-09-21T07:54:38.148823+0000","last_unstale":"2024-09-21T07:54:39.151298+0000","last_undegraded":"2024-09-21T07:54:39.151298+0000","last_fullsized":"2024-09-21T07:54:39.151298+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:40:48.093733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151136+0000","last_change":"2024-09-21T07:54:37.133534+0000","last_active":"2024-09-21T07:54:39.151136+0000","last_peered":"2024-09-21T07:54:39.151136+0000","last_clean":"2024-09-21T07:54:39.151136+0000","last_became_active":"2024-09-21T07:54:37.133170+0000","last_became_peered":"2024-09-21T07:54:37.133170+0000","last_unstale":"2024-09-21T07:54:39.151136+0000","last_undegraded":"2024-09-21T07:54:39.151136+0000","last_fullsized":"2024-09-21T07:54:39.151136+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:48:37.992768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306711+0000","last_change":"2024-09-21T07:54:38.150645+0000","last_active":"2024-09-21T07:54:40.306711+0000","last_peered":"2024-09-21T07:54:40.306711+0000","last_clean":"2024-09-21T07:54:40.306711+0000","last_became_active":"2024-09-21T07:54:38.150309+0000","last_became_peered":"2024-09-21T07:54:38.150309+0000","last_unstale":"2024-09-21T07:54:40.306711+0000","last_undegraded":"2024-09-21T07:54:40.306711+0000","last_fullsized":"2024-09-21T07:54:40.306711+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:35.907070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.15","version":"155'3","reported_seq":18,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.307090+0000","last_change":"2024-09-21T07:54:37.132095+0000","last_active":"2024-09-21T07:54:40.307090+0000","last_peered":"2024-09-21T07:54:40.307090+0000","last_clean":"2024-09-21T07:54:40.307090+0000","last_became_active":"2024-09-21T07:54:37.129665+0000","last_became_peered":"2024-09-21T07:54:37.129665+0000","last_unstale":"2024-09-21T07:54:40.307090+0000","last_undegraded":"2024-09-21T07:54:40.307090+0000","last_fullsized":"2024-09-21T07:54:40.307090+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:30:14.742504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151139+0000","last_change":"2024-09-21T07:54:38.153345+0000","last_active":"2024-09-21T07:54:39.151139+0000","last_peered":"2024-09-21T07:54:39.151139+0000","last_clean":"2024-09-21T07:54:39.151139+0000","last_became_active":"2024-09-21T07:54:38.151739+0000","last_became_peered":"2024-09-21T07:54:38.151739+0000","last_unstale":"2024-09-21T07:54:39.151139+0000","last_undegraded":"2024-09-21T07:54:39.151139+0000","last_fullsized":"2024-09-21T07:54:39.151139+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:31:46.307506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.14","version":"155'6","reported_seq":21,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306824+0000","last_change":"2024-09-21T07:54:37.128556+0000","last_active":"2024-09-21T07:54:40.306824+0000","last_peered":"2024-09-21T07:54:40.306824+0000","last_clean":"2024-09-21T07:54:40.306824+0000","last_became_active":"2024-09-21T07:54:37.128229+0000","last_became_peered":"2024-09-21T07:54:37.128229+0000","last_unstale":"2024-09-21T07:54:40.306824+0000","last_undegraded":"2024-09-21T07:54:40.306824+0000","last_fullsized":"2024-09-21T07:54:40.306824+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:34.080920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150670+0000","last_change":"2024-09-21T07:54:38.161941+0000","last_active":"2024-09-21T07:54:39.150670+0000","last_peered":"2024-09-21T07:54:39.150670+0000","last_clean":"2024-09-21T07:54:39.150670+0000","last_became_active":"2024-09-21T07:54:38.161602+0000","last_became_peered":"2024-09-21T07:54:38.161602+0000","last_unstale":"2024-09-21T07:54:39.150670+0000","last_undegraded":"2024-09-21T07:54:39.150670+0000","last_fullsized":"2024-09-21T07:54:39.150670+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:08:32.786830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.13","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306254+0000","last_change":"2024-09-21T07:54:37.134012+0000","last_active":"2024-09-21T07:54:40.306254+0000","last_peered":"2024-09-21T07:54:40.306254+0000","last_clean":"2024-09-21T07:54:40.306254+0000","last_became_active":"2024-09-21T07:54:37.133019+0000","last_became_peered":"2024-09-21T07:54:37.133019+0000","last_unstale":"2024-09-21T07:54:40.306254+0000","last_undegraded":"2024-09-21T07:54:40.306254+0000","last_fullsized":"2024-09-21T07:54:40.306254+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:14:25.575716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668697+0000","last_change":"2024-09-21T07:54:38.157653+0000","last_active":"2024-09-21T07:54:39.668697+0000","last_peered":"2024-09-21T07:54:39.668697+0000","last_clean":"2024-09-21T07:54:39.668697+0000","last_became_active":"2024-09-21T07:54:38.157326+0000","last_became_peered":"2024-09-21T07:54:38.157326+0000","last_unstale":"2024-09-21T07:54:39.668697+0000","last_undegraded":"2024-09-21T07:54:39.668697+0000","last_fullsized":"2024-09-21T07:54:39.668697+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:03.842999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150668+0000","last_change":"2024-09-21T07:54:37.131349+0000","last_active":"2024-09-21T07:54:39.150668+0000","last_peered":"2024-09-21T07:54:39.150668+0000","last_clean":"2024-09-21T07:54:39.150668+0000","last_became_active":"2024-09-21T07:54:37.130898+0000","last_became_peered":"2024-09-21T07:54:37.130898+0000","last_unstale":"2024-09-21T07:54:39.150668+0000","last_undegraded":"2024-09-21T07:54:39.150668+0000","last_fullsized":"2024-09-21T07:54:39.150668+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:09.118262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305828+0000","last_change":"2024-09-21T07:54:38.149498+0000","last_active":"2024-09-21T07:54:40.305828+0000","last_peered":"2024-09-21T07:54:40.305828+0000","last_clean":"2024-09-21T07:54:40.305828+0000","last_became_active":"2024-09-21T07:54:38.148830+0000","last_became_peered":"2024-09-21T07:54:38.148830+0000","last_unstale":"2024-09-21T07:54:40.305828+0000","last_undegraded":"2024-09-21T07:54:40.305828+0000","last_fullsized":"2024-09-21T07:54:40.305828+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:25:48.203972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306617+0000","last_change":"2024-09-21T07:54:37.125794+0000","last_active":"2024-09-21T07:54:40.306617+0000","last_peered":"2024-09-21T07:54:40.306617+0000","last_clean":"2024-09-21T07:54:40.306617+0000","last_became_active":"2024-09-21T07:54:37.125448+0000","last_became_peered":"2024-09-21T07:54:37.125448+0000","last_unstale":"2024-09-21T07:54:40.306617+0000","last_undegraded":"2024-09-21T07:54:40.306617+0000","last_fullsized":"2024-09-21T07:54:40.306617+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:27:25.229013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306353+0000","last_change":"2024-09-21T07:54:38.153327+0000","last_active":"2024-09-21T07:54:40.306353+0000","last_peered":"2024-09-21T07:54:40.306353+0000","last_clean":"2024-09-21T07:54:40.306353+0000","last_became_active":"2024-09-21T07:54:38.150147+0000","last_became_peered":"2024-09-21T07:54:38.150147+0000","last_unstale":"2024-09-21T07:54:40.306353+0000","last_undegraded":"2024-09-21T07:54:40.306353+0000","last_fullsized":"2024-09-21T07:54:40.306353+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:41.142553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.10","version":"155'8","reported_seq":23,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668367+0000","last_change":"2024-09-21T07:54:37.126507+0000","last_active":"2024-09-21T07:54:39.668367+0000","last_peered":"2024-09-21T07:54:39.668367+0000","last_clean":"2024-09-21T07:54:39.668367+0000","last_became_active":"2024-09-21T07:54:37.126176+0000","last_became_peered":"2024-09-21T07:54:37.126176+0000","last_unstale":"2024-09-21T07:54:39.668367+0000","last_undegraded":"2024-09-21T07:54:39.668367+0000","last_fullsized":"2024-09-21T07:54:39.668367+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:22:55.077145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.151201+0000","last_change":"2024-09-21T07:54:38.154969+0000","last_active":"2024-09-21T07:54:39.151201+0000","last_peered":"2024-09-21T07:54:39.151201+0000","last_clean":"2024-09-21T07:54:39.151201+0000","last_became_active":"2024-09-21T07:54:38.154601+0000","last_became_peered":"2024-09-21T07:54:38.154601+0000","last_unstale":"2024-09-21T07:54:39.151201+0000","last_undegraded":"2024-09-21T07:54:39.151201+0000","last_fullsized":"2024-09-21T07:54:39.151201+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:02.422056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.f","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306967+0000","last_change":"2024-09-21T07:54:37.136015+0000","last_active":"2024-09-21T07:54:40.306967+0000","last_peered":"2024-09-21T07:54:40.306967+0000","last_clean":"2024-09-21T07:54:40.306967+0000","last_became_active":"2024-09-21T07:54:37.134841+0000","last_became_peered":"2024-09-21T07:54:37.134841+0000","last_unstale":"2024-09-21T07:54:40.306967+0000","last_undegraded":"2024-09-21T07:54:40.306967+0000","last_fullsized":"2024-09-21T07:54:40.306967+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:47:02.465129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306846+0000","last_change":"2024-09-21T07:54:38.154353+0000","last_active":"2024-09-21T07:54:40.306846+0000","last_peered":"2024-09-21T07:54:40.306846+0000","last_clean":"2024-09-21T07:54:40.306846+0000","last_became_active":"2024-09-21T07:54:38.153572+0000","last_became_peered":"2024-09-21T07:54:38.153572+0000","last_unstale":"2024-09-21T07:54:40.306846+0000","last_undegraded":"2024-09-21T07:54:40.306846+0000","last_fullsized":"2024-09-21T07:54:40.306846+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:18:38.209311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.e","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131245+0000","last_change":"2024-09-21T07:54:37.124563+0000","last_active":"2024-09-21T07:54:39.131245+0000","last_peered":"2024-09-21T07:54:39.131245+0000","last_clean":"2024-09-21T07:54:39.131245+0000","last_became_active":"2024-09-21T07:54:37.124222+0000","last_became_peered":"2024-09-21T07:54:37.124222+0000","last_unstale":"2024-09-21T07:54:39.131245+0000","last_undegraded":"2024-09-21T07:54:39.131245+0000","last_fullsized":"2024-09-21T07:54:39.131245+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:10.737349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131583+0000","last_change":"2024-09-21T07:54:38.140483+0000","last_active":"2024-09-21T07:54:39.131583+0000","last_peered":"2024-09-21T07:54:39.131583+0000","last_clean":"2024-09-21T07:54:39.131583+0000","last_became_active":"2024-09-21T07:54:38.138949+0000","last_became_peered":"2024-09-21T07:54:38.138949+0000","last_unstale":"2024-09-21T07:54:39.131583+0000","last_undegraded":"2024-09-21T07:54:39.131583+0000","last_fullsized":"2024-09-21T07:54:39.131583+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:12:31.419481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.d","version":"155'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305799+0000","last_change":"2024-09-21T07:54:37.126665+0000","last_active":"2024-09-21T07:54:40.305799+0000","last_peered":"2024-09-21T07:54:40.305799+0000","last_clean":"2024-09-21T07:54:40.305799+0000","last_became_active":"2024-09-21T07:54:37.126361+0000","last_became_peered":"2024-09-21T07:54:37.126361+0000","last_unstale":"2024-09-21T07:54:40.305799+0000","last_undegraded":"2024-09-21T07:54:40.305799+0000","last_fullsized":"2024-09-21T07:54:40.305799+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:45:45.045525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306751+0000","last_change":"2024-09-21T07:54:38.145421+0000","last_active":"2024-09-21T07:54:40.306751+0000","last_peered":"2024-09-21T07:54:40.306751+0000","last_clean":"2024-09-21T07:54:40.306751+0000","last_became_active":"2024-09-21T07:54:38.145102+0000","last_became_peered":"2024-09-21T07:54:38.145102+0000","last_unstale":"2024-09-21T07:54:40.306751+0000","last_undegraded":"2024-09-21T07:54:40.306751+0000","last_fullsized":"2024-09-21T07:54:40.306751+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:05:30.863933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306120+0000","last_change":"2024-09-21T07:54:37.124101+0000","last_active":"2024-09-21T07:54:40.306120+0000","last_peered":"2024-09-21T07:54:40.306120+0000","last_clean":"2024-09-21T07:54:40.306120+0000","last_became_active":"2024-09-21T07:54:37.123761+0000","last_became_peered":"2024-09-21T07:54:37.123761+0000","last_unstale":"2024-09-21T07:54:40.306120+0000","last_undegraded":"2024-09-21T07:54:40.306120+0000","last_fullsized":"2024-09-21T07:54:40.306120+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:56:15.281819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130943+0000","last_change":"2024-09-21T07:54:38.137283+0000","last_active":"2024-09-21T07:54:39.130943+0000","last_peered":"2024-09-21T07:54:39.130943+0000","last_clean":"2024-09-21T07:54:39.130943+0000","last_became_active":"2024-09-21T07:54:38.136700+0000","last_became_peered":"2024-09-21T07:54:38.136700+0000","last_unstale":"2024-09-21T07:54:39.130943+0000","last_undegraded":"2024-09-21T07:54:39.130943+0000","last_fullsized":"2024-09-21T07:54:39.130943+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:06:45.557399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.b","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306731+0000","last_change":"2024-09-21T07:54:37.130376+0000","last_active":"2024-09-21T07:54:40.306731+0000","last_peered":"2024-09-21T07:54:40.306731+0000","last_clean":"2024-09-21T07:54:40.306731+0000","last_became_active":"2024-09-21T07:54:37.129106+0000","last_became_peered":"2024-09-21T07:54:37.129106+0000","last_unstale":"2024-09-21T07:54:40.306731+0000","last_undegraded":"2024-09-21T07:54:40.306731+0000","last_fullsized":"2024-09-21T07:54:40.306731+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:27.509055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.668611+0000","last_change":"2024-09-21T07:54:38.163017+0000","last_active":"2024-09-21T07:54:39.668611+0000","last_peered":"2024-09-21T07:54:39.668611+0000","last_clean":"2024-09-21T07:54:39.668611+0000","last_became_active":"2024-09-21T07:54:38.162690+0000","last_became_peered":"2024-09-21T07:54:38.162690+0000","last_unstale":"2024-09-21T07:54:39.668611+0000","last_undegraded":"2024-09-21T07:54:39.668611+0000","last_fullsized":"2024-09-21T07:54:39.668611+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:39.799812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130889+0000","last_change":"2024-09-21T07:54:37.128878+0000","last_active":"2024-09-21T07:54:39.130889+0000","last_peered":"2024-09-21T07:54:39.130889+0000","last_clean":"2024-09-21T07:54:39.130889+0000","last_became_active":"2024-09-21T07:54:37.128563+0000","last_became_peered":"2024-09-21T07:54:37.128563+0000","last_unstale":"2024-09-21T07:54:39.130889+0000","last_undegraded":"2024-09-21T07:54:39.130889+0000","last_fullsized":"2024-09-21T07:54:39.130889+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:41:12.039826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306188+0000","last_change":"2024-09-21T07:54:38.136244+0000","last_active":"2024-09-21T07:54:40.306188+0000","last_peered":"2024-09-21T07:54:40.306188+0000","last_clean":"2024-09-21T07:54:40.306188+0000","last_became_active":"2024-09-21T07:54:38.135922+0000","last_became_peered":"2024-09-21T07:54:38.135922+0000","last_unstale":"2024-09-21T07:54:40.306188+0000","last_undegraded":"2024-09-21T07:54:40.306188+0000","last_fullsized":"2024-09-21T07:54:40.306188+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:13.175919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130471+0000","last_change":"2024-09-21T07:54:37.132479+0000","last_active":"2024-09-21T07:54:39.130471+0000","last_peered":"2024-09-21T07:54:39.130471+0000","last_clean":"2024-09-21T07:54:39.130471+0000","last_became_active":"2024-09-21T07:54:37.132145+0000","last_became_peered":"2024-09-21T07:54:37.132145+0000","last_unstale":"2024-09-21T07:54:39.130471+0000","last_undegraded":"2024-09-21T07:54:39.130471+0000","last_fullsized":"2024-09-21T07:54:39.130471+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:56:02.830384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.150621+0000","last_change":"2024-09-21T07:54:38.143894+0000","last_active":"2024-09-21T07:54:39.150621+0000","last_peered":"2024-09-21T07:54:39.150621+0000","last_clean":"2024-09-21T07:54:39.150621+0000","last_became_active":"2024-09-21T07:54:38.143567+0000","last_became_peered":"2024-09-21T07:54:38.143567+0000","last_unstale":"2024-09-21T07:54:39.150621+0000","last_undegraded":"2024-09-21T07:54:39.150621+0000","last_fullsized":"2024-09-21T07:54:39.150621+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:58:06.423489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305489+0000","last_change":"2024-09-21T07:54:37.117418+0000","last_active":"2024-09-21T07:54:40.305489+0000","last_peered":"2024-09-21T07:54:40.305489+0000","last_clean":"2024-09-21T07:54:40.305489+0000","last_became_active":"2024-09-21T07:54:37.117096+0000","last_became_peered":"2024-09-21T07:54:37.117096+0000","last_unstale":"2024-09-21T07:54:40.305489+0000","last_undegraded":"2024-09-21T07:54:40.305489+0000","last_fullsized":"2024-09-21T07:54:40.305489+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:08.364653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131977+0000","last_change":"2024-09-21T07:54:38.142681+0000","last_active":"2024-09-21T07:54:39.131977+0000","last_peered":"2024-09-21T07:54:39.131977+0000","last_clean":"2024-09-21T07:54:39.131977+0000","last_became_active":"2024-09-21T07:54:38.142398+0000","last_became_peered":"2024-09-21T07:54:38.142398+0000","last_unstale":"2024-09-21T07:54:39.131977+0000","last_undegraded":"2024-09-21T07:54:39.131977+0000","last_fullsized":"2024-09-21T07:54:39.131977+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:58:11.602913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"155'2","reported_seq":17,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306245+0000","last_change":"2024-09-21T07:54:37.123390+0000","last_active":"2024-09-21T07:54:40.306245+0000","last_peered":"2024-09-21T07:54:40.306245+0000","last_clean":"2024-09-21T07:54:40.306245+0000","last_became_active":"2024-09-21T07:54:37.123079+0000","last_became_peered":"2024-09-21T07:54:37.123079+0000","last_unstale":"2024-09-21T07:54:40.306245+0000","last_undegraded":"2024-09-21T07:54:40.306245+0000","last_fullsized":"2024-09-21T07:54:40.306245+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:03.037587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.132051+0000","last_change":"2024-09-21T07:54:38.134768+0000","last_active":"2024-09-21T07:54:39.132051+0000","last_peered":"2024-09-21T07:54:39.132051+0000","last_clean":"2024-09-21T07:54:39.132051+0000","last_became_active":"2024-09-21T07:54:38.134376+0000","last_became_peered":"2024-09-21T07:54:38.134376+0000","last_unstale":"2024-09-21T07:54:39.132051+0000","last_undegraded":"2024-09-21T07:54:39.132051+0000","last_fullsized":"2024-09-21T07:54:39.132051+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:57.154703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.6","version":"156'1","reported_seq":16,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131883+0000","last_change":"2024-09-21T07:54:37.136890+0000","last_active":"2024-09-21T07:54:39.131883+0000","last_peered":"2024-09-21T07:54:39.131883+0000","last_clean":"2024-09-21T07:54:39.131883+0000","last_became_active":"2024-09-21T07:54:37.136565+0000","last_became_peered":"2024-09-21T07:54:37.136565+0000","last_unstale":"2024-09-21T07:54:39.131883+0000","last_undegraded":"2024-09-21T07:54:39.131883+0000","last_fullsized":"2024-09-21T07:54:39.131883+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:24:10.462635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131496+0000","last_change":"2024-09-21T07:54:38.156378+0000","last_active":"2024-09-21T07:54:39.131496+0000","last_peered":"2024-09-21T07:54:39.131496+0000","last_clean":"2024-09-21T07:54:39.131496+0000","last_became_active":"2024-09-21T07:54:38.154094+0000","last_became_peered":"2024-09-21T07:54:38.154094+0000","last_unstale":"2024-09-21T07:54:39.131496+0000","last_undegraded":"2024-09-21T07:54:39.131496+0000","last_fullsized":"2024-09-21T07:54:39.131496+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:35.907070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131974+0000","last_change":"2024-09-21T07:54:37.133682+0000","last_active":"2024-09-21T07:54:39.131974+0000","last_peered":"2024-09-21T07:54:39.131974+0000","last_clean":"2024-09-21T07:54:39.131974+0000","last_became_active":"2024-09-21T07:54:37.133379+0000","last_became_peered":"2024-09-21T07:54:37.133379+0000","last_unstale":"2024-09-21T07:54:39.131974+0000","last_undegraded":"2024-09-21T07:54:39.131974+0000","last_fullsized":"2024-09-21T07:54:39.131974+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:11:19.669011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.307003+0000","last_change":"2024-09-21T07:54:38.157914+0000","last_active":"2024-09-21T07:54:40.307003+0000","last_peered":"2024-09-21T07:54:40.307003+0000","last_clean":"2024-09-21T07:54:40.307003+0000","last_became_active":"2024-09-21T07:54:38.155018+0000","last_became_peered":"2024-09-21T07:54:38.155018+0000","last_unstale":"2024-09-21T07:54:40.307003+0000","last_undegraded":"2024-09-21T07:54:40.307003+0000","last_fullsized":"2024-09-21T07:54:40.307003+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:32:53.770659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306981+0000","last_change":"2024-09-21T07:54:37.135924+0000","last_active":"2024-09-21T07:54:40.306981+0000","last_peered":"2024-09-21T07:54:40.306981+0000","last_clean":"2024-09-21T07:54:40.306981+0000","last_became_active":"2024-09-21T07:54:37.135615+0000","last_became_peered":"2024-09-21T07:54:37.135615+0000","last_unstale":"2024-09-21T07:54:40.306981+0000","last_undegraded":"2024-09-21T07:54:40.306981+0000","last_fullsized":"2024-09-21T07:54:40.306981+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:30.233548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130989+0000","last_change":"2024-09-21T07:54:38.153321+0000","last_active":"2024-09-21T07:54:39.130989+0000","last_peered":"2024-09-21T07:54:39.130989+0000","last_clean":"2024-09-21T07:54:39.130989+0000","last_became_active":"2024-09-21T07:54:38.153003+0000","last_became_peered":"2024-09-21T07:54:38.153003+0000","last_unstale":"2024-09-21T07:54:39.130989+0000","last_undegraded":"2024-09-21T07:54:39.130989+0000","last_fullsized":"2024-09-21T07:54:39.130989+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:15:17.846050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.131186+0000","last_change":"2024-09-21T07:54:37.129295+0000","last_active":"2024-09-21T07:54:39.131186+0000","last_peered":"2024-09-21T07:54:39.131186+0000","last_clean":"2024-09-21T07:54:39.131186+0000","last_became_active":"2024-09-21T07:54:37.129006+0000","last_became_peered":"2024-09-21T07:54:37.129006+0000","last_unstale":"2024-09-21T07:54:39.131186+0000","last_undegraded":"2024-09-21T07:54:39.131186+0000","last_fullsized":"2024-09-21T07:54:39.131186+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:05:31.892583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305964+0000","last_change":"2024-09-21T07:54:38.150500+0000","last_active":"2024-09-21T07:54:40.305964+0000","last_peered":"2024-09-21T07:54:40.305964+0000","last_clean":"2024-09-21T07:54:40.305964+0000","last_became_active":"2024-09-21T07:54:38.150157+0000","last_became_peered":"2024-09-21T07:54:38.150157+0000","last_unstale":"2024-09-21T07:54:40.305964+0000","last_undegraded":"2024-09-21T07:54:40.305964+0000","last_fullsized":"2024-09-21T07:54:40.305964+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:45:02.433626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130996+0000","last_change":"2024-09-21T07:54:37.129999+0000","last_active":"2024-09-21T07:54:39.130996+0000","last_peered":"2024-09-21T07:54:39.130996+0000","last_clean":"2024-09-21T07:54:39.130996+0000","last_became_active":"2024-09-21T07:54:37.129610+0000","last_became_peered":"2024-09-21T07:54:37.129610+0000","last_unstale":"2024-09-21T07:54:39.130996+0000","last_undegraded":"2024-09-21T07:54:39.130996+0000","last_fullsized":"2024-09-21T07:54:39.130996+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:50:00.940821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130881+0000","last_change":"2024-09-21T07:54:38.144090+0000","last_active":"2024-09-21T07:54:39.130881+0000","last_peered":"2024-09-21T07:54:39.130881+0000","last_clean":"2024-09-21T07:54:39.130881+0000","last_became_active":"2024-09-21T07:54:38.143757+0000","last_became_peered":"2024-09-21T07:54:38.143757+0000","last_unstale":"2024-09-21T07:54:39.130881+0000","last_undegraded":"2024-09-21T07:54:39.130881+0000","last_fullsized":"2024-09-21T07:54:39.130881+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:28:12.810972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130732+0000","last_change":"2024-09-21T07:54:37.129918+0000","last_active":"2024-09-21T07:54:39.130732+0000","last_peered":"2024-09-21T07:54:39.130732+0000","last_clean":"2024-09-21T07:54:39.130732+0000","last_became_active":"2024-09-21T07:54:37.129621+0000","last_became_peered":"2024-09-21T07:54:37.129621+0000","last_unstale":"2024-09-21T07:54:39.130732+0000","last_undegraded":"2024-09-21T07:54:39.130732+0000","last_fullsized":"2024-09-21T07:54:39.130732+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:28:33.094369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.306886+0000","last_change":"2024-09-21T07:54:38.153942+0000","last_active":"2024-09-21T07:54:40.306886+0000","last_peered":"2024-09-21T07:54:40.306886+0000","last_clean":"2024-09-21T07:54:40.306886+0000","last_became_active":"2024-09-21T07:54:38.153570+0000","last_became_peered":"2024-09-21T07:54:38.153570+0000","last_unstale":"2024-09-21T07:54:40.306886+0000","last_undegraded":"2024-09-21T07:54:40.306886+0000","last_fullsized":"2024-09-21T07:54:40.306886+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:45.515792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130454+0000","last_change":"2024-09-21T07:54:37.128531+0000","last_active":"2024-09-21T07:54:39.130454+0000","last_peered":"2024-09-21T07:54:39.130454+0000","last_clean":"2024-09-21T07:54:39.130454+0000","last_became_active":"2024-09-21T07:54:37.127695+0000","last_became_peered":"2024-09-21T07:54:37.127695+0000","last_unstale":"2024-09-21T07:54:39.130454+0000","last_undegraded":"2024-09-21T07:54:39.130454+0000","last_fullsized":"2024-09-21T07:54:39.130454+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:14.889147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":13,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130711+0000","last_change":"2024-09-21T07:54:38.150376+0000","last_active":"2024-09-21T07:54:39.130711+0000","last_peered":"2024-09-21T07:54:39.130711+0000","last_clean":"2024-09-21T07:54:39.130711+0000","last_became_active":"2024-09-21T07:54:38.150069+0000","last_became_peered":"2024-09-21T07:54:38.150069+0000","last_unstale":"2024-09-21T07:54:39.130711+0000","last_undegraded":"2024-09-21T07:54:39.130711+0000","last_fullsized":"2024-09-21T07:54:39.130711+0000","mapping_epoch":154,"log_start":"0'0","ondisk_log_start":"0'0","created":154,"last_epoch_clean":155,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:37.102537+0000","last_clean_scrub_stamp":"2024-09-21T07:54:37.102537+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:01:58.946337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:40.305721+0000","last_change":"2024-09-21T07:54:37.130492+0000","last_active":"2024-09-21T07:54:40.305721+0000","last_peered":"2024-09-21T07:54:40.305721+0000","last_clean":"2024-09-21T07:54:40.305721+0000","last_became_active":"2024-09-21T07:54:37.130184+0000","last_became_peered":"2024-09-21T07:54:37.130184+0000","last_unstale":"2024-09-21T07:54:40.305721+0000","last_undegraded":"2024-09-21T07:54:40.305721+0000","last_fullsized":"2024-09-21T07:54:40.305721+0000","mapping_epoch":153,"log_start":"0'0","ondisk_log_start":"0'0","created":153,"last_epoch_clean":154,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:54:36.096303+0000","last_clean_scrub_stamp":"2024-09-21T07:54:36.096303+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:51:07.369777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":392,"reported_epoch":156,"state":"active+clean","last_fresh":"2024-09-21T07:54:39.130717+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:54:39.130717+0000","last_peered":"2024-09-21T07:54:39.130717+0000","last_clean":"2024-09-21T07:54:39.130717+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:54:39.130717+0000","last_undegraded":"2024-09-21T07:54:39.130717+0000","last_fullsized":"2024-09-21T07:54:39.130717+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542253,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33116,"kb_used_data":1928,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338724,"statfs":{"total":96636764160,"available":96602853376,"internally_reserved":0,"allocated":1974272,"data_stored":865734,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509548,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":864100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509549,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33684,"kb_used_data":2496,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338156,"statfs":{"total":96636764160,"available":96602271744,"internally_reserved":0,"allocated":2555904,"data_stored":1454982,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509549,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":864044,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476847,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33112,"kb_used_data":1924,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338728,"statfs":{"total":96636764160,"available":96602857472,"internally_reserved":0,"allocated":1970176,"data_stored":865692,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476847,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33100,"kb_used_data":1912,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338740,"statfs":{"total":96636764160,"available":96602869760,"internally_reserved":0,"allocated":1957888,"data_stored":864080,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476844,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33676,"kb_used_data":2488,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338164,"statfs":{"total":96636764160,"available":96602279936,"internally_reserved":0,"allocated":2547712,"data_stored":1454378,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476844,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":1916,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":1961984,"data_stored":864626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:54:43.055 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:54:43.055 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:54:43.155 INFO:tasks.ceph.mds.b.smithi175.stderr:2024-09-21T07:54:43.154+0000 7f303d600640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.319+0000 7fb7ae000640 1 -- 172.21.15.88:0/3585629308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a80747e0 msgr2=0x7fb7a8153950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.319+0000 7fb7ae000640 1 --2- 172.21.15.88:0/3585629308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a80747e0 0x7fb7a8153950 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7fb794005fd0 tx=0x7fb79402a5f0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.320+0000 7fb7ae000640 1 -- 172.21.15.88:0/3585629308 shutdown_connections 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.320+0000 7fb7ae000640 1 --2- 172.21.15.88:0/3585629308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a80747e0 0x7fb7a8153950 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.320+0000 7fb7ae000640 1 -- 172.21.15.88:0/3585629308 >> 172.21.15.88:0/3585629308 conn(0x7fb7a8073920 msgr2=0x7fb7a8073d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.320+0000 7fb7ae000640 1 -- 172.21.15.88:0/3585629308 shutdown_connections 2024-09-21T07:54:43.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.320+0000 7fb7ae000640 1 -- 172.21.15.88:0/3585629308 wait complete. 2024-09-21T07:54:43.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.322+0000 7fb7ae000640 1 Processor -- start 2024-09-21T07:54:43.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.322+0000 7fb7ae000640 1 -- start start 2024-09-21T07:54:43.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.323+0000 7fb7ae000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7a806ddb0 0x7fb7a8156980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7ae000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 0x7fb7a8150030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7ae000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7a80747e0 0x7fb7a81f3950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7ae000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8167ed0 con 0x7fb7a806f960 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7ae000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb7a8167d50 con 0x7fb7a806ddb0 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7ae000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb7a8168050 con 0x7fb7a80747e0 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 0x7fb7a8150030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 0x7fb7a8150030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37214/0 (socket says 172.21.15.88:37214) 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 -- 172.21.15.88:0/593732134 learned_addr learned my addr 172.21.15.88:0/593732134 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a7e00640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7a80747e0 0x7fb7a81f3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a7400640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7a806ddb0 0x7fb7a8156980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 -- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7a80747e0 msgr2=0x7fb7a81f3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7a80747e0 0x7fb7a81f3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 -- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7a806ddb0 msgr2=0x7fb7a8156980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7a806ddb0 0x7fb7a8156980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.324+0000 7fb7a6a00640 1 -- 172.21.15.88:0/593732134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7a8156ec0 con 0x7fb7a806f960 2024-09-21T07:54:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.325+0000 7fb7a6a00640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 0x7fb7a8150030 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb794034040 tx=0x7fb7940039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.325+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb79403e030 con 0x7fb7a806f960 2024-09-21T07:54:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.325+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb7940047a0 con 0x7fb7a806f960 2024-09-21T07:54:43.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.325+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7a8153150 con 0x7fb7a806f960 2024-09-21T07:54:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.325+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb794035040 con 0x7fb7a806f960 2024-09-21T07:54:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.325+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7a8152e70 con 0x7fb7a806f960 2024-09-21T07:54:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.326+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7a8153950 con 0x7fb7a806f960 2024-09-21T07:54:43.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.327+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fb794041030 con 0x7fb7a806f960 2024-09-21T07:54:43.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.328+0000 7fb79f400640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb788079400 0x7fb78807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.328+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fb7940bc8c0 con 0x7fb7a806f960 2024-09-21T07:54:43.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.329+0000 7fb7a7400640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb788079400 0x7fb78807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.329+0000 7fb7a7400640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb788079400 0x7fb78807b910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb798000f60 tx=0x7fb798009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.333+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb7a8156ec0 con 0x7fb7a806f960 2024-09-21T07:54:43.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.590+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7fb7a8152c00 con 0x7fb7a806f960 2024-09-21T07:54:43.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.594+0000 7fb79f400640 1 -- 172.21.15.88:0/593732134 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) ==== 190+0+147 (secure 0 0 0) 0x7fb7940bc020 con 0x7fb7a806f960 2024-09-21T07:54:43.596 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:54:43.596 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:54:43.596 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:54:43.596 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:54:43.596 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:54:43.597 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.596+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb788079400 msgr2=0x7fb78807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.596+0000 7fb7ae000640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb788079400 0x7fb78807b910 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fb798000f60 tx=0x7fb798009040 comp rx=0 tx=0).stop 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.596+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 msgr2=0x7fb7a8150030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.596+0000 7fb7ae000640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 0x7fb7a8150030 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fb794034040 tx=0x7fb7940039e0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.596+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 shutdown_connections 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.596+0000 7fb7ae000640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb788079400 0x7fb78807b910 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.597+0000 7fb7ae000640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb7a80747e0 0x7fb7a81f3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.597+0000 7fb7ae000640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7a806f960 0x7fb7a8150030 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.597+0000 7fb7ae000640 1 --2- 172.21.15.88:0/593732134 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb7a806ddb0 0x7fb7a8156980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.597+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 >> 172.21.15.88:0/593732134 conn(0x7fb7a8073920 msgr2=0x7fb7a815df90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.597+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 shutdown_connections 2024-09-21T07:54:43.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.597+0000 7fb7ae000640 1 -- 172.21.15.88:0/593732134 wait complete. 2024-09-21T07:54:43.621 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:54:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 -- 172.21.15.88:0/1882230896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b9e00 msgr2=0x7f5f34162a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 --2- 172.21.15.88:0/1882230896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b9e00 0x7f5f34162a50 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f5f340d2050 tx=0x7f5f2402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 -- 172.21.15.88:0/1882230896 shutdown_connections 2024-09-21T07:54:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 --2- 172.21.15.88:0/1882230896 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340d1580 0x7f5f340d1990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.841 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 --2- 172.21.15.88:0/1882230896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b9e00 0x7f5f34162a50 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 -- 172.21.15.88:0/1882230896 >> 172.21.15.88:0/1882230896 conn(0x7f5f3408bfe0 msgr2=0x7f5f3409d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:43.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 -- 172.21.15.88:0/1882230896 shutdown_connections 2024-09-21T07:54:43.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.839+0000 7f5f33e00640 1 -- 172.21.15.88:0/1882230896 wait complete. 2024-09-21T07:54:43.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.841+0000 7f5f33e00640 1 Processor -- start 2024-09-21T07:54:43.843 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.841+0000 7f5f33e00640 1 -- start start 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 0x7f5f341ebae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340b9e00 0x7f5f341ea6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340d1580 0x7f5f341eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340d44b0 con 0x7f5f340b3160 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5f340d4330 con 0x7f5f340d1580 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5f340d4630 con 0x7f5f340b9e00 2024-09-21T07:54:43.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 0x7f5f341ebae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 0x7f5f341ebae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37224/0 (socket says 172.21.15.88:37224) 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 -- 172.21.15.88:0/83862616 learned_addr learned my addr 172.21.15.88:0/83862616 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f2be00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340b9e00 0x7f5f341ea6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f33400640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340d1580 0x7f5f341eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 -- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340b9e00 msgr2=0x7f5f341ea6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340b9e00 0x7f5f341ea6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 -- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340d1580 msgr2=0x7f5f341eac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340d1580 0x7f5f341eac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.843+0000 7f5f32a00640 1 -- 172.21.15.88:0/83862616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5f341550a0 con 0x7f5f340b3160 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.844+0000 7f5f32a00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 0x7f5f341ebae0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009920 tx=0x7f5f24033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.844+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24031d60 con 0x7f5f340b3160 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.844+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5f24033e10 con 0x7f5f340b3160 2024-09-21T07:54:43.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.844+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f24005280 con 0x7f5f340b3160 2024-09-21T07:54:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.844+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5f340a1d60 con 0x7f5f340b3160 2024-09-21T07:54:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.844+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5f3409f2b0 con 0x7f5f340b3160 2024-09-21T07:54:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.845+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5f340d4bb0 con 0x7f5f340b3160 2024-09-21T07:54:43.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.846+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f5f24005420 con 0x7f5f340b3160 2024-09-21T07:54:43.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.847+0000 7f5f2aa00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f10079400 0x7f5f1007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:43.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.847+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f5f240ba8c0 con 0x7f5f340b3160 2024-09-21T07:54:43.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.848+0000 7f5f2be00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f10079400 0x7f5f1007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:43.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.848+0000 7f5f2be00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f10079400 0x7f5f1007b910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f3409efd0 tx=0x7f5f1c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:43.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:43.852+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5f2403f030 con 0x7f5f340b3160 2024-09-21T07:54:44.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.071+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5f340b4620 con 0x7f5f340b3160 2024-09-21T07:54:44.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.072+0000 7f5f2aa00640 1 -- 172.21.15.88:0/83862616 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 108 v108) ==== 95+0+5239 (secure 0 0 0) 0x7f5f24084840 con 0x7f5f340b3160 2024-09-21T07:54:44.074 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:44.074 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":108,"btime":"2024-09-21T07:54:41:144560+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:37.615557+0000","modified":"2024-09-21T07:54:40.637260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8772":{"gid":8772,"name":"b","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2687096264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2687096264},{"type":"v1","addr":"172.21.15.175:6835","nonce":2687096264}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8772,"qdb_cluster":[8772]},"id":12}]} 2024-09-21T07:54:44.074 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 108 2024-09-21T07:54:44.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f10079400 msgr2=0x7f5f1007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f10079400 0x7f5f1007b910 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5f3409efd0 tx=0x7f5f1c009040 comp rx=0 tx=0).stop 2024-09-21T07:54:44.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 msgr2=0x7f5f341ebae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 0x7f5f341ebae0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f5f24009920 tx=0x7f5f24033c20 comp rx=0 tx=0).stop 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 shutdown_connections 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5f10079400 0x7f5f1007b910 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5f340b9e00 0x7f5f341ea6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5f340d1580 0x7f5f341eac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 --2- 172.21.15.88:0/83862616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5f340b3160 0x7f5f341ebae0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 >> 172.21.15.88:0/83862616 conn(0x7f5f3408bfe0 msgr2=0x7f5f3406e960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 shutdown_connections 2024-09-21T07:54:44.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.074+0000 7f5f33e00640 1 -- 172.21.15.88:0/83862616 wait complete. 2024-09-21T07:54:44.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 108, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:54:37.615557+0000', 'modified': '2024-09-21T07:54:40.637260+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8772}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8772': {'gid': 8772, 'name': 'b', 'rank': 0, 'incarnation': 105, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.175:6835/2687096264', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.175:6801', 'nonce': 2687096264}, {'type': 'v1', 'addr': '172.21.15.175:6835', 'nonce': 2687096264}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 8772, 'qdb_cluster': [8772]} 2024-09-21T07:54:44.504 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:54:44.504 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-09-21T07:54:44.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 -- 172.21.15.88:0/4244855999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68840cfc80 msgr2=0x7f6884156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 --2- 172.21.15.88:0/4244855999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68840cfc80 0x7f6884156e40 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6874009860 tx=0x7f687402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 -- 172.21.15.88:0/4244855999 shutdown_connections 2024-09-21T07:54:44.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 --2- 172.21.15.88:0/4244855999 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f68840cfc80 0x7f6884156e40 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 -- 172.21.15.88:0/4244855999 >> 172.21.15.88:0/4244855999 conn(0x7f688406d3d0 msgr2=0x7f68840b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 -- 172.21.15.88:0/4244855999 shutdown_connections 2024-09-21T07:54:44.729 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.726+0000 7f688ae00640 1 -- 172.21.15.88:0/4244855999 wait complete. 2024-09-21T07:54:44.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.728+0000 7f688ae00640 1 Processor -- start 2024-09-21T07:54:44.730 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.728+0000 7f688ae00640 1 -- start start 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.729+0000 7f688ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 0x7f6884074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f688ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68840cfc80 0x7f68840c8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f688ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f688415ce30 0x7f68840b1340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f688ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68840cd190 con 0x7f68840705b0 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f688ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68840cd010 con 0x7f68840cfc80 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f688ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68840cd310 con 0x7f688415ce30 2024-09-21T07:54:44.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 0x7f6884074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 0x7f6884074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37234/0 (socket says 172.21.15.88:37234) 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 -- 172.21.15.88:0/3712249277 learned_addr learned my addr 172.21.15.88:0/3712249277 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883400640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68840cfc80 0x7f68840c8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6888e00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f688415ce30 0x7f68840b1340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 -- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f688415ce30 msgr2=0x7f68840b1340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f688415ce30 0x7f68840b1340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 -- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68840cfc80 msgr2=0x7f68840c8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68840cfc80 0x7f68840c8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883e00640 1 -- 172.21.15.88:0/3712249277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68841f3a90 con 0x7f68840705b0 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6883400640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68840cfc80 0x7f68840c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.730+0000 7f6888e00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f688415ce30 0x7f68840b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.731+0000 7f6883e00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 0x7f6884074cb0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6874009980 tx=0x7f6874003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.731+0000 7f687be00640 1 -- 172.21.15.88:0/3712249277 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874005c90 con 0x7f68840705b0 2024-09-21T07:54:44.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.731+0000 7f687be00640 1 -- 172.21.15.88:0/3712249277 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6874005020 con 0x7f68840705b0 2024-09-21T07:54:44.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.731+0000 7f687be00640 1 -- 172.21.15.88:0/3712249277 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874005410 con 0x7f68840705b0 2024-09-21T07:54:44.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.731+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68841f3d70 con 0x7f68840705b0 2024-09-21T07:54:44.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.731+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68841f4330 con 0x7f68840705b0 2024-09-21T07:54:44.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.733+0000 7f687be00640 1 -- 172.21.15.88:0/3712249277 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f68740317c0 con 0x7f68840705b0 2024-09-21T07:54:44.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.734+0000 7f687be00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6864079430 0x7f686407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.734+0000 7f687be00640 1 -- 172.21.15.88:0/3712249277 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f68740bd040 con 0x7f68840705b0 2024-09-21T07:54:44.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.735+0000 7f6883400640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6864079430 0x7f686407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.735+0000 7f6891eab740 1 Processor -- start 2024-09-21T07:54:44.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.735+0000 7f6891eab740 1 -- start start 2024-09-21T07:54:44.737 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.735+0000 7f6883400640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6864079430 0x7f686407b940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6870001860 tx=0x7f6870009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6891eab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 0x559193236a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e223b0 0x5591931ebf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x5591931ec4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6891eab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559193175430 con 0x559192e4b270 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x5591931655d0 con 0x559192e2a590 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x559193154a00 con 0x559192e223b0 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6883e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 0x559193236a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6883e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 0x559193236a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37248/0 (socket says 172.21.15.88:37248) 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.736+0000 7f6883e00640 1 -- 172.21.15.88:0/4214056970 learned_addr learned my addr 172.21.15.88:0/4214056970 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6883e00640 1 -- 172.21.15.88:0/4214056970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e223b0 msgr2=0x5591931ebf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6883e00640 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e223b0 0x5591931ebf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6883e00640 1 -- 172.21.15.88:0/4214056970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 msgr2=0x5591931ec4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6883e00640 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x5591931ec4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6883e00640 1 -- 172.21.15.88:0/4214056970 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55919311d520 con 0x559192e4b270 2024-09-21T07:54:44.740 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6883e00640 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 0x559193236a20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6874083ad0 tx=0x7f6874083c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55bf3cbfa55eec68 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6878c00640 1 -- 172.21.15.88:0/4214056970 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874087070 con 0x559192e4b270 2024-09-21T07:54:44.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6878c00640 1 -- 172.21.15.88:0/4214056970 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68740838e0 con 0x559192e4b270 2024-09-21T07:54:44.741 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.737+0000 7f6878c00640 1 -- 172.21.15.88:0/4214056970 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874048040 con 0x559192e4b270 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 -- 172.21.15.88:0/4214056970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 msgr2=0x559193236a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 0x559193236a20 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f6874083ad0 tx=0x7f6874083c60 comp rx=0 tx=0).stop 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 -- 172.21.15.88:0/4214056970 shutdown_connections 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e223b0 0x5591931ebf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x5591931ec4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 --2- 172.21.15.88:0/4214056970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e4b270 0x559193236a20 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 -- 172.21.15.88:0/4214056970 >> 172.21.15.88:0/4214056970 conn(0x559192e3f3c0 msgr2=0x5591931f1970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 -- 172.21.15.88:0/4214056970 shutdown_connections 2024-09-21T07:54:44.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.738+0000 7f6891eab740 1 -- 172.21.15.88:0/4214056970 wait complete. 2024-09-21T07:54:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.739+0000 7f6891eab740 1 -- start start 2024-09-21T07:54:44.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.740+0000 7f6891eab740 1 Processor -- start 2024-09-21T07:54:44.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x559193213170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x55919322e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x55919321ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5591930fb330 con 0x559192e2a590 2024-09-21T07:54:44.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x559193158810 con 0x559192e223b0 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x559192ec5620 con 0x559192e4b270 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6891eab740 10 client.0 _ll_register_callbacks cb 0x559193100420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6883400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x55919322e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6883400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x55919322e980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37258/0 (socket says 172.21.15.88:37258) 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.741+0000 7f6883400640 1 -- 172.21.15.88:0/1165340983 learned_addr learned my addr 172.21.15.88:0/1165340983 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883e00640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x559193213170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6888e00640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x55919321ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883400640 1 -- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 msgr2=0x55919321ac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883400640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x55919321ac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883400640 1 -- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 msgr2=0x559193213170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883400640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x559193213170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883400640 1 -- 172.21.15.88:0/1165340983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559192eb6850 con 0x559192e2a590 2024-09-21T07:54:44.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6883400640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x55919322e980 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6870000fa0 tx=0x7f68700147f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1818a6099452b41 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f686d600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f686d600640 1 -- 172.21.15.88:0/1165340983 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870024060 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f686d600640 1 -- 172.21.15.88:0/1165340983 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f687001f600 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.742+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68740838e0 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f686d600640 1 -- 172.21.15.88:0/1165340983 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f687002a050 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 10 client.8886 fetch_fsmap learned FSMap version 108 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x559192f2d6b0 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f686d600640 1 -- 172.21.15.88:0/1165340983 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68740838e0 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f686d600640 1 -- 172.21.15.88:0/1165340983 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7f687002d020 con 0x559192e2a590 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 10 client.8886 fetch_fsmap finished waiting for FSMap version 108 2024-09-21T07:54:44.747 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 10 client.8886 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:54:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 20 client.8886 populate_metadata read hostname 'smithi088' 2024-09-21T07:54:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x559192e09e40 0x55919324dc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 4 client.8886 mds_command: new command op to 8772 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:54:44.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.743+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 --> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559193167520 con 0x559192e09e40 2024-09-21T07:54:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.744+0000 7f6883e00640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x559192e09e40 0x55919324dc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.744+0000 7f6883e00640 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x559192e09e40 0x55919324dc60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48005b3e7eccac05 server_cookie=8b6358a2a11520fe in_seq=0 out_seq=0 2024-09-21T07:54:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.744+0000 7f686d600640 10 client.8886 ms_handle_connect on v2:172.21.15.175:6801/2687096264 2024-09-21T07:54:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.745+0000 7f686d600640 1 -- 172.21.15.88:0/1165340983 <== mds.0 v2:172.21.15.175:6801/2687096264 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559193167520 con 0x559192e09e40 2024-09-21T07:54:44.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.745+0000 7f686d600640 10 client.8886 handle_command_reply: tid=0 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.745+0000 7f6891eab740 1 client.8886 shutdown 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 msgr2=0x55919322e980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x55919322e980 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f6870000fa0 tx=0x7f68700147f0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 shutdown_connections 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x559192e09e40 0x55919324dc60 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x55919321ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x55919322e980 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f686d600640 0 client.8886 ms_handle_reset on v2:172.21.15.175:6801/2687096264 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 --2- 172.21.15.88:0/1165340983 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x559193213170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 >> 172.21.15.88:0/1165340983 conn(0x559192e3f3c0 msgr2=0x559193236a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 shutdown_connections 2024-09-21T07:54:44.750 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 1 -- 172.21.15.88:0/1165340983 wait complete. 2024-09-21T07:54:44.751 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.746+0000 7f6891eab740 20 client.8886 trim_cache size 0 max 16384 2024-09-21T07:54:44.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.765+0000 7f6891eab740 1 Processor -- start 2024-09-21T07:54:44.767 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.765+0000 7f6891eab740 1 -- start start 2024-09-21T07:54:44.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.766+0000 7f6891eab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 0x55919325e810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.766+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x559193243de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.766+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x559193244320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.766+0000 7f6891eab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68840cbb20 con 0x559192e223b0 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.766+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f68841f6f10 con 0x559192e2a590 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.766+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f68841f7560 con 0x559192e4b270 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 0x55919325e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 0x55919325e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37260/0 (socket says 172.21.15.88:37260) 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 -- 172.21.15.88:0/892453606 learned_addr learned my addr 172.21.15.88:0/892453606 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 -- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 msgr2=0x559193244320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x559193244320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 -- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 msgr2=0x559193243de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.770 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883400640 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x559193243de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x559193243de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 -- 172.21.15.88:0/892453606 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591931f33e0 con 0x559192e223b0 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883400640 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x559193243de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f6883e00640 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 0x55919325e810 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f68740ba4e0 tx=0x7f687402a4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3023620f8568fba8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f686fe00640 1 -- 172.21.15.88:0/892453606 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687409e070 con 0x559192e223b0 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f686fe00640 1 -- 172.21.15.88:0/892453606 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68740888b0 con 0x559192e223b0 2024-09-21T07:54:44.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.767+0000 7f686fe00640 1 -- 172.21.15.88:0/892453606 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874087040 con 0x559192e223b0 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 -- 172.21.15.88:0/892453606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 msgr2=0x55919325e810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 0x55919325e810 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f68740ba4e0 tx=0x7f687402a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 -- 172.21.15.88:0/892453606 shutdown_connections 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x559193244320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e2a590 0x559193243de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 --2- 172.21.15.88:0/892453606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e223b0 0x55919325e810 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 -- 172.21.15.88:0/892453606 >> 172.21.15.88:0/892453606 conn(0x559192e3f3c0 msgr2=0x5591931f0940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 -- 172.21.15.88:0/892453606 shutdown_connections 2024-09-21T07:54:44.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.768+0000 7f6891eab740 1 -- 172.21.15.88:0/892453606 wait complete. 2024-09-21T07:54:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.769+0000 7f6891eab740 1 -- start start 2024-09-21T07:54:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.769+0000 7f6891eab740 1 Processor -- start 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x5591932bf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x559193218480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x5591932189c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55919322eec0 con 0x559192e2a590 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55919324ae10 con 0x559192e223b0 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55919322df20 con 0x559192e4b270 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6891eab740 10 client.0 _ll_register_callbacks cb 0x7f687000dac0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:54:44.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x559193218480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x559193218480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37268/0 (socket says 172.21.15.88:37268) 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 -- 172.21.15.88:0/2135755375 learned_addr learned my addr 172.21.15.88:0/2135755375 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883e00640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x5591932bf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6888e00640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x5591932189c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 -- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 msgr2=0x5591932189c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x5591932189c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 -- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 msgr2=0x5591932bf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.771+0000 7f6883400640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x5591932bf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f6883400640 1 -- 172.21.15.88:0/2135755375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591931ecf40 con 0x559192e2a590 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f6883400640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x559193218480 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f68700046b0 tx=0x7f6870014d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585440d02586cef9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:44.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f686d600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f686d600640 1 -- 172.21.15.88:0/2135755375 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6870032070 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f686d600640 1 -- 172.21.15.88:0/2135755375 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f687002e3f0 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68740888b0 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f686d600640 1 -- 172.21.15.88:0/2135755375 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=108) ==== 24+0+0 (secure 0 0 0) 0x7f687003d050 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f6891eab740 10 client.8892 fetch_fsmap learned FSMap version 108 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.772+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=108,monmap=2+}) -- 0x559193217730 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f686d600640 1 -- 172.21.15.88:0/2135755375 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f68740888b0 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f686d600640 1 -- 172.21.15.88:0/2135755375 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 108) ==== 3502+0+0 (secure 0 0 0) 0x7f6870040020 con 0x559192e2a590 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6891eab740 10 client.8892 fetch_fsmap finished waiting for FSMap version 108 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6891eab740 10 client.8892 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T07:54:44.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6891eab740 20 client.8892 populate_metadata read hostname 'smithi088' 2024-09-21T07:54:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6891eab740 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x55919325d660 0x559193248990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6891eab740 4 client.8892 mds_command: new command op to 8772 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:54:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 --> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] -- command(tid 0: {"prefix": "status"}) -- 0x559193162fd0 con 0x55919325d660 2024-09-21T07:54:44.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.773+0000 7f6883e00640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x55919325d660 0x559193248990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6883e00640 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x55919325d660 0x559193248990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=820c531c6619ab08 server_cookie=2ec2337464783070 in_seq=0 out_seq=0 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f686d600640 10 client.8892 ms_handle_connect on v2:172.21.15.175:6801/2687096264 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f686d600640 1 -- 172.21.15.88:0/2135755375 <== mds.0 v2:172.21.15.175:6801/2687096264 1 ==== command_reply(tid 0: 0 ) ==== 8+0+334 (crc 0 0 0) 0x559193162fd0 con 0x55919325d660 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f686d600640 10 client.8892 handle_command_reply: tid=0 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 client.8892 shutdown 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 msgr2=0x559193218480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.779 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x559193218480 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f68700046b0 tx=0x7f6870014d70 comp rx=0 tx=0).stop 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 shutdown_connections 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:6801/2687096264,v1:172.21.15.175:6835/2687096264] conn(0x55919325d660 0x559193248990 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x559192e4b270 0x5591932189c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f686d600640 0 client.8892 ms_handle_reset on v2:172.21.15.175:6801/2687096264 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559192e2a590 0x559193218480 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 --2- 172.21.15.88:0/2135755375 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x559192e223b0 0x5591932bf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 >> 172.21.15.88:0/2135755375 conn(0x559192e3f3c0 msgr2=0x5591931f0c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 shutdown_connections 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.774+0000 7f6891eab740 1 -- 172.21.15.88:0/2135755375 wait complete. 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f6891eab740 20 client.8892 trim_cache size 0 max 16384 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:54:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 8772, 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6391576069999996, 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 108, 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 156, 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 156, 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 11.691140345000001 2024-09-21T07:54:44.781 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6864079430 msgr2=0x7f686407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6864079430 0x7f686407b940 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f6870001860 tx=0x7f6870009040 comp rx=0 tx=0).stop 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 msgr2=0x7f6884074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 0x7f6884074cb0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f6874009980 tx=0x7f6874003c10 comp rx=0 tx=0).stop 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 shutdown_connections 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6864079430 0x7f686407b940 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f688415ce30 0x7f68840b1340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f68840cfc80 0x7f68840c8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 --2- 172.21.15.88:0/3712249277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68840705b0 0x7f6884074cb0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 >> 172.21.15.88:0/3712249277 conn(0x7f688406d3d0 msgr2=0x7f688415e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 shutdown_connections 2024-09-21T07:54:44.783 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:44.775+0000 7f688ae00640 1 -- 172.21.15.88:0/3712249277 wait complete. 2024-09-21T07:54:44.798 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:45.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 -- 172.21.15.88:0/189317911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 msgr2=0x7f4e0c0cfd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.019 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 --2- 172.21.15.88:0/189317911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0cfd30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f4e0c0b5b30 tx=0x7f4dfc02a560 comp rx=0 tx=0).stop 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 -- 172.21.15.88:0/189317911 shutdown_connections 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 --2- 172.21.15.88:0/189317911 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e0c0b9610 0x7f4e0c0d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 --2- 172.21.15.88:0/189317911 >> v2:172.21.15.175:3300/0 conn(0x7f4e0c1620e0 0x7f4e0c0d0ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 --2- 172.21.15.88:0/189317911 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0cfd30 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 -- 172.21.15.88:0/189317911 >> 172.21.15.88:0/189317911 conn(0x7f4e0c09d820 msgr2=0x7f4e0c08bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 -- 172.21.15.88:0/189317911 shutdown_connections 2024-09-21T07:54:45.020 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.017+0000 7f4e0b200640 1 -- 172.21.15.88:0/189317911 wait complete. 2024-09-21T07:54:45.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.019+0000 7f4e0b200640 1 Processor -- start 2024-09-21T07:54:45.021 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.019+0000 7f4e0b200640 1 -- start start 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e0c0b9610 0x7f4e0c0ca790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e0c1620e0 0x7f4e0c0b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c0b3c50 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4e0c0b3ad0 con 0x7f4e0c1620e0 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4e0c0b3dd0 con 0x7f4e0c0b9610 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.024 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37278/0 (socket says 172.21.15.88:37278) 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 -- 172.21.15.88:0/1202528813 learned_addr learned my addr 172.21.15.88:0/1202528813 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09e00640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e0c0b9610 0x7f4e0c0ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0a800640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e0c1620e0 0x7f4e0c0b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 -- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e0c0b9610 msgr2=0x7f4e0c0ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e0c0b9610 0x7f4e0c0ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 -- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e0c1620e0 msgr2=0x7f4e0c0b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e0c1620e0 0x7f4e0c0b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e09400640 1 -- 172.21.15.88:0/1202528813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e0c0b1120 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.021+0000 7f4e0a800640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e0c1620e0 0x7f4e0c0b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.022+0000 7f4e09400640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0b18b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c970 tx=0x7f4df800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.022+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8007bb0 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.022+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4df8007e40 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.022+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e0c0cb910 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.025 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.022+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4df8005290 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.022+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e0c0ce280 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.023+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e0c0b42f0 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.026 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.024+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f4df8005430 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.026+0000 7f4e02000640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4dec079400 0x7f4dec07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:45.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.026+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f4df809c500 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.026+0000 7f4e09e00640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4dec079400 0x7f4dec07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:45.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.026+0000 7f4e09e00640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4dec079400 0x7f4dec07b910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc033000 tx=0x7f4dfc004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:45.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.030+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4e0c0cb910 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.260+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f4e0c0c7a80 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.262 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.260+0000 7f4e02000640 1 -- 172.21.15.88:0/1202528813 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) ==== 53+0+73 (secure 0 0 0) 0x7f4df8014070 con 0x7f4e0c0b9b80 2024-09-21T07:54:45.263 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:45.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4dec079400 msgr2=0x7f4dec07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4dec079400 0x7f4dec07b910 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc033000 tx=0x7f4dfc004730 comp rx=0 tx=0).stop 2024-09-21T07:54:45.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 msgr2=0x7f4e0c0b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:45.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0b18b0 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f4df800c970 tx=0x7f4df800ce30 comp rx=0 tx=0).stop 2024-09-21T07:54:45.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 shutdown_connections 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4dec079400 0x7f4dec07b910 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4e0c1620e0 0x7f4e0c0b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e0c0b9b80 0x7f4e0c0b18b0 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 --2- 172.21.15.88:0/1202528813 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4e0c0b9610 0x7f4e0c0ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 >> 172.21.15.88:0/1202528813 conn(0x7f4e0c09d820 msgr2=0x7f4e0c0b4500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 shutdown_connections 2024-09-21T07:54:45.266 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45.263+0000 7f4e0b200640 1 -- 172.21.15.88:0/1202528813 wait complete. 2024-09-21T07:54:45.286 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:45.286 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:54:45.287 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:45.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:45.287 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:54:45.323 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:45.323 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:45.323 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:45.324 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42209sec preferred_lft 42209sec 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.325 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:54:45.326 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:45.327 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:45.327 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:45.418 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:54:45.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:45.421 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:54:45.457 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:54:45.457 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:45.457 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:54:45.457 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:54:45.457 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:45.458 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:45.458 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:54:45.458 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:54:45.458 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:54:45.458 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:54:45.458 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:54:45.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:45.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:45 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:45.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:45.656 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:54:45.691 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:54:45.691 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:54:45.692 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:45.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:45.692 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:54:45.789 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:45.789 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:45.823 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:54:45.824 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:45.824 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:45.898 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:54:45.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:45.899 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:46.093 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:46.094 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:54:46.094 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:54:46.114 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:46.114 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:54:46.114 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:54:46.114 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:46.114 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:54:46.149 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:54:46.150 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:54:46.151 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:54:46.151 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:46.151 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:46.151 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:46.151 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:46.154 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:46.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.154 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:46.211 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:46.297 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:46.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:46.316 INFO:teuthology.orchestra.run.smithi088.stdout:client8248 2024-09-21T07:54:46.316 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:46.316 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:46.351 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:54:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 -- 172.21.15.88:0/3038015800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc1518d0 msgr2=0x7f00dc06f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 --2- 172.21.15.88:0/3038015800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc1518d0 0x7f00dc06f9f0 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f00dc074310 tx=0x7f00d002ae00 comp rx=0 tx=0).stop 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 -- 172.21.15.88:0/3038015800 shutdown_connections 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 --2- 172.21.15.88:0/3038015800 >> v2:172.21.15.175:3300/0 conn(0x7f00dc15e3d0 0x7f00dc15e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 --2- 172.21.15.88:0/3038015800 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00dc0735c0 0x7f00dc0739d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 --2- 172.21.15.88:0/3038015800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc1518d0 0x7f00dc06f9f0 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 -- 172.21.15.88:0/3038015800 >> 172.21.15.88:0/3038015800 conn(0x7f00dc0b1e40 msgr2=0x7f00dc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 -- 172.21.15.88:0/3038015800 shutdown_connections 2024-09-21T07:54:46.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.613+0000 7f00e2800640 1 -- 172.21.15.88:0/3038015800 wait complete. 2024-09-21T07:54:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.615+0000 7f00e2800640 1 Processor -- start 2024-09-21T07:54:46.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.615+0000 7f00e2800640 1 -- start start 2024-09-21T07:54:46.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e2800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00dc0735c0 0x7f00dc0b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e2800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00dc1518d0 0x7f00dc1e5170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 0x7f00dc09d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e2800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc0b4630 con 0x7f00dc15e3d0 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e2800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00dc070140 con 0x7f00dc1518d0 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e2800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00dc0b47b0 con 0x7f00dc0735c0 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 0x7f00dc09d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 0x7f00dc09d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37174/0 (socket says 172.21.15.88:37174) 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 -- 172.21.15.88:0/3992107027 learned_addr learned my addr 172.21.15.88:0/3992107027 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1400640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00dc0735c0 0x7f00dc0b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e0a00640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00dc1518d0 0x7f00dc1e5170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 -- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00dc0735c0 msgr2=0x7f00dc0b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00dc0735c0 0x7f00dc0b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 -- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00dc1518d0 msgr2=0x7f00dc1e5170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00dc1518d0 0x7f00dc1e5170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.617+0000 7f00e1e00640 1 -- 172.21.15.88:0/3992107027 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00dc0af640 con 0x7f00dc15e3d0 2024-09-21T07:54:46.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.618+0000 7f00e1e00640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 0x7f00dc09d450 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e990 tx=0x7f00d400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.618+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4004d20 con 0x7f00dc15e3d0 2024-09-21T07:54:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.618+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00d400caa0 con 0x7f00dc15e3d0 2024-09-21T07:54:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.618+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00dc0af920 con 0x7f00dc15e3d0 2024-09-21T07:54:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.618+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00d4005020 con 0x7f00dc15e3d0 2024-09-21T07:54:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.618+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00dc0afee0 con 0x7f00dc15e3d0 2024-09-21T07:54:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.619+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00dc0739d0 con 0x7f00dc15e3d0 2024-09-21T07:54:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.620+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f00d40051c0 con 0x7f00dc15e3d0 2024-09-21T07:54:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.621+0000 7f00d9600640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00bc079430 0x7f00bc07b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.621+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f00d409c8b0 con 0x7f00dc15e3d0 2024-09-21T07:54:46.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.622+0000 7f00e1400640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00bc079430 0x7f00bc07b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:46.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.622+0000 7f00e1400640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00bc079430 0x7f00bc07b940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f00d0034000 tx=0x7f00d0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.626+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00d4016030 con 0x7f00dc15e3d0 2024-09-21T07:54:46.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.851+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f00dc0cf570 con 0x7f00dc15e3d0 2024-09-21T07:54:46.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.852+0000 7f00d9600640 1 -- 172.21.15.88:0/3992107027 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v108) ==== 53+0+73 (secure 0 0 0) 0x7f00d40636e0 con 0x7f00dc15e3d0 2024-09-21T07:54:46.854 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:54:46.856 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00bc079430 msgr2=0x7f00bc07b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00bc079430 0x7f00bc07b940 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f00d0034000 tx=0x7f00d0004730 comp rx=0 tx=0).stop 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 msgr2=0x7f00dc09d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 0x7f00dc09d450 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f00d400e990 tx=0x7f00d400ee50 comp rx=0 tx=0).stop 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 shutdown_connections 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00bc079430 0x7f00bc07b940 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00dc15e3d0 0x7f00dc09d450 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00dc1518d0 0x7f00dc1e5170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 --2- 172.21.15.88:0/3992107027 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00dc0735c0 0x7f00dc0b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 >> 172.21.15.88:0/3992107027 conn(0x7f00dc0b1e40 msgr2=0x7f00dc0d0840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 shutdown_connections 2024-09-21T07:54:46.857 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:46.854+0000 7f00e2800640 1 -- 172.21.15.88:0/3992107027 wait complete. 2024-09-21T07:54:46.882 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:54:46.883 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:54:46.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.883 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:54:46.923 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:54:46.924 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42158sec preferred_lft 42158sec 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:54:46.925 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:54:46.926 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.926 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:54:46.926 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:54:46.926 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:54:46.926 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:54:46.926 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:46.926 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:54:47.021 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:54:47.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.024 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:54:47.061 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:54:47.061 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:54:47.061 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:54:47.061 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:54:47.062 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:54:47.062 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:54:47.062 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:54:47.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:54:47.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:54:47.062 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:54:47.062 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:54:47.179 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:47 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:54:47.250 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:47.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.259 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:54:47.294 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:54:47.294 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:54:47.294 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:54:47.294 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.294 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:54:47.390 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:47.390 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:47.426 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:54:47.426 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:54:47.426 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:47.500 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:54:47.500 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.500 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:47.692 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:47.693 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:54:47.693 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:54:47.712 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:47.712 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:54:47.712 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:54:47.712 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:47.713 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:54:47.742 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:54:47.743 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:54:47.743 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:54:47.743 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:54:47.744 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:54:47.744 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:54:47.746 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:54:47.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:47.747 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:47.802 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:54:47.880 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:47 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:47.883 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:54:47 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:47.891 INFO:teuthology.orchestra.run.smithi175.stdout:client8257 2024-09-21T07:54:47.891 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:47.891 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:47.927 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.157+0000 7f9d8d200640 1 -- 172.21.15.88:0/2701898198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d8806d4b0 msgr2=0x7f9d880b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.157+0000 7f9d8d200640 1 --2- 172.21.15.88:0/2701898198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d8806d4b0 0x7f9d880b4620 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9d74009860 tx=0x7f9d7402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.157+0000 7f9d8d200640 1 -- 172.21.15.88:0/2701898198 shutdown_connections 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.157+0000 7f9d8d200640 1 --2- 172.21.15.88:0/2701898198 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d8806d4b0 0x7f9d880b4620 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.157+0000 7f9d8d200640 1 -- 172.21.15.88:0/2701898198 >> 172.21.15.88:0/2701898198 conn(0x7f9d880b2540 msgr2=0x7f9d880d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.158+0000 7f9d8d200640 1 -- 172.21.15.88:0/2701898198 shutdown_connections 2024-09-21T07:54:48.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.158+0000 7f9d8d200640 1 -- 172.21.15.88:0/2701898198 wait complete. 2024-09-21T07:54:48.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.160+0000 7f9d8d200640 1 Processor -- start 2024-09-21T07:54:48.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.160+0000 7f9d8d200640 1 -- start start 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.161+0000 7f9d8d200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d880af100 0x7f9d881f6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.161+0000 7f9d8d200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d880cc830 0x7f9d881f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.161+0000 7f9d8d200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 0x7f9d88155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.161+0000 7f9d8d200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880b37b0 con 0x7f9d881f2b50 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.161+0000 7f9d8d200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9d880b3630 con 0x7f9d880cc830 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.161+0000 7f9d8d200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9d880b3930 con 0x7f9d880af100 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 0x7f9d88155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 0x7f9d88155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37192/0 (socket says 172.21.15.88:37192) 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 -- 172.21.15.88:0/255714825 learned_addr learned my addr 172.21.15.88:0/255714825 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d85400640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d880cc830 0x7f9d881f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d85e00640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d880af100 0x7f9d881f6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 -- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d880af100 msgr2=0x7f9d881f6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d880af100 0x7f9d881f6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 -- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d880cc830 msgr2=0x7f9d881f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d880cc830 0x7f9d881f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 -- 172.21.15.88:0/255714825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d881f9be0 con 0x7f9d881f2b50 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.162+0000 7f9d86800640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 0x7f9d88155d70 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c700 tx=0x7f9d8000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.163+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d80003b70 con 0x7f9d881f2b50 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.163+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9d80003d10 con 0x7f9d881f2b50 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.163+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d8815d4f0 con 0x7f9d881f2b50 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.163+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d8000aab0 con 0x7f9d881f2b50 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.163+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d881546c0 con 0x7f9d881f2b50 2024-09-21T07:54:48.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.164+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9d88154150 con 0x7f9d881f2b50 2024-09-21T07:54:48.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.165+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f9d8000ac50 con 0x7f9d881f2b50 2024-09-21T07:54:48.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.166+0000 7f9d7e000640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9d68079430 0x7f9d6807b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.166+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f9d8009c490 con 0x7f9d881f2b50 2024-09-21T07:54:48.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.166+0000 7f9d85e00640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9d68079430 0x7f9d6807b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.167+0000 7f9d85e00640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9d68079430 0x7f9d6807b940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9d78007fd0 tx=0x7f9d78007eb0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:48.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.169+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9d881f9be0 con 0x7f9d881f2b50 2024-09-21T07:54:48.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.393+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f9d880d1030 con 0x7f9d881f2b50 2024-09-21T07:54:48.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.610+0000 7f9d7e000640 1 -- 172.21.15.88:0/255714825 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v109) ==== 136+0+0 (secure 0 0 0) 0x7f9d80063640 con 0x7f9d881f2b50 2024-09-21T07:54:48.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9d68079430 msgr2=0x7f9d6807b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9d68079430 0x7f9d6807b940 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f9d78007fd0 tx=0x7f9d78007eb0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 msgr2=0x7f9d88155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 0x7f9d88155d70 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000c700 tx=0x7f9d8000cbc0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 shutdown_connections 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9d68079430 0x7f9d6807b940 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9d881f2b50 0x7f9d88155d70 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9d880cc830 0x7f9d881f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 --2- 172.21.15.88:0/255714825 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9d880af100 0x7f9d881f6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 >> 172.21.15.88:0/255714825 conn(0x7f9d880b2540 msgr2=0x7f9d880c9620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 shutdown_connections 2024-09-21T07:54:48.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.613+0000 7f9d8d200640 1 -- 172.21.15.88:0/255714825 wait complete. 2024-09-21T07:54:48.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:54:48.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 -- 172.21.15.88:0/3964337277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 msgr2=0x7fc600156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 --2- 172.21.15.88:0/3964337277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc600156e40 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009860 tx=0x7fc5f002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 -- 172.21.15.88:0/3964337277 shutdown_connections 2024-09-21T07:54:48.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 --2- 172.21.15.88:0/3964337277 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc600156e40 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 -- 172.21.15.88:0/3964337277 >> 172.21.15.88:0/3964337277 conn(0x7fc60006d3d0 msgr2=0x7fc6000b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:48.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 -- 172.21.15.88:0/3964337277 shutdown_connections 2024-09-21T07:54:48.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.862+0000 7fc605800640 1 -- 172.21.15.88:0/3964337277 wait complete. 2024-09-21T07:54:48.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.864+0000 7fc605800640 1 Processor -- start 2024-09-21T07:54:48.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.864+0000 7fc605800640 1 -- start start 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.865+0000 7fc605800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 0x7fc6000b15e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.865+0000 7fc605800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc6000cc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.865+0000 7fc605800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc60015ce30 0x7fc6001f5aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.865+0000 7fc605800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc6000cd190 con 0x7fc6000705b0 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.865+0000 7fc605800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc6000cd010 con 0x7fc60015ce30 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc605800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc6000cd310 con 0x7fc6000cfc80 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 0x7fc6000b15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 0x7fc6000b15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37202/0 (socket says 172.21.15.88:37202) 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 -- 172.21.15.88:0/3028170608 learned_addr learned my addr 172.21.15.88:0/3028170608 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:48.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fe000640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc6000cc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 -- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 msgr2=0x7fc6000cc000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5ff400640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc60015ce30 0x7fc6001f5aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc6000cc000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 -- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc60015ce30 msgr2=0x7fc6001f5aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc60015ce30 0x7fc6001f5aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fea00640 1 -- 172.21.15.88:0/3028170608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6000ca940 con 0x7fc6000705b0 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5fe000640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc6000cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.866+0000 7fc5ff400640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc60015ce30 0x7fc6001f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.867+0000 7fc5fea00640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 0x7fc6000b15e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009980 tx=0x7fc5f0003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.867+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0005c90 con 0x7fc6000705b0 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.867+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc5f0005020 con 0x7fc6000705b0 2024-09-21T07:54:48.870 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.867+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6000cac20 con 0x7fc6000705b0 2024-09-21T07:54:48.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.867+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5f0005410 con 0x7fc6000705b0 2024-09-21T07:54:48.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.867+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6000cb1e0 con 0x7fc6000705b0 2024-09-21T07:54:48.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.868+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6000cd820 con 0x7fc6000705b0 2024-09-21T07:54:48.871 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.869+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fc5f00317c0 con 0x7fc6000705b0 2024-09-21T07:54:48.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.870+0000 7fc5f6a00640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e0079400 0x7fc5e007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:48.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.870+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fc5f00ba5c0 con 0x7fc6000705b0 2024-09-21T07:54:48.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.871+0000 7fc5fe000640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e0079400 0x7fc5e007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:48.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.871+0000 7fc5fe000640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e0079400 0x7fc5e007b910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001860 tx=0x7fc5ec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:48.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:48.875+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc5f003f030 con 0x7fc6000705b0 2024-09-21T07:54:49.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.082+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fc6000cb9c0 con 0x7fc5e0079400 2024-09-21T07:54:49.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.083+0000 7fc5f6a00640 1 -- 172.21.15.88:0/3028170608 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fc5ec00d7a0 con 0x7fc5e0079400 2024-09-21T07:54:49.086 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:54:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:54:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:54:49.086 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:54:49.087 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:54:49.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e0079400 msgr2=0x7fc5e007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e0079400 0x7fc5e007b910 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc5ec001860 tx=0x7fc5ec009040 comp rx=0 tx=0).stop 2024-09-21T07:54:49.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 msgr2=0x7fc6000b15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 0x7fc6000b15e0 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7fc5f0009980 tx=0x7fc5f0003c10 comp rx=0 tx=0).stop 2024-09-21T07:54:49.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 shutdown_connections 2024-09-21T07:54:49.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc5e0079400 0x7fc5e007b910 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc6000cfc80 0x7fc6000cc000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc60015ce30 0x7fc6001f5aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 --2- 172.21.15.88:0/3028170608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc6000705b0 0x7fc6000b15e0 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 >> 172.21.15.88:0/3028170608 conn(0x7fc60006d3d0 msgr2=0x7fc60015e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 shutdown_connections 2024-09-21T07:54:49.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.086+0000 7fc605800640 1 -- 172.21.15.88:0/3028170608 wait complete. 2024-09-21T07:54:49.110 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:54:49.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa713400640 1 -- 172.21.15.88:0/155349428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f0004e60 msgr2=0x7fa6f00052d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa713400640 1 --2- 172.21.15.88:0/155349428 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6f0004e60 0x7fa6f00052d0 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7fa70c0cb9a0 tx=0x7fa6f0005830 comp rx=0 tx=0).stop 2024-09-21T07:54:49.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa711400640 1 -- 172.21.15.88:0/155349428 reap_dead start 2024-09-21T07:54:49.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa713400640 1 -- 172.21.15.88:0/155349428 shutdown_connections 2024-09-21T07:54:49.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa713400640 1 -- 172.21.15.88:0/155349428 >> 172.21.15.88:0/155349428 conn(0x7fa70c0b2ff0 msgr2=0x7fa70c0b3400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:49.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa713400640 1 -- 172.21.15.88:0/155349428 shutdown_connections 2024-09-21T07:54:49.335 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.332+0000 7fa713400640 1 -- 172.21.15.88:0/155349428 wait complete. 2024-09-21T07:54:49.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.334+0000 7fa713400640 1 Processor -- start 2024-09-21T07:54:49.336 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.334+0000 7fa713400640 1 -- start start 2024-09-21T07:54:49.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa713400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0004e60 0x7fa70c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa713400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 0x7fa70c1f86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa713400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa70c1f8c20 0x7fa70c0cc790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa713400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c0c7540 con 0x7fa70c152d50 2024-09-21T07:54:49.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa713400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa70c0b1100 con 0x7fa70c1f8c20 2024-09-21T07:54:49.339 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa713400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa70c0c76c0 con 0x7fa6f0004e60 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa703e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 0x7fa70c1f86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa703e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 0x7fa70c1f86e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37208/0 (socket says 172.21.15.88:37208) 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa703e00640 1 -- 172.21.15.88:0/1710054580 learned_addr learned my addr 172.21.15.88:0/1710054580 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa710a00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0004e60 0x7fa70c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.336+0000 7fa711400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa70c1f8c20 0x7fa70c0cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa703e00640 1 -- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0004e60 msgr2=0x7fa70c152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa703e00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0004e60 0x7fa70c152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa703e00640 1 -- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa70c1f8c20 msgr2=0x7fa70c0cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa703e00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa70c1f8c20 0x7fa70c0cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa703e00640 1 -- 172.21.15.88:0/1710054580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa70c15be30 con 0x7fa70c152d50 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa710a00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0004e60 0x7fa70c152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa711400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa70c1f8c20 0x7fa70c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa703e00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 0x7fa70c1f86e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400d6a0 tx=0x7fa6f400db60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:49.340 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4017070 con 0x7fa70c152d50 2024-09-21T07:54:49.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa6f4005310 con 0x7fa70c152d50 2024-09-21T07:54:49.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6f4009820 con 0x7fa70c152d50 2024-09-21T07:54:49.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa70c156940 con 0x7fa70c152d50 2024-09-21T07:54:49.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.337+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa70c1570a0 con 0x7fa70c152d50 2024-09-21T07:54:49.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.338+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa70c15bba0 con 0x7fa70c152d50 2024-09-21T07:54:49.341 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.339+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7fa6f40099c0 con 0x7fa70c152d50 2024-09-21T07:54:49.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.340+0000 7fa709600640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e8079400 0x7fa6e807b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:49.342 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.341+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fa6f4099670 con 0x7fa70c152d50 2024-09-21T07:54:49.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.341+0000 7fa710a00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e8079400 0x7fa6e807b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:49.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.341+0000 7fa710a00640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e8079400 0x7fa6e807b910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc001870 tx=0x7fa6fc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:49.347 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.345+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa6f4063540 con 0x7fa70c152d50 2024-09-21T07:54:49.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.529+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fa70c15da10 con 0x7fa70c152d50 2024-09-21T07:54:49.532 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.530+0000 7fa709600640 1 -- 172.21.15.88:0/1710054580 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v50)=0 v50) ==== 122+0+0 (secure 0 0 0) 0x7fa6f4062e40 con 0x7fa70c152d50 2024-09-21T07:54:49.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e8079400 msgr2=0x7fa6e807b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e8079400 0x7fa6e807b910 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fa6fc001870 tx=0x7fa6fc009040 comp rx=0 tx=0).stop 2024-09-21T07:54:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 msgr2=0x7fa70c1f86e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 0x7fa70c1f86e0 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7fa6f400d6a0 tx=0x7fa6f400db60 comp rx=0 tx=0).stop 2024-09-21T07:54:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 shutdown_connections 2024-09-21T07:54:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa6e8079400 0x7fa6e807b910 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.536 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa70c1f8c20 0x7fa70c0cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa70c152d50 0x7fa70c1f86e0 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 --2- 172.21.15.88:0/1710054580 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa6f0004e60 0x7fa70c152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 >> 172.21.15.88:0/1710054580 conn(0x7fa70c0b2ff0 msgr2=0x7fa70c154e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 shutdown_connections 2024-09-21T07:54:49.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:49.532+0000 7fa713400640 1 -- 172.21.15.88:0/1710054580 wait complete. 2024-09-21T07:54:49.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.562 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/_nogroup/subvol68530/0e2ea9b3-aeee-49ff-8efc-27bb620da346) 2024-09-21T07:54:49.630 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.631 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:54:49.669 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:54:49.669 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.669 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/_nogroup/subvol68530/0e2ea9b3-aeee-49ff-8efc-27bb620da346) 2024-09-21T07:54:49.771 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:49.772 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/_nogroup/subvol68530/.meta 2024-09-21T07:54:49.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.831 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p volumes/subvol_grp2020/subvol48061/0e6b2257-845e-4292-a2d5-c37c707ae44d) 2024-09-21T07:54:49.933 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.933 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:54:49.969 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:54:49.969 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:49.969 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data volumes/subvol_grp2020/subvol48061/0e6b2257-845e-4292-a2d5-c37c707ae44d) 2024-09-21T07:54:50.072 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:50.072 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/volumes/subvol_grp2020/subvol48061/.meta 2024-09-21T07:54:50.129 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol68530 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.389+0000 7f56b1c00640 1 -- 172.21.15.88:0/3782183756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac145a40 msgr2=0x7f56ac073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.389+0000 7f56b1c00640 1 --2- 172.21.15.88:0/3782183756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac145a40 0x7f56ac073540 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f5698009830 tx=0x7f569802a5b0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.389+0000 7f56b1c00640 1 -- 172.21.15.88:0/3782183756 shutdown_connections 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.390+0000 7f56b1c00640 1 --2- 172.21.15.88:0/3782183756 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac145a40 0x7f56ac073540 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.390+0000 7f56b1c00640 1 --2- 172.21.15.88:0/3782183756 >> v2:172.21.15.175:3300/0 conn(0x7f56ac145550 0x7f56ac06fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.390+0000 7f56b1c00640 1 -- 172.21.15.88:0/3782183756 >> 172.21.15.88:0/3782183756 conn(0x7f56ac06d1c0 msgr2=0x7f56ac06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:50.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.390+0000 7f56b1c00640 1 -- 172.21.15.88:0/3782183756 shutdown_connections 2024-09-21T07:54:50.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.390+0000 7f56b1c00640 1 -- 172.21.15.88:0/3782183756 wait complete. 2024-09-21T07:54:50.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.392+0000 7f56b1c00640 1 Processor -- start 2024-09-21T07:54:50.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.392+0000 7f56b1c00640 1 -- start start 2024-09-21T07:54:50.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.393+0000 7f56b1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac073a80 0x7f56ac15bea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.393+0000 7f56b1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56ac145550 0x7f56ac138960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56b1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 0x7f56ac135cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56b1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac160a30 con 0x7f56ac145a40 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56b1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f56ac0ce940 con 0x7f56ac145550 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56b1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f56ac160bb0 con 0x7f56ac073a80 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 0x7f56ac135cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 0x7f56ac135cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37216/0 (socket says 172.21.15.88:37216) 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 -- 172.21.15.88:0/470559000 learned_addr learned my addr 172.21.15.88:0/470559000 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56aaa00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac073a80 0x7f56ac15bea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56aa000640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56ac145550 0x7f56ac138960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 -- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac073a80 msgr2=0x7f56ac15bea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac073a80 0x7f56ac15bea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 -- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56ac145550 msgr2=0x7f56ac138960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56ac145550 0x7f56ac138960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.394+0000 7f56ab400640 1 -- 172.21.15.88:0/470559000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ac136200 con 0x7f56ac145a40 2024-09-21T07:54:50.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.395+0000 7f56ab400640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 0x7f56ac135cc0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f56a400c390 tx=0x7f56a400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.395+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4017070 con 0x7f56ac145a40 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.395+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f56a40053c0 con 0x7f56ac145a40 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.395+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ac138100 con 0x7f56ac145a40 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.395+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56a4003830 con 0x7f56ac145a40 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.395+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ac139490 con 0x7f56ac145a40 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.396+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ac06f880 con 0x7f56ac145a40 2024-09-21T07:54:50.399 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.397+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f56a401a070 con 0x7f56ac145a40 2024-09-21T07:54:50.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.398+0000 7f56a2a00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f568c079400 0x7f568c07b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.398+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f56a409b1e0 con 0x7f56ac145a40 2024-09-21T07:54:50.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.398+0000 7f56aaa00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f568c079400 0x7f568c07b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.399+0000 7f56aaa00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f568c079400 0x7f568c07b910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f569c009a60 tx=0x7f569c010040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:50.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.403+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f56ac06f880 con 0x7f56ac145a40 2024-09-21T07:54:50.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.614+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol68530", "target": ["mon-mgr", ""]}) -- 0x7f56ac06fe60 con 0x7f568c079400 2024-09-21T07:54:50.622 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:50.621+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:50.623 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:50.621+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:50.623 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:50.621+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:50.623 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:50.621+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:50.623 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:50.621+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:50.693 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.691+0000 7f56a2a00640 1 -- 172.21.15.88:0/470559000 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+67 (secure 0 0 0) 0x7f56ac06fe60 con 0x7f568c079400 2024-09-21T07:54:50.693 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/_nogroup/subvol68530/0e2ea9b3-aeee-49ff-8efc-27bb620da346 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f568c079400 msgr2=0x7f568c07b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f568c079400 0x7f568c07b910 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f569c009a60 tx=0x7f569c010040 comp rx=0 tx=0).stop 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 msgr2=0x7f56ac135cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 0x7f56ac135cc0 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f56a400c390 tx=0x7f56a400c850 comp rx=0 tx=0).stop 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 shutdown_connections 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f568c079400 0x7f568c07b910 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56ac145a40 0x7f56ac135cc0 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f56ac145550 0x7f56ac138960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 --2- 172.21.15.88:0/470559000 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f56ac073a80 0x7f56ac15bea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 >> 172.21.15.88:0/470559000 conn(0x7f56ac06d1c0 msgr2=0x7f56ac144540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:50.696 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 shutdown_connections 2024-09-21T07:54:50.697 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.693+0000 7f56b1c00640 1 -- 172.21.15.88:0/470559000 wait complete. 2024-09-21T07:54:50.721 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol48061 subvol_grp2020 2024-09-21T07:54:50.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c8800640 1 -- 172.21.15.88:0/1463627006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81a8004d40 msgr2=0x7f81a8005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c8800640 1 --2- 172.21.15.88:0/1463627006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81a8004d40 0x7f81a8005150 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f81bc006fd0 tx=0x7f81a8005830 comp rx=0 tx=0).stop 2024-09-21T07:54:50.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c6800640 1 -- 172.21.15.88:0/1463627006 reap_dead start 2024-09-21T07:54:50.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c8800640 1 -- 172.21.15.88:0/1463627006 shutdown_connections 2024-09-21T07:54:50.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c8800640 1 -- 172.21.15.88:0/1463627006 >> 172.21.15.88:0/1463627006 conn(0x7f81c00b2540 msgr2=0x7f81c00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:50.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c8800640 1 -- 172.21.15.88:0/1463627006 shutdown_connections 2024-09-21T07:54:50.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.944+0000 7f81c8800640 1 -- 172.21.15.88:0/1463627006 wait complete. 2024-09-21T07:54:50.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.946+0000 7f81c8800640 1 Processor -- start 2024-09-21T07:54:50.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.946+0000 7f81c8800640 1 -- start start 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c8800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a8004d40 0x7f81c01f5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c8800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 0x7f81c01f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c8800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81c01f92e0 0x7f81c0097220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c8800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00b3120 con 0x7f81c009deb0 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c8800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81c00b48f0 con 0x7f81a8004d40 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c8800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81c00b32a0 con 0x7f81c01f92e0 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 0x7f81c01f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 0x7f81c01f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37250/0 (socket says 172.21.15.88:37250) 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 -- 172.21.15.88:0/3525167279 learned_addr learned my addr 172.21.15.88:0/3525167279 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:50.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5e00640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a8004d40 0x7f81c01f5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c6800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81c01f92e0 0x7f81c0097220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 -- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81c01f92e0 msgr2=0x7f81c0097220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81c01f92e0 0x7f81c0097220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 -- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a8004d40 msgr2=0x7f81c01f5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.948+0000 7f81c5400640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a8004d40 0x7f81c01f5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81c5400640 1 -- 172.21.15.88:0/3525167279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81c00b3f30 con 0x7f81c009deb0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81c5400640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 0x7f81c01f8da0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f81b0011ae0 tx=0x7f81b0009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b000fcb0 con 0x7f81c009deb0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f81b000fe50 con 0x7f81c009deb0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81b0004280 con 0x7f81c009deb0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81c01f9b70 con 0x7f81c009deb0 2024-09-21T07:54:50.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.949+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81c01544c0 con 0x7f81c009deb0 2024-09-21T07:54:50.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.950+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81c00b41a0 con 0x7f81c009deb0 2024-09-21T07:54:50.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.952+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f81b00035f0 con 0x7f81c009deb0 2024-09-21T07:54:50.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.952+0000 7f81ba000640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f81a4079430 0x7f81a407b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.953+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f81b009cac0 con 0x7f81c009deb0 2024-09-21T07:54:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.953+0000 7f81c5e00640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f81a4079430 0x7f81a407b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:50.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.953+0000 7f81c5e00640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f81a4079430 0x7f81a407b940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009b90 tx=0x7f81b4005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:50.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:50.957+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f81b0063f40 con 0x7f81c009deb0 2024-09-21T07:54:51.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.170+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol48061", "group_name": "subvol_grp2020", "target": ["mon-mgr", ""]}) -- 0x7f81c00aeef0 con 0x7f81a4079430 2024-09-21T07:54:51.228 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.226+0000 7f81ba000640 1 -- 172.21.15.88:0/3525167279 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+73 (secure 0 0 0) 0x7f81c00aeef0 con 0x7f81a4079430 2024-09-21T07:54:51.229 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp2020/subvol48061/0e6b2257-845e-4292-a2d5-c37c707ae44d 2024-09-21T07:54:51.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f81a4079430 msgr2=0x7f81a407b940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.231 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f81a4079430 0x7f81a407b940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009b90 tx=0x7f81b4005f90 comp rx=0 tx=0).stop 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 msgr2=0x7f81c01f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 0x7f81c01f8da0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f81b0011ae0 tx=0x7f81b0009f10 comp rx=0 tx=0).stop 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 shutdown_connections 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f81a4079430 0x7f81a407b940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81c01f92e0 0x7f81c0097220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81c009deb0 0x7f81c01f8da0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 --2- 172.21.15.88:0/3525167279 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81a8004d40 0x7f81c01f5030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 >> 172.21.15.88:0/3525167279 conn(0x7f81c00b2540 msgr2=0x7f81c00989b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 shutdown_connections 2024-09-21T07:54:51.232 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.229+0000 7f81c8800640 1 -- 172.21.15.88:0/3525167279 wait complete. 2024-09-21T07:54:51.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:51.254 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_nogroup/subvol68530/.meta) 2024-09-21T07:54:51.309 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:54:51.310 INFO:teuthology.orchestra.run.smithi088.stdout:version = 2 2024-09-21T07:54:51.310 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:54:51.310 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/_nogroup/subvol68530/0e2ea9b3-aeee-49ff-8efc-27bb620da346 2024-09-21T07:54:51.310 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:54:51.310 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:51.312 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:51.313 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/subvol_grp2020/subvol48061/.meta) 2024-09-21T07:54:51.369 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:54:51.369 INFO:teuthology.orchestra.run.smithi088.stdout:version = 2 2024-09-21T07:54:51.369 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:54:51.369 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/subvol_grp2020/subvol48061/0e6b2257-845e-4292-a2d5-c37c707ae44d 2024-09-21T07:54:51.369 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:54:51.369 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:54:51.372 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol68530 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 -- 172.21.15.88:0/3918925785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10145a40 msgr2=0x7f6b10073540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 --2- 172.21.15.88:0/3918925785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10145a40 0x7f6b10073540 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f6b101464e0 tx=0x7f6afc02a5d0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 -- 172.21.15.88:0/3918925785 shutdown_connections 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 --2- 172.21.15.88:0/3918925785 >> v2:172.21.15.175:3300/0 conn(0x7f6b10073a80 0x7f6b100cd610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 --2- 172.21.15.88:0/3918925785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10145a40 0x7f6b10073540 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 -- 172.21.15.88:0/3918925785 >> 172.21.15.88:0/3918925785 conn(0x7f6b1006d1c0 msgr2=0x7f6b1006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.592+0000 7f6b17e00640 1 -- 172.21.15.88:0/3918925785 shutdown_connections 2024-09-21T07:54:51.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.593+0000 7f6b17e00640 1 -- 172.21.15.88:0/3918925785 wait complete. 2024-09-21T07:54:51.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.595+0000 7f6b17e00640 1 Processor -- start 2024-09-21T07:54:51.597 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.595+0000 7f6b17e00640 1 -- start start 2024-09-21T07:54:51.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.596+0000 7f6b17e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 0x7f6b1013bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.596+0000 7f6b17e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b10145550 0x7f6b101f71d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.596+0000 7f6b17e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10145a40 0x7f6b100aa420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.596+0000 7f6b17e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b100ce0e0 con 0x7f6b10073a80 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.596+0000 7f6b17e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6b100cdf60 con 0x7f6b10145a40 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.596+0000 7f6b17e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6b100ce260 con 0x7f6b10145550 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 0x7f6b1013bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 0x7f6b1013bd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37272/0 (socket says 172.21.15.88:37272) 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 -- 172.21.15.88:0/261327227 learned_addr learned my addr 172.21.15.88:0/261327227 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b14a00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b10145550 0x7f6b101f71d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10145a40 0x7f6b100aa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:51.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 -- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b10145550 msgr2=0x7f6b101f71d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b10145550 0x7f6b101f71d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 -- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10145a40 msgr2=0x7f6b100aa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10145a40 0x7f6b100aa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 -- 172.21.15.88:0/261327227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b1015cf40 con 0x7f6b10073a80 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b15400640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 0x7f6b1013bd70 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c730 tx=0x7f6b0400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.597+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b04016020 con 0x7f6b10073a80 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.598+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6b0400a950 con 0x7f6b10073a80 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.598+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400ada0 con 0x7f6b10073a80 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.598+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b1015d220 con 0x7f6b10073a80 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.598+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b1015d7e0 con 0x7f6b10073a80 2024-09-21T07:54:51.601 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.599+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b100ce780 con 0x7f6b10073a80 2024-09-21T07:54:51.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.600+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 26) ==== 102206+0+0 (secure 0 0 0) 0x7f6b04019020 con 0x7f6b10073a80 2024-09-21T07:54:51.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.601+0000 7f6b0d600640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6af0079400 0x7f6af007b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:51.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.601+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f6b0409bcb0 con 0x7f6b10073a80 2024-09-21T07:54:51.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.601+0000 7f6b14a00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6af0079400 0x7f6af007b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:51.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.602+0000 7f6b14a00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6af0079400 0x7f6af007b910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6afc034000 tx=0x7f6afc004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:51.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.605+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6b04020030 con 0x7f6b10073a80 2024-09-21T07:54:51.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.635+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 27) ==== 102257+0+0 (secure 0 0 0) 0x7f6b0405ec10 con 0x7f6b10073a80 2024-09-21T07:54:51.829 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.827+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol68530", "target": ["mon-mgr", ""]}) -- 0x7f6b1013f330 con 0x7f6af0079400 2024-09-21T07:54:51.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.844+0000 7f6b0d600640 1 -- 172.21.15.88:0/261327227 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f6b1013f330 con 0x7f6af0079400 2024-09-21T07:54:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6af0079400 msgr2=0x7f6af007b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6af0079400 0x7f6af007b910 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f6afc034000 tx=0x7f6afc004730 comp rx=0 tx=0).stop 2024-09-21T07:54:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 msgr2=0x7f6b1013bd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:51.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 0x7f6b1013bd70 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400c730 tx=0x7f6b0400cbf0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 shutdown_connections 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6af0079400 0x7f6af007b910 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6b10145550 0x7f6b101f71d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6b10145a40 0x7f6b100aa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 --2- 172.21.15.88:0/261327227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b10073a80 0x7f6b1013bd70 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 >> 172.21.15.88:0/261327227 conn(0x7f6b1006d1c0 msgr2=0x7f6b1013f650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 shutdown_connections 2024-09-21T07:54:51.850 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:51.847+0000 7f6b17e00640 1 -- 172.21.15.88:0/261327227 wait complete. 2024-09-21T07:54:51.853 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.851+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.853 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.851+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.854 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.851+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.854 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.851+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.854 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.851+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.871 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.868+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.871 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.869+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.871 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.869+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.871 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.869+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.871 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:54:51.869+0000 7facab800640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:54:51.875 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol48061 subvol_grp2020 2024-09-21T07:54:52.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 -- 172.21.15.88:0/2953953258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef940b3ec0 msgr2=0x7fef940b47c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 --2- 172.21.15.88:0/2953953258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef940b3ec0 0x7fef940b47c0 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7fef88005fd0 tx=0x7fef8802a540 comp rx=0 tx=0).stop 2024-09-21T07:54:52.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 -- 172.21.15.88:0/2953953258 shutdown_connections 2024-09-21T07:54:52.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 --2- 172.21.15.88:0/2953953258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef940705b0 0x7fef940af3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 --2- 172.21.15.88:0/2953953258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef940b3ec0 0x7fef940b47c0 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 -- 172.21.15.88:0/2953953258 >> 172.21.15.88:0/2953953258 conn(0x7fef940b1dd0 msgr2=0x7fef940b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:52.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 -- 172.21.15.88:0/2953953258 shutdown_connections 2024-09-21T07:54:52.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.098+0000 7fef9b200640 1 -- 172.21.15.88:0/2953953258 wait complete. 2024-09-21T07:54:52.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.100+0000 7fef9b200640 1 Processor -- start 2024-09-21T07:54:52.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.100+0000 7fef9b200640 1 -- start start 2024-09-21T07:54:52.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef940705b0 0x7fef941545f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef940b3ec0 0x7fef941f9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 0x7fef941421d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef94153a80 con 0x7fef941532e0 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fef94153900 con 0x7fef940b3ec0 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fef94153c00 con 0x7fef940705b0 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 0x7fef941421d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 0x7fef941421d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:37302/0 (socket says 172.21.15.88:37302) 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 -- 172.21.15.88:0/1988737973 learned_addr learned my addr 172.21.15.88:0/1988737973 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef99e00640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef940705b0 0x7fef941545f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef99400640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef940b3ec0 0x7fef941f9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 -- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef940705b0 msgr2=0x7fef941545f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef940705b0 0x7fef941545f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 -- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef940b3ec0 msgr2=0x7fef941f9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef940b3ec0 0x7fef941f9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:52.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.102+0000 7fef9a800640 1 -- 172.21.15.88:0/1988737973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef94152160 con 0x7fef941532e0 2024-09-21T07:54:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.103+0000 7fef9a800640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 0x7fef941421d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c730 tx=0x7fef9000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.103+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef90017070 con 0x7fef941532e0 2024-09-21T07:54:52.106 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.103+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fef90003c60 con 0x7fef941532e0 2024-09-21T07:54:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.103+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef900052d0 con 0x7fef941532e0 2024-09-21T07:54:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.103+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef94155160 con 0x7fef941532e0 2024-09-21T07:54:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.103+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef94155720 con 0x7fef941532e0 2024-09-21T07:54:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.104+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef94154ed0 con 0x7fef941532e0 2024-09-21T07:54:52.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.105+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 27) ==== 102257+0+0 (secure 0 0 0) 0x7fef9001a070 con 0x7fef941532e0 2024-09-21T07:54:52.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.106+0000 7fef8e000640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fef78079490 0x7fef7807b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:52.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.106+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fef90098d00 con 0x7fef941532e0 2024-09-21T07:54:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.107+0000 7fef99e00640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fef78079490 0x7fef7807b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:52.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.107+0000 7fef99e00640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fef78079490 0x7fef7807b9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fef88033000 tx=0x7fef88004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:52.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.111+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fef940b4ec0 con 0x7fef941532e0 2024-09-21T07:54:52.324 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:52.322+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol48061", "group_name": "subvol_grp2020", "target": ["mon-mgr", ""]}) -- 0x7fef940af3c0 con 0x7fef78079490 2024-09-21T07:54:53.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:53.655+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fef90005680 con 0x7fef941532e0 2024-09-21T07:54:57.614 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:57.622 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:54:58.113 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.112+0000 7fef8e000640 1 -- 172.21.15.88:0/1988737973 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fef940af3c0 con 0x7fef78079490 2024-09-21T07:54:58.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fef78079490 msgr2=0x7fef7807b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fef78079490 0x7fef7807b9a0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fef88033000 tx=0x7fef88004730 comp rx=0 tx=0).stop 2024-09-21T07:54:58.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 msgr2=0x7fef941421d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 0x7fef941421d0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7fef9000c730 tx=0x7fef9000cbf0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 shutdown_connections 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fef78079490 0x7fef7807b9a0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef941532e0 0x7fef941421d0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fef940b3ec0 0x7fef941f9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 --2- 172.21.15.88:0/1988737973 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fef940705b0 0x7fef941545f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 >> 172.21.15.88:0/1988737973 conn(0x7fef940b1dd0 msgr2=0x7fef9415f660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 shutdown_connections 2024-09-21T07:54:58.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.115+0000 7fef9b200640 1 -- 172.21.15.88:0/1988737973 wait complete. 2024-09-21T07:54:58.146 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:54:58.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:54:58.236 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:54:58.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:54:58.244 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:54:58.244 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:58.328 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:54:58.425 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:54:58.425 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:58.425 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:58.425 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:58.461 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:58.461 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:58.536 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:58.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc7600640 1 -- 172.21.15.88:0/4000611701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 msgr2=0x7f9ca4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc7600640 1 --2- 172.21.15.88:0/4000611701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9ca4004c10 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c2e0 tx=0x7f9ca4005830 comp rx=0 tx=0).stop 2024-09-21T07:54:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc5600640 1 -- 172.21.15.88:0/4000611701 reap_dead start 2024-09-21T07:54:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc7600640 1 -- 172.21.15.88:0/4000611701 shutdown_connections 2024-09-21T07:54:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc7600640 1 -- 172.21.15.88:0/4000611701 >> 172.21.15.88:0/4000611701 conn(0x7f9cc006d3d0 msgr2=0x7f9cc00b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc7600640 1 -- 172.21.15.88:0/4000611701 shutdown_connections 2024-09-21T07:54:58.797 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.795+0000 7f9cc7600640 1 -- 172.21.15.88:0/4000611701 wait complete. 2024-09-21T07:54:58.798 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.798+0000 7f9cc7600640 1 Processor -- start 2024-09-21T07:54:58.799 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.798+0000 7f9cc7600640 1 -- start start 2024-09-21T07:54:58.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc7600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9cc00c68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc7600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cc009a2d0 0x7f9cc00b15d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc7600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cc009a6e0 0x7f9cc00b1b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc7600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc0071340 con 0x7f9ca4004800 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc7600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9cc00711c0 con 0x7f9cc009a2d0 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc7600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9cc00cfc80 con 0x7f9cc009a6e0 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9cc00c68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc4c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9cc00c68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57428/0 (socket says 172.21.15.88:57428) 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.799+0000 7f9cc4c00640 1 -- 172.21.15.88:0/2194916576 learned_addr learned my addr 172.21.15.88:0/2194916576 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cb7e00640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cc009a2d0 0x7f9cc00b15d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc5600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cc009a6e0 0x7f9cc00b1b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc4c00640 1 -- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cc009a6e0 msgr2=0x7f9cc00b1b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc4c00640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cc009a6e0 0x7f9cc00b1b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc4c00640 1 -- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cc009a2d0 msgr2=0x7f9cc00b15d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:58.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc4c00640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cc009a2d0 0x7f9cc00b15d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc4c00640 1 -- 172.21.15.88:0/2194916576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9cc01f9090 con 0x7f9ca4004800 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc4c00640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9cc00c68f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0011ab0 tx=0x7f9cb0011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb001e070 con 0x7f9ca4004800 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9cb0004460 con 0x7f9ca4004800 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9cc01f9370 con 0x7f9ca4004800 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.800+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000fba0 con 0x7f9ca4004800 2024-09-21T07:54:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.801+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9cc01f9930 con 0x7f9ca4004800 2024-09-21T07:54:58.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.802+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9cc00cf570 con 0x7f9ca4004800 2024-09-21T07:54:58.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.803+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f9cb0021070 con 0x7f9ca4004800 2024-09-21T07:54:58.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.804+0000 7f9cbd600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9c9c0795e0 0x7f9c9c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:58.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.805+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f9cb0099e70 con 0x7f9ca4004800 2024-09-21T07:54:58.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.805+0000 7f9cb7e00640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9c9c0795e0 0x7f9c9c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:58.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.805+0000 7f9cb7e00640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9c9c0795e0 0x7f9c9c07baf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c2e0 tx=0x7f9ca802d000 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:58.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:58.809+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9cb0063740 con 0x7f9ca4004800 2024-09-21T07:54:59.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.009+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f9cc01f56c0 con 0x7f9ca4004800 2024-09-21T07:54:59.011 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.009+0000 7f9cbd600640 1 -- 172.21.15.88:0/2194916576 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7f9cb0068170 con 0x7f9ca4004800 2024-09-21T07:54:59.011 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9c9c0795e0 msgr2=0x7f9c9c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9c9c0795e0 0x7f9c9c07baf0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ca800c2e0 tx=0x7f9ca802d000 comp rx=0 tx=0).stop 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 msgr2=0x7f9cc00c68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9cc00c68f0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0011ab0 tx=0x7f9cb0011f70 comp rx=0 tx=0).stop 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 shutdown_connections 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9c9c0795e0 0x7f9c9c07baf0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.014 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9cc009a6e0 0x7f9cc00b1b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9cc009a2d0 0x7f9cc00b15d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 --2- 172.21.15.88:0/2194916576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ca4004800 0x7f9cc00c68f0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 >> 172.21.15.88:0/2194916576 conn(0x7f9cc006d3d0 msgr2=0x7f9cc015d710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:59.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 shutdown_connections 2024-09-21T07:54:59.015 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.012+0000 7f9cc7600640 1 -- 172.21.15.88:0/2194916576 wait complete. 2024-09-21T07:54:59.039 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:54:59.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:59.039 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:54:59.125 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:54:59.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:59.126 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:54:59.175 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:59.176 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:59.176 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:59.213 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:59.213 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:59.290 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:54:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 -- 172.21.15.88:0/349476541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd358070cc0 msgr2=0x7fd358156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 --2- 172.21.15.88:0/349476541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd358070cc0 0x7fd358156e40 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7fd340009860 tx=0x7fd34002a560 comp rx=0 tx=0).stop 2024-09-21T07:54:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 -- 172.21.15.88:0/349476541 shutdown_connections 2024-09-21T07:54:59.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 --2- 172.21.15.88:0/349476541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd358070cc0 0x7fd358156e40 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 -- 172.21.15.88:0/349476541 >> 172.21.15.88:0/349476541 conn(0x7fd35806d3d0 msgr2=0x7fd3580b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:59.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 -- 172.21.15.88:0/349476541 shutdown_connections 2024-09-21T07:54:59.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.510+0000 7fd35d000640 1 -- 172.21.15.88:0/349476541 wait complete. 2024-09-21T07:54:59.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.512+0000 7fd35d000640 1 Processor -- start 2024-09-21T07:54:59.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.512+0000 7fd35d000640 1 -- start start 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd35d000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580cfc80 0x7fd3581f5530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd35d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 0x7fd35815b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd35d000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358160b90 0x7fd35815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd35d000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580cd100 con 0x7fd35815ce30 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd35d000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3580ccf80 con 0x7fd358160b90 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd35d000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd3580cd280 con 0x7fd3580cfc80 2024-09-21T07:54:59.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd34d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 0x7fd35815b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd34d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 0x7fd35815b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57442/0 (socket says 172.21.15.88:57442) 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd34d400640 1 -- 172.21.15.88:0/1623848660 learned_addr learned my addr 172.21.15.88:0/1623848660 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd355e00640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580cfc80 0x7fd3581f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.514+0000 7fd356800640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358160b90 0x7fd35815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34d400640 1 -- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580cfc80 msgr2=0x7fd3581f5530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34d400640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580cfc80 0x7fd3581f5530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34d400640 1 -- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358160b90 msgr2=0x7fd35815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34d400640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358160b90 0x7fd35815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34d400640 1 -- 172.21.15.88:0/1623848660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3580caae0 con 0x7fd35815ce30 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd355e00640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580cfc80 0x7fd3581f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd356800640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358160b90 0x7fd35815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34d400640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 0x7fd35815b900 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd340009980 tx=0x7fd340033ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340031d60 con 0x7fd35815ce30 2024-09-21T07:54:59.517 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd340033e50 con 0x7fd35815ce30 2024-09-21T07:54:59.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3580cad80 con 0x7fd35815ce30 2024-09-21T07:54:59.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd340005260 con 0x7fd35815ce30 2024-09-21T07:54:59.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.515+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3580cb340 con 0x7fd35815ce30 2024-09-21T07:54:59.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.516+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3580cd790 con 0x7fd35815ce30 2024-09-21T07:54:59.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.517+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fd340005400 con 0x7fd35815ce30 2024-09-21T07:54:59.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.518+0000 7fd34ea00640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd3340795b0 0x7fd33407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:54:59.519 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.518+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fd3400bab40 con 0x7fd35815ce30 2024-09-21T07:54:59.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.519+0000 7fd355e00640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd3340795b0 0x7fd33407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:54:59.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.519+0000 7fd355e00640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd3340795b0 0x7fd33407bac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd348001780 tx=0x7fd348009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:54:59.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.523+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd34003f030 con 0x7fd35815ce30 2024-09-21T07:54:59.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.719+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fd3580afa70 con 0x7fd35815ce30 2024-09-21T07:54:59.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.720+0000 7fd34ea00640 1 -- 172.21.15.88:0/1623848660 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v156) ==== 80+0+0 (secure 0 0 0) 0x7fd340031770 con 0x7fd35815ce30 2024-09-21T07:54:59.721 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:54:59.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd3340795b0 msgr2=0x7fd33407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd3340795b0 0x7fd33407bac0 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7fd348001780 tx=0x7fd348009040 comp rx=0 tx=0).stop 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 msgr2=0x7fd35815b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 0x7fd35815b900 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7fd340009980 tx=0x7fd340033ba0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 shutdown_connections 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd3340795b0 0x7fd33407bac0 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd358160b90 0x7fd35815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd35815ce30 0x7fd35815b900 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 --2- 172.21.15.88:0/1623848660 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd3580cfc80 0x7fd3581f5530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 >> 172.21.15.88:0/1623848660 conn(0x7fd35806d3d0 msgr2=0x7fd3580c69e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 shutdown_connections 2024-09-21T07:54:59.724 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:54:59.722+0000 7fd35d000640 1 -- 172.21.15.88:0/1623848660 wait complete. 2024-09-21T07:54:59.745 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:54:59.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:59.746 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:54:59.810 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:54:59.810 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:54:59.810 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:54:59.847 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:59.848 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:54:59.848 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:59.883 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:54:59.883 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:54:59.884 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:54:59.884 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:54:59.923 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:54:59.923 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 -- 172.21.15.88:0/3268466550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40b3550 msgr2=0x7f36e40b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3268466550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40b3550 0x7f36e40b3960 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f36e4075250 tx=0x7f36dc02a820 comp rx=0 tx=0).stop 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 -- 172.21.15.88:0/3268466550 shutdown_connections 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3268466550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40b3550 0x7f36e40b3960 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3268466550 >> v2:172.21.15.175:3300/0 conn(0x7f36e415d760 0x7f36e41535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 -- 172.21.15.88:0/3268466550 >> 172.21.15.88:0/3268466550 conn(0x7f36e406d730 msgr2=0x7f36e40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 -- 172.21.15.88:0/3268466550 shutdown_connections 2024-09-21T07:55:00.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.143+0000 7f36e8c00640 1 -- 172.21.15.88:0/3268466550 wait complete. 2024-09-21T07:55:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.145+0000 7f36e8c00640 1 Processor -- start 2024-09-21T07:55:00.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.146+0000 7f36e8c00640 1 -- start start 2024-09-21T07:55:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 0x7f36e41f67d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36e40b3550 0x7f36e4159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e8c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36e415d760 0x7f36e4151010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e8c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f36e40b4da0 con 0x7f36e40ad0e0 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e8c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f36e40b4c20 con 0x7f36e415d760 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e8c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f36e40b3ea0 con 0x7f36e40b3550 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 0x7f36e41f67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 0x7f36e41f67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57462/0 (socket says 172.21.15.88:57462) 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.147+0000 7f36e1e00640 1 -- 172.21.15.88:0/3654848955 learned_addr learned my addr 172.21.15.88:0/3654848955 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e2800640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36e415d760 0x7f36e4151010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36d9400640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36e40b3550 0x7f36e4159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e1e00640 1 -- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36e40b3550 msgr2=0x7f36e4159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e1e00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36e40b3550 0x7f36e4159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e1e00640 1 -- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36e415d760 msgr2=0x7f36e4151010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e1e00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36e415d760 0x7f36e4151010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e1e00640 1 -- 172.21.15.88:0/3654848955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36e40afd50 con 0x7f36e40ad0e0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e1e00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 0x7f36e41f67d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f36d400d770 tx=0x7f36d400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d4017070 con 0x7f36e40ad0e0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f36d4009d20 con 0x7f36e40ad0e0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36e40b0010 con 0x7f36e40ad0e0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36d40106c0 con 0x7f36e40ad0e0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.148+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36e40b05d0 con 0x7f36e40ad0e0 2024-09-21T07:55:00.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.149+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36e41572b0 con 0x7f36e40ad0e0 2024-09-21T07:55:00.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.151+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f36d401a070 con 0x7f36e40ad0e0 2024-09-21T07:55:00.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.152+0000 7f36daa00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36c00795e0 0x7f36c007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.152+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f36d409c6a0 con 0x7f36e40ad0e0 2024-09-21T07:55:00.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.152+0000 7f36d9400640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36c00795e0 0x7f36c007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.152+0000 7f36d9400640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36c00795e0 0x7f36c007baf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007fd0 tx=0x7f36cc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:00.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.156+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f36d4063670 con 0x7f36e40ad0e0 2024-09-21T07:55:00.379 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.378+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f36e40b43c0 con 0x7f36e40ad0e0 2024-09-21T07:55:00.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.379+0000 7f36daa00640 1 -- 172.21.15.88:0/3654848955 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5239 (secure 0 0 0) 0x7f36d4062f70 con 0x7f36e40ad0e0 2024-09-21T07:55:00.380 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:00.380 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":109,"btime":"2024-09-21T07:54:48:609789+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:37.615557+0000","modified":"2024-09-21T07:54:40.637260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8772":{"gid":8772,"name":"b","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2687096264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2687096264},{"type":"v1","addr":"172.21.15.175:6835","nonce":2687096264}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8772,"qdb_cluster":[8772]},"id":12}]} 2024-09-21T07:55:00.381 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 109 2024-09-21T07:55:00.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36c00795e0 msgr2=0x7f36c007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.382 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36c00795e0 0x7f36c007baf0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007fd0 tx=0x7f36cc009040 comp rx=0 tx=0).stop 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 msgr2=0x7f36e41f67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 0x7f36e41f67d0 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f36d400d770 tx=0x7f36d400dc30 comp rx=0 tx=0).stop 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 shutdown_connections 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f36c00795e0 0x7f36c007baf0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f36e40b3550 0x7f36e4159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f36e415d760 0x7f36e4151010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 --2- 172.21.15.88:0/3654848955 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36e40ad0e0 0x7f36e41f67d0 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 >> 172.21.15.88:0/3654848955 conn(0x7f36e406d730 msgr2=0x7f36e4152980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 shutdown_connections 2024-09-21T07:55:00.383 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.381+0000 7f36e8c00640 1 -- 172.21.15.88:0/3654848955 wait complete. 2024-09-21T07:55:00.405 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 -- 172.21.15.88:0/577787656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0ad3c0 msgr2=0x7f524c0ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 --2- 172.21.15.88:0/577787656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0ad3c0 0x7f524c0ad7d0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f524c074580 tx=0x7f523802a560 comp rx=0 tx=0).stop 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 -- 172.21.15.88:0/577787656 shutdown_connections 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 --2- 172.21.15.88:0/577787656 >> v2:172.21.15.175:3300/0 conn(0x7f524c0add10 0x7f524c1646e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 --2- 172.21.15.88:0/577787656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0ad3c0 0x7f524c0ad7d0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 -- 172.21.15.88:0/577787656 >> 172.21.15.88:0/577787656 conn(0x7f524c06d730 msgr2=0x7f524c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 -- 172.21.15.88:0/577787656 shutdown_connections 2024-09-21T07:55:00.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.624+0000 7f5251a00640 1 -- 172.21.15.88:0/577787656 wait complete. 2024-09-21T07:55:00.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.627+0000 7f5251a00640 1 Processor -- start 2024-09-21T07:55:00.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.627+0000 7f5251a00640 1 -- start start 2024-09-21T07:55:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f5251a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 0x7f524c0997f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f5251a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c0ad3c0 0x7f524c158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f5251a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524c0add10 0x7f524c158c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f5251a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f524c0712b0 con 0x7f524c0704b0 2024-09-21T07:55:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f5251a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f524c071130 con 0x7f524c0ad3c0 2024-09-21T07:55:00.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f5251a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f524c164c20 con 0x7f524c0add10 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f524aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 0x7f524c0997f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f524aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 0x7f524c0997f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57474/0 (socket says 172.21.15.88:57474) 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.628+0000 7f524aa00640 1 -- 172.21.15.88:0/2365172643 learned_addr learned my addr 172.21.15.88:0/2365172643 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524aa00640 1 -- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524c0add10 msgr2=0x7f524c158c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524b400640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524c0add10 0x7f524c158c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524a000640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c0ad3c0 0x7f524c158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524aa00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524c0add10 0x7f524c158c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524aa00640 1 -- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c0ad3c0 msgr2=0x7f524c158700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524aa00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c0ad3c0 0x7f524c158700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524aa00640 1 -- 172.21.15.88:0/2365172643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f524c0ae8a0 con 0x7f524c0704b0 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524b400640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524c0add10 0x7f524c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524a000640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c0ad3c0 0x7f524c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f524aa00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 0x7f524c0997f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c3f0 tx=0x7f523c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c017070 con 0x7f524c0704b0 2024-09-21T07:55:00.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f523c0035f0 con 0x7f524c0704b0 2024-09-21T07:55:00.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.629+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f523c003a40 con 0x7f524c0704b0 2024-09-21T07:55:00.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.630+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f524c0aee40 con 0x7f524c0704b0 2024-09-21T07:55:00.634 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.630+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f524c163500 con 0x7f524c0704b0 2024-09-21T07:55:00.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.631+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f524c0afb20 con 0x7f524c0704b0 2024-09-21T07:55:00.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.632+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f523c01a070 con 0x7f524c0704b0 2024-09-21T07:55:00.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.633+0000 7f5242a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f522c0795e0 0x7f522c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:00.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.633+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f523c098ba0 con 0x7f524c0704b0 2024-09-21T07:55:00.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.633+0000 7f524a000640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f522c0795e0 0x7f522c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:00.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.634+0000 7f524a000640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f522c0795e0 0x7f522c07baf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5238033000 tx=0x7f5238004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:00.638 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.638+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f523c0628a0 con 0x7f524c0704b0 2024-09-21T07:55:00.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.861+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f524c15e420 con 0x7f524c0704b0 2024-09-21T07:55:00.862 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.861+0000 7f5242a00640 1 -- 172.21.15.88:0/2365172643 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5239 (secure 0 0 0) 0x7f523c0621a0 con 0x7f524c0704b0 2024-09-21T07:55:00.862 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:00.863 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":109,"btime":"2024-09-21T07:54:48:609789+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:37.615557+0000","modified":"2024-09-21T07:54:40.637260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8772":{"gid":8772,"name":"b","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2687096264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2687096264},{"type":"v1","addr":"172.21.15.175:6835","nonce":2687096264}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8772,"qdb_cluster":[8772]},"id":12}]} 2024-09-21T07:55:00.863 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 109 2024-09-21T07:55:00.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f522c0795e0 msgr2=0x7f522c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f522c0795e0 0x7f522c07baf0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5238033000 tx=0x7f5238004730 comp rx=0 tx=0).stop 2024-09-21T07:55:00.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 msgr2=0x7f524c0997f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:00.865 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 0x7f524c0997f0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f523c00c3f0 tx=0x7f523c00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 shutdown_connections 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f522c0795e0 0x7f522c07baf0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f524c0add10 0x7f524c158c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f524c0ad3c0 0x7f524c158700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 --2- 172.21.15.88:0/2365172643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f524c0704b0 0x7f524c0997f0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 >> 172.21.15.88:0/2365172643 conn(0x7f524c06d730 msgr2=0x7f524c151930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 shutdown_connections 2024-09-21T07:55:00.866 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:00.864+0000 7f5251a00640 1 -- 172.21.15.88:0/2365172643 wait complete. 2024-09-21T07:55:00.892 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1c00640 1 -- 172.21.15.88:0/2135448236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03a0004d40 msgr2=0x7f03a00051b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1c00640 1 --2- 172.21.15.88:0/2135448236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03a0004d40 0x7f03a00051b0 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f03b400c6f0 tx=0x7f03a0005830 comp rx=0 tx=0).stop 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1200640 1 -- 172.21.15.88:0/2135448236 reap_dead start 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1c00640 1 -- 172.21.15.88:0/2135448236 shutdown_connections 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1c00640 1 -- 172.21.15.88:0/2135448236 >> 172.21.15.88:0/2135448236 conn(0x7f03bc0b1e40 msgr2=0x7f03bc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1c00640 1 -- 172.21.15.88:0/2135448236 shutdown_connections 2024-09-21T07:55:01.115 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.113+0000 7f03c1c00640 1 -- 172.21.15.88:0/2135448236 wait complete. 2024-09-21T07:55:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.115+0000 7f03c1c00640 1 Processor -- start 2024-09-21T07:55:01.116 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.116+0000 7f03c1c00640 1 -- start start 2024-09-21T07:55:01.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03c1c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a0004d40 0x7f03bc09d110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03c1c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 0x7f03bc141a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03c1c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0ccd20 0x7f03bc1e69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03c1c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc0b2f60 con 0x7f03bc150060 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03c1c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03bc06fbb0 con 0x7f03a0004d40 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03c1c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03bc0b30e0 con 0x7f03bc0ccd20 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 0x7f03bc141a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03b3400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 0x7f03bc141a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57484/0 (socket says 172.21.15.88:57484) 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.117+0000 7f03b3400640 1 -- 172.21.15.88:0/3438205738 learned_addr learned my addr 172.21.15.88:0/3438205738 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03bbe00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a0004d40 0x7f03bc09d110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03c1200640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0ccd20 0x7f03bc1e69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b3400640 1 -- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0ccd20 msgr2=0x7f03bc1e69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b3400640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0ccd20 0x7f03bc1e69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b3400640 1 -- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a0004d40 msgr2=0x7f03bc09d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b3400640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a0004d40 0x7f03bc09d110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b3400640 1 -- 172.21.15.88:0/3438205738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03bc09c350 con 0x7f03bc150060 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03bbe00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a0004d40 0x7f03bc09d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b3400640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 0x7f03bc141a00 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f03a4011b10 tx=0x7f03a4009e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a401e070 con 0x7f03bc150060 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03a400fe40 con 0x7f03bc150060 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03bc09c630 con 0x7f03bc150060 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.118+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03a40042b0 con 0x7f03bc150060 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.119+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03bc074df0 con 0x7f03bc150060 2024-09-21T07:55:01.121 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.119+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03bc1f9430 con 0x7f03bc150060 2024-09-21T07:55:01.122 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.121+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f03a4021070 con 0x7f03bc150060 2024-09-21T07:55:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.122+0000 7f03b8c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03980795b0 0x7f039807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.122+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f03a409a0c0 con 0x7f03bc150060 2024-09-21T07:55:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.122+0000 7f03bbe00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03980795b0 0x7f039807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.123 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.123+0000 7f03bbe00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03980795b0 0x7f039807bac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03ac002230 tx=0x7f03ac009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:01.128 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.127+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03a4063940 con 0x7f03bc150060 2024-09-21T07:55:01.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.315+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f03bc15d670 con 0x7f03bc150060 2024-09-21T07:55:01.317 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.317+0000 7f03b8c00640 1 -- 172.21.15.88:0/3438205738 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16739 (secure 0 0 0) 0x7f03a4068370 con 0x7f03bc150060 2024-09-21T07:55:01.318 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:01.318 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":156,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:39.123067+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:35.511240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:36.571979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:01.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03980795b0 msgr2=0x7f039807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03980795b0 0x7f039807bac0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f03ac002230 tx=0x7f03ac009040 comp rx=0 tx=0).stop 2024-09-21T07:55:01.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 msgr2=0x7f03bc141a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 0x7f03bc141a00 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f03a4011b10 tx=0x7f03a4009e40 comp rx=0 tx=0).stop 2024-09-21T07:55:01.320 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 shutdown_connections 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03980795b0 0x7f039807bac0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03bc0ccd20 0x7f03bc1e69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03bc150060 0x7f03bc141a00 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 --2- 172.21.15.88:0/3438205738 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03a0004d40 0x7f03bc09d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 >> 172.21.15.88:0/3438205738 conn(0x7f03bc0b1e40 msgr2=0x7f03bc0b13b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 shutdown_connections 2024-09-21T07:55:01.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.319+0000 7f03c1c00640 1 -- 172.21.15.88:0/3438205738 wait complete. 2024-09-21T07:55:01.343 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:55:01.343 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 -- 172.21.15.88:0/1291644312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 msgr2=0x7f31340ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 --2- 172.21.15.88:0/1291644312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f31340ad4f0 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7f3120005fd0 tx=0x7f312002a540 comp rx=0 tx=0).stop 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 -- 172.21.15.88:0/1291644312 shutdown_connections 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 --2- 172.21.15.88:0/1291644312 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f31340b3550 0x7f31340b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 --2- 172.21.15.88:0/1291644312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f31340ad4f0 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 -- 172.21.15.88:0/1291644312 >> 172.21.15.88:0/1291644312 conn(0x7f313406d730 msgr2=0x7f31340b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 -- 172.21.15.88:0/1291644312 shutdown_connections 2024-09-21T07:55:01.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.561+0000 7f313ae00640 1 -- 172.21.15.88:0/1291644312 wait complete. 2024-09-21T07:55:01.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.564+0000 7f313ae00640 1 Processor -- start 2024-09-21T07:55:01.565 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.564+0000 7f313ae00640 1 -- start start 2024-09-21T07:55:01.566 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f313ae00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f3134149800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f313ae00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31340b3550 0x7f313409a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f313ae00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313415d760 0x7f313409a810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f313ae00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31340b4da0 con 0x7f31340ad0e0 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f313ae00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f31340b4c20 con 0x7f31340b3550 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f313ae00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f31340b3ea0 con 0x7f313415d760 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.565+0000 7f3133e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f3134149800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31340b3550 0x7f313409a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f3134149800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57500/0 (socket says 172.21.15.88:57500) 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 -- 172.21.15.88:0/3666340495 learned_addr learned my addr 172.21.15.88:0/3666340495 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:01.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3138e00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313415d760 0x7f313409a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 -- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313415d760 msgr2=0x7f313409a810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.568 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313415d760 0x7f313409a810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 -- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31340b3550 msgr2=0x7f313409a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31340b3550 0x7f313409a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 -- 172.21.15.88:0/3666340495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f313409de10 con 0x7f31340ad0e0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.566+0000 7f3133e00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f3134149800 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f312400c3a0 tx=0x7f312400c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.567+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124017070 con 0x7f31340ad0e0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.567+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f31240053c0 con 0x7f31340ad0e0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.567+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f313409e0f0 con 0x7f31340ad0e0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.567+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3124003850 con 0x7f31340ad0e0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.567+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3134156680 con 0x7f31340ad0e0 2024-09-21T07:55:01.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.568+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31340b4370 con 0x7f31340ad0e0 2024-09-21T07:55:01.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.569+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f312401a070 con 0x7f31340ad0e0 2024-09-21T07:55:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.570+0000 7f312be00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f31140795b0 0x7f311407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.570+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f3124098b40 con 0x7f31340ad0e0 2024-09-21T07:55:01.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.570+0000 7f3133400640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f31140795b0 0x7f311407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:01.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.571+0000 7f3133400640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f31140795b0 0x7f311407bac0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3120033000 tx=0x7f3120004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:01.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.575+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f31340ada30 con 0x7f31340ad0e0 2024-09-21T07:55:01.801 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.800+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f313406f910 con 0x7f31340ad0e0 2024-09-21T07:55:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.801+0000 7f312be00640 1 -- 172.21.15.88:0/3666340495 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5239 (secure 0 0 0) 0x7f3124062160 con 0x7f31340ad0e0 2024-09-21T07:55:01.802 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:01.802 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":109,"btime":"2024-09-21T07:54:48:609789+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:37.615557+0000","modified":"2024-09-21T07:54:40.637260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8772":{"gid":8772,"name":"b","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2687096264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2687096264},{"type":"v1","addr":"172.21.15.175:6835","nonce":2687096264}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8772,"qdb_cluster":[8772]},"id":12}]} 2024-09-21T07:55:01.802 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 109 2024-09-21T07:55:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f31140795b0 msgr2=0x7f311407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f31140795b0 0x7f311407bac0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f3120033000 tx=0x7f3120004730 comp rx=0 tx=0).stop 2024-09-21T07:55:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 msgr2=0x7f3134149800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:01.804 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f3134149800 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f312400c3a0 tx=0x7f312400c860 comp rx=0 tx=0).stop 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 shutdown_connections 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f31140795b0 0x7f311407bac0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f313415d760 0x7f313409a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f31340b3550 0x7f313409a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 --2- 172.21.15.88:0/3666340495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31340ad0e0 0x7f3134149800 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 >> 172.21.15.88:0/3666340495 conn(0x7f313406d730 msgr2=0x7f31340b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 shutdown_connections 2024-09-21T07:55:01.805 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:01.803+0000 7f313ae00640 1 -- 172.21.15.88:0/3666340495 wait complete. 2024-09-21T07:55:01.829 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddf5c00640 1 -- 172.21.15.88:0/867451122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 msgr2=0x7fddd4004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddf5c00640 1 --2- 172.21.15.88:0/867451122 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddd4004c10 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7fddf00710d0 tx=0x7fddd4005830 comp rx=0 tx=0).stop 2024-09-21T07:55:02.051 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddef400640 1 -- 172.21.15.88:0/867451122 reap_dead start 2024-09-21T07:55:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddf5c00640 1 -- 172.21.15.88:0/867451122 shutdown_connections 2024-09-21T07:55:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddf5c00640 1 -- 172.21.15.88:0/867451122 >> 172.21.15.88:0/867451122 conn(0x7fddf006d730 msgr2=0x7fddf00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddf5c00640 1 -- 172.21.15.88:0/867451122 shutdown_connections 2024-09-21T07:55:02.052 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.049+0000 7fddf5c00640 1 -- 172.21.15.88:0/867451122 wait complete. 2024-09-21T07:55:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.052+0000 7fddf5c00640 1 Processor -- start 2024-09-21T07:55:02.053 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.052+0000 7fddf5c00640 1 -- start start 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddf5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddf0155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddf5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddf01e5630 0x7fddf0156d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddf5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddf01e5a40 0x7fddf0157250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddf5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fddf00b3720 con 0x7fddd4004800 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddf5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fddf0073850 con 0x7fddf01e5a40 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddf5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fddf00b38a0 con 0x7fddf01e5630 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddeea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddf0155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddeea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddf0155bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57518/0 (socket says 172.21.15.88:57518) 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.053+0000 7fddeea00640 1 -- 172.21.15.88:0/2037486455 learned_addr learned my addr 172.21.15.88:0/2037486455 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:02.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fdde7e00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddf01e5630 0x7fddf0156d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddef400640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddf01e5a40 0x7fddf0157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddeea00640 1 -- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddf01e5630 msgr2=0x7fddf0156d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddeea00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddf01e5630 0x7fddf0156d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddeea00640 1 -- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddf01e5a40 msgr2=0x7fddf0157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddeea00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddf01e5a40 0x7fddf0157250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddeea00640 1 -- 172.21.15.88:0/2037486455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fddf009d250 con 0x7fddd4004800 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddef400640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddf01e5a40 0x7fddf0157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddeea00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddf0155bc0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdde000e9a0 tx=0x7fdde000ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde000cc60 con 0x7fddd4004800 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdde000ce00 con 0x7fddd4004800 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdde0005280 con 0x7fddd4004800 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.054+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fddf009d530 con 0x7fddd4004800 2024-09-21T07:55:02.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.055+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fddf009daf0 con 0x7fddd4004800 2024-09-21T07:55:02.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.056+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fddf00b31e0 con 0x7fddd4004800 2024-09-21T07:55:02.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.057+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fdde0005420 con 0x7fddd4004800 2024-09-21T07:55:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.058+0000 7fdde6a00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddcc0795b0 0x7fddcc07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.058+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7fdde009a150 con 0x7fddd4004800 2024-09-21T07:55:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.058+0000 7fdde7e00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddcc0795b0 0x7fddcc07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.059+0000 7fdde7e00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddcc0795b0 0x7fddcc07bac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fddf00710d0 tx=0x7fddd8038040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:02.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.062+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdde0063dc0 con 0x7fddd4004800 2024-09-21T07:55:02.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.289+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fddf00af0a0 con 0x7fddd4004800 2024-09-21T07:55:02.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.290+0000 7fdde6a00640 1 -- 172.21.15.88:0/2037486455 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 109 v109) ==== 95+0+5239 (secure 0 0 0) 0x7fdde00636c0 con 0x7fddd4004800 2024-09-21T07:55:02.291 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:02.291 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":109,"btime":"2024-09-21T07:54:48:609789+0000","default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[{"mdsmap":{"epoch":108,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:54:37.615557+0000","modified":"2024-09-21T07:54:40.637260+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":8772},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8772":{"gid":8772,"name":"b","rank":0,"incarnation":105,"state":"up:active","state_seq":3,"addr":"172.21.15.175:6835/2687096264","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2687096264},{"type":"v1","addr":"172.21.15.175:6835","nonce":2687096264}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":8772,"qdb_cluster":[8772]},"id":12}]} 2024-09-21T07:55:02.291 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 109 2024-09-21T07:55:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddcc0795b0 msgr2=0x7fddcc07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddcc0795b0 0x7fddcc07bac0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fddf00710d0 tx=0x7fddd8038040 comp rx=0 tx=0).stop 2024-09-21T07:55:02.293 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 msgr2=0x7fddf0155bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddf0155bc0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7fdde000e9a0 tx=0x7fdde000ee60 comp rx=0 tx=0).stop 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 shutdown_connections 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fddcc0795b0 0x7fddcc07bac0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fddf01e5630 0x7fddf0156d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fddf01e5a40 0x7fddf0157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 --2- 172.21.15.88:0/2037486455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fddd4004800 0x7fddf0155bc0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 >> 172.21.15.88:0/2037486455 conn(0x7fddf006d730 msgr2=0x7fddf00b3d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 shutdown_connections 2024-09-21T07:55:02.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.292+0000 7fddf5c00640 1 -- 172.21.15.88:0/2037486455 wait complete. 2024-09-21T07:55:02.319 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 -- 172.21.15.88:0/3495146575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0735c0 msgr2=0x7ff27c0739d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 --2- 172.21.15.88:0/3495146575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0735c0 0x7ff27c0739d0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7ff264009830 tx=0x7ff26402a510 comp rx=0 tx=0).stop 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 -- 172.21.15.88:0/3495146575 shutdown_connections 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 --2- 172.21.15.88:0/3495146575 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27c0b3520 0x7ff27c0b3930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 --2- 172.21.15.88:0/3495146575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0735c0 0x7ff27c0739d0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 -- 172.21.15.88:0/3495146575 >> 172.21.15.88:0/3495146575 conn(0x7ff27c0b1e40 msgr2=0x7ff27c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 -- 172.21.15.88:0/3495146575 shutdown_connections 2024-09-21T07:55:02.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.539+0000 7ff282200640 1 -- 172.21.15.88:0/3495146575 wait complete. 2024-09-21T07:55:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.541+0000 7ff282200640 1 Processor -- start 2024-09-21T07:55:02.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.541+0000 7ff282200640 1 -- start start 2024-09-21T07:55:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff282200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27c0735c0 0x7ff27c15d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.544 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff282200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 0x7ff27c135170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff282200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27c1518d0 0x7ff27c137ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff282200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c06f310 con 0x7ff27c0b3520 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff282200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff27c071160 con 0x7ff27c0735c0 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff282200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff27c0b3e70 con 0x7ff27c1518d0 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 0x7ff27c135170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 0x7ff27c135170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57534/0 (socket says 172.21.15.88:57534) 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 -- 172.21.15.88:0/1931276235 learned_addr learned my addr 172.21.15.88:0/1931276235 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 -- 172.21.15.88:0/1931276235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27c1518d0 msgr2=0x7ff27c137ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27c1518d0 0x7ff27c137ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 -- 172.21.15.88:0/1931276235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27c0735c0 msgr2=0x7ff27c15d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27c0735c0 0x7ff27c15d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.543+0000 7ff273e00640 1 -- 172.21.15.88:0/1931276235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff27c154960 con 0x7ff27c0b3520 2024-09-21T07:55:02.545 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.544+0000 7ff273e00640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 0x7ff27c135170 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff26402ab00 tx=0x7ff264033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.544+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264031d60 con 0x7ff27c0b3520 2024-09-21T07:55:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.544+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff264033e10 con 0x7ff27c0b3520 2024-09-21T07:55:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.544+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff264005260 con 0x7ff27c0b3520 2024-09-21T07:55:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.544+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff27c15cd60 con 0x7ff27c0b3520 2024-09-21T07:55:02.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.544+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff27c149380 con 0x7ff27c0b3520 2024-09-21T07:55:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.545+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff27c0739d0 con 0x7ff27c0b3520 2024-09-21T07:55:02.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.546+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7ff264005400 con 0x7ff27c0b3520 2024-09-21T07:55:02.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.547+0000 7ff279600640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2580795b0 0x7ff25807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.547+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7ff2640bd6a0 con 0x7ff27c0b3520 2024-09-21T07:55:02.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.548+0000 7ff280e00640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2580795b0 0x7ff25807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.548+0000 7ff280e00640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2580795b0 0x7ff25807bac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001510 tx=0x7ff26c007930 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:02.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.552+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7ff26403f030 con 0x7ff27c0b3520 2024-09-21T07:55:02.738 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.736+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff27c0d0c10 con 0x7ff27c0b3520 2024-09-21T07:55:02.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.738+0000 7ff279600640 1 -- 172.21.15.88:0/1931276235 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v156) ==== 74+0+16739 (secure 0 0 0) 0x7ff264084730 con 0x7ff27c0b3520 2024-09-21T07:55:02.739 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:02.739 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":156,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:54:39.123067+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:54:35.511240+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"155","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2024-09-21T07:54:36.571979+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"156","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2580795b0 msgr2=0x7ff25807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2580795b0 0x7ff25807bac0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7ff26c001510 tx=0x7ff26c007930 comp rx=0 tx=0).stop 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 msgr2=0x7ff27c135170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 0x7ff27c135170 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7ff26402ab00 tx=0x7ff264033c20 comp rx=0 tx=0).stop 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 shutdown_connections 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff2580795b0 0x7ff25807bac0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff27c1518d0 0x7ff27c137ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff27c0b3520 0x7ff27c135170 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 --2- 172.21.15.88:0/1931276235 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff27c0735c0 0x7ff27c15d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.742 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 >> 172.21.15.88:0/1931276235 conn(0x7ff27c0b1e40 msgr2=0x7ff27c09dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 shutdown_connections 2024-09-21T07:55:02.743 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.740+0000 7ff282200640 1 -- 172.21.15.88:0/1931276235 wait complete. 2024-09-21T07:55:02.767 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 -- 172.21.15.88:0/142890286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20005320 msgr2=0x7f2a2000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 --2- 172.21.15.88:0/142890286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20005320 0x7f2a2000b9e0 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2a30001f00 tx=0x7f2a20005830 comp rx=0 tx=0).stop 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 -- 172.21.15.88:0/142890286 shutdown_connections 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 --2- 172.21.15.88:0/142890286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a20005320 0x7f2a2000b9e0 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 -- 172.21.15.88:0/142890286 >> 172.21.15.88:0/142890286 conn(0x7f2a3806d3d0 msgr2=0x7f2a380b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 -- 172.21.15.88:0/142890286 shutdown_connections 2024-09-21T07:55:02.989 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.987+0000 7f2a40000640 1 -- 172.21.15.88:0/142890286 wait complete. 2024-09-21T07:55:02.990 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.990+0000 7f2a40000640 1 Processor -- start 2024-09-21T07:55:02.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.990+0000 7f2a40000640 1 -- start start 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a40000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a20005320 0x7f2a380c8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a40000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a380c61c0 0x7f2a3815df60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a40000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 0x7f2a3815e4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a40000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a380709c0 con 0x7f2a380c65d0 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a40000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2a381572c0 con 0x7f2a20005320 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a40000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2a38070b40 con 0x7f2a380c61c0 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a3e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 0x7f2a3815e4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a3e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 0x7f2a3815e4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57554/0 (socket says 172.21.15.88:57554) 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.991+0000 7f2a3e000640 1 -- 172.21.15.88:0/3894555732 learned_addr learned my addr 172.21.15.88:0/3894555732 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3d600640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a20005320 0x7f2a380c8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3cc00640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a380c61c0 0x7f2a3815df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3e000640 1 -- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a380c61c0 msgr2=0x7f2a3815df60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a380c61c0 0x7f2a3815df60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3e000640 1 -- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a20005320 msgr2=0x7f2a380c8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a20005320 0x7f2a380c8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3e000640 1 -- 172.21.15.88:0/3894555732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a381f6aa0 con 0x7f2a380c65d0 2024-09-21T07:55:02.994 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a3e000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 0x7f2a3815e4a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002ada0 tx=0x7f2a30001c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.992+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30003d80 con 0x7f2a380c65d0 2024-09-21T07:55:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.993+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2a30032470 con 0x7f2a380c65d0 2024-09-21T07:55:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.993+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a30032880 con 0x7f2a380c65d0 2024-09-21T07:55:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.993+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a381f6d80 con 0x7f2a380c65d0 2024-09-21T07:55:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.993+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a38075350 con 0x7f2a380c65d0 2024-09-21T07:55:02.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.994+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2a380c9560 con 0x7f2a380c65d0 2024-09-21T07:55:02.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.995+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f2a3003d030 con 0x7f2a380c65d0 2024-09-21T07:55:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.996+0000 7f2a35600640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a180795b0 0x7f2a1807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.996+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(156..156 src has 1..156) ==== 5737+0+0 (secure 0 0 0) 0x7f2a300b93b0 con 0x7f2a380c65d0 2024-09-21T07:55:02.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.996+0000 7f2a3d600640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a180795b0 0x7f2a1807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:02.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:02.997+0000 7f2a3d600640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a180795b0 0x7f2a1807bac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a3809e510 tx=0x7f2a2c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:03.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.000+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2a300829d0 con 0x7f2a380c65d0 2024-09-21T07:55:03.227 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.226+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f2a3815d240 con 0x7f2a380c65d0 2024-09-21T07:55:03.227 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:55:03.226+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:55:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.228+0000 7f2a35600640 1 -- 172.21.15.88:0/3894555732 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v110) ==== 161+0+0 (secure 0 0 0) 0x7f2a30087400 con 0x7f2a380c65d0 2024-09-21T07:55:03.229 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a180795b0 msgr2=0x7f2a1807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a180795b0 0x7f2a1807bac0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2a3809e510 tx=0x7f2a2c005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 msgr2=0x7f2a3815e4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 0x7f2a3815e4a0 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f2a3002ada0 tx=0x7f2a30001c10 comp rx=0 tx=0).stop 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 shutdown_connections 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2a180795b0 0x7f2a1807bac0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2a380c61c0 0x7f2a3815df60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a380c65d0 0x7f2a3815e4a0 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 --2- 172.21.15.88:0/3894555732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2a20005320 0x7f2a380c8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 >> 172.21.15.88:0/3894555732 conn(0x7f2a3806d3d0 msgr2=0x7f2a3806fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 shutdown_connections 2024-09-21T07:55:03.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.232+0000 7f2a40000640 1 -- 172.21.15.88:0/3894555732 wait complete. 2024-09-21T07:55:03.256 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:55:03.373 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.479+0000 7f5bfbe00640 1 -- 172.21.15.88:0/2983346188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 msgr2=0x7f5bfc0cebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.479+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/2983346188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 0x7f5bfc0cebe0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4005fd0 tx=0x7f5bf402aa90 comp rx=0 tx=0).stop 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.480+0000 7f5bfbe00640 1 -- 172.21.15.88:0/2983346188 shutdown_connections 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.480+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/2983346188 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 0x7f5bfc0cebe0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.480+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/2983346188 >> v2:172.21.15.175:3300/0 conn(0x7f5bfc0b30d0 0x7f5bfc0b34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.480+0000 7f5bfbe00640 1 -- 172.21.15.88:0/2983346188 >> 172.21.15.88:0/2983346188 conn(0x7f5bfc06d620 msgr2=0x7f5bfc0b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.480+0000 7f5bfbe00640 1 -- 172.21.15.88:0/2983346188 shutdown_connections 2024-09-21T07:55:03.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.480+0000 7f5bfbe00640 1 -- 172.21.15.88:0/2983346188 wait complete. 2024-09-21T07:55:03.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.482+0000 7f5bfbe00640 1 Processor -- start 2024-09-21T07:55:03.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.482+0000 7f5bfbe00640 1 -- start start 2024-09-21T07:55:03.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.483+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bfc0b30d0 0x7f5bfc1f6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.483+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 0x7f5bfc1f6f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 0x7f5bfc1f74d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfbe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc0cf4f0 con 0x7f5bfc15b670 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfbe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5bfc0cf370 con 0x7f5bfc0b30d0 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfbe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5bfc0cf670 con 0x7f5bfc0ce7d0 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 0x7f5bfc1f74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 0x7f5bfc1f74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57562/0 (socket says 172.21.15.88:57562) 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 -- 172.21.15.88:0/722087534 learned_addr learned my addr 172.21.15.88:0/722087534 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfaa00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bfc0b30d0 0x7f5bfc1f6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bf2000640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 0x7f5bfc1f6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 -- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 msgr2=0x7f5bfc1f6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 0x7f5bfc1f6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 -- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bfc0b30d0 msgr2=0x7f5bfc1f6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:03.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bfc0b30d0 0x7f5bfc1f6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:03.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.484+0000 7f5bfb400640 1 -- 172.21.15.88:0/722087534 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5bfc14fd30 con 0x7f5bfc15b670 2024-09-21T07:55:03.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.485+0000 7f5bfb400640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 0x7f5bfc1f74d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4006010 tx=0x7f5bf4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.485+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4044070 con 0x7f5bfc15b670 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.485+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5bf4004040 con 0x7f5bfc15b670 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.485+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5bfc0b16b0 con 0x7f5bfc15b670 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.485+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5bf4004490 con 0x7f5bfc15b670 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.485+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5bfc0afe30 con 0x7f5bfc15b670 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.486+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5bfc144960 con 0x7f5bfc15b670 2024-09-21T07:55:03.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.487+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f5bf4047070 con 0x7f5bfc15b670 2024-09-21T07:55:03.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.489+0000 7f5bf3e00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5bd80795b0 0x7f5bd807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:03.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.489+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5823+0+0 (secure 0 0 0) 0x7f5bf40ba620 con 0x7f5bfc15b670 2024-09-21T07:55:03.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.489+0000 7f5bfaa00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5bd80795b0 0x7f5bd807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:03.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.490+0000 7f5bfaa00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5bd80795b0 0x7f5bd807bac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0017e0 tx=0x7f5bec009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:03.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.493+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5bf4032030 con 0x7f5bfc15b670 2024-09-21T07:55:03.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:03.717+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5bfc0b34e0 con 0x7f5bfc15b670 2024-09-21T07:55:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.236+0000 7f5bf3e00640 1 -- 172.21.15.88:0/722087534 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v111) ==== 104+0+0 (secure 0 0 0) 0x7f5bf4083c40 con 0x7f5bfc15b670 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5bd80795b0 msgr2=0x7f5bd807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5bd80795b0 0x7f5bd807bac0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f5bec0017e0 tx=0x7f5bec009040 comp rx=0 tx=0).stop 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 msgr2=0x7f5bfc1f74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 0x7f5bfc1f74d0 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f5bf4006010 tx=0x7f5bf4003ba0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 shutdown_connections 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5bd80795b0 0x7f5bd807bac0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.241 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5bfc0ce7d0 0x7f5bfc1f6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5bfc15b670 0x7f5bfc1f74d0 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 --2- 172.21.15.88:0/722087534 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5bfc0b30d0 0x7f5bfc1f6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 >> 172.21.15.88:0/722087534 conn(0x7f5bfc06d620 msgr2=0x7f5bfc150300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:04.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 shutdown_connections 2024-09-21T07:55:04.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.240+0000 7f5bfbe00640 1 -- 172.21.15.88:0/722087534 wait complete. 2024-09-21T07:55:04.264 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:55:04.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 -- 172.21.15.88:0/4020828809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 msgr2=0x7f460806e1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 --2- 172.21.15.88:0/4020828809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f460806e1c0 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f46080b59d0 tx=0x7f460002aa50 comp rx=0 tx=0).stop 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 -- 172.21.15.88:0/4020828809 shutdown_connections 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 --2- 172.21.15.88:0/4020828809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f460806e1c0 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 --2- 172.21.15.88:0/4020828809 >> v2:172.21.15.175:3300/0 conn(0x7f46080745c0 0x7f460815d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 --2- 172.21.15.88:0/4020828809 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4608156db0 0x7f46080b5f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 -- 172.21.15.88:0/4020828809 >> 172.21.15.88:0/4020828809 conn(0x7f4608073890 msgr2=0x7f4608073ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 -- 172.21.15.88:0/4020828809 shutdown_connections 2024-09-21T07:55:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.488+0000 7f460fa00640 1 -- 172.21.15.88:0/4020828809 wait complete. 2024-09-21T07:55:04.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.490+0000 7f460fa00640 1 Processor -- start 2024-09-21T07:55:04.491 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.490+0000 7f460fa00640 1 -- start start 2024-09-21T07:55:04.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460fa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f4608153300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460fa00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46080745c0 0x7f46081474a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460fa00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4608156db0 0x7f46081f5870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460fa00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f46080ce810 con 0x7f460806ddb0 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460fa00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f46080ce690 con 0x7f4608156db0 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460fa00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f46080ce990 con 0x7f46080745c0 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f4608153300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f4608153300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57572/0 (socket says 172.21.15.88:57572) 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.492+0000 7f460d000640 1 -- 172.21.15.88:0/963577533 learned_addr learned my addr 172.21.15.88:0/963577533 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460da00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4608156db0 0x7f46081f5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f4607e00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46080745c0 0x7f46081474a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460d000640 1 -- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46080745c0 msgr2=0x7f46081474a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460d000640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46080745c0 0x7f46081474a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460d000640 1 -- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4608156db0 msgr2=0x7f46081f5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460d000640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4608156db0 0x7f46081f5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460d000640 1 -- 172.21.15.88:0/963577533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46081f6610 con 0x7f460806ddb0 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460da00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4608156db0 0x7f46081f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f4607e00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46080745c0 0x7f46081474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460d000640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f4608153300 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f45f800e7c0 tx=0x7f45f800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f800ccc0 con 0x7f460806ddb0 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f45f8010430 con 0x7f460806ddb0 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46080c5be0 con 0x7f460806ddb0 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45f8010860 con 0x7f460806ddb0 2024-09-21T07:55:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.493+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46080c61a0 con 0x7f460806ddb0 2024-09-21T07:55:04.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.494+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4608153890 con 0x7f460806ddb0 2024-09-21T07:55:04.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.496+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f45f8010a00 con 0x7f460806ddb0 2024-09-21T07:55:04.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.497+0000 7f4604c00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f45e40795b0 0x7f45e407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:04.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.497+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(157..157 src has 1..157) ==== 5823+0+0 (secure 0 0 0) 0x7f45f809a260 con 0x7f460806ddb0 2024-09-21T07:55:04.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.497+0000 7f4607e00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f45e40795b0 0x7f45e407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:04.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.498+0000 7f4607e00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f45e40795b0 0x7f45e407bac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46080c7740 tx=0x7f45f0005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:04.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.501+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f45f8063fa0 con 0x7f460806ddb0 2024-09-21T07:55:04.687 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:04.686+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f46080cef50 con 0x7f460806ddb0 2024-09-21T07:55:05.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.238+0000 7f4604c00640 1 -- 172.21.15.88:0/963577533 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v158) ==== 181+0+0 (secure 0 0 0) 0x7f45f80638a0 con 0x7f460806ddb0 2024-09-21T07:55:05.239 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:55:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f45e40795b0 msgr2=0x7f45e407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f45e40795b0 0x7f45e407bac0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f46080c7740 tx=0x7f45f0005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 msgr2=0x7f4608153300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f4608153300 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f45f800e7c0 tx=0x7f45f800ec80 comp rx=0 tx=0).stop 2024-09-21T07:55:05.242 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 shutdown_connections 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f45e40795b0 0x7f45e407bac0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f46080745c0 0x7f46081474a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4608156db0 0x7f46081f5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 --2- 172.21.15.88:0/963577533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f460806ddb0 0x7f4608153300 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 >> 172.21.15.88:0/963577533 conn(0x7f4608073890 msgr2=0x7f46080d01e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 shutdown_connections 2024-09-21T07:55:05.243 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.241+0000 7f460fa00640 1 -- 172.21.15.88:0/963577533 wait complete. 2024-09-21T07:55:05.268 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.490+0000 7f0cab600640 1 -- 172.21.15.88:0/1999096827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca406d4b0 msgr2=0x7f0ca40b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.490+0000 7f0cab600640 1 --2- 172.21.15.88:0/1999096827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca406d4b0 0x7f0ca40b4620 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f0c90009830 tx=0x7f0c9002a5c0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.490+0000 7f0cab600640 1 -- 172.21.15.88:0/1999096827 shutdown_connections 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.490+0000 7f0cab600640 1 --2- 172.21.15.88:0/1999096827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca406d4b0 0x7f0ca40b4620 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.490+0000 7f0cab600640 1 -- 172.21.15.88:0/1999096827 >> 172.21.15.88:0/1999096827 conn(0x7f0ca40b2540 msgr2=0x7f0ca40d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.491+0000 7f0cab600640 1 -- 172.21.15.88:0/1999096827 shutdown_connections 2024-09-21T07:55:05.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.491+0000 7f0cab600640 1 -- 172.21.15.88:0/1999096827 wait complete. 2024-09-21T07:55:05.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.493+0000 7f0cab600640 1 Processor -- start 2024-09-21T07:55:05.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.493+0000 7f0cab600640 1 -- start start 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.494+0000 7f0cab600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ca40af100 0x7f0ca41e7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.494+0000 7f0cab600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ca40cc830 0x7f0ca40c7d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.494+0000 7f0cab600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 0x7f0ca41f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.494+0000 7f0cab600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40b3780 con 0x7f0ca4097fb0 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.494+0000 7f0cab600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0ca40b3600 con 0x7f0ca40cc830 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.494+0000 7f0cab600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0ca40b3900 con 0x7f0ca40af100 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 0x7f0ca41f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 0x7f0ca41f94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:57580/0 (socket says 172.21.15.88:57580) 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 -- 172.21.15.88:0/3191008431 learned_addr learned my addr 172.21.15.88:0/3191008431 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca3e00640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ca40cc830 0x7f0ca40c7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca8c00640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ca40af100 0x7f0ca41e7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 -- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ca40af100 msgr2=0x7f0ca41e7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ca40af100 0x7f0ca41e7c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 -- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ca40cc830 msgr2=0x7f0ca40c7d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ca40cc830 0x7f0ca40c7d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 -- 172.21.15.88:0/3191008431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ca40cf470 con 0x7f0ca4097fb0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.495+0000 7f0ca9600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 0x7f0ca41f94d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c002230 tx=0x7f0c9c00cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.496+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c003b70 con 0x7f0ca4097fb0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.496+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0c9c003d10 con 0x7f0ca4097fb0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.496+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ca40974d0 con 0x7f0ca4097fb0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.496+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c9c005330 con 0x7f0ca4097fb0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.496+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ca40c82b0 con 0x7f0ca4097fb0 2024-09-21T07:55:05.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.497+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0ca40b3db0 con 0x7f0ca4097fb0 2024-09-21T07:55:05.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.498+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f0c9c0056e0 con 0x7f0ca4097fb0 2024-09-21T07:55:05.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.499+0000 7f0ca0c00640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0c840795b0 0x7f0c8407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:05.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.499+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(158..158 src has 1..158) ==== 5385+0+0 (secure 0 0 0) 0x7f0c9c0994d0 con 0x7f0ca4097fb0 2024-09-21T07:55:05.500 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.499+0000 7f0ca8c00640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0c840795b0 0x7f0c8407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:05.501 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.500+0000 7f0ca8c00640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0c840795b0 0x7f0c8407bac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0c98007fd0 tx=0x7f0c98001370 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:05.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.504+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0c9c062da0 con 0x7f0ca4097fb0 2024-09-21T07:55:05.694 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:05.693+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f0ca40d1030 con 0x7f0ca4097fb0 2024-09-21T07:55:06.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.245+0000 7f0ca0c00640 1 -- 172.21.15.88:0/3191008431 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v159) ==== 169+0+0 (secure 0 0 0) 0x7f0c9c0677d0 con 0x7f0ca4097fb0 2024-09-21T07:55:06.246 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:55:06.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0c840795b0 msgr2=0x7f0c8407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0c840795b0 0x7f0c8407bac0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f0c98007fd0 tx=0x7f0c98001370 comp rx=0 tx=0).stop 2024-09-21T07:55:06.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 msgr2=0x7f0ca41f94d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 0x7f0ca41f94d0 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f0c9c002230 tx=0x7f0c9c00cb90 comp rx=0 tx=0).stop 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 shutdown_connections 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0c840795b0 0x7f0c8407bac0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0ca4097fb0 0x7f0ca41f94d0 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0ca40cc830 0x7f0ca40c7d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 --2- 172.21.15.88:0/3191008431 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0ca40af100 0x7f0ca41e7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 >> 172.21.15.88:0/3191008431 conn(0x7f0ca40b2540 msgr2=0x7f0ca4099820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 shutdown_connections 2024-09-21T07:55:06.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.248+0000 7f0cab600640 1 -- 172.21.15.88:0/3191008431 wait complete. 2024-09-21T07:55:06.276 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:55:06.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 -- 172.21.15.88:0/3025134792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0005330 msgr2=0x7f38f000b9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 --2- 172.21.15.88:0/3025134792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0005330 0x7f38f000b9e0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x7f390c0ce210 tx=0x7f38f0005830 comp rx=0 tx=0).stop 2024-09-21T07:55:06.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 -- 172.21.15.88:0/3025134792 shutdown_connections 2024-09-21T07:55:06.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 --2- 172.21.15.88:0/3025134792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38f0005330 0x7f38f000b9e0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 -- 172.21.15.88:0/3025134792 >> 172.21.15.88:0/3025134792 conn(0x7f390c06d3d0 msgr2=0x7f390c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 -- 172.21.15.88:0/3025134792 shutdown_connections 2024-09-21T07:55:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.496+0000 7f3913800640 1 -- 172.21.15.88:0/3025134792 wait complete. 2024-09-21T07:55:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.498+0000 7f3913800640 1 Processor -- start 2024-09-21T07:55:06.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.499+0000 7f3913800640 1 -- start start 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3913800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38f0005330 0x7f390c0c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3913800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0c4500 0x7f390c0c7df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3913800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 0x7f390c0c8740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3913800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f390c0709c0 con 0x7f390c0c8330 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3913800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f390c157320 con 0x7f390c0c4500 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3913800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f390c070b40 con 0x7f38f0005330 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3911800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 0x7f390c0c8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3911800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 0x7f390c0c8740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60724/0 (socket says 172.21.15.88:60724) 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.500+0000 7f3911800640 1 -- 172.21.15.88:0/2368416059 learned_addr learned my addr 172.21.15.88:0/2368416059 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:06.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3910e00640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38f0005330 0x7f390c0c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3903e00640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0c4500 0x7f390c0c7df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3911800640 1 -- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38f0005330 msgr2=0x7f390c0c3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3911800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38f0005330 0x7f390c0c3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3911800640 1 -- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0c4500 msgr2=0x7f390c0c7df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3911800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0c4500 0x7f390c0c7df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3911800640 1 -- 172.21.15.88:0/2368416059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f390c0c8c80 con 0x7f390c0c8330 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3911800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 0x7f390c0c8740 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f390402b470 tx=0x7f3904004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f390400c280 con 0x7f390c0c8330 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f390402bcc0 con 0x7f390c0c8330 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f390c074dc0 con 0x7f390c0c8330 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3904006340 con 0x7f390c0c8330 2024-09-21T07:55:06.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.501+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f390c0c75f0 con 0x7f390c0c8330 2024-09-21T07:55:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.503+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f390c15d240 con 0x7f390c0c8330 2024-09-21T07:55:06.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.504+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f39040064e0 con 0x7f390c0c8330 2024-09-21T07:55:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.505+0000 7f3909600640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38e80795e0 0x7f38e807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.505+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7f39040b9740 con 0x7f390c0c8330 2024-09-21T07:55:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.506+0000 7f3910e00640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38e80795e0 0x7f38e807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.506+0000 7f3910e00640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38e80795e0 0x7f38e807baf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f390c09e510 tx=0x7f38fc00b260 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.510+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3904083440 con 0x7f390c0c8330 2024-09-21T07:55:06.700 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.699+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f390c0afa70 con 0x7f390c0c8330 2024-09-21T07:55:06.704 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.703+0000 7f3909600640 1 -- 172.21.15.88:0/2368416059 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v51) ==== 71+0+0 (secure 0 0 0) 0x7f3904082d40 con 0x7f390c0c8330 2024-09-21T07:55:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38e80795e0 msgr2=0x7f38e807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38e80795e0 0x7f38e807baf0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f390c09e510 tx=0x7f38fc00b260 comp rx=0 tx=0).stop 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 msgr2=0x7f390c0c8740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 0x7f390c0c8740 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7f390402b470 tx=0x7f3904004c50 comp rx=0 tx=0).stop 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 shutdown_connections 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38e80795e0 0x7f38e807baf0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f390c0c8330 0x7f390c0c8740 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f390c0c4500 0x7f390c0c7df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 --2- 172.21.15.88:0/2368416059 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38f0005330 0x7f390c0c3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 >> 172.21.15.88:0/2368416059 conn(0x7f390c06d3d0 msgr2=0x7f390c06f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 shutdown_connections 2024-09-21T07:55:06.708 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.706+0000 7f3913800640 1 -- 172.21.15.88:0/2368416059 wait complete. 2024-09-21T07:55:06.730 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2' 2024-09-21T07:55:06.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 -- 172.21.15.88:0/1236564613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00ae080 msgr2=0x7f30b0070000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 --2- 172.21.15.88:0/1236564613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00ae080 0x7f30b0070000 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7f30b00ad8e0 tx=0x7f309802a560 comp rx=0 tx=0).stop 2024-09-21T07:55:06.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 -- 172.21.15.88:0/1236564613 shutdown_connections 2024-09-21T07:55:06.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 --2- 172.21.15.88:0/1236564613 >> v2:172.21.15.175:3300/0 conn(0x7f30b0070540 0x7f30b01651b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 --2- 172.21.15.88:0/1236564613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00ae080 0x7f30b0070000 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 -- 172.21.15.88:0/1236564613 >> 172.21.15.88:0/1236564613 conn(0x7f30b00b1e40 msgr2=0x7f30b00b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:06.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 -- 172.21.15.88:0/1236564613 shutdown_connections 2024-09-21T07:55:06.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.952+0000 7f30b4a00640 1 -- 172.21.15.88:0/1236564613 wait complete. 2024-09-21T07:55:06.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.954+0000 7f30b4a00640 1 Processor -- start 2024-09-21T07:55:06.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.954+0000 7f30b4a00640 1 -- start start 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.955+0000 7f30b4a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0070540 0x7f30b0153d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.955+0000 7f30b4a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30b00ae080 0x7f30b0149ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.955+0000 7f30b4a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 0x7f30b0146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.955+0000 7f30b4a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f30b0165900 con 0x7f30b00af050 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.955+0000 7f30b4a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30b00736b0 con 0x7f30b0070540 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.955+0000 7f30b4a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30b0165a80 con 0x7f30b00ae080 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 0x7f30b0146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 0x7f30b0146a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60744/0 (socket says 172.21.15.88:60744) 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 -- 172.21.15.88:0/906207570 learned_addr learned my addr 172.21.15.88:0/906207570 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:06.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30ae800640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0070540 0x7f30b0153d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30ade00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30b00ae080 0x7f30b0149ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 -- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30b00ae080 msgr2=0x7f30b0149ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30b00ae080 0x7f30b0149ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 -- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0070540 msgr2=0x7f30b0153d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0070540 0x7f30b0153d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.956+0000 7f30af200640 1 -- 172.21.15.88:0/906207570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30b01f43b0 con 0x7f30b00af050 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.957+0000 7f30af200640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 0x7f30b0146a00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b440 tx=0x7f30a800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.957+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8015020 con 0x7f30b00af050 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.957+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f30a8011920 con 0x7f30b00af050 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.957+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30b01f4690 con 0x7f30b00af050 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.957+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30a8011d70 con 0x7f30b00af050 2024-09-21T07:55:06.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.957+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30b01497f0 con 0x7f30b00af050 2024-09-21T07:55:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.958+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30b01f4120 con 0x7f30b00af050 2024-09-21T07:55:06.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.959+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f30a8018020 con 0x7f30b00af050 2024-09-21T07:55:06.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.960+0000 7f30a6a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f308c0795e0 0x7f308c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:06.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.960+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7f30a80996c0 con 0x7f30b00af050 2024-09-21T07:55:06.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.960+0000 7f30ae800640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f308c0795e0 0x7f308c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:06.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.961+0000 7f30ae800640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f308c0795e0 0x7f308c07baf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009fd0 tx=0x7f30a000d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:06.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:06.965+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f30a801f020 con 0x7f30b00af050 2024-09-21T07:55:07.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.220+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]} v 0) -- 0x7f30b0070000 con 0x7f30b00af050 2024-09-21T07:55:07.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.247+0000 7f30a6a00640 1 -- 172.21.15.88:0/906207570 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_subvolume_upgrade_v1_to_v2"]}]=0 v508) ==== 145+0+0 (secure 0 0 0) 0x7f30a8062ce0 con 0x7f30b00af050 2024-09-21T07:55:07.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.249+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f308c0795e0 msgr2=0x7f308c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.249+0000 7f30b4a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f308c0795e0 0x7f308c07baf0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f30a0009fd0 tx=0x7f30a000d040 comp rx=0 tx=0).stop 2024-09-21T07:55:07.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 msgr2=0x7f30b0146a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 0x7f30b0146a00 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7f30a800b440 tx=0x7f30a800b900 comp rx=0 tx=0).stop 2024-09-21T07:55:07.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 shutdown_connections 2024-09-21T07:55:07.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f308c0795e0 0x7f308c07baf0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f30b00ae080 0x7f30b0149ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30b00af050 0x7f30b0146a00 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 --2- 172.21.15.88:0/906207570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f30b0070540 0x7f30b0153d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 >> 172.21.15.88:0/906207570 conn(0x7f30b00b1e40 msgr2=0x7f30b0164aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 shutdown_connections 2024-09-21T07:55:07.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.250+0000 7f30b4a00640 1 -- 172.21.15.88:0/906207570 wait complete. 2024-09-21T07:55:07.276 INFO:tasks.cephfs_test_runner:poor man's upgrade test -- theme continues... ... ok 2024-09-21T07:55:07.276 INFO:tasks.cephfs_test_runner:Starting test: test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) Validate unparseable text .meta file on legacy subvol root doesn't break the system 2024-09-21T07:55:07.276 INFO:tasks.cephfs_test_runner:test_unparseable_metafile_on_legacy_to_v1_upgrade (tasks.cephfs.test_volumes.TestMisc) 2024-09-21T07:55:07.277 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 -- 172.21.15.88:0/686606831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 msgr2=0x7fb87006f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 --2- 172.21.15.88:0/686606831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb87006f930 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fb860009830 tx=0x7fb86002a480 comp rx=0 tx=0).stop 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 -- 172.21.15.88:0/686606831 shutdown_connections 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 --2- 172.21.15.88:0/686606831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb87006f930 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 -- 172.21.15.88:0/686606831 >> 172.21.15.88:0/686606831 conn(0x7fb87006d730 msgr2=0x7fb8700b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 -- 172.21.15.88:0/686606831 shutdown_connections 2024-09-21T07:55:07.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.496+0000 7fb876000640 1 -- 172.21.15.88:0/686606831 wait complete. 2024-09-21T07:55:07.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.498+0000 7fb876000640 1 Processor -- start 2024-09-21T07:55:07.499 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.499+0000 7fb876000640 1 -- start start 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb876000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb870152cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb876000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8700730b0 0x7fb870150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb876000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8700b4110 0x7fb87015df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb876000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb87015b380 con 0x7fb8700707b0 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb876000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb87015b200 con 0x7fb8700730b0 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb876000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb87006fe70 con 0x7fb8700b4110 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb86f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb870152cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb86f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb870152cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60770/0 (socket says 172.21.15.88:60770) 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb86f400640 1 -- 172.21.15.88:0/3204581976 learned_addr learned my addr 172.21.15.88:0/3204581976 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.500+0000 7fb86ea00640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8700730b0 0x7fb870150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86fe00640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8700b4110 0x7fb87015df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.502 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86f400640 1 -- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8700b4110 msgr2=0x7fb87015df40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86f400640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8700b4110 0x7fb87015df40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86f400640 1 -- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8700730b0 msgr2=0x7fb870150a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86f400640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8700730b0 0x7fb870150a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86f400640 1 -- 172.21.15.88:0/3204581976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb8701f6030 con 0x7fb8700707b0 2024-09-21T07:55:07.503 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86ea00640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8700730b0 0x7fb870150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb86f400640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb870152cf0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb860009950 tx=0x7fb860033d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860031d60 con 0x7fb8700707b0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fb8600330d0 con 0x7fb8700707b0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8701f6310 con 0x7fb8700707b0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb860033520 con 0x7fb8700707b0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.501+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8701588c0 con 0x7fb8700707b0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.503+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb87006f930 con 0x7fb8700707b0 2024-09-21T07:55:07.504 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.504+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fb8600035f0 con 0x7fb8700707b0 2024-09-21T07:55:07.505 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.505+0000 7fb867400640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8500795b0 0x7fb85007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.505+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7fb8600bad70 con 0x7fb8700707b0 2024-09-21T07:55:07.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.505+0000 7fb86ea00640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8500795b0 0x7fb85007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.506 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.505+0000 7fb86ea00640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8500795b0 0x7fb85007bac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb85c006fd0 tx=0x7fb85c008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.510+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb86003f030 con 0x7fb8700707b0 2024-09-21T07:55:07.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.697+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fb870159930 con 0x7fb8700707b0 2024-09-21T07:55:07.699 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.698+0000 7fb867400640 1 -- 172.21.15.88:0/3204581976 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) ==== 74+0+13100 (secure 0 0 0) 0x7fb860084a70 con 0x7fb8700707b0 2024-09-21T07:55:07.700 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:07.700 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":159,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:06.238909+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":154,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.175:6835/2687096264":"2024-09-22T07:55:03.226899+0000","172.21.15.175:6801/2687096264":"2024-09-22T07:55:03.226899+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8500795b0 msgr2=0x7fb85007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8500795b0 0x7fb85007bac0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fb85c006fd0 tx=0x7fb85c008040 comp rx=0 tx=0).stop 2024-09-21T07:55:07.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 msgr2=0x7fb870152cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb870152cf0 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fb860009950 tx=0x7fb860033d60 comp rx=0 tx=0).stop 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 shutdown_connections 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb8500795b0 0x7fb85007bac0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb8700b4110 0x7fb87015df40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb8700730b0 0x7fb870150a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 --2- 172.21.15.88:0/3204581976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8700707b0 0x7fb870152cf0 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 >> 172.21.15.88:0/3204581976 conn(0x7fb87006d730 msgr2=0x7fb870155d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 shutdown_connections 2024-09-21T07:55:07.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.701+0000 7fb876000640 1 -- 172.21.15.88:0/3204581976 wait complete. 2024-09-21T07:55:07.725 INFO:tasks.ceph_test_case.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T07:45:55.886025+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T07:55:07.725 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T07:55:07.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 -- 172.21.15.88:0/938759695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 msgr2=0x7fbbf00cf9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 --2- 172.21.15.88:0/938759695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf00cf9a0 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0072340 tx=0x7fbbe002a410 comp rx=0 tx=0).stop 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 -- 172.21.15.88:0/938759695 shutdown_connections 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 --2- 172.21.15.88:0/938759695 >> v2:172.21.15.175:3300/0 conn(0x7fbbf0160be0 0x7fbbf006ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 --2- 172.21.15.88:0/938759695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf00cf9a0 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 -- 172.21.15.88:0/938759695 >> 172.21.15.88:0/938759695 conn(0x7fbbf006cf80 msgr2=0x7fbbf006d390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 -- 172.21.15.88:0/938759695 shutdown_connections 2024-09-21T07:55:07.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.951+0000 7fbbf6400640 1 -- 172.21.15.88:0/938759695 wait complete. 2024-09-21T07:55:07.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.953+0000 7fbbf6400640 1 Processor -- start 2024-09-21T07:55:07.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.953+0000 7fbbf6400640 1 -- start start 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf0158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf00d02f0 0x7fbbf01f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf6400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf0160be0 0x7fbbf00b1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf6400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf015d010 con 0x7fbbf0074c00 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf6400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf015ce90 con 0x7fbbf00d02f0 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf6400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbbf015d190 con 0x7fbbf0160be0 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf0158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf0158e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60782/0 (socket says 172.21.15.88:60782) 2024-09-21T07:55:07.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf5000640 1 -- 172.21.15.88:0/2632195650 learned_addr learned my addr 172.21.15.88:0/2632195650 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbe7e00640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf00d02f0 0x7fbbf01f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.955+0000 7fbbf5a00640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf0160be0 0x7fbbf00b1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf5000640 1 -- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf0160be0 msgr2=0x7fbbf00b1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf5000640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf0160be0 0x7fbbf00b1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf5000640 1 -- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf00d02f0 msgr2=0x7fbbf01f2fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf5000640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf00d02f0 0x7fbbf01f2fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf5000640 1 -- 172.21.15.88:0/2632195650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf01f3c00 con 0x7fbbf0074c00 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbe7e00640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf00d02f0 0x7fbbf01f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:07.958 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf5000640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf0158e80 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fbbe002aa00 tx=0x7fbbe0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0031d60 con 0x7fbbf0074c00 2024-09-21T07:55:07.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbe0033e10 con 0x7fbbf0074c00 2024-09-21T07:55:07.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbe0005260 con 0x7fbbf0074c00 2024-09-21T07:55:07.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf01f3ee0 con 0x7fbbf0074c00 2024-09-21T07:55:07.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.956+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf0158680 con 0x7fbbf0074c00 2024-09-21T07:55:07.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.958+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf015d6b0 con 0x7fbbf0074c00 2024-09-21T07:55:07.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.959+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fbbe0005400 con 0x7fbbf0074c00 2024-09-21T07:55:07.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.960+0000 7fbbed600640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbcc0795e0 0x7fbbcc07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:07.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.960+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7fbbe00bab80 con 0x7fbbf0074c00 2024-09-21T07:55:07.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.960+0000 7fbbe7e00640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbcc0795e0 0x7fbbcc07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:07.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.961+0000 7fbbe7e00640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbcc0795e0 0x7fbbcc07baf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af6c0 tx=0x7fbbd8005ec0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:07.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:07.964+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbbe003f030 con 0x7fbbf0074c00 2024-09-21T07:55:08.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.152+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fbbf00cfb40 con 0x7fbbf0074c00 2024-09-21T07:55:08.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.153+0000 7fbbed600640 1 -- 172.21.15.88:0/2632195650 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v159) ==== 93+0+10 (secure 0 0 0) 0x7fbbe0031770 con 0x7fbbf0074c00 2024-09-21T07:55:08.154 INFO:teuthology.orchestra.run.smithi088.stdout:pg_num: 1 2024-09-21T07:55:08.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbcc0795e0 msgr2=0x7fbbcc07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbcc0795e0 0x7fbbcc07baf0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fbbf00af6c0 tx=0x7fbbd8005ec0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 msgr2=0x7fbbf0158e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf0158e80 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7fbbe002aa00 tx=0x7fbbe0033c20 comp rx=0 tx=0).stop 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 shutdown_connections 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbcc0795e0 0x7fbbcc07baf0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbf0160be0 0x7fbbf00b1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbf00d02f0 0x7fbbf01f2fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 --2- 172.21.15.88:0/2632195650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf0074c00 0x7fbbf0158e80 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 >> 172.21.15.88:0/2632195650 conn(0x7fbbf006cf80 msgr2=0x7fbbf00d0700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 shutdown_connections 2024-09-21T07:55:08.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.155+0000 7fbbf6400640 1 -- 172.21.15.88:0/2632195650 wait complete. 2024-09-21T07:55:08.179 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-21T07:55:08.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce35800640 1 -- 172.21.15.88:0/2226823301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce14003a00 msgr2=0x7fce14003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce35800640 1 --2- 172.21.15.88:0/2226823301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce14003a00 0x7fce14003e70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7fce2000bbe0 tx=0x7fce14005830 comp rx=0 tx=0).stop 2024-09-21T07:55:08.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce2f400640 1 -- 172.21.15.88:0/2226823301 reap_dead start 2024-09-21T07:55:08.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce35800640 1 -- 172.21.15.88:0/2226823301 shutdown_connections 2024-09-21T07:55:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce35800640 1 -- 172.21.15.88:0/2226823301 >> 172.21.15.88:0/2226823301 conn(0x7fce3006d730 msgr2=0x7fce300b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce35800640 1 -- 172.21.15.88:0/2226823301 shutdown_connections 2024-09-21T07:55:08.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.399+0000 7fce35800640 1 -- 172.21.15.88:0/2226823301 wait complete. 2024-09-21T07:55:08.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.401+0000 7fce35800640 1 Processor -- start 2024-09-21T07:55:08.402 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.401+0000 7fce35800640 1 -- start start 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce35800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce14003a00 0x7fce3014e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce35800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3014e630 0x7fce301f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce35800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 0x7fce30150790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce35800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fce300b4da0 con 0x7fce301f5020 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce35800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fce300b4c20 con 0x7fce3014e630 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce35800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fce300b3960 con 0x7fce14003a00 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 0x7fce30150790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 0x7fce30150790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60814/0 (socket says 172.21.15.88:60814) 2024-09-21T07:55:08.405 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 -- 172.21.15.88:0/1195496619 learned_addr learned my addr 172.21.15.88:0/1195496619 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce27e00640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3014e630 0x7fce301f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 -- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce14003a00 msgr2=0x7fce3014e0f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce14003a00 0x7fce3014e0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 -- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3014e630 msgr2=0x7fce301f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3014e630 0x7fce301f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 -- 172.21.15.88:0/1195496619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fce3014f8a0 con 0x7fce301f5020 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.403+0000 7fce2f400640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 0x7fce30150790 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fce28005fd0 tx=0x7fce28011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.404+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce2800fd70 con 0x7fce301f5020 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.404+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fce28004b20 con 0x7fce301f5020 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.404+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fce3014fb80 con 0x7fce301f5020 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.404+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fce28003970 con 0x7fce301f5020 2024-09-21T07:55:08.406 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.404+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fce3014cd20 con 0x7fce301f5020 2024-09-21T07:55:08.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.405+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fce3014f610 con 0x7fce301f5020 2024-09-21T07:55:08.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.406+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fce28020020 con 0x7fce301f5020 2024-09-21T07:55:08.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.407+0000 7fce26a00640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fce0c0795b0 0x7fce0c07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:08.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.407+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7fce28099fa0 con 0x7fce301f5020 2024-09-21T07:55:08.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.407+0000 7fce2ea00640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fce0c0795b0 0x7fce0c07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:08.409 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.408+0000 7fce2ea00640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fce0c0795b0 0x7fce0c07bac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce20036000 tx=0x7fce200046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:08.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.412+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fce28063ca0 con 0x7fce301f5020 2024-09-21T07:55:08.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:08.668+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fce301535c0 con 0x7fce301f5020 2024-09-21T07:55:09.261 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.260+0000 7fce26a00640 1 -- 172.21.15.88:0/1195496619 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v510) ==== 166+0+0 (secure 0 0 0) 0x7fce280635a0 con 0x7fce301f5020 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.262+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fce0c0795b0 msgr2=0x7fce0c07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.262+0000 7fce35800640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fce0c0795b0 0x7fce0c07bac0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fce20036000 tx=0x7fce200046c0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.262+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 msgr2=0x7fce30150790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.262+0000 7fce35800640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 0x7fce30150790 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7fce28005fd0 tx=0x7fce28011f90 comp rx=0 tx=0).stop 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 shutdown_connections 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fce0c0795b0 0x7fce0c07bac0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fce301f5020 0x7fce30150790 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.264 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fce3014e630 0x7fce301f4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 --2- 172.21.15.88:0/1195496619 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fce14003a00 0x7fce3014e0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 >> 172.21.15.88:0/1195496619 conn(0x7fce3006d730 msgr2=0x7fce30068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 shutdown_connections 2024-09-21T07:55:09.265 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.263+0000 7fce35800640 1 -- 172.21.15.88:0/1195496619 wait complete. 2024-09-21T07:55:09.287 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config dump 2024-09-21T07:55:09.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 -- 172.21.15.88:0/3011824846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 msgr2=0x7fe5180ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 --2- 172.21.15.88:0/3011824846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe5180ad4f0 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7fe500005fd0 tx=0x7fe50002a540 comp rx=0 tx=0).stop 2024-09-21T07:55:09.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 -- 172.21.15.88:0/3011824846 shutdown_connections 2024-09-21T07:55:09.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 --2- 172.21.15.88:0/3011824846 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b3550 0x7fe5180b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.507 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 --2- 172.21.15.88:0/3011824846 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe5180ad4f0 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 -- 172.21.15.88:0/3011824846 >> 172.21.15.88:0/3011824846 conn(0x7fe51806d730 msgr2=0x7fe5180b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 -- 172.21.15.88:0/3011824846 shutdown_connections 2024-09-21T07:55:09.508 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.506+0000 7fe51e800640 1 -- 172.21.15.88:0/3011824846 wait complete. 2024-09-21T07:55:09.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.508+0000 7fe51e800640 1 Processor -- start 2024-09-21T07:55:09.509 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.508+0000 7fe51e800640 1 -- start start 2024-09-21T07:55:09.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.509+0000 7fe51e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe51806f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.509+0000 7fe51e800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b3550 0x7fe51806f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.509+0000 7fe51e800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51815d760 0x7fe5181544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe51e800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe5180b4da0 con 0x7fe5180ad0e0 2024-09-21T07:55:09.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe51e800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe5180b4c20 con 0x7fe51815d760 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe51e800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe5180b3ea0 con 0x7fe5180b3550 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe51806f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe51806f420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60836/0 (socket says 172.21.15.88:60836) 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 -- 172.21.15.88:0/4010638479 learned_addr learned my addr 172.21.15.88:0/4010638479 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe50fe00640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b3550 0x7fe51806f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517e00640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51815d760 0x7fe5181544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 -- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b3550 msgr2=0x7fe51806f960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b3550 0x7fe51806f960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 -- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51815d760 msgr2=0x7fe5181544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51815d760 0x7fe5181544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.512 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.510+0000 7fe517400640 1 -- 172.21.15.88:0/4010638479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe518155f50 con 0x7fe5180ad0e0 2024-09-21T07:55:09.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.511+0000 7fe517400640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe51806f420 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c3a0 tx=0x7fe50800c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.511+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508017070 con 0x7fe5180ad0e0 2024-09-21T07:55:09.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.511+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe5080053c0 con 0x7fe5180ad0e0 2024-09-21T07:55:09.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.511+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe508003850 con 0x7fe5180ad0e0 2024-09-21T07:55:09.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.511+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe518156230 con 0x7fe5180ad0e0 2024-09-21T07:55:09.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.511+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5181567f0 con 0x7fe5180ad0e0 2024-09-21T07:55:09.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.512+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe5180b4370 con 0x7fe5180ad0e0 2024-09-21T07:55:09.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.513+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fe50801a070 con 0x7fe5180ad0e0 2024-09-21T07:55:09.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.514+0000 7fe50f400640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe4f40795b0 0x7fe4f407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.514+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7fe508098b40 con 0x7fe5180ad0e0 2024-09-21T07:55:09.515 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.514+0000 7fe50fe00640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe4f40795b0 0x7fe4f407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.516 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.515+0000 7fe50fe00640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe4f40795b0 0x7fe4f407bac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe500033000 tx=0x7fe500004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.519+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe5180ada30 con 0x7fe5180ad0e0 2024-09-21T07:55:09.706 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.705+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7fe5181538a0 con 0x7fe5180ad0e0 2024-09-21T07:55:09.707 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.706+0000 7fe50f400640 1 -- 172.21.15.88:0/4010638479 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v51) ==== 59+0+2294 (secure 0 0 0) 0x7fe508062160 con 0x7fe5180ad0e0 2024-09-21T07:55:09.707 INFO:teuthology.orchestra.run.smithi088.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T07:55:09.707 INFO:teuthology.orchestra.run.smithi088.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T07:55:09.707 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced client_mount_timeout 30 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_client 20/20 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_mgr 20/20 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:mgr advanced debug_ms 1/1 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.0 basic osd_mclock_max_capacity_iops_ssd 3435.140052 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.1 basic osd_mclock_max_capacity_iops_ssd 3567.376092 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.2 basic osd_mclock_max_capacity_iops_ssd 3413.387841 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.3 basic osd_mclock_max_capacity_iops_ssd 3407.094046 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.4 basic osd_mclock_max_capacity_iops_ssd 4496.566903 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.5 basic osd_mclock_max_capacity_iops_ssd 4254.415167 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.6 basic osd_mclock_max_capacity_iops_ssd 4301.620844 2024-09-21T07:55:09.708 INFO:teuthology.orchestra.run.smithi088.stdout:osd.7 basic osd_mclock_max_capacity_iops_ssd 5901.126426 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds 20/20 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced debug_ms 2/2 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_frag true 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_debug_scatterstat true 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds dev mds_verify_scatter true 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced client_mount_timeout 600 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_client 20/20 2024-09-21T07:55:09.709 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced debug_ms 1/1 2024-09-21T07:55:09.710 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T07:55:09.710 INFO:teuthology.orchestra.run.smithi088.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe4f40795b0 msgr2=0x7fe4f407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe4f40795b0 0x7fe4f407bac0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe500033000 tx=0x7fe500004730 comp rx=0 tx=0).stop 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 msgr2=0x7fe51806f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe51806f420 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c3a0 tx=0x7fe50800c860 comp rx=0 tx=0).stop 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 shutdown_connections 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe4f40795b0 0x7fe4f407bac0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe5180b3550 0x7fe51806f960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe51815d760 0x7fe5181544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 --2- 172.21.15.88:0/4010638479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5180ad0e0 0x7fe51806f420 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 >> 172.21.15.88:0/4010638479 conn(0x7fe51806d730 msgr2=0x7fe518152b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.708+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 shutdown_connections 2024-09-21T07:55:09.711 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.709+0000 7fe51e800640 1 -- 172.21.15.88:0/4010638479 wait complete. 2024-09-21T07:55:09.732 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:55:09.955 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 -- 172.21.15.88:0/3653968240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 msgr2=0x7fde240ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 --2- 172.21.15.88:0/3653968240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde240ccac0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7fde10009830 tx=0x7fde1002a5b0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 -- 172.21.15.88:0/3653968240 shutdown_connections 2024-09-21T07:55:09.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 --2- 172.21.15.88:0/3653968240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde240ccac0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 -- 172.21.15.88:0/3653968240 >> 172.21.15.88:0/3653968240 conn(0x7fde2406d620 msgr2=0x7fde240b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:09.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 -- 172.21.15.88:0/3653968240 shutdown_connections 2024-09-21T07:55:09.956 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.954+0000 7fde2a400640 1 -- 172.21.15.88:0/3653968240 wait complete. 2024-09-21T07:55:09.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.956+0000 7fde2a400640 1 Processor -- start 2024-09-21T07:55:09.957 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.957+0000 7fde2a400640 1 -- start start 2024-09-21T07:55:09.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde2a400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde240b3f30 0x7fde241f33c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde2a400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde240cd000 0x7fde24144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde2a400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde24145120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde2a400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde240af7e0 con 0x7fde24156ca0 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde2a400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fde24070370 con 0x7fde240b3f30 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde2a400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fde240af960 con 0x7fde240cd000 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde29a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde24145120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde29a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde24145120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60842/0 (socket says 172.21.15.88:60842) 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.958+0000 7fde29a00640 1 -- 172.21.15.88:0/4217571899 learned_addr learned my addr 172.21.15.88:0/4217571899 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29000640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde240b3f30 0x7fde241f33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde23e00640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde240cd000 0x7fde24144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29a00640 1 -- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde240cd000 msgr2=0x7fde24144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29a00640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde240cd000 0x7fde24144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.961 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29a00640 1 -- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde240b3f30 msgr2=0x7fde241f33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29a00640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde240b3f30 0x7fde241f33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29a00640 1 -- 172.21.15.88:0/4217571899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde240b1520 con 0x7fde24156ca0 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29000640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde240b3f30 0x7fde241f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde29a00640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde24145120 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00c3f0 tx=0x7fde1c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011790 con 0x7fde24156ca0 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fde1c017070 con 0x7fde24156ca0 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde2409ac70 con 0x7fde24156ca0 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.959+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde1c011ce0 con 0x7fde24156ca0 2024-09-21T07:55:09.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.960+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde2409af10 con 0x7fde24156ca0 2024-09-21T07:55:09.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.960+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde240ccac0 con 0x7fde24156ca0 2024-09-21T07:55:09.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.962+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fde1c01c460 con 0x7fde24156ca0 2024-09-21T07:55:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.963+0000 7fde20c00640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde040795b0 0x7fde0407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.963+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7fde1c099610 con 0x7fde24156ca0 2024-09-21T07:55:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.963+0000 7fde29000640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde040795b0 0x7fde0407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:09.964 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.964+0000 7fde29000640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde040795b0 0x7fde0407bac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fde14001da0 tx=0x7fde14009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:09.968 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:09.968+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fde1c063280 con 0x7fde24156ca0 2024-09-21T07:55:10.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.155+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7fde240b1220 con 0x7fde24156ca0 2024-09-21T07:55:10.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.159+0000 7fde20c00640 1 -- 172.21.15.88:0/4217571899 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7fde1c062b80 con 0x7fde24156ca0 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde040795b0 msgr2=0x7fde0407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde040795b0 0x7fde0407bac0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fde14001da0 tx=0x7fde14009040 comp rx=0 tx=0).stop 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 msgr2=0x7fde24145120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde24145120 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7fde1c00c3f0 tx=0x7fde1c00c8b0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 shutdown_connections 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fde040795b0 0x7fde0407bac0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fde240cd000 0x7fde24144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde24156ca0 0x7fde24145120 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 --2- 172.21.15.88:0/4217571899 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fde240b3f30 0x7fde241f33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.161+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 >> 172.21.15.88:0/4217571899 conn(0x7fde2406d620 msgr2=0x7fde240af0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.162+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 shutdown_connections 2024-09-21T07:55:10.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.162+0000 7fde2a400640 1 -- 172.21.15.88:0/4217571899 wait complete. 2024-09-21T07:55:10.185 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:55:10.186 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:55:10.262 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:55:10.344 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:55:10.344 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:10.344 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:10.379 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:10.379 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:10.379 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:10.417 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:10.417 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:10.417 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:10.454 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:10.454 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:10.454 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:10.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:10.493 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2024-09-21T07:55:10.493 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 -- 172.21.15.88:0/1476353983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ad0e0 msgr2=0x7f5ea00ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/1476353983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ad0e0 0x7f5ea00ad4f0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f5ea0075250 tx=0x7f5e8802a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 -- 172.21.15.88:0/1476353983 shutdown_connections 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/1476353983 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea0160310 0x7f5ea0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/1476353983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea00ad0e0 0x7f5ea00ad4f0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/1476353983 >> v2:172.21.15.175:3300/0 conn(0x7f5ea015d760 0x7f5ea01535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 -- 172.21.15.88:0/1476353983 >> 172.21.15.88:0/1476353983 conn(0x7f5ea006d730 msgr2=0x7f5ea00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 -- 172.21.15.88:0/1476353983 shutdown_connections 2024-09-21T07:55:10.714 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.712+0000 7f5ea5c00640 1 -- 172.21.15.88:0/1476353983 wait complete. 2024-09-21T07:55:10.715 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.715+0000 7f5ea5c00640 1 Processor -- start 2024-09-21T07:55:10.716 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.715+0000 7f5ea5c00640 1 -- start start 2024-09-21T07:55:10.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5ea5c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea00ad0e0 0x7f5ea0154780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5ea5c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea015d760 0x7f5ea01523d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5ea5c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 0x7f5ea00b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5ea5c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00b4da0 con 0x7f5ea0160310 2024-09-21T07:55:10.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5ea5c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5ea00b4c20 con 0x7f5ea00ad0e0 2024-09-21T07:55:10.718 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5ea5c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5ea0160c60 con 0x7f5ea015d760 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5e9f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 0x7f5ea00b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5e9f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 0x7f5ea00b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60864/0 (socket says 172.21.15.88:60864) 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.716+0000 7f5e9f400640 1 -- 172.21.15.88:0/3146071440 learned_addr learned my addr 172.21.15.88:0/3146071440 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9ea00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea00ad0e0 0x7f5ea0154780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e96000640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea015d760 0x7f5ea01523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9f400640 1 -- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea015d760 msgr2=0x7f5ea01523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9f400640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea015d760 0x7f5ea01523d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9f400640 1 -- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea00ad0e0 msgr2=0x7f5ea0154780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9f400640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea00ad0e0 0x7f5ea0154780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9f400640 1 -- 172.21.15.88:0/3146071440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ea0157580 con 0x7f5ea0160310 2024-09-21T07:55:10.719 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e9f400640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 0x7f5ea00b15b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b970 tx=0x7f5e9800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e98019070 con 0x7f5ea0160310 2024-09-21T07:55:10.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.717+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5e9800cc50 con 0x7f5ea0160310 2024-09-21T07:55:10.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.718+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ea0157860 con 0x7f5ea0160310 2024-09-21T07:55:10.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.718+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e980146a0 con 0x7f5ea0160310 2024-09-21T07:55:10.720 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.718+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ea0157e20 con 0x7f5ea0160310 2024-09-21T07:55:10.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.719+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ea01535c0 con 0x7f5ea0160310 2024-09-21T07:55:10.721 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.720+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f5e9801c070 con 0x7f5ea0160310 2024-09-21T07:55:10.722 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.721+0000 7f5e97400640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e7c0795e0 0x7f5e7c07baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:10.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.721+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(159..159 src has 1..159) ==== 4971+0+0 (secure 0 0 0) 0x7f5e98099410 con 0x7f5ea0160310 2024-09-21T07:55:10.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.722+0000 7f5e9ea00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e7c0795e0 0x7f5e7c07baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:10.723 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.722+0000 7f5e9ea00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e7c0795e0 0x7f5e7c07baf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5e900098e0 tx=0x7f5e90009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:10.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.726+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5e98062b20 con 0x7f5ea0160310 2024-09-21T07:55:10.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.948+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5ea015fc00 con 0x7f5ea0160310 2024-09-21T07:55:10.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.949+0000 7f5e97400640 1 -- 172.21.15.88:0/3146071440 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 111 v111) ==== 95+0+3919 (secure 0 0 0) 0x7f5e98067550 con 0x7f5ea0160310 2024-09-21T07:55:10.950 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:10.950 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":111,"btime":"2024-09-21T07:55:04:229629+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7241,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/1308244278","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1308244278},{"type":"v1","addr":"172.21.15.88:6835","nonce":1308244278}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2924484932","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2924484932},{"type":"v1","addr":"172.21.15.175:6834","nonce":2924484932}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":104},{"gid":7433,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/529940900","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":529940900},{"type":"v1","addr":"172.21.15.175:6835","nonce":529940900}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":111},{"gid":8763,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6834/1636600576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6833","nonce":1636600576},{"type":"v1","addr":"172.21.15.88:6834","nonce":1636600576}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":103}],"filesystems":[]} 2024-09-21T07:55:10.950 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 111 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e7c0795e0 msgr2=0x7f5e7c07baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e7c0795e0 0x7f5e7c07baf0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f5e900098e0 tx=0x7f5e90009060 comp rx=0 tx=0).stop 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 msgr2=0x7f5ea00b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 0x7f5ea00b15b0 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7f5e9800b970 tx=0x7f5e9800be30 comp rx=0 tx=0).stop 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 shutdown_connections 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5e7c0795e0 0x7f5e7c07baf0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5ea015d760 0x7f5ea01523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ea0160310 0x7f5ea00b15b0 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 --2- 172.21.15.88:0/3146071440 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5ea00ad0e0 0x7f5ea0154780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:10.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.951+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 >> 172.21.15.88:0/3146071440 conn(0x7f5ea006d730 msgr2=0x7f5ea00cfb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:10.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.952+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 shutdown_connections 2024-09-21T07:55:10.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:10.952+0000 7f5ea5c00640 1 -- 172.21.15.88:0/3146071440 wait complete. 2024-09-21T07:55:10.975 INFO:tasks.ceph.mds.a:Restarting daemon 2024-09-21T07:55:10.976 INFO:tasks.ceph.mds.a:Stopping old one... 2024-09-21T07:55:10.976 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:55:10.976 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:10.976 INFO:tasks.ceph.mds.c:Restarting daemon 2024-09-21T07:55:10.976 INFO:tasks.ceph.mds.c:Stopping old one... 2024-09-21T07:55:10.977 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:55:10.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:10.977 INFO:tasks.ceph.mds.b:Restarting daemon 2024-09-21T07:55:10.977 INFO:tasks.ceph.mds.b:Stopping old one... 2024-09-21T07:55:10.977 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:55:10.977 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:10.977 INFO:tasks.ceph.mds.d:Restarting daemon 2024-09-21T07:55:10.977 INFO:tasks.ceph.mds.d:Stopping old one... 2024-09-21T07:55:10.978 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:55:10.978 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:10.998 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:55:10.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2024-09-21T07:55:11.001 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:55:11.002 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2024-09-21T07:55:11.003 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:55:11.003 DEBUG:teuthology.orchestra.run.smithi175:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2024-09-21T07:55:11.043 INFO:tasks.ceph.mds.a:Started 2024-09-21T07:55:11.043 DEBUG:teuthology.parallel:result is None 2024-09-21T07:55:11.047 INFO:tasks.ceph.mds.b:Started 2024-09-21T07:55:11.048 DEBUG:teuthology.parallel:result is None 2024-09-21T07:55:11.051 INFO:tasks.ceph.mds.d:Started 2024-09-21T07:55:11.051 DEBUG:teuthology.parallel:result is None 2024-09-21T07:55:11.230 INFO:tasks.ceph.mds.a.smithi088.stdout:starting mds.a at 2024-09-21T07:55:11.245 INFO:tasks.ceph.mds.d.smithi175.stdout:starting mds.d at 2024-09-21T07:55:11.245 INFO:tasks.ceph.mds.b.smithi175.stdout:starting mds.b at 2024-09-21T07:55:11.711 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c is failed for ~0s 2024-09-21T07:55:16.998 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:55:16.998 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2024-09-21T07:55:17.003 INFO:tasks.ceph.mds.c:Started 2024-09-21T07:55:17.004 DEBUG:teuthology.parallel:result is None 2024-09-21T07:55:17.004 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2024-09-21T07:55:17.207 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.241+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2741730523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e6806f280 msgr2=0x7f0e6806f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.241+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2741730523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e6806f280 0x7f0e6806f690 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f0e58005fd0 tx=0x7f0e5802a540 comp rx=0 tx=0).stop 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.241+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2741730523 shutdown_connections 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.241+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2741730523 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e681623a0 0x7f0e681627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.242+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2741730523 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e6806f280 0x7f0e6806f690 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.242+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2741730523 >> 172.21.15.88:0/2741730523 conn(0x7f0e680b1e40 msgr2=0x7f0e680b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.242+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2741730523 shutdown_connections 2024-09-21T07:55:17.244 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.242+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2741730523 wait complete. 2024-09-21T07:55:17.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.244+0000 7f0e6dc00640 1 Processor -- start 2024-09-21T07:55:17.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.244+0000 7f0e6dc00640 1 -- start start 2024-09-21T07:55:17.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.245+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 0x7f0e681f5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:17.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.245+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e68155370 0x7f0e6809bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:17.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.245+0000 7f0e6dc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e681623a0 0x7f0e68153a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:17.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.245+0000 7f0e6dc00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b3a10 con 0x7f0e6806f280 2024-09-21T07:55:17.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.245+0000 7f0e6dc00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0e680b3890 con 0x7f0e681623a0 2024-09-21T07:55:17.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.245+0000 7f0e6dc00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0e680b3b90 con 0x7f0e68155370 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 0x7f0e681f5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 0x7f0e681f5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44784/0 (socket says 172.21.15.88:44784) 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 -- 172.21.15.88:0/2025197532 learned_addr learned my addr 172.21.15.88:0/2025197532 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67400640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e68155370 0x7f0e6809bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e6d200640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e681623a0 0x7f0e68153a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 -- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e68155370 msgr2=0x7f0e6809bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e68155370 0x7f0e6809bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 -- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e681623a0 msgr2=0x7f0e68153a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e681623a0 0x7f0e68153a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:17.249 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 -- 172.21.15.88:0/2025197532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e681f5d50 con 0x7f0e6806f280 2024-09-21T07:55:17.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.246+0000 7f0e67e00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 0x7f0e681f5810 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0e58006010 tx=0x7f0e580038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.247+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e5803e030 con 0x7f0e6806f280 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.247+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0e58003d90 con 0x7f0e6806f280 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.247+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e681f6260 con 0x7f0e6806f280 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.247+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58035040 con 0x7f0e6806f280 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.247+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e68159a00 con 0x7f0e6806f280 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.248+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e681f6780 con 0x7f0e6806f280 2024-09-21T07:55:17.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.249+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f0e58041030 con 0x7f0e6806f280 2024-09-21T07:55:17.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.250+0000 7f0e5fe00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0e48079610 0x7f0e4807bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:17.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.250+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(162..162 src has 1..162) ==== 4971+0+0 (secure 0 0 0) 0x7f0e580b9e90 con 0x7f0e6806f280 2024-09-21T07:55:17.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.251+0000 7f0e67400640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0e48079610 0x7f0e4807bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:17.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.251+0000 7f0e67400640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0e48079610 0x7f0e4807bb20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e68158c60 tx=0x7f0e54005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:17.257 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.255+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0e681f6780 con 0x7f0e6806f280 2024-09-21T07:55:17.452 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:17.450+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) -- 0x7f0e6806f690 con 0x7f0e6806f280 2024-09-21T07:55:18.424 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.c has been restored 2024-09-21T07:55:18.849 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.847+0000 7f0e5fe00640 1 -- 172.21.15.88:0/2025197532 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v164) ==== 97+0+0 (secure 0 0 0) 0x7f0e68162cf0 con 0x7f0e6806f280 2024-09-21T07:55:18.849 INFO:teuthology.orchestra.run.smithi088.stderr: removed all blocklist entries 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.850+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0e48079610 msgr2=0x7f0e4807bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.850+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0e48079610 0x7f0e4807bb20 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f0e68158c60 tx=0x7f0e54005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.850+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 msgr2=0x7f0e681f5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.850+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 0x7f0e681f5810 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7f0e58006010 tx=0x7f0e580038a0 comp rx=0 tx=0).stop 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.850+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 shutdown_connections 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0e48079610 0x7f0e4807bb20 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0e68155370 0x7f0e6809bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0e681623a0 0x7f0e68153a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 --2- 172.21.15.88:0/2025197532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e6806f280 0x7f0e681f5810 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:18.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 >> 172.21.15.88:0/2025197532 conn(0x7f0e680b1e40 msgr2=0x7f0e680b1290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:18.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 shutdown_connections 2024-09-21T07:55:18.854 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:18.851+0000 7f0e6dc00640 1 -- 172.21.15.88:0/2025197532 wait complete. 2024-09-21T07:55:18.882 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.100+0000 7f782c400640 1 -- 172.21.15.88:0/4228716525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240aebc0 msgr2=0x7f7824152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.100+0000 7f782c400640 1 --2- 172.21.15.88:0/4228716525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240aebc0 0x7f7824152920 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f7818005fd0 tx=0x7f781802a540 comp rx=0 tx=0).stop 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.101+0000 7f782c400640 1 -- 172.21.15.88:0/4228716525 shutdown_connections 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.101+0000 7f782c400640 1 --2- 172.21.15.88:0/4228716525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240aebc0 0x7f7824152920 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.101+0000 7f782c400640 1 -- 172.21.15.88:0/4228716525 >> 172.21.15.88:0/4228716525 conn(0x7f782406d730 msgr2=0x7f78240b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.101+0000 7f782c400640 1 -- 172.21.15.88:0/4228716525 shutdown_connections 2024-09-21T07:55:19.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.101+0000 7f782c400640 1 -- 172.21.15.88:0/4228716525 wait complete. 2024-09-21T07:55:19.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.103+0000 7f782c400640 1 Processor -- start 2024-09-21T07:55:19.105 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.103+0000 7f782c400640 1 -- start start 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.104+0000 7f782c400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78240aebc0 0x7f78240c94c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.104+0000 7f782c400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 0x7f78241f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.104+0000 7f782c400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7824153f50 0x7f78241f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.104+0000 7f782c400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78240b4360 con 0x7f78240b2750 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.104+0000 7f782c400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f78240b41e0 con 0x7f78240aebc0 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.104+0000 7f782c400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f78240b44e0 con 0x7f7824153f50 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 0x7f78241f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 0x7f78241f8860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44802/0 (socket says 172.21.15.88:44802) 2024-09-21T07:55:19.108 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 -- 172.21.15.88:0/2168972341 learned_addr learned my addr 172.21.15.88:0/2168972341 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f782a400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7824153f50 0x7f78241f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829a00640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78240aebc0 0x7f78240c94c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 -- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7824153f50 msgr2=0x7f78241f8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7824153f50 0x7f78241f8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 -- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78240aebc0 msgr2=0x7f78240c94c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78240aebc0 0x7f78240c94c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 -- 172.21.15.88:0/2168972341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78240c9f40 con 0x7f78240b2750 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.105+0000 7f7829000640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 0x7f78241f8860 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f781000c3c0 tx=0x7f781000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.106+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810017070 con 0x7f78240b2750 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.106+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f78100035f0 con 0x7f78240b2750 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.106+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7810003a80 con 0x7f78240b2750 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.106+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78240ca220 con 0x7f78240b2750 2024-09-21T07:55:19.109 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.106+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7824074100 con 0x7f78240b2750 2024-09-21T07:55:19.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.107+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78240c9cb0 con 0x7f78240b2750 2024-09-21T07:55:19.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.108+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f781001a070 con 0x7f78240b2750 2024-09-21T07:55:19.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.110+0000 7f7821600640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78040795e0 0x7f780407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.110+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4885+0+0 (secure 0 0 0) 0x7f781009b360 con 0x7f78240b2750 2024-09-21T07:55:19.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.110+0000 7f7829a00640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78040795e0 0x7f780407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.112 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.110+0000 7f7829a00640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78040795e0 0x7f780407baf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7818033000 tx=0x7f7818004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:19.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.115+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7810062820 con 0x7f78240b2750 2024-09-21T07:55:19.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.369+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) -- 0x7f78240b49d0 con 0x7f78240b2750 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.370+0000 7f7821600640 1 -- 172.21.15.88:0/2168972341 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v22) ==== 80+0+5304 (secure 0 0 0) 0x7f7810062120 con 0x7f78240b2750 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: "auth_dump": [ 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.a", 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mCW+RFhAAQrCi+BPHgnj1oYdn1lqa4A==", 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.372 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:55:19.373 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:55:19.373 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.373 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.373 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.373 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.373 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.b", 2024-09-21T07:55:19.374 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mowISMhAAEiFxgsDWiUNYLyBHym8TbQ==", 2024-09-21T07:55:19.374 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.374 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.374 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:55:19.374 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.c", 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mIxZQJBAAtQY9foZ84GX22ZM8bbHl6Q==", 2024-09-21T07:55:19.375 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.376 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mds.d", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mB1FPBBAAlp0pbTQr1VGW4+Wf0MKblQ==", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.0", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB/ee5mPzPZLhAAsmxJRelS3V8s21mCsMPyEg==", 2024-09-21T07:55:19.377 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.378 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.378 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.378 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.378 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.378 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.378 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.1", 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCDee5muiC9AxAAXSH5P2n7Z0xJvSZlYeHZrg==", 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.2", 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCGee5mzP5GEhAASylA2vGNI818Mva88gWKuA==", 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.379 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.3", 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCJee5mFYdGMRAAMCMTjdJxKLvCT97vfuyVUw==", 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.380 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.4", 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCPee5mTYnoMRAAv8Hzxpvmii+5t8XgXA8c2Q==", 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.5", 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCTee5mQGyvBRAATOdRAaEhPtMhAI3JL9b3oQ==", 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.381 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.6", 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCWee5mFWryFBAAWDiLt1qV2sczTK0Zw+V+0Q==", 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.382 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "osd.7", 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQCZee5miGUXJBAAXys1WOVUIsva68KzsWgRXw==", 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow profile osd", 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile osd", 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.0", 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA==", 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.383 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow r", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rw tag cephfs data=cephfs" 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.1", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5m9bPaJRAAy3ZkHVSSZbCtI++eQYR1DQ==", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.384 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.2", 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mWJZXGxAApE4M+JrlaTIzry0ZQc1tFA==", 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:55:19.385 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.386 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.386 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.386 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.3", 2024-09-21T07:55:19.386 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB8ee5mPaA8MBAALR829KoCZNG3nIRMD+rdjA==", 2024-09-21T07:55:19.386 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.386 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow", 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow r", 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow rw", 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow rwx" 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "client.admin", 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB6ee5mKwM2FhAA+NL8Pc/HYkp05L4XD8vkbA==", 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.387 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "mgr": "allow *", 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow *", 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.x", 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5mjAjKCxAAaB96+MeekCaGyIal9MriJA==", 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:55:19.388 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "entity": "mgr.y", 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "key": "AQB7ee5msZ9PABAAddp+UoIud7ZaTeJaUktEBw==", 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": { 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": "allow *", 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "mon": "allow profile mgr", 2024-09-21T07:55:19.389 INFO:teuthology.orchestra.run.smithi088.stdout: "osd": "allow *" 2024-09-21T07:55:19.390 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.390 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.390 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-09-21T07:55:19.390 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78040795e0 msgr2=0x7f780407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78040795e0 0x7f780407baf0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f7818033000 tx=0x7f7818004730 comp rx=0 tx=0).stop 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 msgr2=0x7f78241f8860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 0x7f78241f8860 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f781000c3c0 tx=0x7f781000c880 comp rx=0 tx=0).stop 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 shutdown_connections 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f78040795e0 0x7f780407baf0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7824153f50 0x7f78241f8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78240b2750 0x7f78241f8860 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 --2- 172.21.15.88:0/2168972341 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f78240aebc0 0x7f78240c94c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 >> 172.21.15.88:0/2168972341 conn(0x7f782406d730 msgr2=0x7f782415e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:19.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 shutdown_connections 2024-09-21T07:55:19.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.372+0000 7f782c400640 1 -- 172.21.15.88:0/2168972341 wait complete. 2024-09-21T07:55:19.397 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 -- 172.21.15.88:0/3220035552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 msgr2=0x7f628c004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 --2- 172.21.15.88:0/3220035552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f628c004c10 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7f62940016c0 tx=0x7f628c005830 comp rx=0 tx=0).stop 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 -- 172.21.15.88:0/3220035552 shutdown_connections 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 --2- 172.21.15.88:0/3220035552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f628c004c10 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 -- 172.21.15.88:0/3220035552 >> 172.21.15.88:0/3220035552 conn(0x7f62a806d730 msgr2=0x7f62a80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 -- 172.21.15.88:0/3220035552 shutdown_connections 2024-09-21T07:55:19.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.616+0000 7f62ad400640 1 -- 172.21.15.88:0/3220035552 wait complete. 2024-09-21T07:55:19.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.619+0000 7f62ad400640 1 Processor -- start 2024-09-21T07:55:19.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.619+0000 7f62ad400640 1 -- start start 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62ad400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f62a8150770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62ad400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80afd20 0x7f62a80b0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62ad400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a81e7220 0x7f62a81e7630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62ad400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f62a815f3d0 con 0x7f628c004800 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62ad400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f62a8152cf0 con 0x7f62a80afd20 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62ad400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f62a815f550 con 0x7f62a81e7220 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.620+0000 7f62a6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f62a8150770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f62a8150770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44816/0 (socket says 172.21.15.88:44816) 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 -- 172.21.15.88:0/3085425331 learned_addr learned my addr 172.21.15.88:0/3085425331 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a5e00640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80afd20 0x7f62a80b0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 -- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a81e7220 msgr2=0x7f62a81e7630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a81e7220 0x7f62a81e7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 -- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80afd20 msgr2=0x7f62a80b0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80afd20 0x7f62a80b0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 -- 172.21.15.88:0/3085425331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62a81e7b70 con 0x7f628c004800 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a5e00640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80afd20 0x7f62a80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f62a6800640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f62a8150770 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f62980117b0 tx=0x7f6298011c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.621+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f629801f070 con 0x7f628c004800 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.622+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f629800fe40 con 0x7f628c004800 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.622+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62980126b0 con 0x7f628c004800 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.622+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62a81f5800 con 0x7f628c004800 2024-09-21T07:55:19.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.622+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62a81f5dc0 con 0x7f628c004800 2024-09-21T07:55:19.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.623+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62a815fa20 con 0x7f628c004800 2024-09-21T07:55:19.626 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.624+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f6298022070 con 0x7f628c004800 2024-09-21T07:55:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.625+0000 7f629ea00640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f62880795b0 0x7f628807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.625+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4885+0+0 (secure 0 0 0) 0x7f629809a2b0 con 0x7f628c004800 2024-09-21T07:55:19.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.625+0000 7f62a5e00640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f62880795b0 0x7f628807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:19.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.626+0000 7f62a5e00640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f62880795b0 0x7f628807bac0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6294032000 tx=0x7f6294001190 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:19.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.629+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f62a81e7b70 con 0x7f628c004800 2024-09-21T07:55:19.817 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.815+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f62a8151c80 con 0x7f628c004800 2024-09-21T07:55:19.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.816+0000 7f629ea00640 1 -- 172.21.15.88:0/3085425331 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v164) ==== 84+0+65 (secure 0 0 0) 0x7f6298063930 con 0x7f628c004800 2024-09-21T07:55:19.818 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:19.818 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:55:19.818 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:19.818 INFO:teuthology.orchestra.run.smithi088.stdout: "poolnum": 1, 2024-09-21T07:55:19.818 INFO:teuthology.orchestra.run.smithi088.stdout: "poolname": ".mgr" 2024-09-21T07:55:19.819 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:19.819 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:55:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f62880795b0 msgr2=0x7f628807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f62880795b0 0x7f628807bac0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f6294032000 tx=0x7f6294001190 comp rx=0 tx=0).stop 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 msgr2=0x7f62a8150770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f62a8150770 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7f62980117b0 tx=0x7f6298011c70 comp rx=0 tx=0).stop 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 shutdown_connections 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f62880795b0 0x7f628807bac0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f62a81e7220 0x7f62a81e7630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f62a80afd20 0x7f62a80b0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 --2- 172.21.15.88:0/3085425331 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f628c004800 0x7f62a8150770 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 >> 172.21.15.88:0/3085425331 conn(0x7f62a806d730 msgr2=0x7f62a815c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 shutdown_connections 2024-09-21T07:55:19.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:19.818+0000 7f62ad400640 1 -- 172.21.15.88:0/3085425331 wait complete. 2024-09-21T07:55:19.843 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T07:55:19.844 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 -- 172.21.15.88:0/1725983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 msgr2=0x7f57ec1602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 --2- 172.21.15.88:0/1725983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec1602b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7f57ec06e750 tx=0x7f57e402a820 comp rx=0 tx=0).stop 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 -- 172.21.15.88:0/1725983171 shutdown_connections 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 --2- 172.21.15.88:0/1725983171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec1602b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 --2- 172.21.15.88:0/1725983171 >> v2:172.21.15.175:3300/0 conn(0x7f57ec0705b0 0x7f57ec0b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 -- 172.21.15.88:0/1725983171 >> 172.21.15.88:0/1725983171 conn(0x7f57ec0b1dd0 msgr2=0x7f57ec0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 -- 172.21.15.88:0/1725983171 shutdown_connections 2024-09-21T07:55:20.070 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.067+0000 7f57f3000640 1 -- 172.21.15.88:0/1725983171 wait complete. 2024-09-21T07:55:20.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.069+0000 7f57f3000640 1 Processor -- start 2024-09-21T07:55:20.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.069+0000 7f57f3000640 1 -- start start 2024-09-21T07:55:20.073 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f3000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec0705b0 0x7f57ec1f9920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f3000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec14f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f3000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec0b3ec0 0x7f57ec153dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f3000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec160b80 con 0x7f57ec0af3c0 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f3000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f57ec160a00 con 0x7f57ec0705b0 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f3000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f57ec153c40 con 0x7f57ec0b3ec0 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec14f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f1200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec14f260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44832/0 (socket says 172.21.15.88:44832) 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f1200640 1 -- 172.21.15.88:0/3206253644 learned_addr learned my addr 172.21.15.88:0/3206253644 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f1c00640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec0705b0 0x7f57ec1f9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.071+0000 7f57f2600640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec0b3ec0 0x7f57ec153dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1200640 1 -- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec0b3ec0 msgr2=0x7f57ec153dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1200640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec0b3ec0 0x7f57ec153dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.074 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1200640 1 -- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec0705b0 msgr2=0x7f57ec1f9920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:20.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1200640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec0705b0 0x7f57ec1f9920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1200640 1 -- 172.21.15.88:0/3206253644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57ec09ae30 con 0x7f57ec0af3c0 2024-09-21T07:55:20.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1c00640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec0705b0 0x7f57ec1f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:20.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f1200640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec14f260 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f57d800d780 tx=0x7f57d800dc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:20.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d8017070 con 0x7f57ec0af3c0 2024-09-21T07:55:20.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f57d800de90 con 0x7f57ec0af3c0 2024-09-21T07:55:20.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d80099d0 con 0x7f57ec0af3c0 2024-09-21T07:55:20.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57ec09b110 con 0x7f57ec0af3c0 2024-09-21T07:55:20.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.072+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57ec1f9120 con 0x7f57ec0af3c0 2024-09-21T07:55:20.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.073+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f57ec0b47c0 con 0x7f57ec0af3c0 2024-09-21T07:55:20.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.075+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f57d8009b70 con 0x7f57ec0af3c0 2024-09-21T07:55:20.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.076+0000 7f57e9600640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57cc0795b0 0x7f57cc07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:20.078 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.077+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(164..164 src has 1..164) ==== 4885+0+0 (secure 0 0 0) 0x7f57d8099d20 con 0x7f57ec0af3c0 2024-09-21T07:55:20.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.077+0000 7f57f1c00640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57cc0795b0 0x7f57cc07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:20.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.077+0000 7f57f1c00640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57cc0795b0 0x7f57cc07bac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f57e00013f0 tx=0x7f57e0009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:20.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.081+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f57d8020020 con 0x7f57ec0af3c0 2024-09-21T07:55:20.271 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.269+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f57ec09abc0 con 0x7f57ec0af3c0 2024-09-21T07:55:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.867+0000 7f57e9600640 1 -- 172.21.15.88:0/3206253644 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v165) ==== 138+0+0 (secure 0 0 0) 0x7f57d8063a20 con 0x7f57ec0af3c0 2024-09-21T07:55:20.869 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' created 2024-09-21T07:55:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.870+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57cc0795b0 msgr2=0x7f57cc07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.870+0000 7f57f3000640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57cc0795b0 0x7f57cc07bac0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f57e00013f0 tx=0x7f57e0009040 comp rx=0 tx=0).stop 2024-09-21T07:55:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.870+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 msgr2=0x7f57ec14f260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:20.872 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.870+0000 7f57f3000640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec14f260 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f57d800d780 tx=0x7f57d800dc40 comp rx=0 tx=0).stop 2024-09-21T07:55:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 shutdown_connections 2024-09-21T07:55:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f57cc0795b0 0x7f57cc07bac0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f57ec0b3ec0 0x7f57ec153dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57ec0af3c0 0x7f57ec14f260 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 --2- 172.21.15.88:0/3206253644 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f57ec0705b0 0x7f57ec1f9920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:20.873 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 >> 172.21.15.88:0/3206253644 conn(0x7f57ec0b1dd0 msgr2=0x7f57ec0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 shutdown_connections 2024-09-21T07:55:20.874 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:20.871+0000 7f57f3000640 1 -- 172.21.15.88:0/3206253644 wait complete. 2024-09-21T07:55:20.904 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T07:55:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 -- 172.21.15.88:0/328009745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8145550 msgr2=0x7f11c806fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 --2- 172.21.15.88:0/328009745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8145550 0x7f11c806fb40 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f11c8160280 tx=0x7f11b802a5e0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 -- 172.21.15.88:0/328009745 shutdown_connections 2024-09-21T07:55:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 --2- 172.21.15.88:0/328009745 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c8073a80 0x7f11c8160320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.131 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 --2- 172.21.15.88:0/328009745 >> v2:172.21.15.175:3300/0 conn(0x7f11c8145a40 0x7f11c8073540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 --2- 172.21.15.88:0/328009745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8145550 0x7f11c806fb40 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 -- 172.21.15.88:0/328009745 >> 172.21.15.88:0/328009745 conn(0x7f11c806d1c0 msgr2=0x7f11c806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 -- 172.21.15.88:0/328009745 shutdown_connections 2024-09-21T07:55:21.132 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.129+0000 7f11cd200640 1 -- 172.21.15.88:0/328009745 wait complete. 2024-09-21T07:55:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.131+0000 7f11cd200640 1 Processor -- start 2024-09-21T07:55:21.133 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.131+0000 7f11cd200640 1 -- start start 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11cd200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 0x7f11c81f63e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11cd200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c8145550 0x7f11c813cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11cd200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c8145a40 0x7f11c80b22c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11cd200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8160c30 con 0x7f11c8073a80 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11cd200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f11c8160ab0 con 0x7f11c8145550 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11cd200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f11c8160db0 con 0x7f11c8145a40 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 0x7f11c81f63e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 0x7f11c81f63e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44850/0 (socket says 172.21.15.88:44850) 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 -- 172.21.15.88:0/2921109732 learned_addr learned my addr 172.21.15.88:0/2921109732 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5400640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c8145550 0x7f11c813cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c6800640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c8145a40 0x7f11c80b22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 -- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c8145a40 msgr2=0x7f11c80b22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:21.136 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c8145a40 0x7f11c80b22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 -- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c8145550 msgr2=0x7f11c813cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c8145550 0x7f11c813cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.133+0000 7f11c5e00640 1 -- 172.21.15.88:0/2921109732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11c81f9410 con 0x7f11c8073a80 2024-09-21T07:55:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.134+0000 7f11c5e00640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 0x7f11c81f63e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f11b8034040 tx=0x7f11b80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.134+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b803e030 con 0x7f11c8073a80 2024-09-21T07:55:21.137 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.134+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f11b8004c20 con 0x7f11c8073a80 2024-09-21T07:55:21.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.134+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11b8035040 con 0x7f11c8073a80 2024-09-21T07:55:21.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.134+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11c81f96f0 con 0x7f11c8073a80 2024-09-21T07:55:21.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.134+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11c81f9cb0 con 0x7f11c8073a80 2024-09-21T07:55:21.138 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.135+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f11c80cdf40 con 0x7f11c8073a80 2024-09-21T07:55:21.139 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.137+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f11b8041030 con 0x7f11c8073a80 2024-09-21T07:55:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.138+0000 7f11be000640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11a80795b0 0x7f11a807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.138+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(165..165 src has 1..165) ==== 5255+0+0 (secure 0 0 0) 0x7f11b80ba410 con 0x7f11c8073a80 2024-09-21T07:55:21.140 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.138+0000 7f11c5400640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11a80795b0 0x7f11a807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:21.141 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.139+0000 7f11c5400640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11a80795b0 0x7f11a807bac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f11b40058c0 tx=0x7f11b4005850 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:21.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.142+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f11c80cdf40 con 0x7f11c8073a80 2024-09-21T07:55:21.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.328+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f11c806fb40 con 0x7f11c8073a80 2024-09-21T07:55:21.878 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.875+0000 7f11be000640 1 -- 172.21.15.88:0/2921109732 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v166) ==== 170+0+0 (secure 0 0 0) 0x7f11b80840c0 con 0x7f11c8073a80 2024-09-21T07:55:21.878 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' created 2024-09-21T07:55:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.878+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11a80795b0 msgr2=0x7f11a807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.878+0000 7f11cd200640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11a80795b0 0x7f11a807bac0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f11b40058c0 tx=0x7f11b4005850 comp rx=0 tx=0).stop 2024-09-21T07:55:21.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.878+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 msgr2=0x7f11c81f63e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.878+0000 7f11cd200640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 0x7f11c81f63e0 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f11b8034040 tx=0x7f11b80045e0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 shutdown_connections 2024-09-21T07:55:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11a80795b0 0x7f11a807bac0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f11c8145a40 0x7f11c80b22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f11c8145550 0x7f11c813cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 --2- 172.21.15.88:0/2921109732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11c8073a80 0x7f11c81f63e0 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 >> 172.21.15.88:0/2921109732 conn(0x7f11c806d1c0 msgr2=0x7f11c80adbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 shutdown_connections 2024-09-21T07:55:21.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:21.879+0000 7f11cd200640 1 -- 172.21.15.88:0/2921109732 wait complete. 2024-09-21T07:55:21.922 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T07:55:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 -- 172.21.15.88:0/1828532483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c4156ca0 msgr2=0x7f03c40ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 --2- 172.21.15.88:0/1828532483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c4156ca0 0x7f03c40ccac0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f03b0009be0 tx=0x7f03b002a8b0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 -- 172.21.15.88:0/1828532483 shutdown_connections 2024-09-21T07:55:22.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 --2- 172.21.15.88:0/1828532483 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c4156ca0 0x7f03c40ccac0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 -- 172.21.15.88:0/1828532483 >> 172.21.15.88:0/1828532483 conn(0x7f03c406d620 msgr2=0x7f03c40b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 -- 172.21.15.88:0/1828532483 shutdown_connections 2024-09-21T07:55:22.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.144+0000 7f03c9800640 1 -- 172.21.15.88:0/1828532483 wait complete. 2024-09-21T07:55:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.146+0000 7f03c9800640 1 Processor -- start 2024-09-21T07:55:22.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.146+0000 7f03c9800640 1 -- start start 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.147+0000 7f03c9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c40b3f30 0x7f03c41f3e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.147+0000 7f03c9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c40cd000 0x7f03c4154340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.147+0000 7f03c9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 0x7f03c409d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.147+0000 7f03c9800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f03c40af7e0 con 0x7f03c4156ca0 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f03c40703b0 con 0x7f03c40b3f30 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f03c40af960 con 0x7f03c40cd000 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 0x7f03c409d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 0x7f03c409d510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44862/0 (socket says 172.21.15.88:44862) 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 -- 172.21.15.88:0/1443241865 learned_addr learned my addr 172.21.15.88:0/1443241865 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:22.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c3400640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c40cd000 0x7f03c4154340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:22.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c3e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c40b3f30 0x7f03c41f3e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 -- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c40cd000 msgr2=0x7f03c4154340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c40cd000 0x7f03c4154340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 -- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c40b3f30 msgr2=0x7f03c41f3e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c40b3f30 0x7f03c41f3e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c8e00640 1 -- 172.21.15.88:0/1443241865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03c4154880 con 0x7f03c4156ca0 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.148+0000 7f03c3e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c40b3f30 0x7f03c41f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.149+0000 7f03c8e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 0x7f03c409d510 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f03bc001eb0 tx=0x7f03bc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.149+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc011790 con 0x7f03c4156ca0 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.149+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f03bc017070 con 0x7f03c4156ca0 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.149+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03c40af420 con 0x7f03c4156ca0 2024-09-21T07:55:22.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.149+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03bc011cc0 con 0x7f03c4156ca0 2024-09-21T07:55:22.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.149+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03c409c750 con 0x7f03c4156ca0 2024-09-21T07:55:22.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.150+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03c409c1e0 con 0x7f03c4156ca0 2024-09-21T07:55:22.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.152+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f03bc01c460 con 0x7f03c4156ca0 2024-09-21T07:55:22.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.153+0000 7f03bbe00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03a40795b0 0x7f03a407bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:22.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.153+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(166..166 src has 1..166) ==== 5637+0+0 (secure 0 0 0) 0x7f03bc09a250 con 0x7f03c4156ca0 2024-09-21T07:55:22.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.153+0000 7f03c3e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03a40795b0 0x7f03a407bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:22.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.153+0000 7f03c3e00640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03a40795b0 0x7f03a407bac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001120 tx=0x7f03b4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:22.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.156+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f03c4154880 con 0x7f03c4156ca0 2024-09-21T07:55:22.380 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.378+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f03c40ccac0 con 0x7f03c4156ca0 2024-09-21T07:55:22.879 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.877+0000 7f03bbe00640 1 -- 172.21.15.88:0/1443241865 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 26 and data pool 27 v115) ==== 174+0+0 (secure 0 0 0) 0x7f03bc063f50 con 0x7f03c4156ca0 2024-09-21T07:55:22.880 INFO:teuthology.orchestra.run.smithi088.stderr:new fs with metadata pool 26 and data pool 27 2024-09-21T07:55:22.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03a40795b0 msgr2=0x7f03a407bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03a40795b0 0x7f03a407bac0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f03b4001120 tx=0x7f03b4009040 comp rx=0 tx=0).stop 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 msgr2=0x7f03c409d510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 0x7f03c409d510 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f03bc001eb0 tx=0x7f03bc00c960 comp rx=0 tx=0).stop 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 shutdown_connections 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f03a40795b0 0x7f03a407bac0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03c40cd000 0x7f03c4154340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03c4156ca0 0x7f03c409d510 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.880+0000 7f03c9800640 1 --2- 172.21.15.88:0/1443241865 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f03c40b3f30 0x7f03c41f3e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:22.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.881+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 >> 172.21.15.88:0/1443241865 conn(0x7f03c406d620 msgr2=0x7f03c40d0910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:22.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.881+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 shutdown_connections 2024-09-21T07:55:22.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:22.881+0000 7f03c9800640 1 -- 172.21.15.88:0/1443241865 wait complete. 2024-09-21T07:55:22.936 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 -- 172.21.15.88:0/3029266540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0d1580 msgr2=0x7f225c0d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 --2- 172.21.15.88:0/3029266540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0d1580 0x7f225c0d1990 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f225c0d2050 tx=0x7f225402aa00 comp rx=0 tx=0).stop 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 -- 172.21.15.88:0/3029266540 shutdown_connections 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 --2- 172.21.15.88:0/3029266540 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0d1580 0x7f225c0d1990 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 --2- 172.21.15.88:0/3029266540 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225c0b9e00 0x7f225c162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 -- 172.21.15.88:0/3029266540 >> 172.21.15.88:0/3029266540 conn(0x7f225c08bfe0 msgr2=0x7f225c09d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.154+0000 7f2261400640 1 -- 172.21.15.88:0/3029266540 shutdown_connections 2024-09-21T07:55:23.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.155+0000 7f2261400640 1 -- 172.21.15.88:0/3029266540 wait complete. 2024-09-21T07:55:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.157+0000 7f2261400640 1 Processor -- start 2024-09-21T07:55:23.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.157+0000 7f2261400640 1 -- start start 2024-09-21T07:55:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.158+0000 7f2261400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225c0b3160 0x7f225c15b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.158+0000 7f2261400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0b9e00 0x7f225c15d760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.158+0000 7f2261400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 0x7f225c15dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.158+0000 7f2261400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0d44b0 con 0x7f225c0d1580 2024-09-21T07:55:23.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.158+0000 7f2261400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f225c0d4330 con 0x7f225c0b9e00 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.158+0000 7f2261400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f225c0d4630 con 0x7f225c0b3160 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 0x7f225c15dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 0x7f225c15dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44878/0 (socket says 172.21.15.88:44878) 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 -- 172.21.15.88:0/1416641206 learned_addr learned my addr 172.21.15.88:0/1416641206 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2253400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0b9e00 0x7f225c15d760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f225be00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225c0b3160 0x7f225c15b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 -- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225c0b3160 msgr2=0x7f225c15b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225c0b3160 0x7f225c15b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 -- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0b9e00 msgr2=0x7f225c15d760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0b9e00 0x7f225c15d760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 -- 172.21.15.88:0/1416641206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f225c15ead0 con 0x7f225c0d1580 2024-09-21T07:55:23.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.159+0000 7f2260a00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 0x7f225c15dca0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f225402aff0 tx=0x7f2254003f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.160+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254033050 con 0x7f225c0d1580 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.160+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2254004450 con 0x7f225c0d1580 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.160+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f225c15edb0 con 0x7f225c0d1580 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.160+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2254004820 con 0x7f225c0d1580 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.160+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f225c15be40 con 0x7f225c0d1580 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.161+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f225c15f940 con 0x7f225c0d1580 2024-09-21T07:55:23.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.162+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f2254036070 con 0x7f225c0d1580 2024-09-21T07:55:23.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.163+0000 7f2258c00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22380795b0 0x7f223807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.163+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5737+0+0 (secure 0 0 0) 0x7f22540ba900 con 0x7f225c0d1580 2024-09-21T07:55:23.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.163+0000 7f225be00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22380795b0 0x7f223807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.164+0000 7f225be00640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22380795b0 0x7f223807bac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f224c0095f0 tx=0x7f224c005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:23.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.167+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2254084150 con 0x7f225c0d1580 2024-09-21T07:55:23.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.353+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f225c0d4b50 con 0x7f225c0d1580 2024-09-21T07:55:23.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.354+0000 7f2258c00640 1 -- 172.21.15.88:0/1416641206 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16744 (secure 0 0 0) 0x7f2254088b80 con 0x7f225c0d1580 2024-09-21T07:55:23.357 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:23.357 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":167,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:22.380704+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:55:20.283046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T07:55:21.342418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:23.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22380795b0 msgr2=0x7f223807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22380795b0 0x7f223807bac0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f224c0095f0 tx=0x7f224c005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 msgr2=0x7f225c15dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 0x7f225c15dca0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f225402aff0 tx=0x7f2254003f70 comp rx=0 tx=0).stop 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 shutdown_connections 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f22380795b0 0x7f223807bac0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f225c0d1580 0x7f225c15dca0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f225c0b9e00 0x7f225c15d760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 --2- 172.21.15.88:0/1416641206 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f225c0b3160 0x7f225c15b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 >> 172.21.15.88:0/1416641206 conn(0x7f225c08bfe0 msgr2=0x7f225c06f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 shutdown_connections 2024-09-21T07:55:23.360 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.357+0000 7f2261400640 1 -- 172.21.15.88:0/1416641206 wait complete. 2024-09-21T07:55:23.382 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:23.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.600+0000 7f6c7be00640 1 -- 172.21.15.88:0/1882465620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 msgr2=0x7f6c7c0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.600+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1882465620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 0x7f6c7c0b2f60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009860 tx=0x7f6c6402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.601+0000 7f6c7be00640 1 -- 172.21.15.88:0/1882465620 shutdown_connections 2024-09-21T07:55:23.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.601+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1882465620 >> v2:172.21.15.175:3300/0 conn(0x7f6c7c15a0f0 0x7f6c7c15d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.601+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1882465620 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 0x7f6c7c0b2f60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.601+0000 7f6c7be00640 1 -- 172.21.15.88:0/1882465620 >> 172.21.15.88:0/1882465620 conn(0x7f6c7c0b1e40 msgr2=0x7f6c7c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:23.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.601+0000 7f6c7be00640 1 -- 172.21.15.88:0/1882465620 shutdown_connections 2024-09-21T07:55:23.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.601+0000 7f6c7be00640 1 -- 172.21.15.88:0/1882465620 wait complete. 2024-09-21T07:55:23.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.603+0000 7f6c7be00640 1 Processor -- start 2024-09-21T07:55:23.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.603+0000 7f6c7be00640 1 -- start start 2024-09-21T07:55:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.604+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 0x7f6c7c0ccac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 0x7f6c7c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15a0f0 0x7f6c7c0cb750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c06f410 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7c06f290 con 0x7f6c7c15a0f0 2024-09-21T07:55:23.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6c7c06f590 con 0x7f6c7c0af170 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 0x7f6c7c0ccac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 0x7f6c7c0ccac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44888/0 (socket says 172.21.15.88:44888) 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 -- 172.21.15.88:0/1691231541 learned_addr learned my addr 172.21.15.88:0/1691231541 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c72000640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 0x7f6c7c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7b400640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15a0f0 0x7f6c7c0cb750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 -- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 msgr2=0x7f6c7c0cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 0x7f6c7c0cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 -- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15a0f0 msgr2=0x7f6c7c0cb750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15a0f0 0x7f6c7c0cb750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.605+0000 7f6c7aa00640 1 -- 172.21.15.88:0/1691231541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7c1f60b0 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.606+0000 7f6c7aa00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 0x7f6c7c0ccac0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c460 tx=0x7f6c6c00c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:23.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.606+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0052d0 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.606+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6c6c005490 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.606+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7c1f4f50 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.606+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c012870 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.606+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c7c0ce610 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.607+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c7c06fab0 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.608+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f6c6c012a10 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.609+0000 7f6c73400640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6c580795e0 0x7f6c5807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:23.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.610+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(167..167 src has 1..167) ==== 5737+0+0 (secure 0 0 0) 0x7f6c6c0992a0 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.610+0000 7f6c72000640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6c580795e0 0x7f6c5807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:23.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.610+0000 7f6c72000640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6c580795e0 0x7f6c5807baf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009860 tx=0x7f6c64004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:23.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.614+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6c6c0628a0 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.802 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.800+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6c7c15c090 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.801+0000 7f6c73400640 1 -- 172.21.15.88:0/1691231541 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v167) ==== 74+0+16744 (secure 0 0 0) 0x7f6c6c0672d0 con 0x7f6c7c0ad9c0 2024-09-21T07:55:23.804 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:23.804 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":167,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:22.380704+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:55:20.283046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T07:55:21.342418+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6c580795e0 msgr2=0x7f6c5807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6c580795e0 0x7f6c5807baf0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f6c64009860 tx=0x7f6c64004730 comp rx=0 tx=0).stop 2024-09-21T07:55:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 msgr2=0x7f6c7c0ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:23.806 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 0x7f6c7c0ccac0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c00c460 tx=0x7f6c6c00c920 comp rx=0 tx=0).stop 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 shutdown_connections 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6c580795e0 0x7f6c5807baf0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6c7c0af170 0x7f6c7c0cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6c7c15a0f0 0x7f6c7c0cb750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 --2- 172.21.15.88:0/1691231541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c7c0ad9c0 0x7f6c7c0ccac0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 >> 172.21.15.88:0/1691231541 conn(0x7f6c7c0b1e40 msgr2=0x7f6c7c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 shutdown_connections 2024-09-21T07:55:23.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:23.804+0000 7f6c7be00640 1 -- 172.21.15.88:0/1691231541 wait complete. 2024-09-21T07:55:23.830 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-09-21T07:55:24.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 -- 172.21.15.88:0/2442288967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd415f9e0 msgr2=0x7fbbd4147b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2442288967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd415f9e0 0x7fbbd4147b90 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8009830 tx=0x7fbbc802a3f0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 -- 172.21.15.88:0/2442288967 shutdown_connections 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2442288967 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbd40aee00 0x7fbbd40af210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2442288967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd415f9e0 0x7fbbd4147b90 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 -- 172.21.15.88:0/2442288967 >> 172.21.15.88:0/2442288967 conn(0x7fbbd406d1c0 msgr2=0x7fbbd406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 -- 172.21.15.88:0/2442288967 shutdown_connections 2024-09-21T07:55:24.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.056+0000 7fbbdc800640 1 -- 172.21.15.88:0/2442288967 wait complete. 2024-09-21T07:55:24.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.058+0000 7fbbdc800640 1 Processor -- start 2024-09-21T07:55:24.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.059+0000 7fbbdc800640 1 -- start start 2024-09-21T07:55:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbdc800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbd4073540 0x7fbbd4135850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbdc800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 0x7fbbd41f6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:24.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbdc800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbd415f9e0 0x7fbbd41eaba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbdc800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd406fb60 con 0x7fbbd40aee00 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbdc800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fbbd406f9e0 con 0x7fbbd4073540 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbdc800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fbbd406fce0 con 0x7fbbd415f9e0 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbd9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 0x7fbbd41f6160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbd9400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 0x7fbbd41f6160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44898/0 (socket says 172.21.15.88:44898) 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.060+0000 7fbbd9400640 1 -- 172.21.15.88:0/2774976728 learned_addr learned my addr 172.21.15.88:0/2774976728 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbda800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbd415f9e0 0x7fbbd41eaba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9e00640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbd4073540 0x7fbbd4135850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9400640 1 -- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbd415f9e0 msgr2=0x7fbbd41eaba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9400640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbd415f9e0 0x7fbbd41eaba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9400640 1 -- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbd4073540 msgr2=0x7fbbd4135850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9400640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbd4073540 0x7fbbd4135850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9400640 1 -- 172.21.15.88:0/2774976728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbd4136200 con 0x7fbbd40aee00 2024-09-21T07:55:24.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbd9400640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 0x7fbbd41f6160 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400d6d0 tx=0x7fbbc400db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4017070 con 0x7fbbd40aee00 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fbbc4009b10 con 0x7fbbd40aee00 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbd41364e0 con 0x7fbbd40aee00 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc4010460 con 0x7fbbd40aee00 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.061+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbd4133910 con 0x7fbbd40aee00 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.062+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbd4135f70 con 0x7fbbd40aee00 2024-09-21T07:55:24.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.064+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fbbc401a070 con 0x7fbbd40aee00 2024-09-21T07:55:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.065+0000 7fbbce000640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbb80795e0 0x7fbbb807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.065+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7fbbc4099790 con 0x7fbbd40aee00 2024-09-21T07:55:24.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.066+0000 7fbbd9e00640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbb80795e0 0x7fbbb807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:24.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.066+0000 7fbbd9e00640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbb80795e0 0x7fbbb807baf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:24.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.070+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fbbc4063620 con 0x7fbbd40aee00 2024-09-21T07:55:24.294 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.292+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7fbbd40af210 con 0x7fbbd40aee00 2024-09-21T07:55:24.906 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.904+0000 7fbbce000640 1 -- 172.21.15.88:0/2774976728 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v117) ==== 118+0+0 (secure 0 0 0) 0x7fbbc4062f20 con 0x7fbbd40aee00 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbb80795e0 msgr2=0x7fbbb807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbb80795e0 0x7fbbb807baf0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fbbc8034000 tx=0x7fbbc8004730 comp rx=0 tx=0).stop 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 msgr2=0x7fbbd41f6160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 0x7fbbd41f6160 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7fbbc400d6d0 tx=0x7fbbc400db90 comp rx=0 tx=0).stop 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 shutdown_connections 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fbbb80795e0 0x7fbbb807baf0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fbbd415f9e0 0x7fbbd41eaba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbd40aee00 0x7fbbd41f6160 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 --2- 172.21.15.88:0/2774976728 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fbbd4073540 0x7fbbd4135850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 >> 172.21.15.88:0/2774976728 conn(0x7fbbd406d1c0 msgr2=0x7fbbd40cf720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.908+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 shutdown_connections 2024-09-21T07:55:24.911 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:24.909+0000 7fbbdc800640 1 -- 172.21.15.88:0/2774976728 wait complete. 2024-09-21T07:55:24.937 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2024-09-21T07:55:24.938 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 -- 172.21.15.88:0/3424672831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe914162160 msgr2=0x7fe9140b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 --2- 172.21.15.88:0/3424672831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe914162160 0x7fe9140b8c40 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7fe90c005fd0 tx=0x7fe90c02aa00 comp rx=0 tx=0).stop 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 -- 172.21.15.88:0/3424672831 shutdown_connections 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 --2- 172.21.15.88:0/3424672831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe914162160 0x7fe9140b8c40 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 --2- 172.21.15.88:0/3424672831 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe9140b9810 0x7fe9140b7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 -- 172.21.15.88:0/3424672831 >> 172.21.15.88:0/3424672831 conn(0x7fe9140d46c0 msgr2=0x7fe9140d4ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 -- 172.21.15.88:0/3424672831 shutdown_connections 2024-09-21T07:55:25.163 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.160+0000 7fe919600640 1 -- 172.21.15.88:0/3424672831 wait complete. 2024-09-21T07:55:25.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.162+0000 7fe919600640 1 Processor -- start 2024-09-21T07:55:25.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.163+0000 7fe919600640 1 -- start start 2024-09-21T07:55:25.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe919600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9140b9d80 0x7fe914156160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:25.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe919600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe914162160 0x7fe91415c6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:25.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe919600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 0x7fe914156900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:25.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe919600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe91406f4a0 con 0x7fe91414daf0 2024-09-21T07:55:25.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe919600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe91406f320 con 0x7fe9140b9d80 2024-09-21T07:55:25.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe919600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe91406f620 con 0x7fe914162160 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe913400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 0x7fe914156900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe913400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 0x7fe914156900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44912/0 (socket says 172.21.15.88:44912) 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe913400640 1 -- 172.21.15.88:0/2621807216 learned_addr learned my addr 172.21.15.88:0/2621807216 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.164+0000 7fe912a00640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9140b9d80 0x7fe914156160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe90a000640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe914162160 0x7fe91415c6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe913400640 1 -- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe914162160 msgr2=0x7fe91415c6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe913400640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe914162160 0x7fe91415c6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe913400640 1 -- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9140b9d80 msgr2=0x7fe914156160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe913400640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9140b9d80 0x7fe914156160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe913400640 1 -- 172.21.15.88:0/2621807216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe91414df00 con 0x7fe91414daf0 2024-09-21T07:55:25.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe913400640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 0x7fe914156900 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009d90 tx=0x7fe90c003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:25.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c033050 con 0x7fe91414daf0 2024-09-21T07:55:25.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe90c004450 con 0x7fe91414daf0 2024-09-21T07:55:25.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe90c004860 con 0x7fe91414daf0 2024-09-21T07:55:25.170 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.165+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe91415b650 con 0x7fe91414daf0 2024-09-21T07:55:25.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.166+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe91415bc10 con 0x7fe91414daf0 2024-09-21T07:55:25.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.167+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe91415b3c0 con 0x7fe91414daf0 2024-09-21T07:55:25.171 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.168+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fe90c036070 con 0x7fe91414daf0 2024-09-21T07:55:25.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.169+0000 7fe90b400640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe8f00795e0 0x7fe8f007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:25.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.170+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7fe90c0ba990 con 0x7fe91414daf0 2024-09-21T07:55:25.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.170+0000 7fe912a00640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe8f00795e0 0x7fe8f007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:25.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.170+0000 7fe912a00640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe8f00795e0 0x7fe8f007baf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe90400bb30 tx=0x7fe90400b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:25.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.174+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fe90c0841d0 con 0x7fe91414daf0 2024-09-21T07:55:25.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.394+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) -- 0x7fe9140b9810 con 0x7fe91414daf0 2024-09-21T07:55:25.913 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.911+0000 7fe90b400640 1 -- 172.21.15.88:0/2621807216 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v118) ==== 122+0+0 (secure 0 0 0) 0x7fe90c088c00 con 0x7fe91414daf0 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.913+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe8f00795e0 msgr2=0x7fe8f007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.913+0000 7fe919600640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe8f00795e0 0x7fe8f007baf0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fe90400bb30 tx=0x7fe90400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.913+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 msgr2=0x7fe914156900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.913+0000 7fe919600640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 0x7fe914156900 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7fe90c009d90 tx=0x7fe90c003f10 comp rx=0 tx=0).stop 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 shutdown_connections 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe8f00795e0 0x7fe8f007baf0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe914162160 0x7fe91415c6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe91414daf0 0x7fe914156900 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 --2- 172.21.15.88:0/2621807216 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe9140b9d80 0x7fe914156160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:25.916 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 >> 172.21.15.88:0/2621807216 conn(0x7fe9140d46c0 msgr2=0x7fe91415f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:25.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 shutdown_connections 2024-09-21T07:55:25.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:25.914+0000 7fe919600640 1 -- 172.21.15.88:0/2621807216 wait complete. 2024-09-21T07:55:25.941 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2024-09-21T07:55:25.941 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.157+0000 7fdf65800640 1 -- 172.21.15.88:0/1746937237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600aebc0 msgr2=0x7fdf60152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.157+0000 7fdf65800640 1 --2- 172.21.15.88:0/1746937237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600aebc0 0x7fdf60152920 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7fdf50009830 tx=0x7fdf5002a530 comp rx=0 tx=0).stop 2024-09-21T07:55:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.158+0000 7fdf65800640 1 -- 172.21.15.88:0/1746937237 shutdown_connections 2024-09-21T07:55:26.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.158+0000 7fdf65800640 1 --2- 172.21.15.88:0/1746937237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600aebc0 0x7fdf60152920 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.158+0000 7fdf65800640 1 -- 172.21.15.88:0/1746937237 >> 172.21.15.88:0/1746937237 conn(0x7fdf6006d730 msgr2=0x7fdf600b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.158+0000 7fdf65800640 1 -- 172.21.15.88:0/1746937237 shutdown_connections 2024-09-21T07:55:26.161 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.158+0000 7fdf65800640 1 -- 172.21.15.88:0/1746937237 wait complete. 2024-09-21T07:55:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.160+0000 7fdf65800640 1 Processor -- start 2024-09-21T07:55:26.162 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.160+0000 7fdf65800640 1 -- start start 2024-09-21T07:55:26.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.161+0000 7fdf65800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf600aebc0 0x7fdf6009a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.161+0000 7fdf65800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 0x7fdf60155030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.161+0000 7fdf65800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf60148650 0x7fdf601540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.161+0000 7fdf65800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf600b4360 con 0x7fdf600b2750 2024-09-21T07:55:26.164 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.161+0000 7fdf65800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fdf600b41e0 con 0x7fdf600aebc0 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.161+0000 7fdf65800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fdf600b44e0 con 0x7fdf60148650 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 0x7fdf60155030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 0x7fdf60155030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49830/0 (socket says 172.21.15.88:49830) 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 -- 172.21.15.88:0/2923295533 learned_addr learned my addr 172.21.15.88:0/2923295533 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5f400640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf60148650 0x7fdf601540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5ea00640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf600aebc0 0x7fdf6009a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 -- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf60148650 msgr2=0x7fdf601540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf60148650 0x7fdf601540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 -- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf600aebc0 msgr2=0x7fdf6009a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf600aebc0 0x7fdf6009a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 -- 172.21.15.88:0/2923295533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf60153030 con 0x7fdf600b2750 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5ea00640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf600aebc0 0x7fdf6009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:26.165 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.162+0000 7fdf5e000640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 0x7fdf60155030 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c3d0 tx=0x7fdf4c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:26.166 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.163+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c017070 con 0x7fdf600b2750 2024-09-21T07:55:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.163+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fdf4c0053c0 con 0x7fdf600b2750 2024-09-21T07:55:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.163+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf60153310 con 0x7fdf600b2750 2024-09-21T07:55:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.163+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4c003850 con 0x7fdf600b2750 2024-09-21T07:55:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.163+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf601538d0 con 0x7fdf600b2750 2024-09-21T07:55:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.164+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf600b4990 con 0x7fdf600b2750 2024-09-21T07:55:26.167 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.165+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fdf4c01a070 con 0x7fdf600b2750 2024-09-21T07:55:26.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.166+0000 7fdf56a00640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf400795e0 0x7fdf4007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.166+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7fdf4c098bb0 con 0x7fdf600b2750 2024-09-21T07:55:26.168 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.166+0000 7fdf5ea00640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf400795e0 0x7fdf4007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.169 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.167+0000 7fdf5ea00640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf400795e0 0x7fdf4007baf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdf50034000 tx=0x7fdf50004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:26.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.170+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fdf4c0621d0 con 0x7fdf600b2750 2024-09-21T07:55:26.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.391+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdf60070740 con 0x7fdf600b2750 2024-09-21T07:55:26.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.392+0000 7fdf56a00640 1 -- 172.21.15.88:0/2923295533 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5233 (secure 0 0 0) 0x7fdf4c066c00 con 0x7fdf600b2750 2024-09-21T07:55:26.394 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:26.394 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":118,"btime":"2024-09-21T07:55:25:906694+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:55:22.380663+0000","modified":"2024-09-21T07:55:25.396077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9147":{"gid":9147,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/661983519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":661983519},{"type":"v1","addr":"172.21.15.88:6835","nonce":661983519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9147,"qdb_cluster":[9147]},"id":13}]} 2024-09-21T07:55:26.395 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 118 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.394+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf400795e0 msgr2=0x7fdf4007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.394+0000 7fdf65800640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf400795e0 0x7fdf4007baf0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7fdf50034000 tx=0x7fdf50004730 comp rx=0 tx=0).stop 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.394+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 msgr2=0x7fdf60155030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.394+0000 7fdf65800640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 0x7fdf60155030 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c00c3d0 tx=0x7fdf4c00c890 comp rx=0 tx=0).stop 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 shutdown_connections 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fdf400795e0 0x7fdf4007baf0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fdf60148650 0x7fdf601540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf600b2750 0x7fdf60155030 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 --2- 172.21.15.88:0/2923295533 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fdf600aebc0 0x7fdf6009a2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 >> 172.21.15.88:0/2923295533 conn(0x7fdf6006d730 msgr2=0x7fdf60151f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:26.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 shutdown_connections 2024-09-21T07:55:26.398 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.395+0000 7fdf65800640 1 -- 172.21.15.88:0/2923295533 wait complete. 2024-09-21T07:55:26.419 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:26.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.637+0000 7f9601e00640 1 -- 172.21.15.88:0/3046078403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 msgr2=0x7f95fc1535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.637+0000 7f9601e00640 1 --2- 172.21.15.88:0/3046078403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc1535c0 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7f95fc0b3100 tx=0x7f95ec02a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.637+0000 7f9601e00640 1 -- 172.21.15.88:0/3046078403 shutdown_connections 2024-09-21T07:55:26.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.637+0000 7f9601e00640 1 --2- 172.21.15.88:0/3046078403 >> v2:172.21.15.175:3300/0 conn(0x7f95fc0b3550 0x7f95fc0b3960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.637+0000 7f9601e00640 1 --2- 172.21.15.88:0/3046078403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc1535c0 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.637+0000 7f9601e00640 1 -- 172.21.15.88:0/3046078403 >> 172.21.15.88:0/3046078403 conn(0x7f95fc06d730 msgr2=0x7f95fc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:26.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.638+0000 7f9601e00640 1 -- 172.21.15.88:0/3046078403 shutdown_connections 2024-09-21T07:55:26.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.638+0000 7f9601e00640 1 -- 172.21.15.88:0/3046078403 wait complete. 2024-09-21T07:55:26.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.640+0000 7f9601e00640 1 Processor -- start 2024-09-21T07:55:26.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.640+0000 7f9601e00640 1 -- start start 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f9601e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95fc0ad0e0 0x7f95fc13fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f9601e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95fc0b3550 0x7f95fc14c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f9601e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc14f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f9601e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0b4da0 con 0x7f95fc15d760 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f9601e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f95fc0b4c20 con 0x7f95fc0b3550 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f9601e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f95fc0b3ea0 con 0x7f95fc0ad0e0 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.641+0000 7f95fb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc14f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc14f8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49860/0 (socket says 172.21.15.88:49860) 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 -- 172.21.15.88:0/1232857588 learned_addr learned my addr 172.21.15.88:0/1232857588 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95faa00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95fc0ad0e0 0x7f95fc13fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fa000640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95fc0b3550 0x7f95fc14c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 -- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95fc0ad0e0 msgr2=0x7f95fc13fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95fc0ad0e0 0x7f95fc13fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 -- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95fc0b3550 msgr2=0x7f95fc14c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95fc0b3550 0x7f95fc14c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 -- 172.21.15.88:0/1232857588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95fc14e820 con 0x7f95fc15d760 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95fb400640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc14f8c0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f95f400caa0 tx=0x7f95f400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.642+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f400adf0 con 0x7f95fc15d760 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.643+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f95f400f540 con 0x7f95fc15d760 2024-09-21T07:55:26.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.643+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95f400f990 con 0x7f95fc15d760 2024-09-21T07:55:26.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.643+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95fc14eb00 con 0x7f95fc15d760 2024-09-21T07:55:26.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.643+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95fc14f0c0 con 0x7f95fc15d760 2024-09-21T07:55:26.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.644+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95fc0b43a0 con 0x7f95fc15d760 2024-09-21T07:55:26.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.645+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f95f400fb30 con 0x7f95fc15d760 2024-09-21T07:55:26.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.646+0000 7f95f3400640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f95dc0795b0 0x7f95dc07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:26.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.646+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f95f409a2c0 con 0x7f95fc15d760 2024-09-21T07:55:26.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.646+0000 7f95faa00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f95dc0795b0 0x7f95dc07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:26.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.647+0000 7f95faa00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f95dc0795b0 0x7f95dc07bac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f95ec033000 tx=0x7f95ec004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:26.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.650+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f95f4014030 con 0x7f95fc15d760 2024-09-21T07:55:26.842 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.840+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f95fc0afd50 con 0x7f95fc15d760 2024-09-21T07:55:26.844 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.842+0000 7f95f3400640 1 -- 172.21.15.88:0/1232857588 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v168) ==== 74+0+16731 (secure 0 0 0) 0x7f95f400fe20 con 0x7f95fc15d760 2024-09-21T07:55:26.844 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:26.845 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":168,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:23.891436+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:55:20.283046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T07:55:21.342418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f95dc0795b0 msgr2=0x7f95dc07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f95dc0795b0 0x7f95dc07bac0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f95ec033000 tx=0x7f95ec004730 comp rx=0 tx=0).stop 2024-09-21T07:55:26.846 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 msgr2=0x7f95fc14f8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc14f8c0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7f95f400caa0 tx=0x7f95f400cf60 comp rx=0 tx=0).stop 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 shutdown_connections 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f95dc0795b0 0x7f95dc07bac0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95fc15d760 0x7f95fc14f8c0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f95fc0b3550 0x7f95fc14c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 --2- 172.21.15.88:0/1232857588 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f95fc0ad0e0 0x7f95fc13fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 >> 172.21.15.88:0/1232857588 conn(0x7f95fc06d730 msgr2=0x7f95fc15ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 shutdown_connections 2024-09-21T07:55:26.847 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:26.844+0000 7f9601e00640 1 -- 172.21.15.88:0/1232857588 wait complete. 2024-09-21T07:55:26.873 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:55:26.873 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 -- 172.21.15.88:0/1849576314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc073c10 msgr2=0x7fc7cc06f0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 --2- 172.21.15.88:0/1849576314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc073c10 0x7fc7cc06f0a0 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8005fd0 tx=0x7fc7b802a5d0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 -- 172.21.15.88:0/1849576314 shutdown_connections 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 --2- 172.21.15.88:0/1849576314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc073c10 0x7fc7cc06f0a0 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 -- 172.21.15.88:0/1849576314 >> 172.21.15.88:0/1849576314 conn(0x7fc7cc06d730 msgr2=0x7fc7cc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 -- 172.21.15.88:0/1849576314 shutdown_connections 2024-09-21T07:55:27.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.092+0000 7fc7d2200640 1 -- 172.21.15.88:0/1849576314 wait complete. 2024-09-21T07:55:27.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.094+0000 7fc7d2200640 1 Processor -- start 2024-09-21T07:55:27.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.095+0000 7fc7d2200640 1 -- start start 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7d2200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 0x7fc7cc14f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7d2200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc0af630 0x7fc7cc1f5750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7d2200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7cc1579f0 0x7fc7cc0ade90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7d2200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc0b3df0 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7d2200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc7cc0b3c70 con 0x7fc7cc0af630 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7d2200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc7cc0b3f70 con 0x7fc7cc1579f0 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7cb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 0x7fc7cc14f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7cb400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 0x7fc7cc14f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49878/0 (socket says 172.21.15.88:49878) 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.096+0000 7fc7cb400640 1 -- 172.21.15.88:0/2476180298 learned_addr learned my addr 172.21.15.88:0/2476180298 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7caa00640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc0af630 0x7fc7cc1f5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cbe00640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7cc1579f0 0x7fc7cc0ade90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cb400640 1 -- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7cc1579f0 msgr2=0x7fc7cc0ade90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cb400640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7cc1579f0 0x7fc7cc0ade90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cb400640 1 -- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc0af630 msgr2=0x7fc7cc1f5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cb400640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc0af630 0x7fc7cc1f5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cb400640 1 -- 172.21.15.88:0/2476180298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7cc1f6200 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cbe00640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7cc1579f0 0x7fc7cc0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7cb400640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 0x7fc7cc14f570 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c3a0 tx=0x7fc7bc00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.097+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc017070 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.098+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7cc1f64e0 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.098+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc7bc0053c0 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.098+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7bc003850 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.101 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.098+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7cc14fda0 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.099+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7cc0b4460 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.102 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.100+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fc7bc01a070 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.101+0000 7fc7c3400640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7ac0795b0 0x7fc7ac07bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.101+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7fc7bc09b6c0 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.103 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.101+0000 7fc7caa00640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7ac0795b0 0x7fc7ac07bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.104 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.102+0000 7fc7caa00640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7ac0795b0 0x7fc7ac07bac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8033000 tx=0x7fc7b8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:27.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.106+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc7bc0621e0 con 0x7fc7cc06f5e0 2024-09-21T07:55:27.291 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.289+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc7cc150040 con 0x7fc7ac0795b0 2024-09-21T07:55:27.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.293+0000 7fc7c3400640 1 -- 172.21.15.88:0/2476180298 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+240290 (secure 0 0 0) 0x7fc7cc150040 con 0x7fc7ac0795b0 2024-09-21T07:55:27.298 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:27.299 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7ac0795b0 msgr2=0x7fc7ac07bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7ac0795b0 0x7fc7ac07bac0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fc7b8033000 tx=0x7fc7b8004730 comp rx=0 tx=0).stop 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 msgr2=0x7fc7cc14f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 0x7fc7cc14f570 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fc7bc00c3a0 tx=0x7fc7bc00c860 comp rx=0 tx=0).stop 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 shutdown_connections 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc7ac0795b0 0x7fc7ac07bac0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc7cc1579f0 0x7fc7cc0ade90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc7cc0af630 0x7fc7cc1f5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.301 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 --2- 172.21.15.88:0/2476180298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc7cc06f5e0 0x7fc7cc14f570 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 >> 172.21.15.88:0/2476180298 conn(0x7fc7cc06d730 msgr2=0x7fc7cc0733e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 shutdown_connections 2024-09-21T07:55:27.302 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.297+0000 7fc7d2200640 1 -- 172.21.15.88:0/2476180298 wait complete. 2024-09-21T07:55:27.324 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":243,"stamp":"2024-09-21T07:55:25.759064+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":7,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":266872,"kb_used_data":17340,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754707848,"statfs":{"total":773094113280,"available":772820836352,"internally_reserved":0,"allocated":17756160,"data_stored":8482307,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":10,"apply_latency_ms":10,"commit_latency_ns":10000000,"apply_latency_ns":10000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.889764"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1f","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876987+0000","last_change":"2024-09-21T07:55:20.876987+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876987+0000","last_undegraded":"2024-09-21T07:55:20.876987+0000","last_fullsized":"2024-09-21T07:55:20.876987+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1e","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.879518+0000","last_change":"2024-09-21T07:55:20.879518+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.879518+0000","last_undegraded":"2024-09-21T07:55:20.879518+0000","last_fullsized":"2024-09-21T07:55:20.879518+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1d","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.874935+0000","last_change":"2024-09-21T07:55:20.874935+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.874935+0000","last_undegraded":"2024-09-21T07:55:20.874935+0000","last_fullsized":"2024-09-21T07:55:20.874935+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1c","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.873474+0000","last_change":"2024-09-21T07:55:20.873474+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.873474+0000","last_undegraded":"2024-09-21T07:55:20.873474+0000","last_fullsized":"2024-09-21T07:55:20.873474+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1a","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872246+0000","last_change":"2024-09-21T07:55:20.872246+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872246+0000","last_undegraded":"2024-09-21T07:55:20.872246+0000","last_fullsized":"2024-09-21T07:55:20.872246+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.874077+0000","last_change":"2024-09-21T07:55:20.874077+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.874077+0000","last_undegraded":"2024-09-21T07:55:20.874077+0000","last_fullsized":"2024-09-21T07:55:20.874077+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.873406+0000","last_change":"2024-09-21T07:55:20.873406+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.873406+0000","last_undegraded":"2024-09-21T07:55:20.873406+0000","last_fullsized":"2024-09-21T07:55:20.873406+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.877697+0000","last_change":"2024-09-21T07:55:20.877697+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.877697+0000","last_undegraded":"2024-09-21T07:55:20.877697+0000","last_fullsized":"2024-09-21T07:55:20.877697+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872880+0000","last_change":"2024-09-21T07:55:20.872880+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872880+0000","last_undegraded":"2024-09-21T07:55:20.872880+0000","last_fullsized":"2024-09-21T07:55:20.872880+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.15","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872128+0000","last_change":"2024-09-21T07:55:20.872128+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872128+0000","last_undegraded":"2024-09-21T07:55:20.872128+0000","last_fullsized":"2024-09-21T07:55:20.872128+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.14","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.875573+0000","last_change":"2024-09-21T07:55:20.875573+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.875573+0000","last_undegraded":"2024-09-21T07:55:20.875573+0000","last_fullsized":"2024-09-21T07:55:20.875573+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.13","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.875677+0000","last_change":"2024-09-21T07:55:20.875677+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.875677+0000","last_undegraded":"2024-09-21T07:55:20.875677+0000","last_fullsized":"2024-09-21T07:55:20.875677+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876780+0000","last_change":"2024-09-21T07:55:20.876780+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876780+0000","last_undegraded":"2024-09-21T07:55:20.876780+0000","last_fullsized":"2024-09-21T07:55:20.876780+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872698+0000","last_change":"2024-09-21T07:55:20.872698+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872698+0000","last_undegraded":"2024-09-21T07:55:20.872698+0000","last_fullsized":"2024-09-21T07:55:20.872698+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.10","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.875480+0000","last_change":"2024-09-21T07:55:20.875480+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.875480+0000","last_undegraded":"2024-09-21T07:55:20.875480+0000","last_fullsized":"2024-09-21T07:55:20.875480+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.f","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.873283+0000","last_change":"2024-09-21T07:55:20.873283+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.873283+0000","last_undegraded":"2024-09-21T07:55:20.873283+0000","last_fullsized":"2024-09-21T07:55:20.873283+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.e","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.875799+0000","last_change":"2024-09-21T07:55:20.875799+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.875799+0000","last_undegraded":"2024-09-21T07:55:20.875799+0000","last_fullsized":"2024-09-21T07:55:20.875799+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.d","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876557+0000","last_change":"2024-09-21T07:55:20.876557+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876557+0000","last_undegraded":"2024-09-21T07:55:20.876557+0000","last_fullsized":"2024-09-21T07:55:20.876557+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.873215+0000","last_change":"2024-09-21T07:55:20.873215+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.873215+0000","last_undegraded":"2024-09-21T07:55:20.873215+0000","last_fullsized":"2024-09-21T07:55:20.873215+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.b","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876235+0000","last_change":"2024-09-21T07:55:20.876235+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876235+0000","last_undegraded":"2024-09-21T07:55:20.876235+0000","last_fullsized":"2024-09-21T07:55:20.876235+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872115+0000","last_change":"2024-09-21T07:55:20.872115+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872115+0000","last_undegraded":"2024-09-21T07:55:20.872115+0000","last_fullsized":"2024-09-21T07:55:20.872115+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876737+0000","last_change":"2024-09-21T07:55:20.876737+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876737+0000","last_undegraded":"2024-09-21T07:55:20.876737+0000","last_fullsized":"2024-09-21T07:55:20.876737+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872128+0000","last_change":"2024-09-21T07:55:20.872128+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872128+0000","last_undegraded":"2024-09-21T07:55:20.872128+0000","last_fullsized":"2024-09-21T07:55:20.872128+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.7","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.874583+0000","last_change":"2024-09-21T07:55:20.874583+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.874583+0000","last_undegraded":"2024-09-21T07:55:20.874583+0000","last_fullsized":"2024-09-21T07:55:20.874583+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.6","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.878186+0000","last_change":"2024-09-21T07:55:20.878186+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.878186+0000","last_undegraded":"2024-09-21T07:55:20.878186+0000","last_fullsized":"2024-09-21T07:55:20.878186+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872594+0000","last_change":"2024-09-21T07:55:20.872594+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872594+0000","last_undegraded":"2024-09-21T07:55:20.872594+0000","last_fullsized":"2024-09-21T07:55:20.872594+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872545+0000","last_change":"2024-09-21T07:55:20.872545+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872545+0000","last_undegraded":"2024-09-21T07:55:20.872545+0000","last_fullsized":"2024-09-21T07:55:20.872545+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872815+0000","last_change":"2024-09-21T07:55:20.872815+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872815+0000","last_undegraded":"2024-09-21T07:55:20.872815+0000","last_fullsized":"2024-09-21T07:55:20.872815+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876655+0000","last_change":"2024-09-21T07:55:20.876655+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876655+0000","last_undegraded":"2024-09-21T07:55:20.876655+0000","last_fullsized":"2024-09-21T07:55:20.876655+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.875673+0000","last_change":"2024-09-21T07:55:20.875673+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.875673+0000","last_undegraded":"2024-09-21T07:55:20.875673+0000","last_fullsized":"2024-09-21T07:55:20.875673+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.872771+0000","last_change":"2024-09-21T07:55:20.872771+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.872771+0000","last_undegraded":"2024-09-21T07:55:20.872771+0000","last_fullsized":"2024-09-21T07:55:20.872771+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2024-09-21T07:55:21.869311+0000","last_change":"2024-09-21T07:55:21.869311+0000","last_active":"2024-09-21T07:55:21.869311+0000","last_peered":"2024-09-21T07:55:21.869311+0000","last_clean":"2024-09-21T07:55:21.869311+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:21.869311+0000","last_undegraded":"2024-09-21T07:55:21.869311+0000","last_fullsized":"2024-09-21T07:55:21.869311+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"26.1b","version":"0'0","reported_seq":1,"reported_epoch":165,"state":"creating+peering","last_fresh":"2024-09-21T07:55:20.876752+0000","last_change":"2024-09-21T07:55:20.876752+0000","last_active":"2024-09-21T07:55:20.860949+0000","last_peered":"2024-09-21T07:55:20.860949+0000","last_clean":"2024-09-21T07:55:20.860949+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2024-09-21T07:55:20.876752+0000","last_undegraded":"2024-09-21T07:55:20.876752+0000","last_fullsized":"2024-09-21T07:55:20.876752+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":410,"reported_epoch":165,"state":"active+clean","last_fresh":"2024-09-21T07:55:20.871099+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:55:20.871099+0000","last_peered":"2024-09-21T07:55:20.871099+0000","last_clean":"2024-09-21T07:55:20.871099+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:55:20.871099+0000","last_undegraded":"2024-09-21T07:55:20.871099+0000","last_fullsized":"2024-09-21T07:55:20.871099+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2072576,"data_stored":913427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509556,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33220,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338620,"statfs":{"total":96636764160,"available":96602746880,"internally_reserved":0,"allocated":2072576,"data_stored":913427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509557,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33796,"kb_used_data":2604,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338044,"statfs":{"total":96636764160,"available":96602157056,"internally_reserved":0,"allocated":2666496,"data_stored":1503795,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509557,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":913427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2072576,"data_stored":913427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476855,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33216,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338624,"statfs":{"total":96636764160,"available":96602750976,"internally_reserved":0,"allocated":2072576,"data_stored":913427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476852,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33784,"kb_used_data":2592,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338056,"statfs":{"total":96636764160,"available":96602169344,"internally_reserved":0,"allocated":2654208,"data_stored":1497950,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33212,"kb_used_data":2024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338628,"statfs":{"total":96636764160,"available":96602755072,"internally_reserved":0,"allocated":2072576,"data_stored":913427,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:27.329 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 -- 172.21.15.88:0/3811424957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08e8003a50 msgr2=0x7f08e8003ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 --2- 172.21.15.88:0/3811424957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08e8003a50 0x7f08e8003ec0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f08f400c630 tx=0x7f08e8005830 comp rx=0 tx=0).stop 2024-09-21T07:55:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 -- 172.21.15.88:0/3811424957 shutdown_connections 2024-09-21T07:55:27.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 --2- 172.21.15.88:0/3811424957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f08e8003a50 0x7f08e8003ec0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 -- 172.21.15.88:0/3811424957 >> 172.21.15.88:0/3811424957 conn(0x7f09000b1e40 msgr2=0x7f09000b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 -- 172.21.15.88:0/3811424957 shutdown_connections 2024-09-21T07:55:27.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.584+0000 7f0906e00640 1 -- 172.21.15.88:0/3811424957 wait complete. 2024-09-21T07:55:27.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.586+0000 7f0906e00640 1 Processor -- start 2024-09-21T07:55:27.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.586+0000 7f0906e00640 1 -- start start 2024-09-21T07:55:27.591 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08e8003a50 0x7f090009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 0x7f09001f38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09001f3e00 0x7f0900157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f09000b3340 con 0x7f0900148c20 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f09000b31c0 con 0x7f09001f3e00 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f090015d670 con 0x7f08e8003a50 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 0x7f09001f38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 0x7f09001f38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49890/0 (socket says 172.21.15.88:49890) 2024-09-21T07:55:27.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 -- 172.21.15.88:0/2935477894 learned_addr learned my addr 172.21.15.88:0/2935477894 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905a00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08e8003a50 0x7f090009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906400640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09001f3e00 0x7f0900157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 -- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08e8003a50 msgr2=0x7f090009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08e8003a50 0x7f090009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 -- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09001f3e00 msgr2=0x7f0900157370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09001f3e00 0x7f0900157370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0905000640 1 -- 172.21.15.88:0/2935477894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09001f30d0 con 0x7f0900148c20 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.588+0000 7f0906400640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09001f3e00 0x7f0900157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.589+0000 7f0905000640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 0x7f09001f38c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0117d0 tx=0x7f08ec011c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.589+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec01f070 con 0x7f0900148c20 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.589+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f08ec00fde0 con 0x7f0900148c20 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.589+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0900159590 con 0x7f0900148c20 2024-09-21T07:55:27.593 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.589+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08ec0126b0 con 0x7f0900148c20 2024-09-21T07:55:27.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.589+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0900159b50 con 0x7f0900148c20 2024-09-21T07:55:27.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.590+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0900159300 con 0x7f0900148c20 2024-09-21T07:55:27.594 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.591+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f08ec022070 con 0x7f0900148c20 2024-09-21T07:55:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.592+0000 7f08fd600640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08e00795b0 0x7f08e007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.592+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f08ec09a1a0 con 0x7f0900148c20 2024-09-21T07:55:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.593+0000 7f0905a00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08e00795b0 0x7f08e007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:27.595 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.593+0000 7f0905a00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08e00795b0 0x7f08e007bac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f08f4033000 tx=0x7f08f402d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:27.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.597+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f08ec0637c0 con 0x7f0900148c20 2024-09-21T07:55:27.731 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:27.735 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:27.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.782+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0900070530 con 0x7f08e00795b0 2024-09-21T07:55:27.788 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.786+0000 7f08fd600640 1 -- 172.21.15.88:0/2935477894 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+252482 (secure 0 0 0) 0x7f08f4080a60 con 0x7f08e00795b0 2024-09-21T07:55:27.792 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:27.793 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08e00795b0 msgr2=0x7f08e007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08e00795b0 0x7f08e007bac0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f08f4033000 tx=0x7f08f402d040 comp rx=0 tx=0).stop 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 msgr2=0x7f09001f38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 0x7f09001f38c0 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f08ec0117d0 tx=0x7f08ec011c90 comp rx=0 tx=0).stop 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 shutdown_connections 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f08e00795b0 0x7f08e007bac0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f09001f3e00 0x7f0900157370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0900148c20 0x7f09001f38c0 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 --2- 172.21.15.88:0/2935477894 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f08e8003a50 0x7f090009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:27.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.790+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 >> 172.21.15.88:0/2935477894 conn(0x7f09000b1e40 msgr2=0x7f090006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:27.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.791+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 shutdown_connections 2024-09-21T07:55:27.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:27.791+0000 7f0906e00640 1 -- 172.21.15.88:0/2935477894 wait complete. 2024-09-21T07:55:27.815 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":244,"stamp":"2024-09-21T07:55:27.759449+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":158,"num_write_kb":1642,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":118,"ondisk_log_size":118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":178,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":267244,"kb_used_data":17704,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754707476,"statfs":{"total":773094113280,"available":772820455424,"internally_reserved":0,"allocated":18128896,"data_stored":8648444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001182"},"pg_stats":[{"pgid":"27.3f","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903198+0000","last_change":"2024-09-21T07:55:22.909690+0000","last_active":"2024-09-21T07:55:23.903198+0000","last_peered":"2024-09-21T07:55:23.903198+0000","last_clean":"2024-09-21T07:55:23.903198+0000","last_became_active":"2024-09-21T07:55:22.909162+0000","last_became_peered":"2024-09-21T07:55:22.909162+0000","last_unstale":"2024-09-21T07:55:23.903198+0000","last_undegraded":"2024-09-21T07:55:23.903198+0000","last_fullsized":"2024-09-21T07:55:23.903198+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:53:44.354937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041771+0000","last_change":"2024-09-21T07:55:22.914335+0000","last_active":"2024-09-21T07:55:24.041771+0000","last_peered":"2024-09-21T07:55:24.041771+0000","last_clean":"2024-09-21T07:55:24.041771+0000","last_became_active":"2024-09-21T07:55:22.914065+0000","last_became_peered":"2024-09-21T07:55:22.914065+0000","last_unstale":"2024-09-21T07:55:24.041771+0000","last_undegraded":"2024-09-21T07:55:24.041771+0000","last_fullsized":"2024-09-21T07:55:24.041771+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:11:53.086976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042059+0000","last_change":"2024-09-21T07:55:22.915296+0000","last_active":"2024-09-21T07:55:24.042059+0000","last_peered":"2024-09-21T07:55:24.042059+0000","last_clean":"2024-09-21T07:55:24.042059+0000","last_became_active":"2024-09-21T07:55:22.914947+0000","last_became_peered":"2024-09-21T07:55:22.914947+0000","last_unstale":"2024-09-21T07:55:24.042059+0000","last_undegraded":"2024-09-21T07:55:24.042059+0000","last_fullsized":"2024-09-21T07:55:24.042059+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:25:01.579081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901759+0000","last_change":"2024-09-21T07:55:22.903205+0000","last_active":"2024-09-21T07:55:23.901759+0000","last_peered":"2024-09-21T07:55:23.901759+0000","last_clean":"2024-09-21T07:55:23.901759+0000","last_became_active":"2024-09-21T07:55:22.899579+0000","last_became_peered":"2024-09-21T07:55:22.899579+0000","last_unstale":"2024-09-21T07:55:23.901759+0000","last_undegraded":"2024-09-21T07:55:23.901759+0000","last_fullsized":"2024-09-21T07:55:23.901759+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:49:48.609773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041727+0000","last_change":"2024-09-21T07:55:22.920106+0000","last_active":"2024-09-21T07:55:24.041727+0000","last_peered":"2024-09-21T07:55:24.041727+0000","last_clean":"2024-09-21T07:55:24.041727+0000","last_became_active":"2024-09-21T07:55:22.919378+0000","last_became_peered":"2024-09-21T07:55:22.919378+0000","last_unstale":"2024-09-21T07:55:24.041727+0000","last_undegraded":"2024-09-21T07:55:24.041727+0000","last_fullsized":"2024-09-21T07:55:24.041727+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:58:04.871780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903548+0000","last_change":"2024-09-21T07:55:22.907795+0000","last_active":"2024-09-21T07:55:23.903548+0000","last_peered":"2024-09-21T07:55:23.903548+0000","last_clean":"2024-09-21T07:55:23.903548+0000","last_became_active":"2024-09-21T07:55:22.906669+0000","last_became_peered":"2024-09-21T07:55:22.906669+0000","last_unstale":"2024-09-21T07:55:23.903548+0000","last_undegraded":"2024-09-21T07:55:23.903548+0000","last_fullsized":"2024-09-21T07:55:23.903548+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:12:27.448904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901891+0000","last_change":"2024-09-21T07:55:22.900964+0000","last_active":"2024-09-21T07:55:23.901891+0000","last_peered":"2024-09-21T07:55:23.901891+0000","last_clean":"2024-09-21T07:55:23.901891+0000","last_became_active":"2024-09-21T07:55:22.900316+0000","last_became_peered":"2024-09-21T07:55:22.900316+0000","last_unstale":"2024-09-21T07:55:23.901891+0000","last_undegraded":"2024-09-21T07:55:23.901891+0000","last_fullsized":"2024-09-21T07:55:23.901891+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:34:57.942693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901852+0000","last_change":"2024-09-21T07:55:22.902113+0000","last_active":"2024-09-21T07:55:23.901852+0000","last_peered":"2024-09-21T07:55:23.901852+0000","last_clean":"2024-09-21T07:55:23.901852+0000","last_became_active":"2024-09-21T07:55:22.899250+0000","last_became_peered":"2024-09-21T07:55:22.899250+0000","last_unstale":"2024-09-21T07:55:23.901852+0000","last_undegraded":"2024-09-21T07:55:23.901852+0000","last_fullsized":"2024-09-21T07:55:23.901852+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:53:15.152182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042082+0000","last_change":"2024-09-21T07:55:22.906302+0000","last_active":"2024-09-21T07:55:24.042082+0000","last_peered":"2024-09-21T07:55:24.042082+0000","last_clean":"2024-09-21T07:55:24.042082+0000","last_became_active":"2024-09-21T07:55:22.905997+0000","last_became_peered":"2024-09-21T07:55:22.905997+0000","last_unstale":"2024-09-21T07:55:24.042082+0000","last_undegraded":"2024-09-21T07:55:24.042082+0000","last_fullsized":"2024-09-21T07:55:24.042082+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:01.054827+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041959+0000","last_change":"2024-09-21T07:55:22.917373+0000","last_active":"2024-09-21T07:55:24.041959+0000","last_peered":"2024-09-21T07:55:24.041959+0000","last_clean":"2024-09-21T07:55:24.041959+0000","last_became_active":"2024-09-21T07:55:22.917052+0000","last_became_peered":"2024-09-21T07:55:22.917052+0000","last_unstale":"2024-09-21T07:55:24.041959+0000","last_undegraded":"2024-09-21T07:55:24.041959+0000","last_fullsized":"2024-09-21T07:55:24.041959+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:07:43.712681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.904196+0000","last_change":"2024-09-21T07:55:22.891438+0000","last_active":"2024-09-21T07:55:23.904196+0000","last_peered":"2024-09-21T07:55:23.904196+0000","last_clean":"2024-09-21T07:55:23.904196+0000","last_became_active":"2024-09-21T07:55:22.891105+0000","last_became_peered":"2024-09-21T07:55:22.891105+0000","last_unstale":"2024-09-21T07:55:23.904196+0000","last_undegraded":"2024-09-21T07:55:23.904196+0000","last_fullsized":"2024-09-21T07:55:23.904196+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:17:14.852685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042280+0000","last_change":"2024-09-21T07:55:22.914692+0000","last_active":"2024-09-21T07:55:24.042280+0000","last_peered":"2024-09-21T07:55:24.042280+0000","last_clean":"2024-09-21T07:55:24.042280+0000","last_became_active":"2024-09-21T07:55:22.912164+0000","last_became_peered":"2024-09-21T07:55:22.912164+0000","last_unstale":"2024-09-21T07:55:24.042280+0000","last_undegraded":"2024-09-21T07:55:24.042280+0000","last_fullsized":"2024-09-21T07:55:24.042280+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:23.409160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901810+0000","last_change":"2024-09-21T07:55:22.910987+0000","last_active":"2024-09-21T07:55:23.901810+0000","last_peered":"2024-09-21T07:55:23.901810+0000","last_clean":"2024-09-21T07:55:23.901810+0000","last_became_active":"2024-09-21T07:55:22.908146+0000","last_became_peered":"2024-09-21T07:55:22.908146+0000","last_unstale":"2024-09-21T07:55:23.901810+0000","last_undegraded":"2024-09-21T07:55:23.901810+0000","last_fullsized":"2024-09-21T07:55:23.901810+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:24:10.153237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040764+0000","last_change":"2024-09-21T07:55:22.904614+0000","last_active":"2024-09-21T07:55:24.040764+0000","last_peered":"2024-09-21T07:55:24.040764+0000","last_clean":"2024-09-21T07:55:24.040764+0000","last_became_active":"2024-09-21T07:55:22.904228+0000","last_became_peered":"2024-09-21T07:55:22.904228+0000","last_unstale":"2024-09-21T07:55:24.040764+0000","last_undegraded":"2024-09-21T07:55:24.040764+0000","last_fullsized":"2024-09-21T07:55:24.040764+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:51:56.874532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902171+0000","last_change":"2024-09-21T07:55:22.892173+0000","last_active":"2024-09-21T07:55:23.902171+0000","last_peered":"2024-09-21T07:55:23.902171+0000","last_clean":"2024-09-21T07:55:23.902171+0000","last_became_active":"2024-09-21T07:55:22.891848+0000","last_became_peered":"2024-09-21T07:55:22.891848+0000","last_unstale":"2024-09-21T07:55:23.902171+0000","last_undegraded":"2024-09-21T07:55:23.902171+0000","last_fullsized":"2024-09-21T07:55:23.902171+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:59:22.596526+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902030+0000","last_change":"2024-09-21T07:55:22.904504+0000","last_active":"2024-09-21T07:55:23.902030+0000","last_peered":"2024-09-21T07:55:23.902030+0000","last_clean":"2024-09-21T07:55:23.902030+0000","last_became_active":"2024-09-21T07:55:22.904131+0000","last_became_peered":"2024-09-21T07:55:22.904131+0000","last_unstale":"2024-09-21T07:55:23.902030+0000","last_undegraded":"2024-09-21T07:55:23.902030+0000","last_fullsized":"2024-09-21T07:55:23.902030+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:07.648024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903532+0000","last_change":"2024-09-21T07:55:22.907280+0000","last_active":"2024-09-21T07:55:23.903532+0000","last_peered":"2024-09-21T07:55:23.903532+0000","last_clean":"2024-09-21T07:55:23.903532+0000","last_became_active":"2024-09-21T07:55:22.905692+0000","last_became_peered":"2024-09-21T07:55:22.905692+0000","last_unstale":"2024-09-21T07:55:23.903532+0000","last_undegraded":"2024-09-21T07:55:23.903532+0000","last_fullsized":"2024-09-21T07:55:23.903532+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:08:12.079222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042240+0000","last_change":"2024-09-21T07:55:22.917210+0000","last_active":"2024-09-21T07:55:24.042240+0000","last_peered":"2024-09-21T07:55:24.042240+0000","last_clean":"2024-09-21T07:55:24.042240+0000","last_became_active":"2024-09-21T07:55:22.916886+0000","last_became_peered":"2024-09-21T07:55:22.916886+0000","last_unstale":"2024-09-21T07:55:24.042240+0000","last_undegraded":"2024-09-21T07:55:24.042240+0000","last_fullsized":"2024-09-21T07:55:24.042240+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:18:11.627580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042022+0000","last_change":"2024-09-21T07:55:22.911018+0000","last_active":"2024-09-21T07:55:24.042022+0000","last_peered":"2024-09-21T07:55:24.042022+0000","last_clean":"2024-09-21T07:55:24.042022+0000","last_became_active":"2024-09-21T07:55:22.909477+0000","last_became_peered":"2024-09-21T07:55:22.909477+0000","last_unstale":"2024-09-21T07:55:24.042022+0000","last_undegraded":"2024-09-21T07:55:24.042022+0000","last_fullsized":"2024-09-21T07:55:24.042022+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:24:55.402242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041978+0000","last_change":"2024-09-21T07:55:22.905853+0000","last_active":"2024-09-21T07:55:24.041978+0000","last_peered":"2024-09-21T07:55:24.041978+0000","last_clean":"2024-09-21T07:55:24.041978+0000","last_became_active":"2024-09-21T07:55:22.905549+0000","last_became_peered":"2024-09-21T07:55:22.905549+0000","last_unstale":"2024-09-21T07:55:24.041978+0000","last_undegraded":"2024-09-21T07:55:24.041978+0000","last_fullsized":"2024-09-21T07:55:24.041978+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:38:17.505882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041973+0000","last_change":"2024-09-21T07:55:22.908577+0000","last_active":"2024-09-21T07:55:24.041973+0000","last_peered":"2024-09-21T07:55:24.041973+0000","last_clean":"2024-09-21T07:55:24.041973+0000","last_became_active":"2024-09-21T07:55:22.908242+0000","last_became_peered":"2024-09-21T07:55:22.908242+0000","last_unstale":"2024-09-21T07:55:24.041973+0000","last_undegraded":"2024-09-21T07:55:24.041973+0000","last_fullsized":"2024-09-21T07:55:24.041973+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:30.324173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040745+0000","last_change":"2024-09-21T07:55:22.917028+0000","last_active":"2024-09-21T07:55:24.040745+0000","last_peered":"2024-09-21T07:55:24.040745+0000","last_clean":"2024-09-21T07:55:24.040745+0000","last_became_active":"2024-09-21T07:55:22.916749+0000","last_became_peered":"2024-09-21T07:55:22.916749+0000","last_unstale":"2024-09-21T07:55:24.040745+0000","last_undegraded":"2024-09-21T07:55:24.040745+0000","last_fullsized":"2024-09-21T07:55:24.040745+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:22:07.807317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.900788+0000","last_change":"2024-09-21T07:55:22.906438+0000","last_active":"2024-09-21T07:55:23.900788+0000","last_peered":"2024-09-21T07:55:23.900788+0000","last_clean":"2024-09-21T07:55:23.900788+0000","last_became_active":"2024-09-21T07:55:22.905705+0000","last_became_peered":"2024-09-21T07:55:22.905705+0000","last_unstale":"2024-09-21T07:55:23.900788+0000","last_undegraded":"2024-09-21T07:55:23.900788+0000","last_fullsized":"2024-09-21T07:55:23.900788+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:13:47.188829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040632+0000","last_change":"2024-09-21T07:55:22.891001+0000","last_active":"2024-09-21T07:55:24.040632+0000","last_peered":"2024-09-21T07:55:24.040632+0000","last_clean":"2024-09-21T07:55:24.040632+0000","last_became_active":"2024-09-21T07:55:22.890682+0000","last_became_peered":"2024-09-21T07:55:22.890682+0000","last_unstale":"2024-09-21T07:55:24.040632+0000","last_undegraded":"2024-09-21T07:55:24.040632+0000","last_fullsized":"2024-09-21T07:55:24.040632+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:13:54.100438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902436+0000","last_change":"2024-09-21T07:55:22.899598+0000","last_active":"2024-09-21T07:55:23.902436+0000","last_peered":"2024-09-21T07:55:23.902436+0000","last_clean":"2024-09-21T07:55:23.902436+0000","last_became_active":"2024-09-21T07:55:22.899186+0000","last_became_peered":"2024-09-21T07:55:22.899186+0000","last_unstale":"2024-09-21T07:55:23.902436+0000","last_undegraded":"2024-09-21T07:55:23.902436+0000","last_fullsized":"2024-09-21T07:55:23.902436+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:25.690638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903943+0000","last_change":"2024-09-21T07:55:22.906598+0000","last_active":"2024-09-21T07:55:23.903943+0000","last_peered":"2024-09-21T07:55:23.903943+0000","last_clean":"2024-09-21T07:55:23.903943+0000","last_became_active":"2024-09-21T07:55:22.905988+0000","last_became_peered":"2024-09-21T07:55:22.905988+0000","last_unstale":"2024-09-21T07:55:23.903943+0000","last_undegraded":"2024-09-21T07:55:23.903943+0000","last_fullsized":"2024-09-21T07:55:23.903943+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:58:35.555348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042301+0000","last_change":"2024-09-21T07:55:22.911265+0000","last_active":"2024-09-21T07:55:24.042301+0000","last_peered":"2024-09-21T07:55:24.042301+0000","last_clean":"2024-09-21T07:55:24.042301+0000","last_became_active":"2024-09-21T07:55:22.910949+0000","last_became_peered":"2024-09-21T07:55:22.910949+0000","last_unstale":"2024-09-21T07:55:24.042301+0000","last_undegraded":"2024-09-21T07:55:24.042301+0000","last_fullsized":"2024-09-21T07:55:24.042301+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:24:08.296889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903881+0000","last_change":"2024-09-21T07:55:22.907014+0000","last_active":"2024-09-21T07:55:23.903881+0000","last_peered":"2024-09-21T07:55:23.903881+0000","last_clean":"2024-09-21T07:55:23.903881+0000","last_became_active":"2024-09-21T07:55:22.906226+0000","last_became_peered":"2024-09-21T07:55:22.906226+0000","last_unstale":"2024-09-21T07:55:23.903881+0000","last_undegraded":"2024-09-21T07:55:23.903881+0000","last_fullsized":"2024-09-21T07:55:23.903881+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:31:58.857406+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041066+0000","last_change":"2024-09-21T07:55:22.911206+0000","last_active":"2024-09-21T07:55:24.041066+0000","last_peered":"2024-09-21T07:55:24.041066+0000","last_clean":"2024-09-21T07:55:24.041066+0000","last_became_active":"2024-09-21T07:55:22.910914+0000","last_became_peered":"2024-09-21T07:55:22.910914+0000","last_unstale":"2024-09-21T07:55:24.041066+0000","last_undegraded":"2024-09-21T07:55:24.041066+0000","last_fullsized":"2024-09-21T07:55:24.041066+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:22:14.215192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901361+0000","last_change":"2024-09-21T07:55:22.909494+0000","last_active":"2024-09-21T07:55:23.901361+0000","last_peered":"2024-09-21T07:55:23.901361+0000","last_clean":"2024-09-21T07:55:23.901361+0000","last_became_active":"2024-09-21T07:55:22.908894+0000","last_became_peered":"2024-09-21T07:55:22.908894+0000","last_unstale":"2024-09-21T07:55:23.901361+0000","last_undegraded":"2024-09-21T07:55:23.901361+0000","last_fullsized":"2024-09-21T07:55:23.901361+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:55:24.566586+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041195+0000","last_change":"2024-09-21T07:55:22.913288+0000","last_active":"2024-09-21T07:55:24.041195+0000","last_peered":"2024-09-21T07:55:24.041195+0000","last_clean":"2024-09-21T07:55:24.041195+0000","last_became_active":"2024-09-21T07:55:22.911489+0000","last_became_peered":"2024-09-21T07:55:22.911489+0000","last_unstale":"2024-09-21T07:55:24.041195+0000","last_undegraded":"2024-09-21T07:55:24.041195+0000","last_fullsized":"2024-09-21T07:55:24.041195+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:26:14.066682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040705+0000","last_change":"2024-09-21T07:55:22.898732+0000","last_active":"2024-09-21T07:55:24.040705+0000","last_peered":"2024-09-21T07:55:24.040705+0000","last_clean":"2024-09-21T07:55:24.040705+0000","last_became_active":"2024-09-21T07:55:22.898401+0000","last_became_peered":"2024-09-21T07:55:22.898401+0000","last_unstale":"2024-09-21T07:55:24.040705+0000","last_undegraded":"2024-09-21T07:55:24.040705+0000","last_fullsized":"2024-09-21T07:55:24.040705+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:17:31.003310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042900+0000","last_change":"2024-09-21T07:55:22.920412+0000","last_active":"2024-09-21T07:55:24.042900+0000","last_peered":"2024-09-21T07:55:24.042900+0000","last_clean":"2024-09-21T07:55:24.042900+0000","last_became_active":"2024-09-21T07:55:22.920092+0000","last_became_peered":"2024-09-21T07:55:22.920092+0000","last_unstale":"2024-09-21T07:55:24.042900+0000","last_undegraded":"2024-09-21T07:55:24.042900+0000","last_fullsized":"2024-09-21T07:55:24.042900+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:35.937801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1f","version":"167'4","reported_seq":19,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042264+0000","last_change":"2024-09-21T07:55:21.905003+0000","last_active":"2024-09-21T07:55:24.042264+0000","last_peered":"2024-09-21T07:55:24.042264+0000","last_clean":"2024-09-21T07:55:24.042264+0000","last_became_active":"2024-09-21T07:55:21.904710+0000","last_became_peered":"2024-09-21T07:55:21.904710+0000","last_unstale":"2024-09-21T07:55:24.042264+0000","last_undegraded":"2024-09-21T07:55:24.042264+0000","last_fullsized":"2024-09-21T07:55:24.042264+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:10.644990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902258+0000","last_change":"2024-09-21T07:55:22.904771+0000","last_active":"2024-09-21T07:55:23.902258+0000","last_peered":"2024-09-21T07:55:23.902258+0000","last_clean":"2024-09-21T07:55:23.902258+0000","last_became_active":"2024-09-21T07:55:22.903749+0000","last_became_peered":"2024-09-21T07:55:22.903749+0000","last_unstale":"2024-09-21T07:55:23.902258+0000","last_undegraded":"2024-09-21T07:55:23.902258+0000","last_fullsized":"2024-09-21T07:55:23.902258+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:53.725926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1e","version":"167'1","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041137+0000","last_change":"2024-09-21T07:55:21.895586+0000","last_active":"2024-09-21T07:55:24.041137+0000","last_peered":"2024-09-21T07:55:24.041137+0000","last_clean":"2024-09-21T07:55:24.041137+0000","last_became_active":"2024-09-21T07:55:21.895285+0000","last_became_peered":"2024-09-21T07:55:21.895285+0000","last_unstale":"2024-09-21T07:55:24.041137+0000","last_undegraded":"2024-09-21T07:55:24.041137+0000","last_fullsized":"2024-09-21T07:55:24.041137+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:11:59.462907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041891+0000","last_change":"2024-09-21T07:55:22.914547+0000","last_active":"2024-09-21T07:55:24.041891+0000","last_peered":"2024-09-21T07:55:24.041891+0000","last_clean":"2024-09-21T07:55:24.041891+0000","last_became_active":"2024-09-21T07:55:22.911408+0000","last_became_peered":"2024-09-21T07:55:22.911408+0000","last_unstale":"2024-09-21T07:55:24.041891+0000","last_undegraded":"2024-09-21T07:55:24.041891+0000","last_fullsized":"2024-09-21T07:55:24.041891+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:43:38.600055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1d","version":"167'2","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903633+0000","last_change":"2024-09-21T07:55:21.904575+0000","last_active":"2024-09-21T07:55:23.903633+0000","last_peered":"2024-09-21T07:55:23.903633+0000","last_clean":"2024-09-21T07:55:23.903633+0000","last_became_active":"2024-09-21T07:55:21.904157+0000","last_became_peered":"2024-09-21T07:55:21.904157+0000","last_unstale":"2024-09-21T07:55:23.903633+0000","last_undegraded":"2024-09-21T07:55:23.903633+0000","last_fullsized":"2024-09-21T07:55:23.903633+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:17:53.067219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042233+0000","last_change":"2024-09-21T07:55:22.915507+0000","last_active":"2024-09-21T07:55:24.042233+0000","last_peered":"2024-09-21T07:55:24.042233+0000","last_clean":"2024-09-21T07:55:24.042233+0000","last_became_active":"2024-09-21T07:55:22.914993+0000","last_became_peered":"2024-09-21T07:55:22.914993+0000","last_unstale":"2024-09-21T07:55:24.042233+0000","last_undegraded":"2024-09-21T07:55:24.042233+0000","last_fullsized":"2024-09-21T07:55:24.042233+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:25:28.138449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1c","version":"167'2","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901819+0000","last_change":"2024-09-21T07:55:21.895442+0000","last_active":"2024-09-21T07:55:23.901819+0000","last_peered":"2024-09-21T07:55:23.901819+0000","last_clean":"2024-09-21T07:55:23.901819+0000","last_became_active":"2024-09-21T07:55:21.894992+0000","last_became_peered":"2024-09-21T07:55:21.894992+0000","last_unstale":"2024-09-21T07:55:23.901819+0000","last_undegraded":"2024-09-21T07:55:23.901819+0000","last_fullsized":"2024-09-21T07:55:23.901819+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:03:39.431354+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901072+0000","last_change":"2024-09-21T07:55:22.912955+0000","last_active":"2024-09-21T07:55:23.901072+0000","last_peered":"2024-09-21T07:55:23.901072+0000","last_clean":"2024-09-21T07:55:23.901072+0000","last_became_active":"2024-09-21T07:55:22.912657+0000","last_became_peered":"2024-09-21T07:55:22.912657+0000","last_unstale":"2024-09-21T07:55:23.901072+0000","last_undegraded":"2024-09-21T07:55:23.901072+0000","last_fullsized":"2024-09-21T07:55:23.901072+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:37:08.982995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.1a","version":"167'1","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903200+0000","last_change":"2024-09-21T07:55:21.895484+0000","last_active":"2024-09-21T07:55:23.903200+0000","last_peered":"2024-09-21T07:55:23.903200+0000","last_clean":"2024-09-21T07:55:23.903200+0000","last_became_active":"2024-09-21T07:55:21.895207+0000","last_became_peered":"2024-09-21T07:55:21.895207+0000","last_unstale":"2024-09-21T07:55:23.903200+0000","last_undegraded":"2024-09-21T07:55:23.903200+0000","last_fullsized":"2024-09-21T07:55:23.903200+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:50:13.181445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902878+0000","last_change":"2024-09-21T07:55:22.902070+0000","last_active":"2024-09-21T07:55:23.902878+0000","last_peered":"2024-09-21T07:55:23.902878+0000","last_clean":"2024-09-21T07:55:23.902878+0000","last_became_active":"2024-09-21T07:55:22.900339+0000","last_became_peered":"2024-09-21T07:55:22.900339+0000","last_unstale":"2024-09-21T07:55:23.902878+0000","last_undegraded":"2024-09-21T07:55:23.902878+0000","last_fullsized":"2024-09-21T07:55:23.902878+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:14:15.144185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902551+0000","last_change":"2024-09-21T07:55:21.904143+0000","last_active":"2024-09-21T07:55:23.902551+0000","last_peered":"2024-09-21T07:55:23.902551+0000","last_clean":"2024-09-21T07:55:23.902551+0000","last_became_active":"2024-09-21T07:55:21.903753+0000","last_became_peered":"2024-09-21T07:55:21.903753+0000","last_unstale":"2024-09-21T07:55:23.902551+0000","last_undegraded":"2024-09-21T07:55:23.902551+0000","last_fullsized":"2024-09-21T07:55:23.902551+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:02:12.856019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040864+0000","last_change":"2024-09-21T07:55:22.900180+0000","last_active":"2024-09-21T07:55:24.040864+0000","last_peered":"2024-09-21T07:55:24.040864+0000","last_clean":"2024-09-21T07:55:24.040864+0000","last_became_active":"2024-09-21T07:55:22.899927+0000","last_became_peered":"2024-09-21T07:55:22.899927+0000","last_unstale":"2024-09-21T07:55:24.040864+0000","last_undegraded":"2024-09-21T07:55:24.040864+0000","last_fullsized":"2024-09-21T07:55:24.040864+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:06:54.137662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.900918+0000","last_change":"2024-09-21T07:55:21.905109+0000","last_active":"2024-09-21T07:55:23.900918+0000","last_peered":"2024-09-21T07:55:23.900918+0000","last_clean":"2024-09-21T07:55:23.900918+0000","last_became_active":"2024-09-21T07:55:21.904789+0000","last_became_peered":"2024-09-21T07:55:21.904789+0000","last_unstale":"2024-09-21T07:55:23.900918+0000","last_undegraded":"2024-09-21T07:55:23.900918+0000","last_fullsized":"2024-09-21T07:55:23.900918+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:09:44.157479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902295+0000","last_change":"2024-09-21T07:55:22.906070+0000","last_active":"2024-09-21T07:55:23.902295+0000","last_peered":"2024-09-21T07:55:23.902295+0000","last_clean":"2024-09-21T07:55:23.902295+0000","last_became_active":"2024-09-21T07:55:22.902729+0000","last_became_peered":"2024-09-21T07:55:22.902729+0000","last_unstale":"2024-09-21T07:55:23.902295+0000","last_undegraded":"2024-09-21T07:55:23.902295+0000","last_fullsized":"2024-09-21T07:55:23.902295+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:46.904162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041924+0000","last_change":"2024-09-21T07:55:21.910639+0000","last_active":"2024-09-21T07:55:24.041924+0000","last_peered":"2024-09-21T07:55:24.041924+0000","last_clean":"2024-09-21T07:55:24.041924+0000","last_became_active":"2024-09-21T07:55:21.910351+0000","last_became_peered":"2024-09-21T07:55:21.910351+0000","last_unstale":"2024-09-21T07:55:24.041924+0000","last_undegraded":"2024-09-21T07:55:24.041924+0000","last_fullsized":"2024-09-21T07:55:24.041924+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:11:30.237576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041214+0000","last_change":"2024-09-21T07:55:22.908686+0000","last_active":"2024-09-21T07:55:24.041214+0000","last_peered":"2024-09-21T07:55:24.041214+0000","last_clean":"2024-09-21T07:55:24.041214+0000","last_became_active":"2024-09-21T07:55:22.904455+0000","last_became_peered":"2024-09-21T07:55:22.904455+0000","last_unstale":"2024-09-21T07:55:24.041214+0000","last_undegraded":"2024-09-21T07:55:24.041214+0000","last_fullsized":"2024-09-21T07:55:24.041214+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:28:27.461980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902008+0000","last_change":"2024-09-21T07:55:21.901763+0000","last_active":"2024-09-21T07:55:23.902008+0000","last_peered":"2024-09-21T07:55:23.902008+0000","last_clean":"2024-09-21T07:55:23.902008+0000","last_became_active":"2024-09-21T07:55:21.901457+0000","last_became_peered":"2024-09-21T07:55:21.901457+0000","last_unstale":"2024-09-21T07:55:23.902008+0000","last_undegraded":"2024-09-21T07:55:23.902008+0000","last_fullsized":"2024-09-21T07:55:23.902008+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:49:22.757414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041751+0000","last_change":"2024-09-21T07:55:22.915358+0000","last_active":"2024-09-21T07:55:24.041751+0000","last_peered":"2024-09-21T07:55:24.041751+0000","last_clean":"2024-09-21T07:55:24.041751+0000","last_became_active":"2024-09-21T07:55:22.913798+0000","last_became_peered":"2024-09-21T07:55:22.913798+0000","last_unstale":"2024-09-21T07:55:24.041751+0000","last_undegraded":"2024-09-21T07:55:24.041751+0000","last_fullsized":"2024-09-21T07:55:24.041751+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:46:30.818534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.15","version":"167'3","reported_seq":18,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903776+0000","last_change":"2024-09-21T07:55:21.900343+0000","last_active":"2024-09-21T07:55:23.903776+0000","last_peered":"2024-09-21T07:55:23.903776+0000","last_clean":"2024-09-21T07:55:23.903776+0000","last_became_active":"2024-09-21T07:55:21.899387+0000","last_became_peered":"2024-09-21T07:55:21.899387+0000","last_unstale":"2024-09-21T07:55:23.903776+0000","last_undegraded":"2024-09-21T07:55:23.903776+0000","last_fullsized":"2024-09-21T07:55:23.903776+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:27:16.220879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903430+0000","last_change":"2024-09-21T07:55:22.902710+0000","last_active":"2024-09-21T07:55:23.903430+0000","last_peered":"2024-09-21T07:55:23.903430+0000","last_clean":"2024-09-21T07:55:23.903430+0000","last_became_active":"2024-09-21T07:55:22.902413+0000","last_became_peered":"2024-09-21T07:55:22.902413+0000","last_unstale":"2024-09-21T07:55:23.903430+0000","last_undegraded":"2024-09-21T07:55:23.903430+0000","last_fullsized":"2024-09-21T07:55:23.903430+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:05:22.858039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.14","version":"167'6","reported_seq":21,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041891+0000","last_change":"2024-09-21T07:55:21.899493+0000","last_active":"2024-09-21T07:55:24.041891+0000","last_peered":"2024-09-21T07:55:24.041891+0000","last_clean":"2024-09-21T07:55:24.041891+0000","last_became_active":"2024-09-21T07:55:21.899084+0000","last_became_peered":"2024-09-21T07:55:21.899084+0000","last_unstale":"2024-09-21T07:55:24.041891+0000","last_undegraded":"2024-09-21T07:55:24.041891+0000","last_fullsized":"2024-09-21T07:55:24.041891+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:16:13.860855+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040765+0000","last_change":"2024-09-21T07:55:22.898813+0000","last_active":"2024-09-21T07:55:24.040765+0000","last_peered":"2024-09-21T07:55:24.040765+0000","last_clean":"2024-09-21T07:55:24.040765+0000","last_became_active":"2024-09-21T07:55:22.898452+0000","last_became_peered":"2024-09-21T07:55:22.898452+0000","last_unstale":"2024-09-21T07:55:24.040765+0000","last_undegraded":"2024-09-21T07:55:24.040765+0000","last_fullsized":"2024-09-21T07:55:24.040765+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:17:33.700585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.13","version":"167'2","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041966+0000","last_change":"2024-09-21T07:55:21.906247+0000","last_active":"2024-09-21T07:55:24.041966+0000","last_peered":"2024-09-21T07:55:24.041966+0000","last_clean":"2024-09-21T07:55:24.041966+0000","last_became_active":"2024-09-21T07:55:21.905901+0000","last_became_peered":"2024-09-21T07:55:21.905901+0000","last_unstale":"2024-09-21T07:55:24.041966+0000","last_undegraded":"2024-09-21T07:55:24.041966+0000","last_fullsized":"2024-09-21T07:55:24.041966+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:39:10.644990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041838+0000","last_change":"2024-09-21T07:55:22.912496+0000","last_active":"2024-09-21T07:55:24.041838+0000","last_peered":"2024-09-21T07:55:24.041838+0000","last_clean":"2024-09-21T07:55:24.041838+0000","last_became_active":"2024-09-21T07:55:22.911905+0000","last_became_peered":"2024-09-21T07:55:22.911905+0000","last_unstale":"2024-09-21T07:55:24.041838+0000","last_undegraded":"2024-09-21T07:55:24.041838+0000","last_fullsized":"2024-09-21T07:55:24.041838+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:55:28.938565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040899+0000","last_change":"2024-09-21T07:55:21.902093+0000","last_active":"2024-09-21T07:55:24.040899+0000","last_peered":"2024-09-21T07:55:24.040899+0000","last_clean":"2024-09-21T07:55:24.040899+0000","last_became_active":"2024-09-21T07:55:21.901806+0000","last_became_peered":"2024-09-21T07:55:21.901806+0000","last_unstale":"2024-09-21T07:55:24.040899+0000","last_undegraded":"2024-09-21T07:55:24.040899+0000","last_fullsized":"2024-09-21T07:55:24.040899+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:54:33.819791+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041341+0000","last_change":"2024-09-21T07:55:22.903607+0000","last_active":"2024-09-21T07:55:24.041341+0000","last_peered":"2024-09-21T07:55:24.041341+0000","last_clean":"2024-09-21T07:55:24.041341+0000","last_became_active":"2024-09-21T07:55:22.903203+0000","last_became_peered":"2024-09-21T07:55:22.903203+0000","last_unstale":"2024-09-21T07:55:24.041341+0000","last_undegraded":"2024-09-21T07:55:24.041341+0000","last_fullsized":"2024-09-21T07:55:24.041341+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:26:27.856157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902689+0000","last_change":"2024-09-21T07:55:21.894028+0000","last_active":"2024-09-21T07:55:23.902689+0000","last_peered":"2024-09-21T07:55:23.902689+0000","last_clean":"2024-09-21T07:55:23.902689+0000","last_became_active":"2024-09-21T07:55:21.890415+0000","last_became_peered":"2024-09-21T07:55:21.890415+0000","last_unstale":"2024-09-21T07:55:23.902689+0000","last_undegraded":"2024-09-21T07:55:23.902689+0000","last_fullsized":"2024-09-21T07:55:23.902689+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:57:02.674755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902416+0000","last_change":"2024-09-21T07:55:22.902918+0000","last_active":"2024-09-21T07:55:23.902416+0000","last_peered":"2024-09-21T07:55:23.902416+0000","last_clean":"2024-09-21T07:55:23.902416+0000","last_became_active":"2024-09-21T07:55:22.902577+0000","last_became_peered":"2024-09-21T07:55:22.902577+0000","last_unstale":"2024-09-21T07:55:23.902416+0000","last_undegraded":"2024-09-21T07:55:23.902416+0000","last_fullsized":"2024-09-21T07:55:23.902416+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:30:27.756453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.10","version":"167'8","reported_seq":23,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902590+0000","last_change":"2024-09-21T07:55:21.895482+0000","last_active":"2024-09-21T07:55:23.902590+0000","last_peered":"2024-09-21T07:55:23.902590+0000","last_clean":"2024-09-21T07:55:23.902590+0000","last_became_active":"2024-09-21T07:55:21.895173+0000","last_became_peered":"2024-09-21T07:55:21.895173+0000","last_unstale":"2024-09-21T07:55:23.902590+0000","last_undegraded":"2024-09-21T07:55:23.902590+0000","last_fullsized":"2024-09-21T07:55:23.902590+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:18:47.075581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042799+0000","last_change":"2024-09-21T07:55:22.918491+0000","last_active":"2024-09-21T07:55:24.042799+0000","last_peered":"2024-09-21T07:55:24.042799+0000","last_clean":"2024-09-21T07:55:24.042799+0000","last_became_active":"2024-09-21T07:55:22.917293+0000","last_became_peered":"2024-09-21T07:55:22.917293+0000","last_unstale":"2024-09-21T07:55:24.042799+0000","last_undegraded":"2024-09-21T07:55:24.042799+0000","last_fullsized":"2024-09-21T07:55:24.042799+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:42.196802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.f","version":"167'1","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903966+0000","last_change":"2024-09-21T07:55:21.900135+0000","last_active":"2024-09-21T07:55:23.903966+0000","last_peered":"2024-09-21T07:55:23.903966+0000","last_clean":"2024-09-21T07:55:23.903966+0000","last_became_active":"2024-09-21T07:55:21.899792+0000","last_became_peered":"2024-09-21T07:55:21.899792+0000","last_unstale":"2024-09-21T07:55:23.903966+0000","last_undegraded":"2024-09-21T07:55:23.903966+0000","last_fullsized":"2024-09-21T07:55:23.903966+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:22:22.400798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902406+0000","last_change":"2024-09-21T07:55:22.912601+0000","last_active":"2024-09-21T07:55:23.902406+0000","last_peered":"2024-09-21T07:55:23.902406+0000","last_clean":"2024-09-21T07:55:23.902406+0000","last_became_active":"2024-09-21T07:55:22.912299+0000","last_became_peered":"2024-09-21T07:55:22.912299+0000","last_unstale":"2024-09-21T07:55:23.902406+0000","last_undegraded":"2024-09-21T07:55:23.902406+0000","last_fullsized":"2024-09-21T07:55:23.902406+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:15:12.135422+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.e","version":"167'2","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041328+0000","last_change":"2024-09-21T07:55:21.906124+0000","last_active":"2024-09-21T07:55:24.041328+0000","last_peered":"2024-09-21T07:55:24.041328+0000","last_clean":"2024-09-21T07:55:24.041328+0000","last_became_active":"2024-09-21T07:55:21.905853+0000","last_became_peered":"2024-09-21T07:55:21.905853+0000","last_unstale":"2024-09-21T07:55:24.041328+0000","last_undegraded":"2024-09-21T07:55:24.041328+0000","last_fullsized":"2024-09-21T07:55:24.041328+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:43.918136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042120+0000","last_change":"2024-09-21T07:55:22.912506+0000","last_active":"2024-09-21T07:55:24.042120+0000","last_peered":"2024-09-21T07:55:24.042120+0000","last_clean":"2024-09-21T07:55:24.042120+0000","last_became_active":"2024-09-21T07:55:22.911532+0000","last_became_peered":"2024-09-21T07:55:22.911532+0000","last_unstale":"2024-09-21T07:55:24.042120+0000","last_undegraded":"2024-09-21T07:55:24.042120+0000","last_fullsized":"2024-09-21T07:55:24.042120+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:46:09.063302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.d","version":"167'1","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040979+0000","last_change":"2024-09-21T07:55:21.901107+0000","last_active":"2024-09-21T07:55:24.040979+0000","last_peered":"2024-09-21T07:55:24.040979+0000","last_clean":"2024-09-21T07:55:24.040979+0000","last_became_active":"2024-09-21T07:55:21.900794+0000","last_became_peered":"2024-09-21T07:55:21.900794+0000","last_unstale":"2024-09-21T07:55:24.040979+0000","last_undegraded":"2024-09-21T07:55:24.040979+0000","last_fullsized":"2024-09-21T07:55:24.040979+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:44:39.347391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.904253+0000","last_change":"2024-09-21T07:55:22.896555+0000","last_active":"2024-09-21T07:55:23.904253+0000","last_peered":"2024-09-21T07:55:23.904253+0000","last_clean":"2024-09-21T07:55:23.904253+0000","last_became_active":"2024-09-21T07:55:22.896078+0000","last_became_peered":"2024-09-21T07:55:22.896078+0000","last_unstale":"2024-09-21T07:55:23.904253+0000","last_undegraded":"2024-09-21T07:55:23.904253+0000","last_fullsized":"2024-09-21T07:55:23.904253+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:03:09.546147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901984+0000","last_change":"2024-09-21T07:55:21.904652+0000","last_active":"2024-09-21T07:55:23.901984+0000","last_peered":"2024-09-21T07:55:23.901984+0000","last_clean":"2024-09-21T07:55:23.901984+0000","last_became_active":"2024-09-21T07:55:21.902707+0000","last_became_peered":"2024-09-21T07:55:21.902707+0000","last_unstale":"2024-09-21T07:55:23.901984+0000","last_undegraded":"2024-09-21T07:55:23.901984+0000","last_fullsized":"2024-09-21T07:55:23.901984+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:39:25.988845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041603+0000","last_change":"2024-09-21T07:55:22.909063+0000","last_active":"2024-09-21T07:55:24.041603+0000","last_peered":"2024-09-21T07:55:24.041603+0000","last_clean":"2024-09-21T07:55:24.041603+0000","last_became_active":"2024-09-21T07:55:22.907687+0000","last_became_peered":"2024-09-21T07:55:22.907687+0000","last_unstale":"2024-09-21T07:55:24.041603+0000","last_undegraded":"2024-09-21T07:55:24.041603+0000","last_fullsized":"2024-09-21T07:55:24.041603+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:07:27.372642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.b","version":"167'2","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040595+0000","last_change":"2024-09-21T07:55:21.898206+0000","last_active":"2024-09-21T07:55:24.040595+0000","last_peered":"2024-09-21T07:55:24.040595+0000","last_clean":"2024-09-21T07:55:24.040595+0000","last_became_active":"2024-09-21T07:55:21.897876+0000","last_became_peered":"2024-09-21T07:55:21.897876+0000","last_unstale":"2024-09-21T07:55:24.040595+0000","last_undegraded":"2024-09-21T07:55:24.040595+0000","last_fullsized":"2024-09-21T07:55:24.040595+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:09:16.392654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903249+0000","last_change":"2024-09-21T07:55:22.911203+0000","last_active":"2024-09-21T07:55:23.903249+0000","last_peered":"2024-09-21T07:55:23.903249+0000","last_clean":"2024-09-21T07:55:23.903249+0000","last_became_active":"2024-09-21T07:55:22.910276+0000","last_became_peered":"2024-09-21T07:55:22.910276+0000","last_unstale":"2024-09-21T07:55:23.903249+0000","last_undegraded":"2024-09-21T07:55:23.903249+0000","last_fullsized":"2024-09-21T07:55:23.903249+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:10:52.415163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902661+0000","last_change":"2024-09-21T07:55:21.900347+0000","last_active":"2024-09-21T07:55:23.902661+0000","last_peered":"2024-09-21T07:55:23.902661+0000","last_clean":"2024-09-21T07:55:23.902661+0000","last_became_active":"2024-09-21T07:55:21.899787+0000","last_became_peered":"2024-09-21T07:55:21.899787+0000","last_unstale":"2024-09-21T07:55:23.902661+0000","last_undegraded":"2024-09-21T07:55:23.902661+0000","last_fullsized":"2024-09-21T07:55:23.902661+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:01:58.474180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.900679+0000","last_change":"2024-09-21T07:55:22.903655+0000","last_active":"2024-09-21T07:55:23.900679+0000","last_peered":"2024-09-21T07:55:23.900679+0000","last_clean":"2024-09-21T07:55:23.900679+0000","last_became_active":"2024-09-21T07:55:22.898937+0000","last_became_peered":"2024-09-21T07:55:22.898937+0000","last_unstale":"2024-09-21T07:55:23.900679+0000","last_undegraded":"2024-09-21T07:55:23.900679+0000","last_fullsized":"2024-09-21T07:55:23.900679+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:32:31.074280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041033+0000","last_change":"2024-09-21T07:55:21.907918+0000","last_active":"2024-09-21T07:55:24.041033+0000","last_peered":"2024-09-21T07:55:24.041033+0000","last_clean":"2024-09-21T07:55:24.041033+0000","last_became_active":"2024-09-21T07:55:21.907562+0000","last_became_peered":"2024-09-21T07:55:21.907562+0000","last_unstale":"2024-09-21T07:55:24.041033+0000","last_undegraded":"2024-09-21T07:55:24.041033+0000","last_fullsized":"2024-09-21T07:55:24.041033+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:29:39.634957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.900841+0000","last_change":"2024-09-21T07:55:22.906077+0000","last_active":"2024-09-21T07:55:23.900841+0000","last_peered":"2024-09-21T07:55:23.900841+0000","last_clean":"2024-09-21T07:55:23.900841+0000","last_became_active":"2024-09-21T07:55:22.905055+0000","last_became_peered":"2024-09-21T07:55:22.905055+0000","last_unstale":"2024-09-21T07:55:23.900841+0000","last_undegraded":"2024-09-21T07:55:23.900841+0000","last_fullsized":"2024-09-21T07:55:23.900841+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:48:40.228172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902733+0000","last_change":"2024-09-21T07:55:21.892595+0000","last_active":"2024-09-21T07:55:23.902733+0000","last_peered":"2024-09-21T07:55:23.902733+0000","last_clean":"2024-09-21T07:55:23.902733+0000","last_became_active":"2024-09-21T07:55:21.892204+0000","last_became_peered":"2024-09-21T07:55:21.892204+0000","last_unstale":"2024-09-21T07:55:23.902733+0000","last_undegraded":"2024-09-21T07:55:23.902733+0000","last_fullsized":"2024-09-21T07:55:23.902733+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:31:28.204312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042932+0000","last_change":"2024-09-21T07:55:22.918905+0000","last_active":"2024-09-21T07:55:24.042932+0000","last_peered":"2024-09-21T07:55:24.042932+0000","last_clean":"2024-09-21T07:55:24.042932+0000","last_became_active":"2024-09-21T07:55:22.917757+0000","last_became_peered":"2024-09-21T07:55:22.917757+0000","last_unstale":"2024-09-21T07:55:24.042932+0000","last_undegraded":"2024-09-21T07:55:24.042932+0000","last_fullsized":"2024-09-21T07:55:24.042932+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:04:45.078912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.7","version":"167'2","reported_seq":17,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903677+0000","last_change":"2024-09-21T07:55:21.897497+0000","last_active":"2024-09-21T07:55:23.903677+0000","last_peered":"2024-09-21T07:55:23.903677+0000","last_clean":"2024-09-21T07:55:23.903677+0000","last_became_active":"2024-09-21T07:55:21.897099+0000","last_became_peered":"2024-09-21T07:55:21.897099+0000","last_unstale":"2024-09-21T07:55:23.903677+0000","last_undegraded":"2024-09-21T07:55:23.903677+0000","last_fullsized":"2024-09-21T07:55:23.903677+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:54.646880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903786+0000","last_change":"2024-09-21T07:55:22.906033+0000","last_active":"2024-09-21T07:55:23.903786+0000","last_peered":"2024-09-21T07:55:23.903786+0000","last_clean":"2024-09-21T07:55:23.903786+0000","last_became_active":"2024-09-21T07:55:22.905290+0000","last_became_peered":"2024-09-21T07:55:22.905290+0000","last_unstale":"2024-09-21T07:55:23.903786+0000","last_undegraded":"2024-09-21T07:55:23.903786+0000","last_fullsized":"2024-09-21T07:55:23.903786+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:47:08.075335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.6","version":"167'1","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041262+0000","last_change":"2024-09-21T07:55:21.901607+0000","last_active":"2024-09-21T07:55:24.041262+0000","last_peered":"2024-09-21T07:55:24.041262+0000","last_clean":"2024-09-21T07:55:24.041262+0000","last_became_active":"2024-09-21T07:55:21.901262+0000","last_became_peered":"2024-09-21T07:55:21.901262+0000","last_unstale":"2024-09-21T07:55:24.041262+0000","last_undegraded":"2024-09-21T07:55:24.041262+0000","last_fullsized":"2024-09-21T07:55:24.041262+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:54:30.055314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041673+0000","last_change":"2024-09-21T07:55:22.912958+0000","last_active":"2024-09-21T07:55:24.041673+0000","last_peered":"2024-09-21T07:55:24.041673+0000","last_clean":"2024-09-21T07:55:24.041673+0000","last_became_active":"2024-09-21T07:55:22.911354+0000","last_became_peered":"2024-09-21T07:55:22.911354+0000","last_unstale":"2024-09-21T07:55:24.041673+0000","last_undegraded":"2024-09-21T07:55:24.041673+0000","last_fullsized":"2024-09-21T07:55:24.041673+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:20:51.473317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.904289+0000","last_change":"2024-09-21T07:55:21.902875+0000","last_active":"2024-09-21T07:55:23.904289+0000","last_peered":"2024-09-21T07:55:23.904289+0000","last_clean":"2024-09-21T07:55:23.904289+0000","last_became_active":"2024-09-21T07:55:21.902548+0000","last_became_peered":"2024-09-21T07:55:21.902548+0000","last_unstale":"2024-09-21T07:55:23.904289+0000","last_undegraded":"2024-09-21T07:55:23.904289+0000","last_fullsized":"2024-09-21T07:55:23.904289+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:07:29.315811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.042399+0000","last_change":"2024-09-21T07:55:22.913715+0000","last_active":"2024-09-21T07:55:24.042399+0000","last_peered":"2024-09-21T07:55:24.042399+0000","last_clean":"2024-09-21T07:55:24.042399+0000","last_became_active":"2024-09-21T07:55:22.913417+0000","last_became_peered":"2024-09-21T07:55:22.913417+0000","last_unstale":"2024-09-21T07:55:24.042399+0000","last_undegraded":"2024-09-21T07:55:24.042399+0000","last_fullsized":"2024-09-21T07:55:24.042399+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T17:23:16.502026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903397+0000","last_change":"2024-09-21T07:55:21.895484+0000","last_active":"2024-09-21T07:55:23.903397+0000","last_peered":"2024-09-21T07:55:23.903397+0000","last_clean":"2024-09-21T07:55:23.903397+0000","last_became_active":"2024-09-21T07:55:21.895160+0000","last_became_peered":"2024-09-21T07:55:21.895160+0000","last_unstale":"2024-09-21T07:55:23.903397+0000","last_undegraded":"2024-09-21T07:55:23.903397+0000","last_fullsized":"2024-09-21T07:55:23.903397+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:02:42.704749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.903112+0000","last_change":"2024-09-21T07:55:22.906045+0000","last_active":"2024-09-21T07:55:23.903112+0000","last_peered":"2024-09-21T07:55:23.903112+0000","last_clean":"2024-09-21T07:55:23.903112+0000","last_became_active":"2024-09-21T07:55:22.905152+0000","last_became_peered":"2024-09-21T07:55:22.905152+0000","last_unstale":"2024-09-21T07:55:23.903112+0000","last_undegraded":"2024-09-21T07:55:23.903112+0000","last_fullsized":"2024-09-21T07:55:23.903112+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:52:50.654801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901138+0000","last_change":"2024-09-21T07:55:21.904748+0000","last_active":"2024-09-21T07:55:23.901138+0000","last_peered":"2024-09-21T07:55:23.901138+0000","last_clean":"2024-09-21T07:55:23.901138+0000","last_became_active":"2024-09-21T07:55:21.904005+0000","last_became_peered":"2024-09-21T07:55:21.904005+0000","last_unstale":"2024-09-21T07:55:23.901138+0000","last_undegraded":"2024-09-21T07:55:23.901138+0000","last_fullsized":"2024-09-21T07:55:23.901138+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:56:53.882908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.040926+0000","last_change":"2024-09-21T07:55:22.916588+0000","last_active":"2024-09-21T07:55:24.040926+0000","last_peered":"2024-09-21T07:55:24.040926+0000","last_clean":"2024-09-21T07:55:24.040926+0000","last_became_active":"2024-09-21T07:55:22.916271+0000","last_became_peered":"2024-09-21T07:55:22.916271+0000","last_unstale":"2024-09-21T07:55:24.040926+0000","last_undegraded":"2024-09-21T07:55:24.040926+0000","last_fullsized":"2024-09-21T07:55:24.040926+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:32.076142+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041707+0000","last_change":"2024-09-21T07:55:21.908045+0000","last_active":"2024-09-21T07:55:24.041707+0000","last_peered":"2024-09-21T07:55:24.041707+0000","last_clean":"2024-09-21T07:55:24.041707+0000","last_became_active":"2024-09-21T07:55:21.907601+0000","last_became_peered":"2024-09-21T07:55:21.907601+0000","last_unstale":"2024-09-21T07:55:24.041707+0000","last_undegraded":"2024-09-21T07:55:24.041707+0000","last_fullsized":"2024-09-21T07:55:24.041707+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:24:43.790181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.902507+0000","last_change":"2024-09-21T07:55:22.901510+0000","last_active":"2024-09-21T07:55:23.902507+0000","last_peered":"2024-09-21T07:55:23.902507+0000","last_clean":"2024-09-21T07:55:23.902507+0000","last_became_active":"2024-09-21T07:55:22.901036+0000","last_became_peered":"2024-09-21T07:55:22.901036+0000","last_unstale":"2024-09-21T07:55:23.902507+0000","last_undegraded":"2024-09-21T07:55:23.902507+0000","last_fullsized":"2024-09-21T07:55:23.902507+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:52:01.042222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041158+0000","last_change":"2024-09-21T07:55:21.901463+0000","last_active":"2024-09-21T07:55:24.041158+0000","last_peered":"2024-09-21T07:55:24.041158+0000","last_clean":"2024-09-21T07:55:24.041158+0000","last_became_active":"2024-09-21T07:55:21.901122+0000","last_became_peered":"2024-09-21T07:55:21.901122+0000","last_unstale":"2024-09-21T07:55:24.041158+0000","last_undegraded":"2024-09-21T07:55:24.041158+0000","last_fullsized":"2024-09-21T07:55:24.041158+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T15:23:43.918136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901006+0000","last_change":"2024-09-21T07:55:22.905326+0000","last_active":"2024-09-21T07:55:23.901006+0000","last_peered":"2024-09-21T07:55:23.901006+0000","last_clean":"2024-09-21T07:55:23.901006+0000","last_became_active":"2024-09-21T07:55:22.904651+0000","last_became_peered":"2024-09-21T07:55:22.904651+0000","last_unstale":"2024-09-21T07:55:23.901006+0000","last_undegraded":"2024-09-21T07:55:23.901006+0000","last_fullsized":"2024-09-21T07:55:23.901006+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:41:32.860507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":15,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.900991+0000","last_change":"2024-09-21T07:55:21.905287+0000","last_active":"2024-09-21T07:55:23.900991+0000","last_peered":"2024-09-21T07:55:23.900991+0000","last_clean":"2024-09-21T07:55:23.900991+0000","last_became_active":"2024-09-21T07:55:21.904988+0000","last_became_peered":"2024-09-21T07:55:21.904988+0000","last_unstale":"2024-09-21T07:55:23.900991+0000","last_undegraded":"2024-09-21T07:55:23.900991+0000","last_fullsized":"2024-09-21T07:55:23.900991+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:20.228061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1a","version":"0'0","reported_seq":13,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041568+0000","last_change":"2024-09-21T07:55:22.907976+0000","last_active":"2024-09-21T07:55:24.041568+0000","last_peered":"2024-09-21T07:55:24.041568+0000","last_clean":"2024-09-21T07:55:24.041568+0000","last_became_active":"2024-09-21T07:55:22.907291+0000","last_became_peered":"2024-09-21T07:55:22.907291+0000","last_unstale":"2024-09-21T07:55:24.041568+0000","last_undegraded":"2024-09-21T07:55:24.041568+0000","last_fullsized":"2024-09-21T07:55:24.041568+0000","mapping_epoch":166,"log_start":"0'0","ondisk_log_start":"0'0","created":166,"last_epoch_clean":167,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:21.869311+0000","last_clean_scrub_stamp":"2024-09-21T07:55:21.869311+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T18:33:13.812615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1b","version":"167'1","reported_seq":16,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:24.041000+0000","last_change":"2024-09-21T07:55:21.900512+0000","last_active":"2024-09-21T07:55:24.041000+0000","last_peered":"2024-09-21T07:55:24.041000+0000","last_clean":"2024-09-21T07:55:24.041000+0000","last_became_active":"2024-09-21T07:55:21.900185+0000","last_became_peered":"2024-09-21T07:55:21.900185+0000","last_unstale":"2024-09-21T07:55:24.041000+0000","last_undegraded":"2024-09-21T07:55:24.041000+0000","last_fullsized":"2024-09-21T07:55:24.041000+0000","mapping_epoch":165,"log_start":"0'0","ondisk_log_start":"0'0","created":165,"last_epoch_clean":166,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:55:20.860949+0000","last_clean_scrub_stamp":"2024-09-21T07:55:20.860949+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T19:38:12.070351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"74'79","reported_seq":416,"reported_epoch":168,"state":"active+clean","last_fresh":"2024-09-21T07:55:23.901048+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:55:23.901048+0000","last_peered":"2024-09-21T07:55:23.901048+0000","last_clean":"2024-09-21T07:55:23.901048+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:55:23.901048+0000","last_undegraded":"2024-09-21T07:55:23.901048+0000","last_fullsized":"2024-09-21T07:55:23.901048+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542262,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33260,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338580,"statfs":{"total":96636764160,"available":96602705920,"internally_reserved":0,"allocated":2113536,"data_stored":932883,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509557,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2113536,"data_stored":932931,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509558,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33852,"kb_used_data":2656,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337988,"statfs":{"total":96636764160,"available":96602099712,"internally_reserved":0,"allocated":2719744,"data_stored":1523957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509558,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33252,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338588,"statfs":{"total":96636764160,"available":96602714112,"internally_reserved":0,"allocated":2113536,"data_stored":932951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476856,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33268,"kb_used_data":2072,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338572,"statfs":{"total":96636764160,"available":96602697728,"internally_reserved":0,"allocated":2121728,"data_stored":934473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476856,"num_pgs":14,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33276,"kb_used_data":2080,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338564,"statfs":{"total":96636764160,"available":96602689536,"internally_reserved":0,"allocated":2129920,"data_stored":934577,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33828,"kb_used_data":2640,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338012,"statfs":{"total":96636764160,"available":96602124288,"internally_reserved":0,"allocated":2703360,"data_stored":1523229,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476853,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33256,"kb_used_data":2064,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338584,"statfs":{"total":96636764160,"available":96602710016,"internally_reserved":0,"allocated":2113536,"data_stored":933443,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1716,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:27.819 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:55:27.819 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2024-09-21T07:55:27.864 INFO:tasks.ceph.mds.c.smithi088.stderr:2024-09-21T07:55:27.862+0000 7fbef7400640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T07:55:28.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 -- 172.21.15.88:0/217716346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf700735e0 msgr2=0x7faf700739f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 --2- 172.21.15.88:0/217716346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf700735e0 0x7faf700739f0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009830 tx=0x7faf5c02a510 comp rx=0 tx=0).stop 2024-09-21T07:55:28.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 -- 172.21.15.88:0/217716346 shutdown_connections 2024-09-21T07:55:28.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 --2- 172.21.15.88:0/217716346 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf700735e0 0x7faf700739f0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 --2- 172.21.15.88:0/217716346 >> v2:172.21.15.175:3300/0 conn(0x7faf70145450 0x7faf7006f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 -- 172.21.15.88:0/217716346 >> 172.21.15.88:0/217716346 conn(0x7faf7006d1c0 msgr2=0x7faf7006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:28.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 -- 172.21.15.88:0/217716346 shutdown_connections 2024-09-21T07:55:28.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.081+0000 7faf76a00640 1 -- 172.21.15.88:0/217716346 wait complete. 2024-09-21T07:55:28.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.083+0000 7faf76a00640 1 Processor -- start 2024-09-21T07:55:28.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.083+0000 7faf76a00640 1 -- start start 2024-09-21T07:55:28.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf76a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf700735e0 0x7faf701351d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf76a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 0x7faf701f5e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.087 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf76a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf70160320 0x7faf7015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf76a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf700ce620 con 0x7faf70145450 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf76a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faf70071020 con 0x7faf700735e0 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf76a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faf700ce7a0 con 0x7faf70160320 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 0x7faf701f5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 0x7faf701f5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49896/0 (socket says 172.21.15.88:49896) 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 -- 172.21.15.88:0/4221968818 learned_addr learned my addr 172.21.15.88:0/4221968818 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6fe00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf700735e0 0x7faf701351d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf74a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf70160320 0x7faf7015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 -- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf70160320 msgr2=0x7faf7015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf70160320 0x7faf7015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 -- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf700735e0 msgr2=0x7faf701351d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf700735e0 0x7faf701351d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.085+0000 7faf6f400640 1 -- 172.21.15.88:0/4221968818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf70134fe0 con 0x7faf70145450 2024-09-21T07:55:28.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.086+0000 7faf6f400640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 0x7faf701f5e20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009950 tx=0x7faf5c033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:28.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.086+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c005c90 con 0x7faf70145450 2024-09-21T07:55:28.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.086+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faf5c031e50 con 0x7faf70145450 2024-09-21T07:55:28.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.086+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf7015dd20 con 0x7faf70145450 2024-09-21T07:55:28.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.086+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf5c005260 con 0x7faf70145450 2024-09-21T07:55:28.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.086+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf7015e2e0 con 0x7faf70145450 2024-09-21T07:55:28.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.087+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf7015da90 con 0x7faf70145450 2024-09-21T07:55:28.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.088+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7faf5c005400 con 0x7faf70145450 2024-09-21T07:55:28.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.089+0000 7faf67e00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf500795b0 0x7faf5007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.090+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7faf5c0ba9a0 con 0x7faf70145450 2024-09-21T07:55:28.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.090+0000 7faf6fe00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf500795b0 0x7faf5007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.090+0000 7faf6fe00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf500795b0 0x7faf5007bac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf60009ac0 tx=0x7faf60005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:28.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.094+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faf5c03f030 con 0x7faf70145450 2024-09-21T07:55:28.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.350+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) -- 0x7faf70161050 con 0x7faf70145450 2024-09-21T07:55:28.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.353+0000 7faf67e00640 1 -- 172.21.15.88:0/4221968818 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v23) ==== 190+0+147 (secure 0 0 0) 0x7faf5c084590 con 0x7faf70145450 2024-09-21T07:55:28.355 INFO:teuthology.orchestra.run.smithi088.stdout:[client.0] 2024-09-21T07:55:28.356 INFO:teuthology.orchestra.run.smithi088.stdout: key = AQB8ee5m0ff1EBAA4A2GUVv4rkGV+VT39uc2kA== 2024-09-21T07:55:28.356 INFO:teuthology.orchestra.run.smithi088.stdout: caps mds = "allow" 2024-09-21T07:55:28.356 INFO:teuthology.orchestra.run.smithi088.stdout: caps mon = "allow r" 2024-09-21T07:55:28.356 INFO:teuthology.orchestra.run.smithi088.stdout: caps osd = "allow rw tag cephfs data=cephfs" 2024-09-21T07:55:28.356 INFO:teuthology.orchestra.run.smithi088.stderr:updated caps for client.0 2024-09-21T07:55:28.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf500795b0 msgr2=0x7faf5007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf500795b0 0x7faf5007bac0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7faf60009ac0 tx=0x7faf60005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:28.358 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 msgr2=0x7faf701f5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 0x7faf701f5e20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7faf5c009950 tx=0x7faf5c033c20 comp rx=0 tx=0).stop 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 shutdown_connections 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faf500795b0 0x7faf5007bac0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faf70160320 0x7faf7015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf70145450 0x7faf701f5e20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 --2- 172.21.15.88:0/4221968818 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faf700735e0 0x7faf701351d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 >> 172.21.15.88:0/4221968818 conn(0x7faf7006d1c0 msgr2=0x7faf700cea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 shutdown_connections 2024-09-21T07:55:28.359 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.356+0000 7faf76a00640 1 -- 172.21.15.88:0/4221968818 wait complete. 2024-09-21T07:55:28.380 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.638+0000 7f129ea00640 1 -- 172.21.15.88:0/795107229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980aebc0 msgr2=0x7f1298152920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.638+0000 7f129ea00640 1 --2- 172.21.15.88:0/795107229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980aebc0 0x7f1298152920 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f1288009830 tx=0x7f128802a480 comp rx=0 tx=0).stop 2024-09-21T07:55:28.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.639+0000 7f129ea00640 1 -- 172.21.15.88:0/795107229 shutdown_connections 2024-09-21T07:55:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.639+0000 7f129ea00640 1 --2- 172.21.15.88:0/795107229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f129815fae0 0x7f12980b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.639+0000 7f129ea00640 1 --2- 172.21.15.88:0/795107229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980aebc0 0x7f1298152920 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.639+0000 7f129ea00640 1 -- 172.21.15.88:0/795107229 >> 172.21.15.88:0/795107229 conn(0x7f129806d730 msgr2=0x7f12980b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.639+0000 7f129ea00640 1 -- 172.21.15.88:0/795107229 shutdown_connections 2024-09-21T07:55:28.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.639+0000 7f129ea00640 1 -- 172.21.15.88:0/795107229 wait complete. 2024-09-21T07:55:28.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.641+0000 7f129ea00640 1 Processor -- start 2024-09-21T07:55:28.643 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.641+0000 7f129ea00640 1 -- start start 2024-09-21T07:55:28.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.642+0000 7f129ea00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12980aebc0 0x7f1298145370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.642+0000 7f129ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 0x7f1298141aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.642+0000 7f129ea00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1298151b40 0x7f1298141fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.642+0000 7f129ea00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12980b4190 con 0x7f12980b2750 2024-09-21T07:55:28.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.642+0000 7f129ea00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12980b4010 con 0x7f1298151b40 2024-09-21T07:55:28.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.642+0000 7f129ea00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f12980b4310 con 0x7f12980aebc0 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 0x7f1298141aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 0x7f1298141aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49924/0 (socket says 172.21.15.88:49924) 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 -- 172.21.15.88:0/2941345137 learned_addr learned my addr 172.21.15.88:0/2941345137 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f129ca00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1298151b40 0x7f1298141fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f1297e00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12980aebc0 0x7f1298145370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 -- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12980aebc0 msgr2=0x7f1298145370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12980aebc0 0x7f1298145370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 -- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1298151b40 msgr2=0x7f1298141fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1298151b40 0x7f1298141fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.643+0000 7f128f400640 1 -- 172.21.15.88:0/2941345137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12981458b0 con 0x7f12980b2750 2024-09-21T07:55:28.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.644+0000 7f128f400640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 0x7f1298141aa0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f128000c430 tx=0x7f128000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:28.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.644+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280011790 con 0x7f12980b2750 2024-09-21T07:55:28.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.644+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1280017070 con 0x7f12980b2750 2024-09-21T07:55:28.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.644+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12980cc740 con 0x7f12980b2750 2024-09-21T07:55:28.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.644+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1280011ce0 con 0x7f12980b2750 2024-09-21T07:55:28.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.644+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12980ccd00 con 0x7f12980b2750 2024-09-21T07:55:28.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.645+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f12981426f0 con 0x7f12980b2750 2024-09-21T07:55:28.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.646+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f128001c460 con 0x7f12980b2750 2024-09-21T07:55:28.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.647+0000 7f1294c00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f12740795e0 0x7f127407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:28.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.647+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f1280099520 con 0x7f12980b2750 2024-09-21T07:55:28.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.647+0000 7f1297e00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f12740795e0 0x7f127407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:28.650 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.648+0000 7f1297e00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f12740795e0 0x7f127407baf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1288034000 tx=0x7f1288004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:28.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.652+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1280062b40 con 0x7f12980b2750 2024-09-21T07:55:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.875+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f129813f780 con 0x7f12980b2750 2024-09-21T07:55:28.877 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.875+0000 7f1294c00640 1 -- 172.21.15.88:0/2941345137 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) ==== 95+0+5233 (secure 0 0 0) 0x7f1280067570 con 0x7f12980b2750 2024-09-21T07:55:28.878 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:28.878 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":118,"btime":"2024-09-21T07:55:25:906694+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:55:22.380663+0000","modified":"2024-09-21T07:55:25.396077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9147":{"gid":9147,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/661983519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":661983519},{"type":"v1","addr":"172.21.15.88:6835","nonce":661983519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9147,"qdb_cluster":[9147]},"id":13}]} 2024-09-21T07:55:28.878 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 118 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f12740795e0 msgr2=0x7f127407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f12740795e0 0x7f127407baf0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f1288034000 tx=0x7f1288004730 comp rx=0 tx=0).stop 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 msgr2=0x7f1298141aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 0x7f1298141aa0 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f128000c430 tx=0x7f128000c8f0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 shutdown_connections 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f12740795e0 0x7f127407baf0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.880 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1298151b40 0x7f1298141fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12980b2750 0x7f1298141aa0 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 --2- 172.21.15.88:0/2941345137 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12980aebc0 0x7f1298145370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:28.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 >> 172.21.15.88:0/2941345137 conn(0x7f129806d730 msgr2=0x7f129815b240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:28.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 shutdown_connections 2024-09-21T07:55:28.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:28.878+0000 7f129ea00640 1 -- 172.21.15.88:0/2941345137 wait complete. 2024-09-21T07:55:29.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-09-21T07:55:22.380663+0000', 'modified': '2024-09-21T07:55:25.396077+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9147}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9147': {'gid': 9147, 'name': 'c', 'rank': 0, 'incarnation': 115, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6835/661983519', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 661983519}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 661983519}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 9147, 'qdb_cluster': [9147]} 2024-09-21T07:55:29.308 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T07:55:29.308 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2024-09-21T07:55:29.528 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 -- 172.21.15.88:0/1029044518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc15a0f0 msgr2=0x7f4cbc15d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 --2- 172.21.15.88:0/1029044518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc15a0f0 0x7f4cbc15d670 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4005fd0 tx=0x7f4cb402a970 comp rx=0 tx=0).stop 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 -- 172.21.15.88:0/1029044518 shutdown_connections 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 --2- 172.21.15.88:0/1029044518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc15a0f0 0x7f4cbc15d670 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 --2- 172.21.15.88:0/1029044518 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc0ad9c0 0x7f4cbc0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 -- 172.21.15.88:0/1029044518 >> 172.21.15.88:0/1029044518 conn(0x7f4cbc0b1e40 msgr2=0x7f4cbc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 -- 172.21.15.88:0/1029044518 shutdown_connections 2024-09-21T07:55:29.529 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.526+0000 7f4cc2000640 1 -- 172.21.15.88:0/1029044518 wait complete. 2024-09-21T07:55:29.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.528+0000 7f4cc2000640 1 Processor -- start 2024-09-21T07:55:29.530 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.528+0000 7f4cc2000640 1 -- start start 2024-09-21T07:55:29.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc2000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cbc0ad9c0 0x7f4cbc072e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc2000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 0x7f4cbc1ec290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc2000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc15a0f0 0x7f4cbc1f5740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc2000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f410 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.533 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc2000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f290 con 0x7f4cbc0ad9c0 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc2000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cbc06f590 con 0x7f4cbc15a0f0 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 0x7f4cbc1ec290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 0x7f4cbc1ec290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49946/0 (socket says 172.21.15.88:49946) 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cb3e00640 1 -- 172.21.15.88:0/2239376060 learned_addr learned my addr 172.21.15.88:0/2239376060 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cbc0ad9c0 0x7f4cbc072e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.530+0000 7f4cc1600640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc15a0f0 0x7f4cbc1f5740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb3e00640 1 -- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc15a0f0 msgr2=0x7f4cbc1f5740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc15a0f0 0x7f4cbc1f5740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb3e00640 1 -- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cbc0ad9c0 msgr2=0x7f4cbc072e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cbc0ad9c0 0x7f4cbc072e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb3e00640 1 -- 172.21.15.88:0/2239376060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cbc0c9320 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cbc0ad9c0 0x7f4cbc072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:29.534 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cc1600640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc15a0f0 0x7f4cbc1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 0x7f4cbc1ec290 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b3f0 tx=0x7f4ca400b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb9600640 1 -- 172.21.15.88:0/2239376060 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4015020 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb9600640 1 -- 172.21.15.88:0/2239376060 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ca4011920 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cbc0c9820 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cb9600640 1 -- 172.21.15.88:0/2239376060 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4011d70 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.531+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cbc0c9de0 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.535 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.534+0000 7f4cb9600640 1 -- 172.21.15.88:0/2239376060 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f4ca4018020 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.535+0000 7f4cb9600640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4c980795b0 0x7f4c9807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.535+0000 7f4cb9600640 1 -- 172.21.15.88:0/2239376060 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f4ca40994a0 con 0x7f4cbc0b2f60 2024-09-21T07:55:29.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.535+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4c980795b0 0x7f4c9807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.535+0000 7f4cca7c8740 1 Processor -- start 2024-09-21T07:55:29.537 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.535+0000 7f4cca7c8740 1 -- start start 2024-09-21T07:55:29.538 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.535+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4c980795b0 0x7f4c9807bac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00ab40 tx=0x7f4cac00a520 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6b8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6b90a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2c7590 0x55d43d689150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d43d612430 con 0x55d43d2e8270 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d43d6025d0 con 0x55d43d2bf3b0 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55d43d5f1a00 con 0x55d43d2c7590 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6b8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6b8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49950/0 (socket says 172.21.15.88:49950) 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 -- 172.21.15.88:0/894539042 learned_addr learned my addr 172.21.15.88:0/894539042 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:29.540 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6b90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc1600640 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2c7590 0x55d43d689150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 -- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2c7590 msgr2=0x55d43d689150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2c7590 0x55d43d689150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 -- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 msgr2=0x55d43d6b90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6b90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.537+0000 7f4cc0c00640 1 -- 172.21.15.88:0/894539042 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d43d5ba520 con 0x55d43d2e8270 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.538+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6b8b60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00ee30 tx=0x7f4cac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f67197465d429cce server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.538+0000 7f4cb1200640 1 -- 172.21.15.88:0/894539042 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac021070 con 0x55d43d2e8270 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.538+0000 7f4cb1200640 1 -- 172.21.15.88:0/894539042 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cac01d950 con 0x55d43d2e8270 2024-09-21T07:55:29.541 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.538+0000 7f4cb1200640 1 -- 172.21.15.88:0/894539042 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cac01dd00 con 0x55d43d2e8270 2024-09-21T07:55:29.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 -- 172.21.15.88:0/894539042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 msgr2=0x55d43d6b8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6b8b60 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00ee30 tx=0x7f4cac011000 comp rx=0 tx=0).stop 2024-09-21T07:55:29.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 -- 172.21.15.88:0/894539042 shutdown_connections 2024-09-21T07:55:29.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2c7590 0x55d43d689150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.542 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6b90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/894539042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6b8b60 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 -- 172.21.15.88:0/894539042 >> 172.21.15.88:0/894539042 conn(0x55d43d2dc3c0 msgr2=0x55d43d68eb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 -- 172.21.15.88:0/894539042 shutdown_connections 2024-09-21T07:55:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.539+0000 7f4cca7c8740 1 -- 172.21.15.88:0/894539042 wait complete. 2024-09-21T07:55:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.540+0000 7f4cca7c8740 1 -- start start 2024-09-21T07:55:29.543 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.540+0000 7f4cca7c8740 1 Processor -- start 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.541+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6cdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.541+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6cd6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2e8270 0x55d43d6d49e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d43d598330 con 0x55d43d2c7590 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d43d5f5810 con 0x55d43d2bf3b0 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55d43d362620 con 0x55d43d2e8270 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cca7c8740 10 client.0 _ll_register_callbacks cb 0x55d43d59d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6cd6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6cd6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49964/0 (socket says 172.21.15.88:49964) 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3248744258 learned_addr learned my addr 172.21.15.88:0/3248744258 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:29.546 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cc1600640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2e8270 0x55d43d6d49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6cdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2e8270 msgr2=0x55d43d6d49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2e8270 0x55d43d6d49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 msgr2=0x55d43d6cdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6cdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3248744258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d43d353850 con 0x55d43d2c7590 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cc1600640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2e8270 0x55d43d6d49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6cd6c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4062a30 tx=0x7f4ca4063000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64d31ffd2e67a78 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.542+0000 7f4ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4ca9600640 1 -- 172.21.15.88:0/3248744258 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4078dc0 con 0x55d43d2c7590 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4ca9600640 1 -- 172.21.15.88:0/3248744258 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f4ca405e3f0 con 0x55d43d2c7590 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cac01d950 con 0x55d43d2c7590 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4ca9600640 1 -- 172.21.15.88:0/3248744258 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f4ca40625b0 con 0x55d43d2c7590 2024-09-21T07:55:29.547 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 10 client.9264 fetch_fsmap learned FSMap version 118 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55d43d3ca6b0 con 0x55d43d2c7590 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4ca9600640 1 -- 172.21.15.88:0/3248744258 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4cac01d950 con 0x55d43d2c7590 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4ca9600640 1 -- 172.21.15.88:0/3248744258 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 118) ==== 3502+0+0 (secure 0 0 0) 0x7f4ca4061770 con 0x55d43d2c7590 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 10 client.9264 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 10 client.9264 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 20 client.9264 populate_metadata read hostname 'smithi088' 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d2a6e40 0x55d43d6b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 4 client.9264 mds_command: new command op to 9147 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.543+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 --> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d43d604520 con 0x55d43d2a6e40 2024-09-21T07:55:29.548 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.544+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d2a6e40 0x55d43d6b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.549 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.544+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d2a6e40 0x55d43d6b4de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=597b241461104dcf server_cookie=7a91d3ad8d7abe9f in_seq=0 out_seq=0 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.544+0000 7f4ca9600640 10 client.9264 ms_handle_connect on v2:172.21.15.88:6834/661983519 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4ca9600640 1 -- 172.21.15.88:0/3248744258 <== mds.0 v2:172.21.15.88:6834/661983519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d43d604520 con 0x55d43d2a6e40 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4ca9600640 10 client.9264 handle_command_reply: tid=0 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 client.9264 shutdown 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 msgr2=0x55d43d6cd6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6cd6c0 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4ca4062a30 tx=0x7f4ca4063000 comp rx=0 tx=0).stop 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 shutdown_connections 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d2a6e40 0x55d43d6b4de0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2e8270 0x55d43d6d49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6cd6c0 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4ca9600640 0 client.9264 ms_handle_reset on v2:172.21.15.88:6834/661983519 2024-09-21T07:55:29.550 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3248744258 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2bf3b0 0x55d43d6cdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.545+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 >> 172.21.15.88:0/3248744258 conn(0x55d43d2dc3c0 msgr2=0x55d43d6b00b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.546+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 shutdown_connections 2024-09-21T07:55:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.546+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3248744258 wait complete. 2024-09-21T07:55:29.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.546+0000 7f4cca7c8740 20 client.9264 trim_cache size 0 max 16384 2024-09-21T07:55:29.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.565+0000 7f4cca7c8740 1 Processor -- start 2024-09-21T07:55:29.567 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.565+0000 7f4cca7c8740 1 -- start start 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.566+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d6afce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.566+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6b4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.566+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2e8270 0x55d43d6b52a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.566+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc201110 con 0x55d43d2c7590 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.566+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4cbc1f90d0 con 0x55d43d2e8270 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.566+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4cbc1f9720 con 0x55d43d2bf3b0 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6b4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6b4d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49974/0 (socket says 172.21.15.88:49974) 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3138062716 learned_addr learned my addr 172.21.15.88:0/3138062716 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:29.569 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d6afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cc1600640 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2e8270 0x55d43d6b52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 msgr2=0x55d43d6afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d6afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2e8270 msgr2=0x55d43d6b52a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2e8270 0x55d43d6b52a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.570 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 -- 172.21.15.88:0/3138062716 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d43d68a9a0 con 0x55d43d2c7590 2024-09-21T07:55:29.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6b4d60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b3c0 tx=0x7f4ca407df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5a0c02bc2d0d837 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cabe00640 1 -- 172.21.15.88:0/3138062716 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4062aa0 con 0x55d43d2c7590 2024-09-21T07:55:29.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cabe00640 1 -- 172.21.15.88:0/3138062716 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ca4067580 con 0x55d43d2c7590 2024-09-21T07:55:29.571 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.567+0000 7f4cabe00640 1 -- 172.21.15.88:0/3138062716 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ca4078e20 con 0x55d43d2c7590 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3138062716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 msgr2=0x55d43d6b4d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6b4d60 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b3c0 tx=0x7f4ca407df70 comp rx=0 tx=0).stop 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3138062716 shutdown_connections 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2e8270 0x55d43d6b52a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2c7590 0x55d43d6b4d60 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/3138062716 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d6afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3138062716 >> 172.21.15.88:0/3138062716 conn(0x55d43d2dc3c0 msgr2=0x55d43d6d57f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3138062716 shutdown_connections 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.568+0000 7f4cca7c8740 1 -- 172.21.15.88:0/3138062716 wait complete. 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.569+0000 7f4cca7c8740 1 -- start start 2024-09-21T07:55:29.572 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.569+0000 7f4cca7c8740 1 Processor -- start 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d7525b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2c7590 0x55d43d6d57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6d5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d43d6b1d50 con 0x55d43d2e8270 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x55d43d6b5c80 con 0x55d43d2c7590 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x55d43d6b2370 con 0x55d43d2bf3b0 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cca7c8740 10 client.0 _ll_register_callbacks cb 0x7f4ca405ef40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6d5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6d5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:49986/0 (socket says 172.21.15.88:49986) 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 -- 172.21.15.88:0/1507052880 learned_addr learned my addr 172.21.15.88:0/1507052880 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cb3e00640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2c7590 0x55d43d6d57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d7525b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 -- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 msgr2=0x55d43d7525b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d7525b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 -- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2c7590 msgr2=0x55d43d6d57f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.571+0000 7f4cc1600640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2c7590 0x55d43d6d57f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4cc1600640 1 -- 172.21.15.88:0/1507052880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d43d68e220 con 0x55d43d2e8270 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4cc1600640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6d5d30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006010 tx=0x7f4cb4030b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827357a53b33160c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4ca9600640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-09-21T07:55:29.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4ca9600640 1 -- 172.21.15.88:0/1507052880 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb4034040 con 0x55d43d2e8270 2024-09-21T07:55:29.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4ca9600640 1 -- 172.21.15.88:0/1507052880 <== mon.0 v2:172.21.15.88:3300/0 2 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f4cb4004240 con 0x55d43d2e8270 2024-09-21T07:55:29.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ca4067580 con 0x55d43d2e8270 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4ca9600640 1 -- 172.21.15.88:0/1507052880 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) ==== 24+0+0 (secure 0 0 0) 0x7f4cb4050050 con 0x55d43d2e8270 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4cca7c8740 10 client.9270 fetch_fsmap learned FSMap version 118 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.572+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) -- 0x55d43d6ca530 con 0x55d43d2e8270 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4ca9600640 1 -- 172.21.15.88:0/1507052880 <== mon.0 v2:172.21.15.88:3300/0 4 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ca4067580 con 0x55d43d2e8270 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4ca9600640 1 -- 172.21.15.88:0/1507052880 <== mon.0 v2:172.21.15.88:3300/0 5 ==== fsmap(e 118) ==== 3502+0+0 (secure 0 0 0) 0x7f4cb4053020 con 0x55d43d2e8270 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cca7c8740 10 client.9270 fetch_fsmap finished waiting for FSMap version 118 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cca7c8740 10 client.9270 resolve_mds: resolved name 'c' to daemon mds.c 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cca7c8740 20 client.9270 populate_metadata read hostname 'smithi088' 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d6c9ea0 0x55d43d6e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cca7c8740 4 client.9270 mds_command: new command op to 9147 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 --> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] -- command(tid 0: {"prefix": "status"}) -- 0x55d43d6cf1e0 con 0x55d43d6c9ea0 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d6c9ea0 0x55d43d6e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4cc0c00640 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d6c9ea0 0x55d43d6e4e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=270e152efee9d7f2 server_cookie=cf43f0eeadf11588 in_seq=0 out_seq=0 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.573+0000 7f4ca9600640 10 client.9270 ms_handle_connect on v2:172.21.15.88:6834/661983519 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4ca9600640 1 -- 172.21.15.88:0/1507052880 <== mds.0 v2:172.21.15.88:6834/661983519 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55d43d6cf1e0 con 0x55d43d6c9ea0 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4ca9600640 10 client.9270 handle_command_reply: tid=0 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 client.9270 shutdown 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 msgr2=0x55d43d6d5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6d5d30 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7f4cb4006010 tx=0x7f4cb4030b90 comp rx=0 tx=0).stop 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 shutdown_connections 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:6834/661983519,v1:172.21.15.88:6835/661983519] conn(0x55d43d6c9ea0 0x55d43d6e4e60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d43d2e8270 0x55d43d6d5d30 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x55d43d2c7590 0x55d43d6d57f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 --2- 172.21.15.88:0/1507052880 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x55d43d2bf3b0 0x55d43d7525b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 >> 172.21.15.88:0/1507052880 conn(0x55d43d2dc3c0 msgr2=0x55d43d6b5e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4ca9600640 0 client.9270 ms_handle_reset on v2:172.21.15.88:6834/661983519 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 shutdown_connections 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 1 -- 172.21.15.88:0/1507052880 wait complete. 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.574+0000 7f4cca7c8740 20 client.9270 trim_cache size 0 max 16384 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout: "cluster_fsid": "c1afec4d-fdf7-414b-83b4-ffa85a98fc94", 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout: "whoami": 0, 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout: "id": 9147, 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout: "want_state": "up:active", 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout: "state": "up:active", 2024-09-21T07:55:29.580 INFO:teuthology.orchestra.run.smithi088.stdout: "fs_name": "cephfs", 2024-09-21T07:55:29.581 INFO:teuthology.orchestra.run.smithi088.stdout: "rank_uptime": 6.6952711090000001, 2024-09-21T07:55:29.581 INFO:teuthology.orchestra.run.smithi088.stdout: "mdsmap_epoch": 118, 2024-09-21T07:55:29.581 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch": 168, 2024-09-21T07:55:29.581 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_epoch_barrier": 168, 2024-09-21T07:55:29.581 INFO:teuthology.orchestra.run.smithi088.stdout: "uptime": 12.36336097 2024-09-21T07:55:29.581 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4c980795b0 msgr2=0x7f4c9807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4c980795b0 0x7f4c9807bac0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f4cac00ab40 tx=0x7f4cac00a520 comp rx=0 tx=0).stop 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 msgr2=0x7f4cbc1ec290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 0x7f4cbc1ec290 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4ca400b3f0 tx=0x7f4ca400b8b0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 shutdown_connections 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4c980795b0 0x7f4c9807bac0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4cbc15a0f0 0x7f4cbc1f5740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cbc0b2f60 0x7f4cbc1ec290 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 --2- 172.21.15.88:0/2239376060 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4cbc0ad9c0 0x7f4cbc072e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 >> 172.21.15.88:0/2239376060 conn(0x7f4cbc0b1e40 msgr2=0x7f4cbc0b35c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 shutdown_connections 2024-09-21T07:55:29.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.575+0000 7f4cc2000640 1 -- 172.21.15.88:0/2239376060 wait complete. 2024-09-21T07:55:29.598 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 -- 172.21.15.88:0/4061005124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b015d760 msgr2=0x7f02b01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 --2- 172.21.15.88:0/4061005124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b015d760 0x7f02b01535c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f02a0009830 tx=0x7f02a002a410 comp rx=0 tx=0).stop 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 -- 172.21.15.88:0/4061005124 shutdown_connections 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 --2- 172.21.15.88:0/4061005124 >> v2:172.21.15.175:3300/0 conn(0x7f02b00ad0e0 0x7f02b00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 --2- 172.21.15.88:0/4061005124 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b015d760 0x7f02b01535c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 -- 172.21.15.88:0/4061005124 >> 172.21.15.88:0/4061005124 conn(0x7f02b006d730 msgr2=0x7f02b00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 -- 172.21.15.88:0/4061005124 shutdown_connections 2024-09-21T07:55:29.819 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.816+0000 7f02b6a00640 1 -- 172.21.15.88:0/4061005124 wait complete. 2024-09-21T07:55:29.820 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.818+0000 7f02b6a00640 1 Processor -- start 2024-09-21T07:55:29.821 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.819+0000 7f02b6a00640 1 -- start start 2024-09-21T07:55:29.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b00ad0e0 0x7f02b015e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 0x7f02b0156190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02b6a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02b0160310 0x7f02b0146450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02b6a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02b006f220 con 0x7f02b015d760 2024-09-21T07:55:29.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02b6a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f02b006f0a0 con 0x7f02b0160310 2024-09-21T07:55:29.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02b6a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f02b00b4c20 con 0x7f02b00ad0e0 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02af400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 0x7f02b0156190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02af400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 0x7f02b0156190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50002/0 (socket says 172.21.15.88:50002) 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.820+0000 7f02af400640 1 -- 172.21.15.88:0/3233826707 learned_addr learned my addr 172.21.15.88:0/3233826707 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02afe00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b00ad0e0 0x7f02b015e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02b4a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02b0160310 0x7f02b0146450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02af400640 1 -- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b00ad0e0 msgr2=0x7f02b015e0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.824 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02af400640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b00ad0e0 0x7f02b015e0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02af400640 1 -- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02b0160310 msgr2=0x7f02b0146450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02af400640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02b0160310 0x7f02b0146450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02af400640 1 -- 172.21.15.88:0/3233826707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02b0156e10 con 0x7f02b015d760 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02af400640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 0x7f02b0156190 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f029c00e790 tx=0x7f029c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c00eea0 con 0x7f02b015d760 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f029c00ccc0 con 0x7f02b015d760 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.821+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02b01f9220 con 0x7f02b015d760 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.822+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f029c004280 con 0x7f02b015d760 2024-09-21T07:55:29.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.822+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02b01f97e0 con 0x7f02b015d760 2024-09-21T07:55:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.823+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f02b0156b80 con 0x7f02b015d760 2024-09-21T07:55:29.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.824+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f029c0035f0 con 0x7f02b015d760 2024-09-21T07:55:29.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.825+0000 7f02a7e00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02900795b0 0x7f029007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.826+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f029c09a1f0 con 0x7f02b015d760 2024-09-21T07:55:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.826+0000 7f02afe00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02900795b0 0x7f029007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:29.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.826+0000 7f02afe00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02900795b0 0x7f029007bac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f02a0034000 tx=0x7f02a0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:29.832 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:29.830+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f029c063de0 con 0x7f02b015d760 2024-09-21T07:55:30.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.057+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f02b0160fa0 con 0x7f02b015d760 2024-09-21T07:55:30.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.057+0000 7f02a7e00640 1 -- 172.21.15.88:0/3233826707 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v118) ==== 53+0+73 (secure 0 0 0) 0x7f029c0636e0 con 0x7f02b015d760 2024-09-21T07:55:30.060 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:55:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02900795b0 msgr2=0x7f029007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:30.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02900795b0 0x7f029007bac0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f02a0034000 tx=0x7f02a0004730 comp rx=0 tx=0).stop 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 msgr2=0x7f02b0156190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 0x7f02b0156190 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f029c00e790 tx=0x7f029c00ec50 comp rx=0 tx=0).stop 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 shutdown_connections 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02900795b0 0x7f029007bac0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f02b0160310 0x7f02b0146450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02b015d760 0x7f02b0156190 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 --2- 172.21.15.88:0/3233826707 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f02b00ad0e0 0x7f02b015e0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 >> 172.21.15.88:0/3233826707 conn(0x7f02b006d730 msgr2=0x7f02b0160720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 shutdown_connections 2024-09-21T07:55:30.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30.060+0000 7f02b6a00640 1 -- 172.21.15.88:0/3233826707 wait complete. 2024-09-21T07:55:30.084 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:55:30.084 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:55:30.084 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi088.front.sepia.ceph.com 2024-09-21T07:55:30.085 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T07:55:30.085 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T07:55:30.085 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:55:30.085 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:55:30.085 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:55:30.085 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:55:30.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.085 DEBUG:teuthology.orchestra.run.smithi088:> ip addr 2024-09-21T07:55:30.121 INFO:teuthology.orchestra.run.smithi088.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:55:30.121 INFO:teuthology.orchestra.run.smithi088.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 ::1/128 scope host 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f0 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:d8:a8:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout: altname enp7s0f1 2024-09-21T07:55:30.122 INFO:teuthology.orchestra.run.smithi088.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2c brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether 0c:c4:7a:8f:d4:2d brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: inet 172.21.15.88/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft 42164sec preferred_lft 42164sec 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ec4:7aff:fe8f:d42d/64 scope link noprefixroute 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:55:30.123 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::eca8:10ff:febb:4dd9/64 scope link 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether da:2e:42:1b:4a:52 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::d82e:42ff:fe1b:4a52/64 scope link 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: link/ether ee:dd:1e:d9:a7:83 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: inet6 fe80::ecdd:1eff:fed9:a783/64 scope link 2024-09-21T07:55:30.124 INFO:teuthology.orchestra.run.smithi088.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:30.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.125 DEBUG:teuthology.orchestra.run.smithi088:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:55:30.215 INFO:teuthology.orchestra.run.smithi088.stdout:1 2024-09-21T07:55:30.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.218 DEBUG:teuthology.orchestra.run.smithi088:> route 2024-09-21T07:55:30.255 INFO:teuthology.orchestra.run.smithi088.stdout:Kernel IP routing table 2024-09-21T07:55:30.255 INFO:teuthology.orchestra.run.smithi088.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:55:30.255 INFO:teuthology.orchestra.run.smithi088.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:55:30.255 INFO:teuthology.orchestra.run.smithi088.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:55:30.255 INFO:teuthology.orchestra.run.smithi088.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:55:30.255 DEBUG:teuthology.orchestra.run.smithi088:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:55:30.255 DEBUG:teuthology.orchestra.run.smithi088:> set -e 2024-09-21T07:55:30.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:55:30.255 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:55:30.256 DEBUG:teuthology.orchestra.run.smithi088:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:55:30.256 DEBUG:teuthology.orchestra.run.smithi088:> ') 2024-09-21T07:55:30.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:55:30.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:55:30.449 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.449 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:55:30.485 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:55:30.485 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:55:30.485 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:55:30.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.485 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link set brx.0 up 2024-09-21T07:55:30.582 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:55:30.582 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T07:55:30.617 INFO:teuthology.orchestra.run.smithi088.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T07:55:30.617 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:55:30.617 DEBUG:teuthology.orchestra.run.smithi088:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:55:30.692 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:55:30.692 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.692 DEBUG:teuthology.orchestra.run.smithi088:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:55:30.886 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:55:30.886 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:55:30.886 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:55:30.908 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:55:30.908 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:55:30.908 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:55:30.908 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:55:30.908 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs". 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:invalid new device string format 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:55:30.936 INFO:teuthology.orchestra.run.smithi088.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,key=0,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:55:30.937 INFO:teuthology.orchestra.run.smithi088.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T07:55:30.937 INFO:teuthology.orchestra.run.smithi088.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:55:30.938 INFO:teuthology.orchestra.run.smithi088.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:55:30.938 INFO:teuthology.orchestra.run.smithi088.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:55:30.938 INFO:teuthology.orchestra.run.smithi088.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:55:30.940 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:55:30.940 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:30.941 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T07:55:30.996 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:55:31.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:55:31.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:55:31.091 INFO:teuthology.orchestra.run.smithi088.stdout:client9279 2024-09-21T07:55:31.092 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:31.092 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:31.127 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2024-09-21T07:55:31.386 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 -- 172.21.15.88:0/3705443433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f030815a0f0 msgr2=0x7f030815d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 --2- 172.21.15.88:0/3705443433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f030815a0f0 0x7f030815d670 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f0308051590 tx=0x7f030002a980 comp rx=0 tx=0).stop 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 -- 172.21.15.88:0/3705443433 shutdown_connections 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 --2- 172.21.15.88:0/3705443433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f030815a0f0 0x7f030815d670 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 --2- 172.21.15.88:0/3705443433 >> v2:172.21.15.175:3300/0 conn(0x7f03080ad9c0 0x7f03080af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 -- 172.21.15.88:0/3705443433 >> 172.21.15.88:0/3705443433 conn(0x7f03080b1e40 msgr2=0x7f03080b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 -- 172.21.15.88:0/3705443433 shutdown_connections 2024-09-21T07:55:31.387 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.384+0000 7f030ca00640 1 -- 172.21.15.88:0/3705443433 wait complete. 2024-09-21T07:55:31.388 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.386+0000 7f030ca00640 1 Processor -- start 2024-09-21T07:55:31.389 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.387+0000 7f030ca00640 1 -- start start 2024-09-21T07:55:31.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f030ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 0x7f03081517d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f030ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03080b2f60 0x7f0308156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f030ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f030815a0f0 0x7f03080cee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f030ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f030806f290 con 0x7f03080ad9c0 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f030ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f030815a5e0 con 0x7f030815a0f0 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f030ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f030806f410 con 0x7f03080b2f60 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f0307400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 0x7f03081517d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f0307400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 0x7f03081517d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50016/0 (socket says 172.21.15.88:50016) 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.388+0000 7f0307400640 1 -- 172.21.15.88:0/1671166969 learned_addr learned my addr 172.21.15.88:0/1671166969 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307e00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f030815a0f0 0x7f03080cee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f02fea00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03080b2f60 0x7f0308156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307400640 1 -- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03080b2f60 msgr2=0x7f0308156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:31.392 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307400640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03080b2f60 0x7f0308156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307400640 1 -- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f030815a0f0 msgr2=0x7f03080cee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307400640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f030815a0f0 0x7f03080cee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307400640 1 -- 172.21.15.88:0/1671166969 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0308151d10 con 0x7f03080ad9c0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f02fea00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03080b2f60 0x7f0308156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f0307400640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 0x7f03081517d0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f02f800d770 tx=0x7f02f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f8017070 con 0x7f03080ad9c0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f02f8009cc0 con 0x7f03080ad9c0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03080cdd60 con 0x7f03080ad9c0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.389+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f80106c0 con 0x7f03080ad9c0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.390+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03080ce320 con 0x7f03080ad9c0 2024-09-21T07:55:31.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.390+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f030806f930 con 0x7f03080ad9c0 2024-09-21T07:55:31.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.392+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f02f801a070 con 0x7f03080ad9c0 2024-09-21T07:55:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.393+0000 7f02ffe00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02e40795e0 0x7f02e407baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.393+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f02f8099940 con 0x7f03080ad9c0 2024-09-21T07:55:31.395 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.393+0000 7f02fea00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02e40795e0 0x7f02e407baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:31.396 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.394+0000 7f02fea00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02e40795e0 0x7f02e407baf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f030809b3c0 tx=0x7f02f0008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:31.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.397+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f02f80635b0 con 0x7f03080ad9c0 2024-09-21T07:55:31.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.616+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f03080707e0 con 0x7f03080ad9c0 2024-09-21T07:55:31.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.617+0000 7f02ffe00640 1 -- 172.21.15.88:0/1671166969 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v118) ==== 53+0+73 (secure 0 0 0) 0x7f02f8062eb0 con 0x7f03080ad9c0 2024-09-21T07:55:31.619 INFO:teuthology.orchestra.run.smithi088.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02e40795e0 msgr2=0x7f02e407baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02e40795e0 0x7f02e407baf0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f030809b3c0 tx=0x7f02f0008040 comp rx=0 tx=0).stop 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 msgr2=0x7f03081517d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 0x7f03081517d0 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f02f800d770 tx=0x7f02f800dc30 comp rx=0 tx=0).stop 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 shutdown_connections 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f02e40795e0 0x7f02e407baf0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f03080b2f60 0x7f0308156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f030815a0f0 0x7f03080cee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 --2- 172.21.15.88:0/1671166969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f03080ad9c0 0x7f03081517d0 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 >> 172.21.15.88:0/1671166969 conn(0x7f03080b1e40 msgr2=0x7f030815b1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 shutdown_connections 2024-09-21T07:55:31.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:31.619+0000 7f030ca00640 1 -- 172.21.15.88:0/1671166969 wait complete. 2024-09-21T07:55:31.643 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-09-21T07:55:31.643 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T07:55:31.643 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi175.front.sepia.ceph.com 2024-09-21T07:55:31.643 INFO:tasks.cephfs.mount:self.client.name = client.1 2024-09-21T07:55:31.643 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2024-09-21T07:55:31.644 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-09-21T07:55:31.644 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2024-09-21T07:55:31.644 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T07:55:31.644 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:55:31.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:31.644 DEBUG:teuthology.orchestra.run.smithi175:> ip addr 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 ::1/128 scope host 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ee brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:31.685 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f0 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:d9:39:ef brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: altname enp7s0f1 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f0 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 0c:c4:7a:88:71:f1 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: inet 172.21.15.175/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft 42113sec preferred_lft 42113sec 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::ec4:7aff:fe88:71f1/64 scope link noprefixroute 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff 2024-09-21T07:55:31.686 INFO:teuthology.orchestra.run.smithi175.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::407f:e3ff:fe5e:2169/64 scope link 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether b2:c6:4a:b5:c3:f6 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::b0c6:4aff:feb5:c3f6/64 scope link 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: link/ether 02:e8:76:03:3e:e3 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: inet6 fe80::e8:76ff:fe03:3ee3/64 scope link 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run.smithi175.stdout: valid_lft forever preferred_lft forever 2024-09-21T07:55:31.687 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:31.688 DEBUG:teuthology.orchestra.run.smithi175:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T07:55:31.783 INFO:teuthology.orchestra.run.smithi175.stdout:1 2024-09-21T07:55:31.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:31.787 DEBUG:teuthology.orchestra.run.smithi175:> route 2024-09-21T07:55:31.823 INFO:teuthology.orchestra.run.smithi175.stdout:Kernel IP routing table 2024-09-21T07:55:31.823 INFO:teuthology.orchestra.run.smithi175.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T07:55:31.823 INFO:teuthology.orchestra.run.smithi175.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-09-21T07:55:31.823 INFO:teuthology.orchestra.run.smithi175.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-09-21T07:55:31.824 INFO:teuthology.orchestra.run.smithi175.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T07:55:31.824 DEBUG:teuthology.orchestra.run.smithi175:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T07:55:31.824 DEBUG:teuthology.orchestra.run.smithi175:> set -e 2024-09-21T07:55:31.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-09-21T07:55:31.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-09-21T07:55:31.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-09-21T07:55:31.824 DEBUG:teuthology.orchestra.run.smithi175:> ') 2024-09-21T07:55:31.944 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:55:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T07:55:32.017 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:55:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:55:32.024 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:32.024 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:55:32.059 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:55:32.059 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:55:32.060 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.1' netns for '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:55:32.060 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:32.060 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link set brx.0 up 2024-09-21T07:55:32.155 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:55:32.156 DEBUG:teuthology.orchestra.run.smithi175:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2024-09-21T07:55:32.190 INFO:teuthology.orchestra.run.smithi175.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2024-09-21T07:55:32.191 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T07:55:32.191 DEBUG:teuthology.orchestra.run.smithi175:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:55:32.265 INFO:tasks.cephfs.kernel_mount:mounting using device: :/ 2024-09-21T07:55:32.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:32.266 DEBUG:teuthology.orchestra.run.smithi175:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:55:32.458 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:55:32.458 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:55:32.458 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:55:32.478 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:55:32.478 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:55:32.478 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:55:32.478 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:55:32.478 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=1,mds_namespace=cephfs 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs". 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:invalid new device string format 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:3300,172.21.15.175:3300,172.21.15.175:3301" 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: switching to using v1 address with old syntax 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: resolved to: "172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790" 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.88:6789,172.21.15.175:6789,172.21.15.175:6790:/ 2024-09-21T07:55:32.505 INFO:teuthology.orchestra.run.smithi175.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,mds_namespace=cephfs,key=1,fsid=c1afec4d-fdf7-414b-83b4-ffa85a98fc94" will pass to kernel 2024-09-21T07:55:32.507 INFO:teuthology.orchestra.run.smithi175.stdout:mount: /home/ubuntu/cephtest/mnt.1 does not contain SELinux labels. 2024-09-21T07:55:32.507 INFO:teuthology.orchestra.run.smithi175.stdout: You just mounted a file system that supports labels which does not 2024-09-21T07:55:32.507 INFO:teuthology.orchestra.run.smithi175.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T07:55:32.507 INFO:teuthology.orchestra.run.smithi175.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T07:55:32.507 INFO:teuthology.orchestra.run.smithi175.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T07:55:32.510 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T07:55:32.510 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:32.510 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2024-09-21T07:55:32.566 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest/mnt.1 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T07:55:32.643 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:55:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:55:32.646 INFO:teuthology.orchestra.run.smithi175.stderr:2024-09-21T07:55:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:55:32.654 INFO:teuthology.orchestra.run.smithi175.stdout:client8542 2024-09-21T07:55:32.654 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:32.654 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:32.690 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 -- 172.21.15.88:0/3863401266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58580cadc0 msgr2=0x7f585808b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 --2- 172.21.15.88:0/3863401266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58580cadc0 0x7f585808b1d0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7f5850005fd0 tx=0x7f585002a9a0 comp rx=0 tx=0).stop 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 -- 172.21.15.88:0/3863401266 shutdown_connections 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 --2- 172.21.15.88:0/3863401266 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f58580cadc0 0x7f585808b1d0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 -- 172.21.15.88:0/3863401266 >> 172.21.15.88:0/3863401266 conn(0x7f58580b2220 msgr2=0x7f58580d0060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 -- 172.21.15.88:0/3863401266 shutdown_connections 2024-09-21T07:55:32.920 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.917+0000 7f585ec00640 1 -- 172.21.15.88:0/3863401266 wait complete. 2024-09-21T07:55:32.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.919+0000 7f585ec00640 1 Processor -- start 2024-09-21T07:55:32.921 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.920+0000 7f585ec00640 1 -- start start 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ec00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585816b150 0x7f5858206ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ec00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 0x7f585808a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ec00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58580ad830 0x7f58580adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ec00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5858177230 con 0x7f585816b560 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ec00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f58581770b0 con 0x7f585816b150 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ec00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f58581773b0 con 0x7f58580ad830 2024-09-21T07:55:32.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 0x7f585808a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 0x7f585808a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50036/0 (socket says 172.21.15.88:50036) 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585ce00640 1 -- 172.21.15.88:0/3980186194 learned_addr learned my addr 172.21.15.88:0/3980186194 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.921+0000 7f585d800640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585816b150 0x7f5858206ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585e200640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58580ad830 0x7f58580adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585ce00640 1 -- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58580ad830 msgr2=0x7f58580adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585ce00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58580ad830 0x7f58580adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585ce00640 1 -- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585816b150 msgr2=0x7f5858206ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585ce00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585816b150 0x7f5858206ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585ce00640 1 -- 172.21.15.88:0/3980186194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58580ae180 con 0x7f585816b560 2024-09-21T07:55:32.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f585ce00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 0x7f585808a5f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f584400c4f0 tx=0x7f584400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:32.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844017070 con 0x7f585816b560 2024-09-21T07:55:32.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f58440035f0 con 0x7f585816b560 2024-09-21T07:55:32.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.922+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5844003a40 con 0x7f585816b560 2024-09-21T07:55:32.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.923+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58580ae920 con 0x7f585816b560 2024-09-21T07:55:32.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.923+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5858169330 con 0x7f585816b560 2024-09-21T07:55:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.924+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f585806f710 con 0x7f585816b560 2024-09-21T07:55:32.927 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.925+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f5844003be0 con 0x7f585816b560 2024-09-21T07:55:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.926+0000 7f5855600640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f58380795b0 0x7f583807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.926+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f5844098be0 con 0x7f585816b560 2024-09-21T07:55:32.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.926+0000 7f585d800640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f58380795b0 0x7f583807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:32.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.927+0000 7f585d800640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f58380795b0 0x7f583807bac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f584c000f60 tx=0x7f584c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:32.932 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:32.931+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f58580ae180 con 0x7f585816b560 2024-09-21T07:55:33.159 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.157+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) -- 0x7f58580aeaf0 con 0x7f585816b560 2024-09-21T07:55:33.943 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.941+0000 7f5855600640 1 -- 172.21.15.88:0/3980186194 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v119) ==== 136+0+0 (secure 0 0 0) 0x7f584409b020 con 0x7f585816b560 2024-09-21T07:55:33.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f58380795b0 msgr2=0x7f583807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f58380795b0 0x7f583807bac0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f584c000f60 tx=0x7f584c009040 comp rx=0 tx=0).stop 2024-09-21T07:55:33.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 msgr2=0x7f585808a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:33.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 0x7f585808a5f0 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7f584400c4f0 tx=0x7f584400c9b0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 shutdown_connections 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f58380795b0 0x7f583807bac0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f58580ad830 0x7f58580adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f585816b560 0x7f585808a5f0 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 --2- 172.21.15.88:0/3980186194 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f585816b150 0x7f5858206ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 >> 172.21.15.88:0/3980186194 conn(0x7f58580b2220 msgr2=0x7f58581778e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 shutdown_connections 2024-09-21T07:55:33.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:33.944+0000 7f585ec00640 1 -- 172.21.15.88:0/3980186194 wait complete. 2024-09-21T07:55:33.970 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2024-09-21T07:55:34.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 -- 172.21.15.88:0/4175518090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 msgr2=0x7fa79015d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 --2- 172.21.15.88:0/4175518090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa79015d240 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fa790071220 tx=0x7fa78802a980 comp rx=0 tx=0).stop 2024-09-21T07:55:34.198 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 -- 172.21.15.88:0/4175518090 shutdown_connections 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 --2- 172.21.15.88:0/4175518090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa79015d240 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 --2- 172.21.15.88:0/4175518090 >> v2:172.21.15.175:3300/0 conn(0x7fa7900705b0 0x7fa7900709c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 --2- 172.21.15.88:0/4175518090 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 0x7fa790156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 -- 172.21.15.88:0/4175518090 >> 172.21.15.88:0/4175518090 conn(0x7fa79006d3d0 msgr2=0x7fa7900b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 -- 172.21.15.88:0/4175518090 shutdown_connections 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.189+0000 7fa795000640 1 -- 172.21.15.88:0/4175518090 wait complete. 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.192+0000 7fa795000640 1 Processor -- start 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.192+0000 7fa795000640 1 -- start start 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa795000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7900705b0 0x7fa79015a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa795000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 0x7fa7901f9250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa795000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa7901f9790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa795000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900cce40 con 0x7fa79015ce30 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa795000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fa7900cccc0 con 0x7fa7900705b0 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa795000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fa7900ccfc0 con 0x7fa7900cfc80 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa78e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa7901f9790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa78e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa7901f9790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50054/0 (socket says 172.21.15.88:50054) 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.193+0000 7fa78e800640 1 -- 172.21.15.88:0/3132800229 learned_addr learned my addr 172.21.15.88:0/3132800229 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78de00640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7900705b0 0x7fa79015a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78d400640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 0x7fa7901f9250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78e800640 1 -- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 msgr2=0x7fa7901f9250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.200 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78e800640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 0x7fa7901f9250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78e800640 1 -- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7900705b0 msgr2=0x7fa79015a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78e800640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7900705b0 0x7fa79015a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78e800640 1 -- 172.21.15.88:0/3132800229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7900c6180 con 0x7fa79015ce30 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78de00640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7900705b0 0x7fa79015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78d400640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 0x7fa7901f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa78e800640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa7901f9790 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa788033040 tx=0x7fa788003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788034040 con 0x7fa79015ce30 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fa788004540 con 0x7fa79015ce30 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa788045480 con 0x7fa79015ce30 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.194+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7900c6460 con 0x7fa79015ce30 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.195+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7900c6a20 con 0x7fa79015ce30 2024-09-21T07:55:34.201 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.196+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7900cd550 con 0x7fa79015ce30 2024-09-21T07:55:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.197+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7fa788037020 con 0x7fa79015ce30 2024-09-21T07:55:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.198+0000 7fa786000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa7700795b0 0x7fa77007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.202 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.198+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7fa7880bb1c0 con 0x7fa79015ce30 2024-09-21T07:55:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.198+0000 7fa78de00640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa7700795b0 0x7fa77007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.199+0000 7fa78de00640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa7700795b0 0x7fa77007bac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa79009e510 tx=0x7fa780005f10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:34.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.202+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fa790070f00 con 0x7fa79015ce30 2024-09-21T07:55:34.413 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.411+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) -- 0x7fa7900afa70 con 0x7fa7700795b0 2024-09-21T07:55:34.415 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.413+0000 7fa786000640 1 -- 172.21.15.88:0/3132800229 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+41 (secure 0 0 0) 0x7fa7900afa70 con 0x7fa7700795b0 2024-09-21T07:55:34.415 INFO:teuthology.orchestra.run.smithi088.stdout:[ 2024-09-21T07:55:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: { 2024-09-21T07:55:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: "name": "cephfs" 2024-09-21T07:55:34.415 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-09-21T07:55:34.415 INFO:teuthology.orchestra.run.smithi088.stdout:] 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa7700795b0 msgr2=0x7fa77007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa7700795b0 0x7fa77007bac0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa79009e510 tx=0x7fa780005f10 comp rx=0 tx=0).stop 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 msgr2=0x7fa7901f9790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa7901f9790 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fa788033040 tx=0x7fa788003f10 comp rx=0 tx=0).stop 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 shutdown_connections 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fa7700795b0 0x7fa77007bac0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fa7900cfc80 0x7fa7901f9250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.415+0000 7fa795000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa79015ce30 0x7fa7901f9790 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.416+0000 7fa795000640 1 --2- 172.21.15.88:0/3132800229 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fa7900705b0 0x7fa79015a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.416+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 >> 172.21.15.88:0/3132800229 conn(0x7fa79006d3d0 msgr2=0x7fa79015ebc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.416+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 shutdown_connections 2024-09-21T07:55:34.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.416+0000 7fa795000640 1 -- 172.21.15.88:0/3132800229 wait complete. 2024-09-21T07:55:34.440 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 -- 172.21.15.88:0/4190796854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb4145450 msgr2=0x7f3cb406f180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 --2- 172.21.15.88:0/4190796854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb4145450 0x7f3cb406f180 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4005fd0 tx=0x7f3ca402a5d0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 -- 172.21.15.88:0/4190796854 shutdown_connections 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 --2- 172.21.15.88:0/4190796854 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb40cd610 0x7f3cb40cda20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 --2- 172.21.15.88:0/4190796854 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb4145450 0x7f3cb406f180 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 -- 172.21.15.88:0/4190796854 >> 172.21.15.88:0/4190796854 conn(0x7f3cb406d1c0 msgr2=0x7f3cb406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 -- 172.21.15.88:0/4190796854 shutdown_connections 2024-09-21T07:55:34.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.660+0000 7f3cb9800640 1 -- 172.21.15.88:0/4190796854 wait complete. 2024-09-21T07:55:34.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.662+0000 7f3cb9800640 1 Processor -- start 2024-09-21T07:55:34.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.663+0000 7f3cb9800640 1 -- start start 2024-09-21T07:55:34.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb9800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 0x7f3cb415c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb9800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb40cd610 0x7f3cb4140110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.666 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb9800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb4145450 0x7f3cb41f4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb9800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb40ce2f0 con 0x7f3cb40735e0 2024-09-21T07:55:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb9800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3cb40ce170 con 0x7f3cb40cd610 2024-09-21T07:55:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb9800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3cb40ce470 con 0x7f3cb4145450 2024-09-21T07:55:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 0x7f3cb415c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb2a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 0x7f3cb415c490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50068/0 (socket says 172.21.15.88:50068) 2024-09-21T07:55:34.667 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.664+0000 7f3cb2a00640 1 -- 172.21.15.88:0/3884675695 learned_addr learned my addr 172.21.15.88:0/3884675695 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb3400640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb4145450 0x7f3cb41f4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3caa000640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb40cd610 0x7f3cb4140110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb2a00640 1 -- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb4145450 msgr2=0x7f3cb41f4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb2a00640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb4145450 0x7f3cb41f4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb2a00640 1 -- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb40cd610 msgr2=0x7f3cb4140110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb2a00640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb40cd610 0x7f3cb4140110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb2a00640 1 -- 172.21.15.88:0/3884675695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3cb413eff0 con 0x7f3cb40735e0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb3400640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb4145450 0x7f3cb41f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3caa000640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb40cd610 0x7f3cb4140110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb2a00640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 0x7f3cb415c490 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4034040 tx=0x7f3ca40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca403e030 con 0x7f3cb40735e0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3ca4003ae0 con 0x7f3cb40735e0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4035040 con 0x7f3cb40735e0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.665+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3cb413f2d0 con 0x7f3cb40735e0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.666+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3cb415de20 con 0x7f3cb40735e0 2024-09-21T07:55:34.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.667+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3cb40cda20 con 0x7f3cb40735e0 2024-09-21T07:55:34.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.668+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f3ca4041030 con 0x7f3cb40735e0 2024-09-21T07:55:34.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.669+0000 7f3cab400640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3c900795b0 0x7f3c9007bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:34.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.669+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f3ca40ba0e0 con 0x7f3cb40735e0 2024-09-21T07:55:34.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.669+0000 7f3caa000640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3c900795b0 0x7f3c9007bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:34.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.670+0000 7f3caa000640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3c900795b0 0x7f3c9007bac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4073b60 tx=0x7f3c9c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:34.675 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.673+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3ca4083de0 con 0x7f3cb40735e0 2024-09-21T07:55:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.862+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) -- 0x7f3cb4145be0 con 0x7f3cb40735e0 2024-09-21T07:55:34.864 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.862+0000 7f3cab400640 1 -- 172.21.15.88:0/3884675695 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) ==== 122+0+0 (secure 0 0 0) 0x7f3ca40836e0 con 0x7f3cb40735e0 2024-09-21T07:55:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3c900795b0 msgr2=0x7f3c9007bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3c900795b0 0x7f3c9007bac0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cb4073b60 tx=0x7f3c9c009040 comp rx=0 tx=0).stop 2024-09-21T07:55:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 msgr2=0x7f3cb415c490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 0x7f3cb415c490 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4034040 tx=0x7f3ca40035f0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 shutdown_connections 2024-09-21T07:55:34.867 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3c900795b0 0x7f3c9007bac0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3cb4145450 0x7f3cb41f4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3cb40cd610 0x7f3cb4140110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 --2- 172.21.15.88:0/3884675695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3cb40735e0 0x7f3cb415c490 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 >> 172.21.15.88:0/3884675695 conn(0x7f3cb406d1c0 msgr2=0x7f3cb40b00f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 shutdown_connections 2024-09-21T07:55:34.868 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:34.865+0000 7f3cb9800640 1 -- 172.21.15.88:0/3884675695 wait complete. 2024-09-21T07:55:34.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:34.890 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo mkdir -p ./volumes/subvol_grp65700/subvol96590) 2024-09-21T07:55:34.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:34.954 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec getfattr --only-values -n ceph.dir.layout.pool .) 2024-09-21T07:55:34.991 INFO:teuthology.orchestra.run.smithi088.stdout:cephfs_data 2024-09-21T07:55:34.991 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:34.991 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo setfattr -n ceph.dir.layout.pool -v cephfs_data ./volumes/subvol_grp65700/subvol96590) 2024-09-21T07:55:35.091 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:35.092 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/mnt.0/./volumes/subvol_grp65700/subvol96590/.meta 2024-09-21T07:55:35.151 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol96590 subvol_grp65700 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 -- 172.21.15.88:0/471743618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0af100 msgr2=0x7f0aac0af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 --2- 172.21.15.88:0/471743618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0af100 0x7f0aac0af510 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4005fd0 tx=0x7f0aa402aa90 comp rx=0 tx=0).stop 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 -- 172.21.15.88:0/471743618 shutdown_connections 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 --2- 172.21.15.88:0/471743618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0af100 0x7f0aac0af510 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 --2- 172.21.15.88:0/471743618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac06d4b0 0x7f0aac0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 -- 172.21.15.88:0/471743618 >> 172.21.15.88:0/471743618 conn(0x7f0aac0b2540 msgr2=0x7f0aac0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 -- 172.21.15.88:0/471743618 shutdown_connections 2024-09-21T07:55:35.418 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.415+0000 7f0ab1400640 1 -- 172.21.15.88:0/471743618 wait complete. 2024-09-21T07:55:35.419 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.417+0000 7f0ab1400640 1 Processor -- start 2024-09-21T07:55:35.420 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.418+0000 7f0ab1400640 1 -- start start 2024-09-21T07:55:35.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0ab1400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0af100 0x7f0aac0ad320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0ab1400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 0x7f0aac1f2790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.422 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0ab1400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b1260 0x7f0aac0b1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0ab1400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac0b3670 con 0x7f0aac0cc830 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0ab1400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f0aac0b34f0 con 0x7f0aac0b1260 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0ab1400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f0aac0b37f0 con 0x7f0aac0af100 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0aa1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 0x7f0aac1f2790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0aa1e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 0x7f0aac1f2790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:50076/0 (socket says 172.21.15.88:50076) 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.419+0000 7f0aa1e00640 1 -- 172.21.15.88:0/3590362412 learned_addr learned my addr 172.21.15.88:0/3590362412 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aaa800640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0af100 0x7f0aac0ad320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aab200640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b1260 0x7f0aac0b1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa1e00640 1 -- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0af100 msgr2=0x7f0aac0ad320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa1e00640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0af100 0x7f0aac0ad320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa1e00640 1 -- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b1260 msgr2=0x7f0aac0b1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa1e00640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b1260 0x7f0aac0b1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa1e00640 1 -- 172.21.15.88:0/3590362412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aac0b1bb0 con 0x7f0aac0cc830 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa1e00640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 0x7f0aac1f2790 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d700 tx=0x7f0a9400dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:35.423 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94017070 con 0x7f0aac0cc830 2024-09-21T07:55:35.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f0a94004310 con 0x7f0aac0cc830 2024-09-21T07:55:35.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aac1f8880 con 0x7f0aac0cc830 2024-09-21T07:55:35.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a94009a80 con 0x7f0aac0cc830 2024-09-21T07:55:35.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.420+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aac1f8b20 con 0x7f0aac0cc830 2024-09-21T07:55:35.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.421+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aac099b10 con 0x7f0aac0cc830 2024-09-21T07:55:35.424 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.423+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f0a94009c20 con 0x7f0aac0cc830 2024-09-21T07:55:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.424+0000 7f0aa3400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a880795e0 0x7f0a8807baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.426 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.424+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f0a940999d0 con 0x7f0aac0cc830 2024-09-21T07:55:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.424+0000 7f0aaa800640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a880795e0 0x7f0a8807baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.427 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.425+0000 7f0aaa800640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a880795e0 0x7f0a8807baf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0015e0 tx=0x7f0a9c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:35.431 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.429+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0a94063600 con 0x7f0aac0cc830 2024-09-21T07:55:35.642 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.640+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol96590", "group_name": "subvol_grp65700", "target": ["mon-mgr", ""]}) -- 0x7f0aac0b3d10 con 0x7f0a880795e0 2024-09-21T07:55:35.649 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:35.647+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:35.649 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:35.647+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:35.649 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:35.647+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:35.649 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:35.647+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:35.649 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:35.647+0000 7faca9000640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:35.689 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.686+0000 7f0aa3400640 1 -- 172.21.15.88:0/3590362412 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f0aac0b3d10 con 0x7f0a880795e0 2024-09-21T07:55:35.689 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp65700/subvol96590 2024-09-21T07:55:35.691 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a880795e0 msgr2=0x7f0a8807baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a880795e0 0x7f0a8807baf0 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0a9c0015e0 tx=0x7f0a9c009040 comp rx=0 tx=0).stop 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 msgr2=0x7f0aac1f2790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 0x7f0aac1f2790 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f0a9400d700 tx=0x7f0a9400dbc0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 shutdown_connections 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0a880795e0 0x7f0a8807baf0 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f0aac0b1260 0x7f0aac0b1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0aac0cc830 0x7f0aac1f2790 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 --2- 172.21.15.88:0/3590362412 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f0aac0af100 0x7f0aac0ad320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 >> 172.21.15.88:0/3590362412 conn(0x7f0aac0b2540 msgr2=0x7f0aac074380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 shutdown_connections 2024-09-21T07:55:35.692 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.689+0000 7f0ab1400640 1 -- 172.21.15.88:0/3590362412 wait complete. 2024-09-21T07:55:35.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs subvol96590 subvol_grp65700 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903fe00640 1 -- 172.21.15.88:0/1527035980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 msgr2=0x7f9024004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903fe00640 1 --2- 172.21.15.88:0/1527035980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9024004c10 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f902c009b50 tx=0x7f9024005830 comp rx=0 tx=0).stop 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903f400640 1 -- 172.21.15.88:0/1527035980 reap_dead start 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903fe00640 1 -- 172.21.15.88:0/1527035980 shutdown_connections 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903fe00640 1 -- 172.21.15.88:0/1527035980 >> 172.21.15.88:0/1527035980 conn(0x7f90400cd460 msgr2=0x7f90400cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903fe00640 1 -- 172.21.15.88:0/1527035980 shutdown_connections 2024-09-21T07:55:35.942 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.939+0000 7f903fe00640 1 -- 172.21.15.88:0/1527035980 wait complete. 2024-09-21T07:55:35.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.942+0000 7f903fe00640 1 Processor -- start 2024-09-21T07:55:35.944 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.942+0000 7f903fe00640 1 -- start start 2024-09-21T07:55:35.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.943+0000 7f903fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9040153d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.943+0000 7f903fe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9040155fc0 0x7f904015e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.943+0000 7f903fe00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90401563d0 0x7f90401f6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.943+0000 7f903fe00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f90400b64c0 con 0x7f9024004800 2024-09-21T07:55:35.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.943+0000 7f903fe00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f90400b6340 con 0x7f9040155fc0 2024-09-21T07:55:35.946 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.943+0000 7f903fe00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f904015d060 con 0x7f90401563d0 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9040153d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9040153d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43474/0 (socket says 172.21.15.88:43474) 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 -- 172.21.15.88:0/295657794 learned_addr learned my addr 172.21.15.88:0/295657794 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903e000640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9040155fc0 0x7f904015e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903f400640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90401563d0 0x7f90401f6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 -- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90401563d0 msgr2=0x7f90401f6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90401563d0 0x7f90401f6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 -- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9040155fc0 msgr2=0x7f904015e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9040155fc0 0x7f904015e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:35.947 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 -- 172.21.15.88:0/295657794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9040156b30 con 0x7f9024004800 2024-09-21T07:55:35.948 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.944+0000 7f903ea00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9040153d30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f9030000fb0 tx=0x7f90300039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:35.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.947+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030015070 con 0x7f9024004800 2024-09-21T07:55:35.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.947+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9030011920 con 0x7f9024004800 2024-09-21T07:55:35.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.947+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9030011d70 con 0x7f9024004800 2024-09-21T07:55:35.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.947+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9040148c10 con 0x7f9024004800 2024-09-21T07:55:35.949 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.947+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f904015e0d0 con 0x7f9024004800 2024-09-21T07:55:35.950 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.948+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f904015df00 con 0x7f9024004800 2024-09-21T07:55:35.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.950+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f9030018070 con 0x7f9024004800 2024-09-21T07:55:35.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.951+0000 7f9036a00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f90200795e0 0x7f902007baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:35.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.951+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(168..168 src has 1..168) ==== 5737+0+0 (secure 0 0 0) 0x7f903009a3e0 con 0x7f9024004800 2024-09-21T07:55:35.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.951+0000 7f903e000640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f90200795e0 0x7f902007baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:35.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.952+0000 7f903e000640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f90200795e0 0x7f902007baf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f902c009b50 tx=0x7f902c02d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:35.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:35.958+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f90400b6670 con 0x7f9024004800 2024-09-21T07:55:36.172 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.170+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "subvol96590", "group_name": "subvol_grp65700", "target": ["mon-mgr", ""]}) -- 0x7f9040149130 con 0x7f90200795e0 2024-09-21T07:55:36.176 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.174+0000 7f9036a00640 1 -- 172.21.15.88:0/295657794 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9040149130 con 0x7f90200795e0 2024-09-21T07:55:36.176 INFO:teuthology.orchestra.run.smithi088.stdout:/volumes/subvol_grp65700/subvol96590 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.176+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f90200795e0 msgr2=0x7f902007baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.176+0000 7f903fe00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f90200795e0 0x7f902007baf0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f902c009b50 tx=0x7f902c02d040 comp rx=0 tx=0).stop 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.176+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 msgr2=0x7f9040153d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9040153d30 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f9030000fb0 tx=0x7f90300039c0 comp rx=0 tx=0).stop 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 shutdown_connections 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f90200795e0 0x7f902007baf0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:36.179 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f90401563d0 0x7f90401f6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9040155fc0 0x7f904015e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 --2- 172.21.15.88:0/295657794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9024004800 0x7f9040153d30 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 >> 172.21.15.88:0/295657794 conn(0x7f90400cd460 msgr2=0x7f904015a830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 shutdown_connections 2024-09-21T07:55:36.180 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:36.177+0000 7f903fe00640 1 -- 172.21.15.88:0/295657794 wait complete. 2024-09-21T07:55:36.204 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:36.204 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec sudo cat ./volumes/_legacy/d591655c135be40e2e27012d8236f723.meta) 2024-09-21T07:55:37.219 INFO:teuthology.orchestra.run.smithi088.stdout:[GLOBAL] 2024-09-21T07:55:37.219 INFO:teuthology.orchestra.run.smithi088.stdout:version = 1 2024-09-21T07:55:37.220 INFO:teuthology.orchestra.run.smithi088.stdout:type = subvolume 2024-09-21T07:55:37.220 INFO:teuthology.orchestra.run.smithi088.stdout:path = /volumes/subvol_grp65700/subvol96590 2024-09-21T07:55:37.220 INFO:teuthology.orchestra.run.smithi088.stdout:state = complete 2024-09-21T07:55:37.220 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:37.223 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume rm cephfs subvol96590 subvol_grp65700 2024-09-21T07:55:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 -- 172.21.15.88:0/2248263743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640af810 msgr2=0x7f76640b4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 --2- 172.21.15.88:0/2248263743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640af810 0x7f76640b4620 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f765c009be0 tx=0x7f765c02a8b0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 -- 172.21.15.88:0/2248263743 shutdown_connections 2024-09-21T07:55:37.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 --2- 172.21.15.88:0/2248263743 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640af810 0x7f76640b4620 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 -- 172.21.15.88:0/2248263743 >> 172.21.15.88:0/2248263743 conn(0x7f76640b2540 msgr2=0x7f76640d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:37.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 -- 172.21.15.88:0/2248263743 shutdown_connections 2024-09-21T07:55:37.449 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.446+0000 7f766b800640 1 -- 172.21.15.88:0/2248263743 wait complete. 2024-09-21T07:55:37.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.448+0000 7f766b800640 1 Processor -- start 2024-09-21T07:55:37.450 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.448+0000 7f766b800640 1 -- start start 2024-09-21T07:55:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f766b800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 0x7f76641f97f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f766b800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640cc830 0x7f76640b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f766b800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76640b1510 0x7f76641ea470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f766b800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76640b3750 con 0x7f76640af810 2024-09-21T07:55:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f766b800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f76640b35d0 con 0x7f76640b1510 2024-09-21T07:55:37.453 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f766b800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f76640b38d0 con 0x7f76640cc830 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 0x7f76641f97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 0x7f76641f97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43488/0 (socket says 172.21.15.88:43488) 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 -- 172.21.15.88:0/1282622924 learned_addr learned my addr 172.21.15.88:0/1282622924 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7669800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76640b1510 0x7f76641ea470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7663e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640cc830 0x7f76640b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 -- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640cc830 msgr2=0x7f76640b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640cc830 0x7f76640b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 -- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76640b1510 msgr2=0x7f76641ea470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76640b1510 0x7f76641ea470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7668e00640 1 -- 172.21.15.88:0/1282622924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76641f5990 con 0x7f76640af810 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7669800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76640b1510 0x7f76641ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:37.454 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.450+0000 7f7663e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640cc830 0x7f76640b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.451+0000 7f7668e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 0x7f76641f97f0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f765800c460 tx=0x7f765800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.451+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76580052d0 con 0x7f76640af810 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.451+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7658005490 con 0x7f76640af810 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.451+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76641f5c70 con 0x7f76640af810 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.451+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7658012870 con 0x7f76640af810 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.451+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76641f6230 con 0x7f76640af810 2024-09-21T07:55:37.455 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.452+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76640d1030 con 0x7f76640af810 2024-09-21T07:55:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.453+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 28) ==== 102359+0+0 (secure 0 0 0) 0x7f7658012a10 con 0x7f76640af810 2024-09-21T07:55:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.454+0000 7f7660c00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f76480795b0 0x7f764807bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.454+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(169..169 src has 1..169) ==== 5824+0+0 (secure 0 0 0) 0x7f76580993b0 con 0x7f76640af810 2024-09-21T07:55:37.456 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.455+0000 7f7663e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f76480795b0 0x7f764807bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:37.457 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.455+0000 7f7663e00640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f76480795b0 0x7f764807bac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7654000c00 tx=0x7f7654009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:37.461 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.459+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f76580630d0 con 0x7f76640af810 2024-09-21T07:55:37.669 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.667+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolume rm", "vol_name": "cephfs", "sub_name": "subvol96590", "group_name": "subvol_grp65700", "target": ["mon-mgr", ""]}) -- 0x7f7664069690 con 0x7f76480795b0 2024-09-21T07:55:37.698 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.696+0000 7f7660c00640 1 -- 172.21.15.88:0/1282622924 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f7664069690 con 0x7f76480795b0 2024-09-21T07:55:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f76480795b0 msgr2=0x7f764807bac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f76480795b0 0x7f764807bac0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f7654000c00 tx=0x7f7654009040 comp rx=0 tx=0).stop 2024-09-21T07:55:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 msgr2=0x7f76641f97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:37.702 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 0x7f76641f97f0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f765800c460 tx=0x7f765800c920 comp rx=0 tx=0).stop 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 shutdown_connections 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f76480795b0 0x7f764807bac0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f76640cc830 0x7f76640b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f76640b1510 0x7f76641ea470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 --2- 172.21.15.88:0/1282622924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76640af810 0x7f76641f97f0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 >> 172.21.15.88:0/1282622924 conn(0x7f76640b2540 msgr2=0x7f7664099820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 shutdown_connections 2024-09-21T07:55:37.703 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37.700+0000 7f766b800640 1 -- 172.21.15.88:0/1282622924 wait complete. 2024-09-21T07:55:37.705 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.703+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.705 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.703+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.705 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.703+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.705 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.703+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.705 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.703+0000 7facacc00640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.723 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.721+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.723 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.721+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.723 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.721+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.723 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.721+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.723 INFO:tasks.ceph.mgr.y.smithi088.stderr:2024-09-21T07:55:37.721+0000 7facac200640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T07:55:37.732 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'stat -c %h /home/ubuntu/cephtest/mnt.0/./volumes/_deleting') 2024-09-21T07:55:37.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T07:55:37.817 INFO:teuthology.orchestra.run.smithi088.stdout:2 2024-09-21T07:55:37.818 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:37 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T07:55:37.826 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup rm cephfs subvol_grp65700 2024-09-21T07:55:38.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 -- 172.21.15.88:0/4268493994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 msgr2=0x7f5cd00ccac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 --2- 172.21.15.88:0/4268493994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd00ccac0 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7f5cd0075230 tx=0x7f5cbc02a560 comp rx=0 tx=0).stop 2024-09-21T07:55:38.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 -- 172.21.15.88:0/4268493994 shutdown_connections 2024-09-21T07:55:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 --2- 172.21.15.88:0/4268493994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cd00cd000 0x7f5cd00af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 --2- 172.21.15.88:0/4268493994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd00ccac0 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 --2- 172.21.15.88:0/4268493994 >> v2:172.21.15.175:3300/0 conn(0x7f5cd00b3f30 0x7f5cd0156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 -- 172.21.15.88:0/4268493994 >> 172.21.15.88:0/4268493994 conn(0x7f5cd006d620 msgr2=0x7f5cd00b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 -- 172.21.15.88:0/4268493994 shutdown_connections 2024-09-21T07:55:38.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.057+0000 7f5cd6200640 1 -- 172.21.15.88:0/4268493994 wait complete. 2024-09-21T07:55:38.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.059+0000 7f5cd6200640 1 Processor -- start 2024-09-21T07:55:38.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.059+0000 7f5cd6200640 1 -- start start 2024-09-21T07:55:38.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd6200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd00b3f30 0x7f5cd01e5ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd6200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cd00cd000 0x7f5cd015f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd6200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd0152860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd6200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd00af790 con 0x7f5cd015d6e0 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd6200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5cd00af610 con 0x7f5cd00b3f30 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd6200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5cd00af910 con 0x7f5cd00cd000 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd0152860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd5800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd0152860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43544/0 (socket says 172.21.15.88:43544) 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd5800640 1 -- 172.21.15.88:0/1223079181 learned_addr learned my addr 172.21.15.88:0/1223079181 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5ccfe00640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cd00cd000 0x7f5cd015f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.061+0000 7f5cd4e00640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd00b3f30 0x7f5cd01e5ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd5800640 1 -- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cd00cd000 msgr2=0x7f5cd015f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd5800640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cd00cd000 0x7f5cd015f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd5800640 1 -- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd00b3f30 msgr2=0x7f5cd01e5ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd5800640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd00b3f30 0x7f5cd01e5ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd5800640 1 -- 172.21.15.88:0/1223079181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cd01f6780 con 0x7f5cd015d6e0 2024-09-21T07:55:38.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd5800640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd0152860 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800e9c0 tx=0x7f5cc800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8004d20 con 0x7f5cd015d6e0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5cc800cb20 con 0x7f5cd015d6e0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cd01e50c0 con 0x7f5cd015d6e0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5cc8005020 con 0x7f5cd015d6e0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.062+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cd01532c0 con 0x7f5cd015d6e0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.063+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5cd006fd20 con 0x7f5cd015d6e0 2024-09-21T07:55:38.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.065+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) ==== 102410+0+0 (secure 0 0 0) 0x7f5cc80051c0 con 0x7f5cd015d6e0 2024-09-21T07:55:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.066+0000 7f5cccc00640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cb0079640 0x7f5cb007bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.066+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f5cc8099eb0 con 0x7f5cd015d6e0 2024-09-21T07:55:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.066+0000 7f5cd4e00640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cb0079640 0x7f5cb007bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.066+0000 7f5cd4e00640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cb0079640 0x7f5cb007bb50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005fd0 tx=0x7f5cc400b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:38.072 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.070+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5cc8016030 con 0x7f5cd015d6e0 2024-09-21T07:55:38.282 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.280+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup rm", "vol_name": "cephfs", "group_name": "subvol_grp65700", "target": ["mon-mgr", ""]}) -- 0x7f5cd0153560 con 0x7f5cb0079640 2024-09-21T07:55:38.286 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.284+0000 7f5cccc00640 1 -- 172.21.15.88:0/1223079181 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f5cd0153560 con 0x7f5cb0079640 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cb0079640 msgr2=0x7f5cb007bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cb0079640 0x7f5cb007bb50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f5cc4005fd0 tx=0x7f5cc400b2e0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 msgr2=0x7f5cd0152860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd0152860 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7f5cc800e9c0 tx=0x7f5cc800ee80 comp rx=0 tx=0).stop 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 shutdown_connections 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5cb0079640 0x7f5cb007bb50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5cd00cd000 0x7f5cd015f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5cd015d6e0 0x7f5cd0152860 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 --2- 172.21.15.88:0/1223079181 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5cd00b3f30 0x7f5cd01e5ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.286+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 >> 172.21.15.88:0/1223079181 conn(0x7f5cd006d620 msgr2=0x7f5cd00ae860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:38.289 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.287+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 shutdown_connections 2024-09-21T07:55:38.290 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.287+0000 7f5cd6200640 1 -- 172.21.15.88:0/1223079181 wait complete. 2024-09-21T07:55:38.311 INFO:teuthology.nuke:Clearing teuthology firewall rules... 2024-09-21T07:55:38.311 DEBUG:teuthology.orchestra.run.smithi088:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:55:38.401 DEBUG:teuthology.orchestra.run.smithi175:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2024-09-21T07:55:38.509 INFO:teuthology.nuke:Cleared teuthology firewall rules. 2024-09-21T07:55:38.509 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:55:38.509 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:38.509 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:38.545 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:38.545 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:38.620 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:55:38.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 -- 172.21.15.88:0/2236164055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81080705b0 msgr2=0x7f81080709c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 --2- 172.21.15.88:0/2236164055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81080705b0 0x7f81080709c0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009830 tx=0x7f80f002a480 comp rx=0 tx=0).stop 2024-09-21T07:55:38.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 -- 172.21.15.88:0/2236164055 shutdown_connections 2024-09-21T07:55:38.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 --2- 172.21.15.88:0/2236164055 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f81080705b0 0x7f81080709c0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 --2- 172.21.15.88:0/2236164055 >> v2:172.21.15.175:3300/0 conn(0x7f81080cfc80 0x7f8108156e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 -- 172.21.15.88:0/2236164055 >> 172.21.15.88:0/2236164055 conn(0x7f810806d3d0 msgr2=0x7f81080b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:38.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 -- 172.21.15.88:0/2236164055 shutdown_connections 2024-09-21T07:55:38.884 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.881+0000 7f810ca00640 1 -- 172.21.15.88:0/2236164055 wait complete. 2024-09-21T07:55:38.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.883+0000 7f810ca00640 1 Processor -- start 2024-09-21T07:55:38.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.883+0000 7f810ca00640 1 -- start start 2024-09-21T07:55:38.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f810ca00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81080705b0 0x7f810809dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f810ca00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 0x7f8108074120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f810ca00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f810815ce30 0x7f810809af50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f810ca00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81080cced0 con 0x7f81080cfc80 2024-09-21T07:55:38.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f810ca00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f81080d0170 con 0x7f81080705b0 2024-09-21T07:55:38.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f810ca00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f81080cd050 con 0x7f810815ce30 2024-09-21T07:55:38.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f80fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 0x7f8108074120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f80fd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 0x7f8108074120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43546/0 (socket says 172.21.15.88:43546) 2024-09-21T07:55:38.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f80fd400640 1 -- 172.21.15.88:0/4134222080 learned_addr learned my addr 172.21.15.88:0/4134222080 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:38.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f8105e00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81080705b0 0x7f810809dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f8106800640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f810815ce30 0x7f810809af50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f80fd400640 1 -- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f810815ce30 msgr2=0x7f810809af50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.885+0000 7f80fd400640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f810815ce30 0x7f810809af50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.891 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fd400640 1 -- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81080705b0 msgr2=0x7f810809dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:38.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fd400640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81080705b0 0x7f810809dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:38.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fd400640 1 -- 172.21.15.88:0/4134222080 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81081f6ed0 con 0x7f81080cfc80 2024-09-21T07:55:38.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f8105e00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81080705b0 0x7f810809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:38.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fd400640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 0x7f8108074120 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009950 tx=0x7f80f0033c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:38.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0005c90 con 0x7f81080cfc80 2024-09-21T07:55:38.892 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f80f0031e50 con 0x7f81080cfc80 2024-09-21T07:55:38.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81081f7190 con 0x7f81080cfc80 2024-09-21T07:55:38.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80f0005260 con 0x7f81080cfc80 2024-09-21T07:55:38.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.886+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81081f7750 con 0x7f81080cfc80 2024-09-21T07:55:38.893 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.887+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81080cd570 con 0x7f81080cfc80 2024-09-21T07:55:38.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.889+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) ==== 102410+0+0 (secure 0 0 0) 0x7f80f0005400 con 0x7f81080cfc80 2024-09-21T07:55:38.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.890+0000 7f80fea00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e4079670 0x7f80e407bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:38.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.890+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f80f00ba760 con 0x7f81080cfc80 2024-09-21T07:55:38.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.890+0000 7f8105e00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e4079670 0x7f80e407bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:38.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.891+0000 7f8105e00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e4079670 0x7f80e407bb80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f810809e510 tx=0x7f80f8005f90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:38.896 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:38.895+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f80f003f030 con 0x7f81080cfc80 2024-09-21T07:55:39.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.105+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f810815d630 con 0x7f81080cfc80 2024-09-21T07:55:39.107 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.105+0000 7f80fea00640 1 -- 172.21.15.88:0/4134222080 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) ==== 80+0+0 (secure 0 0 0) 0x7f80f00846a0 con 0x7f81080cfc80 2024-09-21T07:55:39.107 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e4079670 msgr2=0x7f80e407bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e4079670 0x7f80e407bb80 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f810809e510 tx=0x7f80f8005f90 comp rx=0 tx=0).stop 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 msgr2=0x7f8108074120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 0x7f8108074120 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7f80f0009950 tx=0x7f80f0033c20 comp rx=0 tx=0).stop 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 shutdown_connections 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f80e4079670 0x7f80e407bb80 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f810815ce30 0x7f810809af50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81080cfc80 0x7f8108074120 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 --2- 172.21.15.88:0/4134222080 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f81080705b0 0x7f810809dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.110 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 >> 172.21.15.88:0/4134222080 conn(0x7f810806d3d0 msgr2=0x7f810815ee50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:39.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.107+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 shutdown_connections 2024-09-21T07:55:39.111 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.108+0000 7f810ca00640 1 -- 172.21.15.88:0/4134222080 wait complete. 2024-09-21T07:55:39.132 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T07:55:39.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:39.133 DEBUG:teuthology.orchestra.run.smithi088:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T07:55:39.226 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:55:39.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:39.227 DEBUG:teuthology.orchestra.run.smithi088:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T07:55:39.274 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:55:39.274 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:39.274 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:39.312 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:39.312 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:39.389 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-09-21T07:55:39.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 -- 172.21.15.88:0/2984119008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c806e2f0 msgr2=0x7f00c806f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 --2- 172.21.15.88:0/2984119008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c806e2f0 0x7f00c806f9a0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f00c0005fd0 tx=0x7f00c002ab20 comp rx=0 tx=0).stop 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 -- 172.21.15.88:0/2984119008 shutdown_connections 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 --2- 172.21.15.88:0/2984119008 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c806e2f0 0x7f00c806f9a0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 --2- 172.21.15.88:0/2984119008 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c80ce640 0x7f00c806ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 -- 172.21.15.88:0/2984119008 >> 172.21.15.88:0/2984119008 conn(0x7f00c8073940 msgr2=0x7f00c8073d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 -- 172.21.15.88:0/2984119008 shutdown_connections 2024-09-21T07:55:39.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.608+0000 7f00d0000640 1 -- 172.21.15.88:0/2984119008 wait complete. 2024-09-21T07:55:39.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.610+0000 7f00d0000640 1 Processor -- start 2024-09-21T07:55:39.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.610+0000 7f00d0000640 1 -- start start 2024-09-21T07:55:39.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00d0000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c806e2f0 0x7f00c81f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:39.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00d0000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c80ce640 0x7f00c8068e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:39.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00d0000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 0x7f00c80693c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:39.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00d0000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f00c806fee0 con 0x7f00c8153880 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00d0000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f00c8075f30 con 0x7f00c80ce640 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00d0000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f00c8070060 con 0x7f00c806e2f0 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00ce000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 0x7f00c80693c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00ce000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 0x7f00c80693c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43548/0 (socket says 172.21.15.88:43548) 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00cd600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c806e2f0 0x7f00c81f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00ce000640 1 -- 172.21.15.88:0/2617622979 learned_addr learned my addr 172.21.15.88:0/2617622979 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:39.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.612+0000 7f00ccc00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c80ce640 0x7f00c8068e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00ce000640 1 -- 172.21.15.88:0/2617622979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c806e2f0 msgr2=0x7f00c81f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00ce000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c806e2f0 0x7f00c81f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00ce000640 1 -- 172.21.15.88:0/2617622979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c80ce640 msgr2=0x7f00c8068e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00ce000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c80ce640 0x7f00c8068e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00ce000640 1 -- 172.21.15.88:0/2617622979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f00c81f6440 con 0x7f00c8153880 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00ce000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 0x7f00c80693c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f00c0006010 tx=0x7f00c0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0044070 con 0x7f00c8153880 2024-09-21T07:55:39.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f00c0004040 con 0x7f00c8153880 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f00c81f6720 con 0x7f00c8153880 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f00c0004490 con 0x7f00c8153880 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.613+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00c81f6ce0 con 0x7f00c8153880 2024-09-21T07:55:39.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.615+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f00c8071da0 con 0x7f00c8153880 2024-09-21T07:55:39.618 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.616+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 29) ==== 102410+0+0 (secure 0 0 0) 0x7f00c0004630 con 0x7f00c8153880 2024-09-21T07:55:39.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.617+0000 7f00c5600640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00a8079640 0x7f00a807bb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:39.619 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.617+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f00c00b9eb0 con 0x7f00c8153880 2024-09-21T07:55:39.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.618+0000 7f00cd600640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00a8079640 0x7f00a807bb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:39.620 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.618+0000 7f00cd600640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00a8079640 0x7f00a807bb50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0017e0 tx=0x7f00bc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:39.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.622+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f00c8071da0 con 0x7f00c8153880 2024-09-21T07:55:39.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.821+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f00c81f9630 con 0x7f00c8153880 2024-09-21T07:55:39.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.821+0000 7f00c5600640 1 -- 172.21.15.88:0/2617622979 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v170) ==== 80+0+0 (secure 0 0 0) 0x7f00c00bee20 con 0x7f00c8153880 2024-09-21T07:55:39.823 INFO:teuthology.orchestra.run.smithi088.stderr:listed 0 entries 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.823+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00a8079640 msgr2=0x7f00a807bb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.823+0000 7f00d0000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00a8079640 0x7f00a807bb50 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f00bc0017e0 tx=0x7f00bc009040 comp rx=0 tx=0).stop 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.823+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 msgr2=0x7f00c80693c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:39.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.823+0000 7f00d0000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 0x7f00c80693c0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f00c0006010 tx=0x7f00c0003ba0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 shutdown_connections 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f00a8079640 0x7f00a807bb50 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00c8153880 0x7f00c80693c0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f00c80ce640 0x7f00c8068e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 --2- 172.21.15.88:0/2617622979 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f00c806e2f0 0x7f00c81f5840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 >> 172.21.15.88:0/2617622979 conn(0x7f00c8073940 msgr2=0x7f00c8152f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 shutdown_connections 2024-09-21T07:55:39.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:39.824+0000 7f00d0000640 1 -- 172.21.15.88:0/2617622979 wait complete. 2024-09-21T07:55:39.847 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2024-09-21T07:55:39.847 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:39.847 DEBUG:teuthology.orchestra.run.smithi175:> sudo umount /home/ubuntu/cephtest/mnt.1 2024-09-21T07:55:39.911 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:55:39.912 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:39.912 DEBUG:teuthology.orchestra.run.smithi175:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2024-09-21T07:55:39.948 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:55:39.949 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:39.949 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:39.985 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:39.985 INFO:tasks.cephfs.mount:Terminating background process 2024-09-21T07:55:39.985 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:39.985 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:40.025 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:40.025 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.243+0000 7f1f0f000640 1 -- 172.21.15.88:0/3382560142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080ad3c0 msgr2=0x7f1f080ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.243+0000 7f1f0f000640 1 --2- 172.21.15.88:0/3382560142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080ad3c0 0x7f1f080ad7d0 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400a940 tx=0x7f1ef402b7b0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.244+0000 7f1f0f000640 1 -- 172.21.15.88:0/3382560142 shutdown_connections 2024-09-21T07:55:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.244+0000 7f1f0f000640 1 --2- 172.21.15.88:0/3382560142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080ad3c0 0x7f1f080ad7d0 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.246 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.244+0000 7f1f0f000640 1 -- 172.21.15.88:0/3382560142 >> 172.21.15.88:0/3382560142 conn(0x7f1f0806d730 msgr2=0x7f1f080b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.244+0000 7f1f0f000640 1 -- 172.21.15.88:0/3382560142 shutdown_connections 2024-09-21T07:55:40.247 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.244+0000 7f1f0f000640 1 -- 172.21.15.88:0/3382560142 wait complete. 2024-09-21T07:55:40.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.246+0000 7f1f0f000640 1 Processor -- start 2024-09-21T07:55:40.248 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.246+0000 7f1f0f000640 1 -- start start 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.247+0000 7f1f0f000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f080704b0 0x7f1f08149a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.247+0000 7f1f0f000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f080ad3c0 0x7f1f081f7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0f000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 0x7f1f081f7770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0f000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f080712f0 con 0x7f1f080add10 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0f000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f1f08071170 con 0x7f1f080704b0 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0f000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f1f08165e00 con 0x7f1f080ad3c0 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 0x7f1f081f7770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.250 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 0x7f1f081f7770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43562/0 (socket says 172.21.15.88:43562) 2024-09-21T07:55:40.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 -- 172.21.15.88:0/1397475185 learned_addr learned my addr 172.21.15.88:0/1397475185 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:40.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f07400640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f080ad3c0 0x7f1f081f7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.251 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f07e00640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f080704b0 0x7f1f08149a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 -- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f080ad3c0 msgr2=0x7f1f081f7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f080ad3c0 0x7f1f081f7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 -- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f080704b0 msgr2=0x7f1f08149a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f080704b0 0x7f1f08149a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f0d000640 1 -- 172.21.15.88:0/1397475185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f08149f80 con 0x7f1f080add10 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.248+0000 7f1f07e00640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f080704b0 0x7f1f08149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.249+0000 7f1f0d000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 0x7f1f081f7770 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b9e0 tx=0x7f1f0000bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.249+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f00003b70 con 0x7f1f080add10 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.249+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1f00003e00 con 0x7f1f080add10 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.249+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f0000cb80 con 0x7f1f080add10 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.249+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f0815cf20 con 0x7f1f080add10 2024-09-21T07:55:40.252 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.249+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f0815d4e0 con 0x7f1f080add10 2024-09-21T07:55:40.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.250+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f081f7f60 con 0x7f1f080add10 2024-09-21T07:55:40.253 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.251+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f1f00004000 con 0x7f1f080add10 2024-09-21T07:55:40.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.252+0000 7f1effe00640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ee8079760 0x7f1ee807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.252+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f1f00099670 con 0x7f1f080add10 2024-09-21T07:55:40.254 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.253+0000 7f1f07e00640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ee8079760 0x7f1ee807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.255 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.253+0000 7f1f07e00640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ee8079760 0x7f1ee807bc70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8000f60 tx=0x7f1ef8009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:40.258 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.257+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f1f00062d00 con 0x7f1f080add10 2024-09-21T07:55:40.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.479+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1f080ad7d0 con 0x7f1f080add10 2024-09-21T07:55:40.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.480+0000 7f1effe00640 1 -- 172.21.15.88:0/1397475185 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+5233 (secure 0 0 0) 0x7f1f00067730 con 0x7f1f080add10 2024-09-21T07:55:40.482 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:40.482 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":119,"btime":"2024-09-21T07:55:33:933850+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:55:22.380663+0000","modified":"2024-09-21T07:55:25.396077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9147":{"gid":9147,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/661983519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":661983519},{"type":"v1","addr":"172.21.15.88:6835","nonce":661983519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9147,"qdb_cluster":[9147]},"id":13}]} 2024-09-21T07:55:40.483 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 119 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.482+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ee8079760 msgr2=0x7f1ee807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.482+0000 7f1f0f000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ee8079760 0x7f1ee807bc70 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8000f60 tx=0x7f1ef8009040 comp rx=0 tx=0).stop 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.482+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 msgr2=0x7f1f081f7770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.482+0000 7f1f0f000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 0x7f1f081f7770 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f1f0000b9e0 tx=0x7f1f0000bea0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 shutdown_connections 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f1ee8079760 0x7f1ee807bc70 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f1f080ad3c0 0x7f1f081f7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f080add10 0x7f1f081f7770 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 --2- 172.21.15.88:0/1397475185 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f1f080704b0 0x7f1f08149a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 >> 172.21.15.88:0/1397475185 conn(0x7f1f0806d730 msgr2=0x7f1f08152cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:40.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 shutdown_connections 2024-09-21T07:55:40.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.483+0000 7f1f0f000640 1 -- 172.21.15.88:0/1397475185 wait complete. 2024-09-21T07:55:40.506 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:40.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.723+0000 7faec7000640 1 -- 172.21.15.88:0/4076745638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 msgr2=0x7faec01535c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.726 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.723+0000 7faec7000640 1 --2- 172.21.15.88:0/4076745638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec01535c0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7faec0072d60 tx=0x7faeb002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 -- 172.21.15.88:0/4076745638 shutdown_connections 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 --2- 172.21.15.88:0/4076745638 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec0160310 0x7faec0160720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 --2- 172.21.15.88:0/4076745638 >> v2:172.21.15.175:3300/0 conn(0x7faec00ad0e0 0x7faec00ad4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 --2- 172.21.15.88:0/4076745638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec01535c0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 -- 172.21.15.88:0/4076745638 >> 172.21.15.88:0/4076745638 conn(0x7faec006d730 msgr2=0x7faec00b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 -- 172.21.15.88:0/4076745638 shutdown_connections 2024-09-21T07:55:40.727 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.724+0000 7faec7000640 1 -- 172.21.15.88:0/4076745638 wait complete. 2024-09-21T07:55:40.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.726+0000 7faec7000640 1 Processor -- start 2024-09-21T07:55:40.728 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.726+0000 7faec7000640 1 -- start start 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.727+0000 7faec7000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ad0e0 0x7faec01f8d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.727+0000 7faec7000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec0156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.727+0000 7faec7000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec0160310 0x7faec0073040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.727+0000 7faec7000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faec00b4da0 con 0x7faec015d760 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.727+0000 7faec7000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faec00b4c20 con 0x7faec00ad0e0 2024-09-21T07:55:40.731 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.727+0000 7faec7000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faec0160c60 con 0x7faec0160310 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ad0e0 0x7faec01f8d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebfe00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ad0e0 0x7faec01f8d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.175:3300/0 says I am v2:172.21.15.88:38320/0 (socket says 172.21.15.88:38320) 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec0156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebfe00640 1 -- 172.21.15.88:0/2188176400 learned_addr learned my addr 172.21.15.88:0/2188176400 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faec5000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec0160310 0x7faec0073040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 -- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec0160310 msgr2=0x7faec0073040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec0160310 0x7faec0073040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 -- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ad0e0 msgr2=0x7faec01f8d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ad0e0 0x7faec01f8d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 -- 172.21.15.88:0/2188176400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faec01f93e0 con 0x7faec015d760 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.728+0000 7faebf400640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec0156cc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d650 tx=0x7faeac00db10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:40.732 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.729+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac017070 con 0x7faec015d760 2024-09-21T07:55:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.729+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faeac004310 con 0x7faec015d760 2024-09-21T07:55:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.729+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faec01f96c0 con 0x7faec015d760 2024-09-21T07:55:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.729+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faeac009a80 con 0x7faec015d760 2024-09-21T07:55:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.729+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faec00736f0 con 0x7faec015d760 2024-09-21T07:55:40.733 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.730+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faec01f5730 con 0x7faec015d760 2024-09-21T07:55:40.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.731+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7faeac009c20 con 0x7faec015d760 2024-09-21T07:55:40.734 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.732+0000 7faeb7e00640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea0079760 0x7faea007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:40.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.732+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7faeac0998b0 con 0x7faec015d760 2024-09-21T07:55:40.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.732+0000 7faebfe00640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea0079760 0x7faea007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:40.735 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.733+0000 7faebfe00640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea0079760 0x7faea007bc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009230 tx=0x7faeb0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:40.739 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.737+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faeac062e80 con 0x7faec015d760 2024-09-21T07:55:40.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.960+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7faec01535c0 con 0x7faec015d760 2024-09-21T07:55:40.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.960+0000 7faeb7e00640 1 -- 172.21.15.88:0/2188176400 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+5233 (secure 0 0 0) 0x7faeac0678b0 con 0x7faec015d760 2024-09-21T07:55:40.963 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:40.963 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":119,"btime":"2024-09-21T07:55:33:933850+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:55:22.380663+0000","modified":"2024-09-21T07:55:25.396077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9147":{"gid":9147,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/661983519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":661983519},{"type":"v1","addr":"172.21.15.88:6835","nonce":661983519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9147,"qdb_cluster":[9147]},"id":13}]} 2024-09-21T07:55:40.963 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 119 2024-09-21T07:55:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea0079760 msgr2=0x7faea007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea0079760 0x7faea007bc70 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7faeb0009230 tx=0x7faeb0004730 comp rx=0 tx=0).stop 2024-09-21T07:55:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 msgr2=0x7faec0156cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:40.965 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec0156cc0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7faeac00d650 tx=0x7faeac00db10 comp rx=0 tx=0).stop 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 shutdown_connections 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faea0079760 0x7faea007bc70 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faec0160310 0x7faec0073040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faec015d760 0x7faec0156cc0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 --2- 172.21.15.88:0/2188176400 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faec00ad0e0 0x7faec01f8d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 >> 172.21.15.88:0/2188176400 conn(0x7faec006d730 msgr2=0x7faec00b1d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 shutdown_connections 2024-09-21T07:55:40.966 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:40.963+0000 7faec7000640 1 -- 172.21.15.88:0/2188176400 wait complete. 2024-09-21T07:55:40.988 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:41.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.205+0000 7f10e2e00640 1 -- 172.21.15.88:0/620223078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0b3550 msgr2=0x7f10dc0b3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.205+0000 7f10e2e00640 1 --2- 172.21.15.88:0/620223078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0b3550 0x7f10dc0b3960 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f10d4005fd0 tx=0x7f10d402a820 comp rx=0 tx=0).stop 2024-09-21T07:55:41.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.205+0000 7f10e2e00640 1 -- 172.21.15.88:0/620223078 shutdown_connections 2024-09-21T07:55:41.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.205+0000 7f10e2e00640 1 --2- 172.21.15.88:0/620223078 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0b3550 0x7f10dc0b3960 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.208 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.205+0000 7f10e2e00640 1 --2- 172.21.15.88:0/620223078 >> v2:172.21.15.175:3300/0 conn(0x7f10dc15d760 0x7f10dc1535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.205+0000 7f10e2e00640 1 -- 172.21.15.88:0/620223078 >> 172.21.15.88:0/620223078 conn(0x7f10dc06d730 msgr2=0x7f10dc0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:41.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.206+0000 7f10e2e00640 1 -- 172.21.15.88:0/620223078 shutdown_connections 2024-09-21T07:55:41.209 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.206+0000 7f10e2e00640 1 -- 172.21.15.88:0/620223078 wait complete. 2024-09-21T07:55:41.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.208+0000 7f10e2e00640 1 Processor -- start 2024-09-21T07:55:41.210 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.208+0000 7f10e2e00640 1 -- start start 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.209+0000 7f10e2e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0ad0e0 0x7f10dc14dfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.209+0000 7f10e2e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc0b3550 0x7f10dc0749d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.209+0000 7f10e2e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 0x7f10dc1565a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.209+0000 7f10e2e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0b4da0 con 0x7f10dc15d760 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.209+0000 7f10e2e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f10dc0b4c20 con 0x7f10dc0b3550 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.209+0000 7f10e2e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f10dc0b3ea0 con 0x7f10dc0ad0e0 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 0x7f10dc1565a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 0x7f10dc1565a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43592/0 (socket says 172.21.15.88:43592) 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 -- 172.21.15.88:0/325028719 learned_addr learned my addr 172.21.15.88:0/325028719 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:41.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10dbe00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0ad0e0 0x7f10dc14dfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10db400640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc0b3550 0x7f10dc0749d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 -- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0ad0e0 msgr2=0x7f10dc14dfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0ad0e0 0x7f10dc14dfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 -- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc0b3550 msgr2=0x7f10dc0749d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc0b3550 0x7f10dc0749d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 -- 172.21.15.88:0/325028719 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10dc15c650 con 0x7f10dc15d760 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.210+0000 7f10e0e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 0x7f10dc1565a0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009cd0 tx=0x7f10d4032c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.211+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4037040 con 0x7f10dc15d760 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.211+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f10d4042070 con 0x7f10dc15d760 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.211+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10d4003e60 con 0x7f10dc15d760 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.211+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10dc15bad0 con 0x7f10dc15d760 2024-09-21T07:55:41.214 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.211+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10dc15c090 con 0x7f10dc15d760 2024-09-21T07:55:41.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.212+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10dc15b840 con 0x7f10dc15d760 2024-09-21T07:55:41.215 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.213+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f10d4004280 con 0x7f10dc15d760 2024-09-21T07:55:41.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.214+0000 7f10d3e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f10bc079760 0x7f10bc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.215+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f10d40ba650 con 0x7f10dc15d760 2024-09-21T07:55:41.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.215+0000 7f10dbe00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f10bc079760 0x7f10bc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.215+0000 7f10dbe00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f10bc079760 0x7f10bc07bc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f10cc005fd0 tx=0x7f10cc009260 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:41.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.219+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f10d4084330 con 0x7f10dc15d760 2024-09-21T07:55:41.407 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.405+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f10dc0b4330 con 0x7f10dc15d760 2024-09-21T07:55:41.408 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.406+0000 7f10d3e00640 1 -- 172.21.15.88:0/325028719 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16932 (secure 0 0 0) 0x7f10d4083c30 con 0x7f10dc15d760 2024-09-21T07:55:41.408 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:41.409 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":170,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:37.953367+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:55:20.283046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T07:55:21.342418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"27.b","mappings":[{"from":5,"to":2}]},{"pgid":"27.1d","mappings":[{"from":7,"to":2}]},{"pgid":"27.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:41.410 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f10bc079760 msgr2=0x7f10bc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f10bc079760 0x7f10bc07bc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f10cc005fd0 tx=0x7f10cc009260 comp rx=0 tx=0).stop 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 msgr2=0x7f10dc1565a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 0x7f10dc1565a0 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009cd0 tx=0x7f10d4032c60 comp rx=0 tx=0).stop 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 shutdown_connections 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f10bc079760 0x7f10bc07bc70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10dc15d760 0x7f10dc1565a0 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f10dc0b3550 0x7f10dc0749d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 --2- 172.21.15.88:0/325028719 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f10dc0ad0e0 0x7f10dc14dfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 >> 172.21.15.88:0/325028719 conn(0x7f10dc06d730 msgr2=0x7f10dc15e9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 shutdown_connections 2024-09-21T07:55:41.411 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.408+0000 7f10e2e00640 1 -- 172.21.15.88:0/325028719 wait complete. 2024-09-21T07:55:41.436 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:55:41.437 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:41.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 -- 172.21.15.88:0/3210443211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800d1580 msgr2=0x7f2c800d1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 --2- 172.21.15.88:0/3210443211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800d1580 0x7f2c800d1990 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f2c8008d440 tx=0x7f2c7802aa30 comp rx=0 tx=0).stop 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 -- 172.21.15.88:0/3210443211 shutdown_connections 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 --2- 172.21.15.88:0/3210443211 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800d1580 0x7f2c800d1990 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 --2- 172.21.15.88:0/3210443211 >> v2:172.21.15.175:3300/0 conn(0x7f2c800b9e00 0x7f2c80162a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 -- 172.21.15.88:0/3210443211 >> 172.21.15.88:0/3210443211 conn(0x7f2c8008bfe0 msgr2=0x7f2c8009d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 -- 172.21.15.88:0/3210443211 shutdown_connections 2024-09-21T07:55:41.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.653+0000 7f2c85c00640 1 -- 172.21.15.88:0/3210443211 wait complete. 2024-09-21T07:55:41.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.655+0000 7f2c85c00640 1 Processor -- start 2024-09-21T07:55:41.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.655+0000 7f2c85c00640 1 -- start start 2024-09-21T07:55:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 0x7f2c800cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.659 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c800b9e00 0x7f2c800cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1580 0x7f2c800c8f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2c800d44b0 con 0x7f2c800b3160 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2c800d4330 con 0x7f2c800b9e00 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2c800d4630 con 0x7f2c800d1580 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 0x7f2c800cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c7fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 0x7f2c800cd9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43618/0 (socket says 172.21.15.88:43618) 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c7fe00640 1 -- 172.21.15.88:0/1143930776 learned_addr learned my addr 172.21.15.88:0/1143930776 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c7f400640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c800b9e00 0x7f2c800cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.657+0000 7f2c85200640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1580 0x7f2c800c8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7fe00640 1 -- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1580 msgr2=0x7f2c800c8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7fe00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1580 0x7f2c800c8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7fe00640 1 -- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c800b9e00 msgr2=0x7f2c800cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7fe00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c800b9e00 0x7f2c800cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.660 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7fe00640 1 -- 172.21.15.88:0/1143930776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c800a23f0 con 0x7f2c800b3160 2024-09-21T07:55:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7f400640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c800b9e00 0x7f2c800cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c7fe00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 0x7f2c800cd9d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000d770 tx=0x7f2c7000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:41.661 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70017070 con 0x7f2c800b3160 2024-09-21T07:55:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2c70009d20 con 0x7f2c800b3160 2024-09-21T07:55:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c70004280 con 0x7f2c800b3160 2024-09-21T07:55:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c800c8150 con 0x7f2c800b3160 2024-09-21T07:55:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.658+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c800c8710 con 0x7f2c800b3160 2024-09-21T07:55:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.659+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c800d4b50 con 0x7f2c800b3160 2024-09-21T07:55:41.662 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.661+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f2c70005020 con 0x7f2c800b3160 2024-09-21T07:55:41.663 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.662+0000 7f2c77e00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2c60079760 0x7f2c6007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:41.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.662+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f2c70099d00 con 0x7f2c800b3160 2024-09-21T07:55:41.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.662+0000 7f2c7f400640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2c60079760 0x7f2c6007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:41.664 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.662+0000 7f2c7f400640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2c60079760 0x7f2c6007bc70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c800b2b60 tx=0x7f2c6c009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:41.668 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.666+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2c70020020 con 0x7f2c800b3160 2024-09-21T07:55:41.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.892+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2c800b4620 con 0x7f2c800b3160 2024-09-21T07:55:41.895 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.893+0000 7f2c77e00640 1 -- 172.21.15.88:0/1143930776 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+5233 (secure 0 0 0) 0x7f2c700639e0 con 0x7f2c800b3160 2024-09-21T07:55:41.895 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:41.895 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":119,"btime":"2024-09-21T07:55:33:933850+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:55:22.380663+0000","modified":"2024-09-21T07:55:25.396077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9147":{"gid":9147,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/661983519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":661983519},{"type":"v1","addr":"172.21.15.88:6835","nonce":661983519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9147,"qdb_cluster":[9147]},"id":13}]} 2024-09-21T07:55:41.896 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 119 2024-09-21T07:55:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.895+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2c60079760 msgr2=0x7f2c6007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.895+0000 7f2c85c00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2c60079760 0x7f2c6007bc70 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2c800b2b60 tx=0x7f2c6c009040 comp rx=0 tx=0).stop 2024-09-21T07:55:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 msgr2=0x7f2c800cd9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 0x7f2c800cd9d0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f2c7000d770 tx=0x7f2c7000dc30 comp rx=0 tx=0).stop 2024-09-21T07:55:41.898 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 shutdown_connections 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2c60079760 0x7f2c6007bc70 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2c800d1580 0x7f2c800c8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2c800b9e00 0x7f2c800cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 --2- 172.21.15.88:0/1143930776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c800b3160 0x7f2c800cd9d0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 >> 172.21.15.88:0/1143930776 conn(0x7f2c8008bfe0 msgr2=0x7f2c8006f730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 shutdown_connections 2024-09-21T07:55:41.899 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:41.896+0000 7f2c85c00640 1 -- 172.21.15.88:0/1143930776 wait complete. 2024-09-21T07:55:41.924 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:42.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 -- 172.21.15.88:0/3787216980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 msgr2=0x7f12200d0150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 --2- 172.21.15.88:0/3787216980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200d0150 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1218009be0 tx=0x7f121802a820 comp rx=0 tx=0).stop 2024-09-21T07:55:42.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 -- 172.21.15.88:0/3787216980 shutdown_connections 2024-09-21T07:55:42.144 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 --2- 172.21.15.88:0/3787216980 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200d0150 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 -- 172.21.15.88:0/3787216980 >> 172.21.15.88:0/3787216980 conn(0x7f12200d44a0 msgr2=0x7f12200d48b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:42.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 -- 172.21.15.88:0/3787216980 shutdown_connections 2024-09-21T07:55:42.145 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.141+0000 7f1225600640 1 -- 172.21.15.88:0/3787216980 wait complete. 2024-09-21T07:55:42.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.143+0000 7f1225600640 1 Processor -- start 2024-09-21T07:55:42.146 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.144+0000 7f1225600640 1 -- start start 2024-09-21T07:55:42.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1225600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12200b9840 0x7f12200b6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1225600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.150 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1225600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 0x7f12200b5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1225600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12200d10a0 con 0x7f12200d4fc0 2024-09-21T07:55:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1225600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f12200d0f20 con 0x7f12200b9840 2024-09-21T07:55:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1225600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f12200d1220 con 0x7f12200cfd40 2024-09-21T07:55:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1224c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 0x7f12200b5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1224c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 0x7f12200b5490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43628/0 (socket says 172.21.15.88:43628) 2024-09-21T07:55:42.151 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.145+0000 7f1224c00640 1 -- 172.21.15.88:0/704668519 learned_addr learned my addr 172.21.15.88:0/704668519 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f121fe00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12200b9840 0x7f12200b6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1224c00640 1 -- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 msgr2=0x7f12200b4f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1217e00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1224c00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200b4f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1224c00640 1 -- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12200b9840 msgr2=0x7f12200b6370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1224c00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12200b9840 0x7f12200b6370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.152 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1224c00640 1 -- 172.21.15.88:0/704668519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12200b68b0 con 0x7f12200d4fc0 2024-09-21T07:55:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f121fe00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12200b9840 0x7f12200b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1217e00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f1224c00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 0x7f12200b5490 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1218009be0 tx=0x7f12180305d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218041030 con 0x7f12200d4fc0 2024-09-21T07:55:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f1218003970 con 0x7f12200d4fc0 2024-09-21T07:55:42.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.146+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218036040 con 0x7f12200d4fc0 2024-09-21T07:55:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.147+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12200ce210 con 0x7f12200d4fc0 2024-09-21T07:55:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.147+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122014e5f0 con 0x7f12200d4fc0 2024-09-21T07:55:42.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.148+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f122008cc70 con 0x7f12200d4fc0 2024-09-21T07:55:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.149+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f1218044070 con 0x7f12200d4fc0 2024-09-21T07:55:42.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.150+0000 7f121cc00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11fc079760 0x7f11fc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.150+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f12180ba830 con 0x7f12200d4fc0 2024-09-21T07:55:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.150+0000 7f121fe00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11fc079760 0x7f11fc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.151+0000 7f121fe00640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11fc079760 0x7f11fc07bc70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12200b3ed0 tx=0x7f1210009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:42.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.154+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f122008cc70 con 0x7f12200d4fc0 2024-09-21T07:55:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.379+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f12200d0150 con 0x7f12200d4fc0 2024-09-21T07:55:42.381 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.379+0000 7f121cc00640 1 -- 172.21.15.88:0/704668519 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 119 v119) ==== 95+0+5233 (secure 0 0 0) 0x7f12200b68b0 con 0x7f12200d4fc0 2024-09-21T07:55:42.381 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:42.382 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":119,"btime":"2024-09-21T07:55:33:933850+0000","default_fscid":13,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-09-21T07:55:22.380663+0000","modified":"2024-09-21T07:55:25.396077+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":9147},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9147":{"gid":9147,"name":"c","rank":0,"incarnation":115,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6835/661983519","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":661983519},{"type":"v1","addr":"172.21.15.88:6835","nonce":661983519}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":9147,"qdb_cluster":[9147]},"id":13}]} 2024-09-21T07:55:42.382 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 119 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.381+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11fc079760 msgr2=0x7f11fc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.381+0000 7f1225600640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11fc079760 0x7f11fc07bc70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f12200b3ed0 tx=0x7f1210009040 comp rx=0 tx=0).stop 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.381+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 msgr2=0x7f12200b5490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.381+0000 7f1225600640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 0x7f12200b5490 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7f1218009be0 tx=0x7f12180305d0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 shutdown_connections 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f11fc079760 0x7f11fc07bc70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f12200cfd40 0x7f12200b4f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12200d4fc0 0x7f12200b5490 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 --2- 172.21.15.88:0/704668519 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f12200b9840 0x7f12200b6370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.384 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 >> 172.21.15.88:0/704668519 conn(0x7f12200d44a0 msgr2=0x7f1220068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 shutdown_connections 2024-09-21T07:55:42.385 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.382+0000 7f1225600640 1 -- 172.21.15.88:0/704668519 wait complete. 2024-09-21T07:55:42.406 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.620+0000 7fd730000640 1 -- 172.21.15.88:0/2202159269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 msgr2=0x7fd7280b3750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.620+0000 7fd730000640 1 --2- 172.21.15.88:0/2202159269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd7280b3750 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fd72806f3c0 tx=0x7fd72002a980 comp rx=0 tx=0).stop 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.620+0000 7fd730000640 1 -- 172.21.15.88:0/2202159269 shutdown_connections 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.620+0000 7fd730000640 1 --2- 172.21.15.88:0/2202159269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd7280b3750 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.620+0000 7fd730000640 1 --2- 172.21.15.88:0/2202159269 >> v2:172.21.15.175:3300/0 conn(0x7fd728152920 0x7fd72815f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.620+0000 7fd730000640 1 -- 172.21.15.88:0/2202159269 >> 172.21.15.88:0/2202159269 conn(0x7fd72806d730 msgr2=0x7fd7280b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.621+0000 7fd730000640 1 -- 172.21.15.88:0/2202159269 shutdown_connections 2024-09-21T07:55:42.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.621+0000 7fd730000640 1 -- 172.21.15.88:0/2202159269 wait complete. 2024-09-21T07:55:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.623+0000 7fd730000640 1 Processor -- start 2024-09-21T07:55:42.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.623+0000 7fd730000640 1 -- start start 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.624+0000 7fd730000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7280aebc0 0x7fd7280ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.624+0000 7fd730000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd728152920 0x7fd7280b0f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.624+0000 7fd730000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd728154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.624+0000 7fd730000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd7280b4130 con 0x7fd72815f910 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.624+0000 7fd730000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd7280b3fb0 con 0x7fd728152920 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.624+0000 7fd730000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd7280b42b0 con 0x7fd7280aebc0 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd728154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd728154700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43646/0 (socket says 172.21.15.88:43646) 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 -- 172.21.15.88:0/3148883639 learned_addr learned my addr 172.21.15.88:0/3148883639 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:42.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72d600640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7280aebc0 0x7fd7280ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72cc00640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd728152920 0x7fd7280b0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 -- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7280aebc0 msgr2=0x7fd7280ceaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7280aebc0 0x7fd7280ceaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 -- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd728152920 msgr2=0x7fd7280b0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd728152920 0x7fd7280b0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 -- 172.21.15.88:0/3148883639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7280cefe0 con 0x7fd72815f910 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72cc00640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd728152920 0x7fd7280b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.625+0000 7fd72e000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd728154700 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd720009d60 tx=0x7fd720003f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.626+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720034040 con 0x7fd72815f910 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.626+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd720004540 con 0x7fd72815f910 2024-09-21T07:55:42.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.626+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7281f9230 con 0x7fd72815f910 2024-09-21T07:55:42.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.626+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd720045480 con 0x7fd72815f910 2024-09-21T07:55:42.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.626+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7281f97f0 con 0x7fd72815f910 2024-09-21T07:55:42.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.627+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7280b19b0 con 0x7fd72815f910 2024-09-21T07:55:42.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.628+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fd720037020 con 0x7fd72815f910 2024-09-21T07:55:42.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.629+0000 7fd725600640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd708079760 0x7fd70807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:42.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.629+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7fd7200baff0 con 0x7fd72815f910 2024-09-21T07:55:42.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.629+0000 7fd72d600640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd708079760 0x7fd70807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:42.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.630+0000 7fd72d600640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd708079760 0x7fd70807bc70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd71c006fd0 tx=0x7fd71c009060 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:42.635 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.633+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd72815fd20 con 0x7fd72815f910 2024-09-21T07:55:42.822 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.820+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd7280b4d30 con 0x7fd72815f910 2024-09-21T07:55:42.823 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.822+0000 7fd725600640 1 -- 172.21.15.88:0/3148883639 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v170) ==== 74+0+16932 (secure 0 0 0) 0x7fd720084eb0 con 0x7fd72815f910 2024-09-21T07:55:42.824 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:42.824 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":170,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:37.953367+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"cephfs_metadata","create_time":"2024-09-21T07:55:20.283046+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"167","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"cephfs_data","create_time":"2024-09-21T07:55:21.342418+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"168","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"27.b","mappings":[{"from":5,"to":2}]},{"pgid":"27.1d","mappings":[{"from":7,"to":2}]},{"pgid":"27.39","mappings":[{"from":5,"to":6}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:42.826 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd708079760 msgr2=0x7fd70807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd708079760 0x7fd70807bc70 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fd71c006fd0 tx=0x7fd71c009060 comp rx=0 tx=0).stop 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 msgr2=0x7fd728154700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd728154700 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fd720009d60 tx=0x7fd720003f10 comp rx=0 tx=0).stop 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 shutdown_connections 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd708079760 0x7fd70807bc70 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd72815f910 0x7fd728154700 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd728152920 0x7fd7280b0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 --2- 172.21.15.88:0/3148883639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd7280aebc0 0x7fd7280ceaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 >> 172.21.15.88:0/3148883639 conn(0x7fd72806d730 msgr2=0x7fd728159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 shutdown_connections 2024-09-21T07:55:42.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:42.824+0000 7fd730000640 1 -- 172.21.15.88:0/3148883639 wait complete. 2024-09-21T07:55:42.852 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-09-21T07:55:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 -- 172.21.15.88:0/2930990639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 msgr2=0x7f30700b39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 --2- 172.21.15.88:0/2930990639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f30700b39c0 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7f3068005fd0 tx=0x7f306802a970 comp rx=0 tx=0).stop 2024-09-21T07:55:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 -- 172.21.15.88:0/2930990639 shutdown_connections 2024-09-21T07:55:43.075 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 --2- 172.21.15.88:0/2930990639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f30700b39c0 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 --2- 172.21.15.88:0/2930990639 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f307006f710 0x7f30700b94b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 -- 172.21.15.88:0/2930990639 >> 172.21.15.88:0/2930990639 conn(0x7f307008dc30 msgr2=0x7f307009d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 -- 172.21.15.88:0/2930990639 shutdown_connections 2024-09-21T07:55:43.076 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.073+0000 7f306f200640 1 -- 172.21.15.88:0/2930990639 wait complete. 2024-09-21T07:55:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.075+0000 7f306f200640 1 Processor -- start 2024-09-21T07:55:43.077 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.075+0000 7f306f200640 1 -- start start 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.076+0000 7f306f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f307006f710 0x7f30700cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.076+0000 7f306f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f307015c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.076+0000 7f306f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070161fa0 0x7f30700cd900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.076+0000 7f306f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f30700d2a20 con 0x7f30700d13d0 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.076+0000 7f306f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f30700d28a0 con 0x7f307006f710 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.076+0000 7f306f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f30700d2ba0 con 0x7f3070161fa0 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f307015c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f307015c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43654/0 (socket says 172.21.15.88:43654) 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 -- 172.21.15.88:0/3481093931 learned_addr learned my addr 172.21.15.88:0/3481093931 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:43.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306de00640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f307006f710 0x7f30700cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306e800640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070161fa0 0x7f30700cd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 -- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070161fa0 msgr2=0x7f30700cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070161fa0 0x7f30700cd900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 -- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f307006f710 msgr2=0x7f30700cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f307006f710 0x7f30700cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306d400640 1 -- 172.21.15.88:0/3481093931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f307015cad0 con 0x7f30700d13d0 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.077+0000 7f306de00640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f307006f710 0x7f30700cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.078+0000 7f306d400640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f307015c590 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b3f0 tx=0x7f305c00b8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.078+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c015020 con 0x7f30700d13d0 2024-09-21T07:55:43.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.078+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f305c011920 con 0x7f30700d13d0 2024-09-21T07:55:43.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.078+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30700ce680 con 0x7f30700d13d0 2024-09-21T07:55:43.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.078+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f305c011d70 con 0x7f30700d13d0 2024-09-21T07:55:43.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.078+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30700a1f50 con 0x7f30700d13d0 2024-09-21T07:55:43.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.079+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30700d3050 con 0x7f30700d13d0 2024-09-21T07:55:43.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.081+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f305c018020 con 0x7f30700d13d0 2024-09-21T07:55:43.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.082+0000 7f3066000640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3050079760 0x7f305007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.082+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(170..170 src has 1..170) ==== 5824+0+0 (secure 0 0 0) 0x7f305c099500 con 0x7f30700d13d0 2024-09-21T07:55:43.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.082+0000 7f306de00640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3050079760 0x7f305007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.085 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.083+0000 7f306de00640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3050079760 0x7f305007bc70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f306000bbb0 tx=0x7f3060005ec0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:43.088 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.086+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f305c01f030 con 0x7f30700d13d0 2024-09-21T07:55:43.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.307+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f30700d0aa0 con 0x7f30700d13d0 2024-09-21T07:55:43.310 INFO:tasks.ceph.mon.a.smithi088.stderr:2024-09-21T07:55:43.308+0000 7fad26000640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T07:55:43.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.310+0000 7f3066000640 1 -- 172.21.15.88:0/3481093931 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v120) ==== 161+0+0 (secure 0 0 0) 0x7f305c063220 con 0x7f30700d13d0 2024-09-21T07:55:43.312 INFO:teuthology.orchestra.run.smithi088.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T07:55:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.313+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3050079760 msgr2=0x7f305007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.313+0000 7f306f200640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3050079760 0x7f305007bc70 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f306000bbb0 tx=0x7f3060005ec0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.313+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 msgr2=0x7f307015c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.315 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.313+0000 7f306f200640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f307015c590 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f305c00b3f0 tx=0x7f305c00b8b0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 shutdown_connections 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3050079760 0x7f305007bc70 secure :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f306000bbb0 tx=0x7f3060005ec0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3070161fa0 0x7f30700cd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30700d13d0 0x7f307015c590 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 --2- 172.21.15.88:0/3481093931 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f307006f710 0x7f30700cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 >> 172.21.15.88:0/3481093931 conn(0x7f307008dc30 msgr2=0x7f3070161220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 shutdown_connections 2024-09-21T07:55:43.316 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.314+0000 7f306f200640 1 -- 172.21.15.88:0/3481093931 wait complete. 2024-09-21T07:55:43.345 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T07:55:43.457 INFO:tasks.ceph.mds.c.smithi088.stdout:starting mds.c at 2024-09-21T07:55:43.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 -- 172.21.15.88:0/3356712929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000b3f30 msgr2=0x7f3e00156ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.573 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 --2- 172.21.15.88:0/3356712929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000b3f30 0x7f3e00156ca0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f3df0005fd0 tx=0x7f3df002a5d0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 -- 172.21.15.88:0/3356712929 shutdown_connections 2024-09-21T07:55:43.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 --2- 172.21.15.88:0/3356712929 >> v2:172.21.15.175:3300/0 conn(0x7f3e000cd1d0 0x7f3e000af0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 --2- 172.21.15.88:0/3356712929 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000b3f30 0x7f3e00156ca0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 -- 172.21.15.88:0/3356712929 >> 172.21.15.88:0/3356712929 conn(0x7f3e0006d620 msgr2=0x7f3e000b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:43.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 -- 172.21.15.88:0/3356712929 shutdown_connections 2024-09-21T07:55:43.574 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.571+0000 7f3e05600640 1 -- 172.21.15.88:0/3356712929 wait complete. 2024-09-21T07:55:43.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.573+0000 7f3e05600640 1 Processor -- start 2024-09-21T07:55:43.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.573+0000 7f3e05600640 1 -- start start 2024-09-21T07:55:43.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.574+0000 7f3e05600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e000b3f30 0x7f3e0014bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.574+0000 7f3e05600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000cd1d0 0x7f3e001446b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e05600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 0x7f3e0009ca50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e05600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3e000af9a0 con 0x7f3e0015d6e0 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e05600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f3e000af820 con 0x7f3e000b3f30 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e05600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f3e000afb20 con 0x7f3e000cd1d0 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 0x7f3e0009ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 0x7f3e0009ca50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43692/0 (socket says 172.21.15.88:43692) 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 -- 172.21.15.88:0/2027313557 learned_addr learned my addr 172.21.15.88:0/2027313557 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3dffe00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e000b3f30 0x7f3e0014bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3df7400640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000cd1d0 0x7f3e001446b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 -- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000cd1d0 msgr2=0x7f3e001446b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000cd1d0 0x7f3e001446b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 -- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e000b3f30 msgr2=0x7f3e0014bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:43.578 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e000b3f30 0x7f3e0014bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:43.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3e04c00640 1 -- 172.21.15.88:0/2027313557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e00145420 con 0x7f3e0015d6e0 2024-09-21T07:55:43.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3dffe00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e000b3f30 0x7f3e0014bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:43.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.575+0000 7f3df7400640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000cd1d0 0x7f3e001446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.576+0000 7f3e04c00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 0x7f3e0009ca50 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3df800eab0 tx=0x7f3df800ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.576+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df800cea0 con 0x7f3e0015d6e0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.576+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3df8004b20 con 0x7f3e0015d6e0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.576+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3df8005020 con 0x7f3e0015d6e0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.576+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e00145700 con 0x7f3e0015d6e0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.576+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e00145cc0 con 0x7f3e0015d6e0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.577+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e00145190 con 0x7f3e0015d6e0 2024-09-21T07:55:43.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.579+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f3df80051c0 con 0x7f3e0015d6e0 2024-09-21T07:55:43.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.580+0000 7f3dfcc00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3ddc079760 0x7f3ddc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:43.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.580+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5910+0+0 (secure 0 0 0) 0x7f3df809a340 con 0x7f3e0015d6e0 2024-09-21T07:55:43.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.580+0000 7f3dffe00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3ddc079760 0x7f3ddc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:43.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.580+0000 7f3dffe00640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3ddc079760 0x7f3ddc07bc70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3df0033000 tx=0x7f3df0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:43.586 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.584+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3df8064260 con 0x7f3e0015d6e0 2024-09-21T07:55:43.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:43.807+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f3e0015c940 con 0x7f3e0015d6e0 2024-09-21T07:55:44.318 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.316+0000 7f3dfcc00640 1 -- 172.21.15.88:0/2027313557 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v121) ==== 104+0+0 (secure 0 0 0) 0x7f3df8063b60 con 0x7f3e0015d6e0 2024-09-21T07:55:44.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3ddc079760 msgr2=0x7f3ddc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:44.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3ddc079760 0x7f3ddc07bc70 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f3df0033000 tx=0x7f3df0004730 comp rx=0 tx=0).stop 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 msgr2=0x7f3e0009ca50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 0x7f3e0009ca50 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f3df800eab0 tx=0x7f3df800ef70 comp rx=0 tx=0).stop 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 shutdown_connections 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f3ddc079760 0x7f3ddc07bc70 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3e000cd1d0 0x7f3e001446b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3e0015d6e0 0x7f3e0009ca50 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 --2- 172.21.15.88:0/2027313557 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f3e000b3f30 0x7f3e0014bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 >> 172.21.15.88:0/2027313557 conn(0x7f3e0006d620 msgr2=0x7f3e000b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 shutdown_connections 2024-09-21T07:55:44.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.319+0000 7f3e05600640 1 -- 172.21.15.88:0/2027313557 wait complete. 2024-09-21T07:55:44.350 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 -- 172.21.15.88:0/1327339280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80af3c0 msgr2=0x7faac81602b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 --2- 172.21.15.88:0/1327339280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80af3c0 0x7faac81602b0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7faac806ddf0 tx=0x7faac002aa30 comp rx=0 tx=0).stop 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 -- 172.21.15.88:0/1327339280 shutdown_connections 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 --2- 172.21.15.88:0/1327339280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80af3c0 0x7faac81602b0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 --2- 172.21.15.88:0/1327339280 >> v2:172.21.15.175:3300/0 conn(0x7faac80b3ec0 0x7faac80b47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 -- 172.21.15.88:0/1327339280 >> 172.21.15.88:0/1327339280 conn(0x7faac80b1dd0 msgr2=0x7faac80b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 -- 172.21.15.88:0/1327339280 shutdown_connections 2024-09-21T07:55:44.575 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.572+0000 7faace800640 1 -- 172.21.15.88:0/1327339280 wait complete. 2024-09-21T07:55:44.576 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.575+0000 7faace800640 1 Processor -- start 2024-09-21T07:55:44.577 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.575+0000 7faace800640 1 -- start start 2024-09-21T07:55:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faace800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 0x7faac81457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:44.579 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faace800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac80af3c0 0x7faac814d310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faace800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac80b3ec0 0x7faac814d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faace800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faac8160b80 con 0x7faac80705b0 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faace800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7faac8160a00 con 0x7faac80b3ec0 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faace800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7faac8154f10 con 0x7faac80af3c0 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faacd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 0x7faac81457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faacd400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 0x7faac81457a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43702/0 (socket says 172.21.15.88:43702) 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.576+0000 7faacd400640 1 -- 172.21.15.88:0/3753906461 learned_addr learned my addr 172.21.15.88:0/3753906461 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacde00640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac80b3ec0 0x7faac814d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacca00640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac80af3c0 0x7faac814d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacd400640 1 -- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac80af3c0 msgr2=0x7faac814d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacd400640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac80af3c0 0x7faac814d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.580 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacd400640 1 -- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac80b3ec0 msgr2=0x7faac814d850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacd400640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac80b3ec0 0x7faac814d850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacd400640 1 -- 172.21.15.88:0/3753906461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac81505a0 con 0x7faac80705b0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacca00640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac80af3c0 0x7faac814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faacd400640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 0x7faac81457a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faabc00e790 tx=0x7faabc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc00eea0 con 0x7faac80705b0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7faabc00cdb0 con 0x7faac80705b0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faabc010690 con 0x7faac80705b0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.577+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac8150e10 con 0x7faac80705b0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.578+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac81513d0 con 0x7faac80705b0 2024-09-21T07:55:44.581 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.579+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac81602b0 con 0x7faac80705b0 2024-09-21T07:55:44.582 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.580+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7faabc010830 con 0x7faac80705b0 2024-09-21T07:55:44.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.581+0000 7faac5600640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faaa8079790 0x7faaa807bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:44.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.581+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(171..171 src has 1..171) ==== 5910+0+0 (secure 0 0 0) 0x7faabc09a010 con 0x7faac80705b0 2024-09-21T07:55:44.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.581+0000 7faacca00640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faaa8079790 0x7faaa807bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:44.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.582+0000 7faacca00640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faaa8079790 0x7faaa807bca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faac8150840 tx=0x7faab4009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:44.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.585+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7faabc063f90 con 0x7faac80705b0 2024-09-21T07:55:44.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:44.771+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7faac81517b0 con 0x7faac80705b0 2024-09-21T07:55:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.318+0000 7faac5600640 1 -- 172.21.15.88:0/3753906461 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v172) ==== 181+0+0 (secure 0 0 0) 0x7faabc063890 con 0x7faac80705b0 2024-09-21T07:55:45.321 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_metadata' removed 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faaa8079790 msgr2=0x7faaa807bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faaa8079790 0x7faaa807bca0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7faac8150840 tx=0x7faab4009040 comp rx=0 tx=0).stop 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 msgr2=0x7faac81457a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 0x7faac81457a0 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7faabc00e790 tx=0x7faabc00ec50 comp rx=0 tx=0).stop 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 shutdown_connections 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7faaa8079790 0x7faaa807bca0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7faac80af3c0 0x7faac814d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7faac80b3ec0 0x7faac814d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 --2- 172.21.15.88:0/3753906461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac80705b0 0x7faac81457a0 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.322+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 >> 172.21.15.88:0/3753906461 conn(0x7faac80b1dd0 msgr2=0x7faac815def0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.323+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 shutdown_connections 2024-09-21T07:55:45.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.323+0000 7faace800640 1 -- 172.21.15.88:0/3753906461 wait complete. 2024-09-21T07:55:45.349 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 -- 172.21.15.88:0/3550808733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 msgr2=0x7f2ce8004c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3550808733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c10 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f2cec009b50 tx=0x7f2ce8005830 comp rx=0 tx=0).stop 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 -- 172.21.15.88:0/3550808733 shutdown_connections 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3550808733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2ce8004c10 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 -- 172.21.15.88:0/3550808733 >> 172.21.15.88:0/3550808733 conn(0x7f2d000b1dd0 msgr2=0x7f2d000b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 -- 172.21.15.88:0/3550808733 shutdown_connections 2024-09-21T07:55:45.583 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.580+0000 7f2d06a00640 1 -- 172.21.15.88:0/3550808733 wait complete. 2024-09-21T07:55:45.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.583+0000 7f2d06a00640 1 Processor -- start 2024-09-21T07:55:45.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.583+0000 7f2d06a00640 1 -- start start 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.584+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2d001463f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.584+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d00146ab0 0x7f2d0009aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.584+0000 7f2d06a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d0009da80 0x7f2d00148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.584+0000 7f2d06a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d00153460 con 0x7f2ce8004800 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.584+0000 7f2d06a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f2d001532e0 con 0x7f2d0009da80 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.584+0000 7f2d06a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f2d001535e0 con 0x7f2d00146ab0 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2d001463f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2d001463f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:43720/0 (socket says 172.21.15.88:43720) 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 -- 172.21.15.88:0/3726182618 learned_addr learned my addr 172.21.15.88:0/3726182618 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d04c00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d00146ab0 0x7f2d0009aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d06000640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d0009da80 0x7f2d00148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:45.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 -- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d00146ab0 msgr2=0x7f2d0009aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d00146ab0 0x7f2d0009aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 -- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d0009da80 msgr2=0x7f2d00148b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d0009da80 0x7f2d00148b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 -- 172.21.15.88:0/3726182618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0009c9e0 con 0x7f2ce8004800 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d06000640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d0009da80 0x7f2d00148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d04c00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d00146ab0 0x7f2d0009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.585+0000 7f2d05600640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2d001463f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4006440 tx=0x7f2cf40049d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.586+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4015020 con 0x7f2ce8004800 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.586+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f2cf4011960 con 0x7f2ce8004800 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.586+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cf4011d70 con 0x7f2ce8004800 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.586+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0009ccc0 con 0x7f2ce8004800 2024-09-21T07:55:45.589 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.586+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0009d280 con 0x7f2ce8004800 2024-09-21T07:55:45.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.587+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d00153b40 con 0x7f2ce8004800 2024-09-21T07:55:45.590 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.588+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f2cf4018020 con 0x7f2ce8004800 2024-09-21T07:55:45.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.590+0000 7f2cfd600640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2ce0079760 0x7f2ce007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:45.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.590+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(172..172 src has 1..172) ==== 5472+0+0 (secure 0 0 0) 0x7f2cf409a030 con 0x7f2ce8004800 2024-09-21T07:55:45.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.590+0000 7f2d04c00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2ce0079760 0x7f2ce007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:45.592 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.591+0000 7f2d04c00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2ce0079760 0x7f2ce007bc70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2cec035000 tx=0x7f2cec0046c0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:45.596 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.594+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f2cf401f030 con 0x7f2ce8004800 2024-09-21T07:55:45.784 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:45.782+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f2d000b0500 con 0x7f2ce8004800 2024-09-21T07:55:46.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.325+0000 7f2cfd600640 1 -- 172.21.15.88:0/3726182618 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v173) ==== 169+0+0 (secure 0 0 0) 0x7f2d000b4ec0 con 0x7f2ce8004800 2024-09-21T07:55:46.327 INFO:teuthology.orchestra.run.smithi088.stderr:pool 'cephfs_data' removed 2024-09-21T07:55:46.331 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2ce0079760 msgr2=0x7f2ce007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2ce0079760 0x7f2ce007bc70 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f2cec035000 tx=0x7f2cec0046c0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 msgr2=0x7f2d001463f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2d001463f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f2cf4006440 tx=0x7f2cf40049d0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 shutdown_connections 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f2ce0079760 0x7f2ce007bc70 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f2d00146ab0 0x7f2d0009aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f2d0009da80 0x7f2d00148b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 --2- 172.21.15.88:0/3726182618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ce8004800 0x7f2d001463f0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.329+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 >> 172.21.15.88:0/3726182618 conn(0x7f2d000b1dd0 msgr2=0x7f2d000b0b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.330+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 shutdown_connections 2024-09-21T07:55:46.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.330+0000 7f2d06a00640 1 -- 172.21.15.88:0/3726182618 wait complete. 2024-09-21T07:55:46.355 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config reset 31 2024-09-21T07:55:46.584 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 -- 172.21.15.88:0/3321239144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f941006f280 msgr2=0x7f941006f690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.622 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 --2- 172.21.15.88:0/3321239144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f941006f280 0x7f941006f690 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f941006f220 tx=0x7f940402a560 comp rx=0 tx=0).stop 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 -- 172.21.15.88:0/3321239144 shutdown_connections 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 --2- 172.21.15.88:0/3321239144 >> v2:172.21.15.175:3300/0 conn(0x7f94101623a0 0x7f94101627b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 --2- 172.21.15.88:0/3321239144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f941006f280 0x7f941006f690 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 -- 172.21.15.88:0/3321239144 >> 172.21.15.88:0/3321239144 conn(0x7f94100b1e40 msgr2=0x7f94100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 -- 172.21.15.88:0/3321239144 shutdown_connections 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.575+0000 7f9416a00640 1 -- 172.21.15.88:0/3321239144 wait complete. 2024-09-21T07:55:46.623 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.577+0000 7f9416a00640 1 Processor -- start 2024-09-21T07:55:46.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.577+0000 7f9416a00640 1 -- start start 2024-09-21T07:55:46.624 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f941006f280 0x7f94101479c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:46.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9410155370 0x7f9410157d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:46.625 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 0x7f94101582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:46.627 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94100b3890 con 0x7f94101623a0 2024-09-21T07:55:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9410073e30 con 0x7f941006f280 2024-09-21T07:55:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f94100b3a10 con 0x7f9410155370 2024-09-21T07:55:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 0x7f94101582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 0x7f94101582c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44512/0 (socket says 172.21.15.88:44512) 2024-09-21T07:55:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9416000640 1 -- 172.21.15.88:0/2576205362 learned_addr learned my addr 172.21.15.88:0/2576205362 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:46.628 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9415600640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f941006f280 0x7f94101479c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:46.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.579+0000 7f9414c00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9410155370 0x7f9410157d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:46.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416000640 1 -- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9410155370 msgr2=0x7f9410157d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416000640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9410155370 0x7f9410157d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.629 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416000640 1 -- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f941006f280 msgr2=0x7f94101479c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416000640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f941006f280 0x7f94101479c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416000640 1 -- 172.21.15.88:0/2576205362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94101f9a70 con 0x7f94101623a0 2024-09-21T07:55:46.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416000640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 0x7f94101582c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f940800c760 tx=0x7f940800cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:46.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9408017070 con 0x7f94101623a0 2024-09-21T07:55:46.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f940800ce10 con 0x7f94101623a0 2024-09-21T07:55:46.630 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94101f91d0 con 0x7f94101623a0 2024-09-21T07:55:46.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94080052b0 con 0x7f94101623a0 2024-09-21T07:55:46.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.580+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94101f9790 con 0x7f94101623a0 2024-09-21T07:55:46.631 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.581+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f94101f8f40 con 0x7f94101623a0 2024-09-21T07:55:46.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.583+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f940801a070 con 0x7f94101623a0 2024-09-21T07:55:46.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.584+0000 7f940d600640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f93f0079760 0x7f93f007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:46.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.584+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f9408098dc0 con 0x7f94101623a0 2024-09-21T07:55:46.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.584+0000 7f9415600640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f93f0079760 0x7f93f007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:46.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.584+0000 7f9415600640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f93f0079760 0x7f93f007bc70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9404033000 tx=0x7f9404004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:46.632 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.588+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9408062400 con 0x7f94101623a0 2024-09-21T07:55:46.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.776+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "config reset", "num": 31} v 0) -- 0x7f941006ee00 con 0x7f94101623a0 2024-09-21T07:55:46.782 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.780+0000 7f940d600640 1 -- 172.21.15.88:0/2576205362 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "config reset", "num": 31}]=0 v53) ==== 71+0+0 (secure 0 0 0) 0x7f941006ee00 con 0x7f94101623a0 2024-09-21T07:55:46.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f93f0079760 msgr2=0x7f93f007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f93f0079760 0x7f93f007bc70 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9404033000 tx=0x7f9404004730 comp rx=0 tx=0).stop 2024-09-21T07:55:46.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 msgr2=0x7f94101582c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:46.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 0x7f94101582c0 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f940800c760 tx=0x7f940800cc20 comp rx=0 tx=0).stop 2024-09-21T07:55:46.785 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 shutdown_connections 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f93f0079760 0x7f93f007bc70 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9410155370 0x7f9410157d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94101623a0 0x7f94101582c0 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 --2- 172.21.15.88:0/2576205362 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f941006f280 0x7f94101479c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 >> 172.21.15.88:0/2576205362 conn(0x7f94100b1e40 msgr2=0x7f94100b11c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 shutdown_connections 2024-09-21T07:55:46.786 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:46.783+0000 7f9416a00640 1 -- 172.21.15.88:0/2576205362 wait complete. 2024-09-21T07:55:46.806 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade' 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 -- 172.21.15.88:0/1761015541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 msgr2=0x7fd0841678a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 --2- 172.21.15.88:0/1761015541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841678a0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7fd07c005fd0 tx=0x7fd07c02a820 comp rx=0 tx=0).stop 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 -- 172.21.15.88:0/1761015541 shutdown_connections 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 --2- 172.21.15.88:0/1761015541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841678a0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 -- 172.21.15.88:0/1761015541 >> 172.21.15.88:0/1761015541 conn(0x7fd08406d730 msgr2=0x7fd0840b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 -- 172.21.15.88:0/1761015541 shutdown_connections 2024-09-21T07:55:47.028 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.025+0000 7fd08c000640 1 -- 172.21.15.88:0/1761015541 wait complete. 2024-09-21T07:55:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.027+0000 7fd08c000640 1 Processor -- start 2024-09-21T07:55:47.029 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.028+0000 7fd08c000640 1 -- start start 2024-09-21T07:55:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08c000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0840ad0e0 0x7fd0840b0340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:47.031 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08c000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd08415d760 0x7fd0840999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08c000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08c000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd084167de0 con 0x7fd084165480 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08c000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd0840b4e30 con 0x7fd08415d760 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08c000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd084167f60 con 0x7fd0840ad0e0 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08a000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841668d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44540/0 (socket says 172.21.15.88:44540) 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.029+0000 7fd08a000640 1 -- 172.21.15.88:0/4157758570 learned_addr learned my addr 172.21.15.88:0/4157758570 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd088c00640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd08415d760 0x7fd0840999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:47.032 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd089600640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0840ad0e0 0x7fd0840b0340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd08a000640 1 -- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0840ad0e0 msgr2=0x7fd0840b0340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:47.033 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd08a000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0840ad0e0 0x7fd0840b0340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd08a000640 1 -- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd08415d760 msgr2=0x7fd0840999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd08a000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd08415d760 0x7fd0840999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd08a000640 1 -- 172.21.15.88:0/4157758570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd084099ee0 con 0x7fd084165480 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd089600640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0840ad0e0 0x7fd0840b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd08a000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841668d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009cd0 tx=0x7fd07c032d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.030+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c037040 con 0x7fd084165480 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.031+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd07c042070 con 0x7fd084165480 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.031+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd07c004280 con 0x7fd084165480 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.031+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0841588a0 con 0x7fd084165480 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.031+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0841585c0 con 0x7fd084165480 2024-09-21T07:55:47.034 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.032+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd084159f70 con 0x7fd084165480 2024-09-21T07:55:47.035 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.033+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fd07c004420 con 0x7fd084165480 2024-09-21T07:55:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.034+0000 7fd081600640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd064079760 0x7fd06407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.034+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fd07c0ba7b0 con 0x7fd084165480 2024-09-21T07:55:47.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.034+0000 7fd089600640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd064079760 0x7fd06407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:47.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.035+0000 7fd089600640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd064079760 0x7fd06407bc70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd078006fd0 tx=0x7fd078008040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:47.040 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.039+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd084099ee0 con 0x7fd084165480 2024-09-21T07:55:47.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.305+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]} v 0) -- 0x7fd084159d20 con 0x7fd084165480 2024-09-21T07:55:47.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.327+0000 7fd081600640 1 -- 172.21.15.88:0/4157758570 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_volumes.TestMisc.test_unparseable_metafile_on_legacy_to_v1_upgrade"]}]=0 v544) ==== 163+0+0 (secure 0 0 0) 0x7fd084159d20 con 0x7fd084165480 2024-09-21T07:55:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd064079760 msgr2=0x7fd06407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd064079760 0x7fd06407bc70 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fd078006fd0 tx=0x7fd078008040 comp rx=0 tx=0).stop 2024-09-21T07:55:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 msgr2=0x7fd0841668d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841668d0 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fd07c009cd0 tx=0x7fd07c032d10 comp rx=0 tx=0).stop 2024-09-21T07:55:47.332 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 shutdown_connections 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd064079760 0x7fd06407bc70 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd084165480 0x7fd0841668d0 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd08415d760 0x7fd0840999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 --2- 172.21.15.88:0/4157758570 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd0840ad0e0 0x7fd0840b0340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 >> 172.21.15.88:0/4157758570 conn(0x7fd08406d730 msgr2=0x7fd0840702f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 shutdown_connections 2024-09-21T07:55:47.333 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:47.330+0000 7fd08c000640 1 -- 172.21.15.88:0/4157758570 wait complete. 2024-09-21T07:55:47.355 INFO:tasks.cephfs_test_runner:Validate unparseable text .meta file on legacy subvol root doesn't break the system ... ok 2024-09-21T07:55:47.355 INFO:tasks.cephfs_test_runner: 2024-09-21T07:55:47.355 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2024-09-21T07:55:47.356 INFO:tasks.cephfs_test_runner:Ran 11 tests in 552.200s 2024-09-21T07:55:47.356 INFO:tasks.cephfs_test_runner: 2024-09-21T07:55:47.356 INFO:tasks.cephfs_test_runner:OK 2024-09-21T07:55:47.356 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2024-09-21T07:55:47.364 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T07:55:47.382 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T07:55:47.383 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:47.383 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:47.418 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:47.418 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-09-21T07:55:47.418 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:47.493 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052628,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248984k,nr_inodes=812246,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497972k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2024-09-21T07:55:47.493 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:47.493 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:47.533 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:47.533 DEBUG:teuthology.orchestra.run.smithi175:> set -ex 2024-09-21T07:55:47.533 DEBUG:teuthology.orchestra.run.smithi175:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T07:55:47.612 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4052617,mode=755 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 none /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 ramfs /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248976k,nr_inodes=812244,mode=700,uid=1000,gid=1333 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,seclabel,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497956k,nr_inodes=819200,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2024-09-21T07:55:47.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:47.613 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:55:47.648 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T07:55:47.648 INFO:teuthology.orchestra.run.smithi088.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2024-09-21T07:55:47.648 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:47.648 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T07:55:47.746 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:47.746 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2024-09-21T07:55:48.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.008 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T07:55:48.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.299 DEBUG:teuthology.orchestra.run.smithi088:> ip netns list 2024-09-21T07:55:48.335 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.335 DEBUG:teuthology.orchestra.run.smithi088:> sudo ip link delete ceph-brx 2024-09-21T07:55:48.435 INFO:teuthology.orchestra.run.smithi088.stderr:Cannot find device "ceph-brx" 2024-09-21T07:55:48.437 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:48.438 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.438 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:55:48.477 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2024-09-21T07:55:48.477 INFO:teuthology.orchestra.run.smithi175.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2024-09-21T07:55:48.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.478 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2024-09-21T07:55:48.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.581 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2024-09-21T07:55:48.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:48.877 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-09-21T07:55:49.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:49.180 DEBUG:teuthology.orchestra.run.smithi175:> ip netns list 2024-09-21T07:55:49.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T07:55:49.216 DEBUG:teuthology.orchestra.run.smithi175:> sudo ip link delete ceph-brx 2024-09-21T07:55:49.315 INFO:teuthology.orchestra.run.smithi175.stderr:Cannot find device "ceph-brx" 2024-09-21T07:55:49.318 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:55:49.318 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-09-21T07:55:49.326 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-09-21T07:55:49.326 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 -- 172.21.15.88:0/3753805738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c152920 msgr2=0x7f935c15f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 --2- 172.21.15.88:0/3753805738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c152920 0x7f935c15f3d0 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f9348009830 tx=0x7f934802a480 comp rx=0 tx=0).stop 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 -- 172.21.15.88:0/3753805738 shutdown_connections 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 --2- 172.21.15.88:0/3753805738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c152920 0x7f935c15f3d0 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 -- 172.21.15.88:0/3753805738 >> 172.21.15.88:0/3753805738 conn(0x7f935c06d730 msgr2=0x7f935c0b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 -- 172.21.15.88:0/3753805738 shutdown_connections 2024-09-21T07:55:49.551 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.549+0000 7f9361800640 1 -- 172.21.15.88:0/3753805738 wait complete. 2024-09-21T07:55:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.551+0000 7f9361800640 1 Processor -- start 2024-09-21T07:55:49.552 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.552+0000 7f9361800640 1 -- start start 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f9361800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 0x7f935c14f550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f9361800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c152920 0x7f935c0c9960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f9361800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f935c15f910 0x7f935c1582c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f9361800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f935c0b4390 con 0x7f935c0aebc0 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f9361800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f935c0b4210 con 0x7f935c152920 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f9361800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f935c0b4510 con 0x7f935c15f910 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f935aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 0x7f935c14f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f935aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 0x7f935c14f550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44566/0 (socket says 172.21.15.88:44566) 2024-09-21T07:55:49.555 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.553+0000 7f935aa00640 1 -- 172.21.15.88:0/3881353239 learned_addr learned my addr 172.21.15.88:0/3881353239 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935a000640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c152920 0x7f935c0c9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935aa00640 1 -- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f935c15f910 msgr2=0x7f935c1582c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935aa00640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f935c15f910 0x7f935c1582c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935aa00640 1 -- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c152920 msgr2=0x7f935c0c9960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935aa00640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c152920 0x7f935c0c9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935aa00640 1 -- 172.21.15.88:0/3881353239 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f935c1f5db0 con 0x7f935c0aebc0 2024-09-21T07:55:49.556 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f935aa00640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 0x7f935c14f550 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c3a0 tx=0x7f934c00c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c011790 con 0x7f935c0aebc0 2024-09-21T07:55:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f934c017070 con 0x7f935c0aebc0 2024-09-21T07:55:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f935c156640 con 0x7f935c0aebc0 2024-09-21T07:55:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f934c011cc0 con 0x7f935c0aebc0 2024-09-21T07:55:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.554+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f935c156c00 con 0x7f935c0aebc0 2024-09-21T07:55:49.557 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.555+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f935c0b49c0 con 0x7f935c0aebc0 2024-09-21T07:55:49.558 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.557+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f934c01c460 con 0x7f935c0aebc0 2024-09-21T07:55:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.558+0000 7f9352a00640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f933c079790 0x7f933c07bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.558+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f934c099680 con 0x7f935c0aebc0 2024-09-21T07:55:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.558+0000 7f935a000640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f933c079790 0x7f933c07bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:49.559 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.559+0000 7f935a000640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f933c079790 0x7f933c07bca0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9348034000 tx=0x7f9348004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:49.563 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.562+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f934c063340 con 0x7f935c0aebc0 2024-09-21T07:55:49.745 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.744+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f935c0d0fc0 con 0x7f933c079790 2024-09-21T07:55:49.746 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.745+0000 7f9352a00640 1 -- 172.21.15.88:0/3881353239 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13503 (secure 0 0 0) 0x7f935c0d0fc0 con 0x7f933c079790 2024-09-21T07:55:49.746 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:49.746 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:55:49.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f933c079790 msgr2=0x7f933c07bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f933c079790 0x7f933c07bca0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f9348034000 tx=0x7f9348004730 comp rx=0 tx=0).stop 2024-09-21T07:55:49.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 msgr2=0x7f935c14f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.748 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 0x7f935c14f550 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f934c00c3a0 tx=0x7f934c00c860 comp rx=0 tx=0).stop 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 shutdown_connections 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f933c079790 0x7f933c07bca0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f935c15f910 0x7f935c1582c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f935c152920 0x7f935c0c9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 --2- 172.21.15.88:0/3881353239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f935c0aebc0 0x7f935c14f550 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 >> 172.21.15.88:0/3881353239 conn(0x7f935c06d730 msgr2=0x7f935c0b1c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 shutdown_connections 2024-09-21T07:55:49.749 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.747+0000 7f9361800640 1 -- 172.21.15.88:0/3881353239 wait complete. 2024-09-21T07:55:49.770 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":259,"stamp":"2024-09-21T07:55:47.763913+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":267600,"kb_used_data":18088,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754707120,"statfs":{"total":773094113280,"available":772820090880,"internally_reserved":0,"allocated":18522112,"data_stored":8850632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-56069,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-81,"num_write_kb":-87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001229"},"pg_stats":[{"pgid":"1.0","version":"74'79","reported_seq":424,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T07:55:45.528513+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:55:45.528513+0000","last_peered":"2024-09-21T07:55:45.528513+0000","last_clean":"2024-09-21T07:55:45.528513+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:55:45.528513+0000","last_undegraded":"2024-09-21T07:55:45.528513+0000","last_fullsized":"2024-09-21T07:55:45.528513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542266,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509562,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2772992,"data_stored":1553338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509562,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2756608,"data_stored":1547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:49.771 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.989+0000 7f9fa8e00640 1 -- 172.21.15.88:0/3036631682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa4152920 msgr2=0x7f9fa415f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.989+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/3036631682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa4152920 0x7f9fa415f3d0 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7f9fa4160420 tx=0x7f9f9002a560 comp rx=0 tx=0).stop 2024-09-21T07:55:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.990+0000 7f9fa8e00640 1 -- 172.21.15.88:0/3036631682 shutdown_connections 2024-09-21T07:55:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.990+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/3036631682 >> v2:172.21.15.175:3300/0 conn(0x7f9fa415f910 0x7f9fa40b3750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.991 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.990+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/3036631682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa4152920 0x7f9fa415f3d0 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.990+0000 7f9fa8e00640 1 -- 172.21.15.88:0/3036631682 >> 172.21.15.88:0/3036631682 conn(0x7f9fa406d730 msgr2=0x7f9fa40b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:49.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.990+0000 7f9fa8e00640 1 -- 172.21.15.88:0/3036631682 shutdown_connections 2024-09-21T07:55:49.992 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.990+0000 7f9fa8e00640 1 -- 172.21.15.88:0/3036631682 wait complete. 2024-09-21T07:55:49.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.992+0000 7f9fa8e00640 1 Processor -- start 2024-09-21T07:55:49.993 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.992+0000 7f9fa8e00640 1 -- start start 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.993+0000 7f9fa8e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa40aebc0 0x7f9fa41f6570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.993+0000 7f9fa8e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fa4152920 0x7f9fa41434b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.993+0000 7f9fa8e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 0x7f9fa40ceaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.993+0000 7f9fa8e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa40b40d0 con 0x7f9fa415f910 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.993+0000 7f9fa8e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f9fa40b3f50 con 0x7f9fa40aebc0 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.993+0000 7f9fa8e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f9fa40b4250 con 0x7f9fa4152920 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 0x7f9fa40ceaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 0x7f9fa40ceaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44576/0 (socket says 172.21.15.88:44576) 2024-09-21T07:55:49.995 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 -- 172.21.15.88:0/4196279765 learned_addr learned my addr 172.21.15.88:0/4196279765 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:49.996 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa1400640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fa4152920 0x7f9fa41434b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa1e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa40aebc0 0x7f9fa41f6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 -- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fa4152920 msgr2=0x7f9fa41434b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fa4152920 0x7f9fa41434b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 -- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa40aebc0 msgr2=0x7f9fa41f6570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa40aebc0 0x7f9fa41f6570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 -- 172.21.15.88:0/4196279765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fa40cda00 con 0x7f9fa415f910 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa1e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa40aebc0 0x7f9fa41f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa1400640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fa4152920 0x7f9fa41434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.994+0000 7f9fa2800640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 0x7f9fa40ceaa0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00eab0 tx=0x7f9f9c00ef70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.995+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c00cea0 con 0x7f9fa415f910 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.995+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9f9c004b20 con 0x7f9fa415f910 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.995+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fa40cdce0 con 0x7f9fa415f910 2024-09-21T07:55:49.997 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.995+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f9c005020 con 0x7f9fa415f910 2024-09-21T07:55:49.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.995+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fa40ce2a0 con 0x7f9fa415f910 2024-09-21T07:55:49.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.996+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fa40b47e0 con 0x7f9fa415f910 2024-09-21T07:55:49.998 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.997+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f9f9c0051c0 con 0x7f9fa415f910 2024-09-21T07:55:50.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.999+0000 7f9f9a000640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f84079760 0x7f9f8407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.999+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f9f9c09a410 con 0x7f9fa415f910 2024-09-21T07:55:50.000 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.999+0000 7f9fa1e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f84079760 0x7f9f8407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.001 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:49.999+0000 7f9fa1e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f84079760 0x7f9f8407bc70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9f94000f60 tx=0x7f9f94009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:50.004 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.003+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f9f9c0642a0 con 0x7f9fa415f910 2024-09-21T07:55:50.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.186+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f9fa40b0bf0 con 0x7f9f84079760 2024-09-21T07:55:50.187 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.186+0000 7f9f9a000640 1 -- 172.21.15.88:0/4196279765 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13503 (secure 0 0 0) 0x7f9fa40b0bf0 con 0x7f9f84079760 2024-09-21T07:55:50.188 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:50.188 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:55:50.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f84079760 msgr2=0x7f9f8407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f84079760 0x7f9f8407bc70 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f9f94000f60 tx=0x7f9f94009040 comp rx=0 tx=0).stop 2024-09-21T07:55:50.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 msgr2=0x7f9fa40ceaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.190 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 0x7f9fa40ceaa0 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f9f9c00eab0 tx=0x7f9f9c00ef70 comp rx=0 tx=0).stop 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 shutdown_connections 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f9f84079760 0x7f9f8407bc70 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f9fa4152920 0x7f9fa41434b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fa415f910 0x7f9fa40ceaa0 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 --2- 172.21.15.88:0/4196279765 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9fa40aebc0 0x7f9fa41f6570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 >> 172.21.15.88:0/4196279765 conn(0x7f9fa406d730 msgr2=0x7f9fa4155730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 shutdown_connections 2024-09-21T07:55:50.191 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.189+0000 7f9fa8e00640 1 -- 172.21.15.88:0/4196279765 wait complete. 2024-09-21T07:55:50.213 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-21T07:55:49.764275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":267600,"kb_used_data":18088,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754707120,"statfs":{"total":773094113280,"available":772820090880,"internally_reserved":0,"allocated":18522112,"data_stored":8850632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-56069,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-81,"num_write_kb":-87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.451795"},"pg_stats":[{"pgid":"1.0","version":"74'79","reported_seq":424,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T07:55:45.528513+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:55:45.528513+0000","last_peered":"2024-09-21T07:55:45.528513+0000","last_clean":"2024-09-21T07:55:45.528513+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:55:45.528513+0000","last_undegraded":"2024-09-21T07:55:45.528513+0000","last_fullsized":"2024-09-21T07:55:45.528513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542266,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509562,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2772992,"data_stored":1553338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509562,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2756608,"data_stored":1547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:50.214 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-09-21T07:55:50.214 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 -- 172.21.15.88:0/629898939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80af170 msgr2=0x7f38c80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.435 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 --2- 172.21.15.88:0/629898939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80af170 0x7f38c80b2f60 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f38c80cfe20 tx=0x7f38b002a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 -- 172.21.15.88:0/629898939 shutdown_connections 2024-09-21T07:55:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 --2- 172.21.15.88:0/629898939 >> v2:172.21.15.175:3300/0 conn(0x7f38c815a0f0 0x7f38c815d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 --2- 172.21.15.88:0/629898939 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80af170 0x7f38c80b2f60 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 -- 172.21.15.88:0/629898939 >> 172.21.15.88:0/629898939 conn(0x7f38c80b1e40 msgr2=0x7f38c80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 -- 172.21.15.88:0/629898939 shutdown_connections 2024-09-21T07:55:50.436 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.434+0000 7f38ce200640 1 -- 172.21.15.88:0/629898939 wait complete. 2024-09-21T07:55:50.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.436+0000 7f38ce200640 1 Processor -- start 2024-09-21T07:55:50.437 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.436+0000 7f38ce200640 1 -- start start 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38ce200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 0x7f38c814fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38ce200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80af170 0x7f38c8159140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38ce200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c815a0f0 0x7f38c8156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38ce200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38c806f290 con 0x7f38c80ad9c0 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38ce200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f38c8070c80 con 0x7f38c80af170 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38ce200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f38c806f410 con 0x7f38c815a0f0 2024-09-21T07:55:50.440 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38cce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 0x7f38c814fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38cce00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 0x7f38c814fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44588/0 (socket says 172.21.15.88:44588) 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38cce00640 1 -- 172.21.15.88:0/520107527 learned_addr learned my addr 172.21.15.88:0/520107527 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.438+0000 7f38bfe00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80af170 0x7f38c8159140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38cce00640 1 -- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c815a0f0 msgr2=0x7f38c8156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38cce00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c815a0f0 0x7f38c8156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38cce00640 1 -- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80af170 msgr2=0x7f38c8159140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38cce00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80af170 0x7f38c8159140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38cce00640 1 -- 172.21.15.88:0/520107527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38c8150120 con 0x7f38c80ad9c0 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38bfe00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80af170 0x7f38c8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38cce00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 0x7f38c814fbe0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f38b800c3d0 tx=0x7f38b800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8017070 con 0x7f38c80ad9c0 2024-09-21T07:55:50.441 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f38b80053c0 con 0x7f38c80ad9c0 2024-09-21T07:55:50.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38c8159680 con 0x7f38c80ad9c0 2024-09-21T07:55:50.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38b8003850 con 0x7f38c80ad9c0 2024-09-21T07:55:50.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.439+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38c80743a0 con 0x7f38c80ad9c0 2024-09-21T07:55:50.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.440+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38c806f930 con 0x7f38c80ad9c0 2024-09-21T07:55:50.442 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.442+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f38b801a070 con 0x7f38c80ad9c0 2024-09-21T07:55:50.443 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.442+0000 7f38c5600640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38a4079760 0x7f38a407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.443+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f38b8098ca0 con 0x7f38c80ad9c0 2024-09-21T07:55:50.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.443+0000 7f38bfe00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38a4079760 0x7f38a407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.444 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.444+0000 7f38bfe00640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38a4079760 0x7f38a407bc70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38b0033000 tx=0x7f38b0004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:50.448 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.447+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f38c8150120 con 0x7f38c80ad9c0 2024-09-21T07:55:50.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.632+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f38c8074d40 con 0x7f38a4079760 2024-09-21T07:55:50.633 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.632+0000 7f38c5600640 1 -- 172.21.15.88:0/520107527 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13503 (secure 0 0 0) 0x7f38c8074d40 con 0x7f38a4079760 2024-09-21T07:55:50.634 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:50.634 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:55:50.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38a4079760 msgr2=0x7f38a407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38a4079760 0x7f38a407bc70 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f38b0033000 tx=0x7f38b0004730 comp rx=0 tx=0).stop 2024-09-21T07:55:50.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 msgr2=0x7f38c814fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 0x7f38c814fbe0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f38b800c3d0 tx=0x7f38b800c890 comp rx=0 tx=0).stop 2024-09-21T07:55:50.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 shutdown_connections 2024-09-21T07:55:50.636 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38a4079760 0x7f38a407bc70 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f38c815a0f0 0x7f38c8156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f38c80af170 0x7f38c8159140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 --2- 172.21.15.88:0/520107527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38c80ad9c0 0x7f38c814fbe0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 >> 172.21.15.88:0/520107527 conn(0x7f38c80b1e40 msgr2=0x7f38c8155b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:50.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 shutdown_connections 2024-09-21T07:55:50.637 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.635+0000 7f38ce200640 1 -- 172.21.15.88:0/520107527 wait complete. 2024-09-21T07:55:50.660 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2024-09-21T07:55:49.764275+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":172,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":267600,"kb_used_data":18088,"kb_used_omap":12,"kb_used_meta":249331,"kb_avail":754707120,"statfs":{"total":773094113280,"available":772820090880,"internally_reserved":0,"allocated":18522112,"data_stored":8850632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":255315505},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-56069,"num_objects":-23,"num_object_clones":0,"num_object_copies":-46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-23,"num_whiteouts":0,"num_read":-4,"num_read_kb":-4,"num_write":-81,"num_write_kb":-87,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.451795"},"pg_stats":[{"pgid":"1.0","version":"74'79","reported_seq":424,"reported_epoch":172,"state":"active+clean","last_fresh":"2024-09-21T07:55:45.528513+0000","last_change":"2024-09-21T07:45:57.326597+0000","last_active":"2024-09-21T07:55:45.528513+0000","last_peered":"2024-09-21T07:55:45.528513+0000","last_clean":"2024-09-21T07:55:45.528513+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:55:45.528513+0000","last_undegraded":"2024-09-21T07:55:45.528513+0000","last_fullsized":"2024-09-21T07:55:45.528513+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T07:45:56.074859+0000","last_clean_scrub_stamp":"2024-09-21T07:45:56.074859+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T16:01:58.226920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542266,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33296,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338544,"statfs":{"total":96636764160,"available":96602669056,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509561,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509562,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33900,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337940,"statfs":{"total":96636764160,"available":96602050560,"internally_reserved":0,"allocated":2772992,"data_stored":1553338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509562,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476860,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476860,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33304,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338536,"statfs":{"total":96636764160,"available":96602660864,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476857,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33880,"kb_used_data":2692,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337960,"statfs":{"total":96636764160,"available":96602071040,"internally_reserved":0,"allocated":2756608,"data_stored":1547694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476857,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33300,"kb_used_data":2112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338540,"statfs":{"total":96636764160,"available":96602664960,"internally_reserved":0,"allocated":2162688,"data_stored":957326,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:50.661 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T07:55:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 -- 172.21.15.88:0/3106979399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3740af630 msgr2=0x7fd374073500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 --2- 172.21.15.88:0/3106979399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3740af630 0x7fd374073500 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fd364009830 tx=0x7fd36402a5c0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 -- 172.21.15.88:0/3106979399 shutdown_connections 2024-09-21T07:55:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 --2- 172.21.15.88:0/3106979399 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd3740af630 0x7fd374073500 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.881 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 -- 172.21.15.88:0/3106979399 >> 172.21.15.88:0/3106979399 conn(0x7fd37406d730 msgr2=0x7fd3740b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:50.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 -- 172.21.15.88:0/3106979399 shutdown_connections 2024-09-21T07:55:50.882 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.880+0000 7fd378c00640 1 -- 172.21.15.88:0/3106979399 wait complete. 2024-09-21T07:55:50.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.882+0000 7fd378c00640 1 Processor -- start 2024-09-21T07:55:50.883 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.882+0000 7fd378c00640 1 -- start start 2024-09-21T07:55:50.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.883+0000 7fd378c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37406f7b0 0x7fd3741595a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.883+0000 7fd378c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd374073a40 0x7fd37409c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.883+0000 7fd378c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 0x7fd3741f6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.885 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.883+0000 7fd378c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd3740b3d90 con 0x7fd3740af630 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.883+0000 7fd378c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fd3740b3c10 con 0x7fd37406f7b0 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.883+0000 7fd378c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fd3740b3f10 con 0x7fd374073a40 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 0x7fd3741f6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 0x7fd3741f6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44592/0 (socket says 172.21.15.88:44592) 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 -- 172.21.15.88:0/1123278159 learned_addr learned my addr 172.21.15.88:0/1123278159 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd371e00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37406f7b0 0x7fd3741595a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd371400640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd374073a40 0x7fd37409c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 -- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd374073a40 msgr2=0x7fd37409c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd374073a40 0x7fd37409c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 -- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37406f7b0 msgr2=0x7fd3741595a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:50.886 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37406f7b0 0x7fd3741595a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:50.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 -- 172.21.15.88:0/1123278159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3741f5fa0 con 0x7fd3740af630 2024-09-21T07:55:50.887 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.884+0000 7fd372800640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 0x7fd3741f6e10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00caa0 tx=0x7fd36c00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.885+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c016020 con 0x7fd3740af630 2024-09-21T07:55:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.885+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fd36c00acf0 con 0x7fd3740af630 2024-09-21T07:55:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.885+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd36c012680 con 0x7fd3740af630 2024-09-21T07:55:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.885+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3741f6280 con 0x7fd3740af630 2024-09-21T07:55:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.885+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3741f6840 con 0x7fd3740af630 2024-09-21T07:55:50.888 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.886+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd3740b43c0 con 0x7fd3740af630 2024-09-21T07:55:50.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.887+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fd36c019020 con 0x7fd3740af630 2024-09-21T07:55:50.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.888+0000 7fd36a000640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd354079790 0x7fd35407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:50.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.888+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fd36c099e30 con 0x7fd3740af630 2024-09-21T07:55:50.889 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.888+0000 7fd371e00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd354079790 0x7fd35407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:50.890 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.889+0000 7fd371e00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd354079790 0x7fd35407bca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd364034000 tx=0x7fd364004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:50.894 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:50.893+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fd36c020020 con 0x7fd3740af630 2024-09-21T07:55:51.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.078+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd3740b1e70 con 0x7fd3740af630 2024-09-21T07:55:51.080 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.079+0000 7fd36a000640 1 -- 172.21.15.88:0/1123278159 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) ==== 74+0+13096 (secure 0 0 0) 0x7fd374073e50 con 0x7fd3740af630 2024-09-21T07:55:51.080 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:51.081 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":173,"fsid":"c1afec4d-fdf7-414b-83b4-ffa85a98fc94","created":"2024-09-21T07:45:38.755045+0000","modified":"2024-09-21T07:55:46.320387+0000","last_up_change":"2024-09-21T07:45:55.070768+0000","last_in_change":"2024-09-21T07:45:44.433149+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T07:45:55.886025+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c4f0fa6c-e896-4720-9487-4bd90e732c8f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6810","nonce":2605666815}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6814","nonce":2605666815}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6819","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6821","nonce":2605666815}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":2605666815},{"type":"v1","addr":"172.21.15.88:6818","nonce":2605666815}]},"public_addr":"172.21.15.88:6810/2605666815","cluster_addr":"172.21.15.88:6814/2605666815","heartbeat_back_addr":"172.21.15.88:6821/2605666815","heartbeat_front_addr":"172.21.15.88:6818/2605666815","state":["exists","up"]},{"osd":1,"uuid":"1a85d167-4a6b-49c0-a642-3935989cfde3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6801","nonce":3750368380}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6803","nonce":3750368380}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6807","nonce":3750368380}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":3750368380},{"type":"v1","addr":"172.21.15.88:6805","nonce":3750368380}]},"public_addr":"172.21.15.88:6801/3750368380","cluster_addr":"172.21.15.88:6803/3750368380","heartbeat_back_addr":"172.21.15.88:6807/3750368380","heartbeat_front_addr":"172.21.15.88:6805/3750368380","state":["exists","up"]},{"osd":2,"uuid":"7ecabd95-42fd-4a37-9c85-7a603ad4f623","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6809","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6811","nonce":4047473449}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6813","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6815","nonce":4047473449}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6823","nonce":4047473449}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6817","nonce":4047473449},{"type":"v1","addr":"172.21.15.88:6820","nonce":4047473449}]},"public_addr":"172.21.15.88:6811/4047473449","cluster_addr":"172.21.15.88:6815/4047473449","heartbeat_back_addr":"172.21.15.88:6823/4047473449","heartbeat_front_addr":"172.21.15.88:6820/4047473449","state":["exists","up"]},{"osd":3,"uuid":"9f106eba-bb27-41e1-8bc9-78e321c28923","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":16,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6825","nonce":1227287750}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6827","nonce":1227287750}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6831","nonce":1227287750}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1227287750},{"type":"v1","addr":"172.21.15.88:6829","nonce":1227287750}]},"public_addr":"172.21.15.88:6825/1227287750","cluster_addr":"172.21.15.88:6827/1227287750","heartbeat_back_addr":"172.21.15.88:6831/1227287750","heartbeat_front_addr":"172.21.15.88:6829/1227287750","state":["exists","up"]},{"osd":4,"uuid":"7cc1d1f3-890e-4a86-9cbd-9376658e53b3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6810","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6811","nonce":1473903548}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6812","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6813","nonce":1473903548}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6816","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6817","nonce":1473903548}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6814","nonce":1473903548},{"type":"v1","addr":"172.21.15.175:6815","nonce":1473903548}]},"public_addr":"172.21.15.175:6811/1473903548","cluster_addr":"172.21.15.175:6813/1473903548","heartbeat_back_addr":"172.21.15.175:6817/1473903548","heartbeat_front_addr":"172.21.15.175:6815/1473903548","state":["exists","up"]},{"osd":5,"uuid":"ab03d263-cece-4d04-aeff-abb758a02353","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6821","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6823","nonce":4267764075}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6825","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6827","nonce":4267764075}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6832","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6833","nonce":4267764075}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6829","nonce":4267764075},{"type":"v1","addr":"172.21.15.175:6831","nonce":4267764075}]},"public_addr":"172.21.15.175:6823/4267764075","cluster_addr":"172.21.15.175:6827/4267764075","heartbeat_back_addr":"172.21.15.175:6833/4267764075","heartbeat_front_addr":"172.21.15.175:6831/4267764075","state":["exists","up"]},{"osd":6,"uuid":"dd7287f4-2961-40e0-ac15-cfd409cce208","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6818","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6819","nonce":443779952}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6820","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6822","nonce":443779952}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6828","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6830","nonce":443779952}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6824","nonce":443779952},{"type":"v1","addr":"172.21.15.175:6826","nonce":443779952}]},"public_addr":"172.21.15.175:6819/443779952","cluster_addr":"172.21.15.175:6822/443779952","heartbeat_back_addr":"172.21.15.175:6830/443779952","heartbeat_front_addr":"172.21.15.175:6826/443779952","state":["exists","up"]},{"osd":7,"uuid":"0e813d39-967a-403e-8dea-c31d2392cd7a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":166,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6802","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6803","nonce":1199047837}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6804","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6805","nonce":1199047837}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6808","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6809","nonce":1199047837}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6806","nonce":1199047837},{"type":"v1","addr":"172.21.15.175:6807","nonce":1199047837}]},"public_addr":"172.21.15.175:6803/1199047837","cluster_addr":"172.21.15.175:6805/1199047837","heartbeat_back_addr":"172.21.15.175:6809/1199047837","heartbeat_front_addr":"172.21.15.175:6807/1199047837","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.185075+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.098361+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.237384+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.590586+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.060924+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.327607+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:51.081652+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T07:45:50.677845+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.88:6835/661983519":"2024-09-22T07:55:43.309270+0000","172.21.15.88:6834/661983519":"2024-09-22T07:55:43.309270+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":19,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T07:55:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.081+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd354079790 msgr2=0x7fd35407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.081+0000 7fd378c00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd354079790 0x7fd35407bca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd364034000 tx=0x7fd364004730 comp rx=0 tx=0).stop 2024-09-21T07:55:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 msgr2=0x7fd3741f6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 0x7fd3741f6e10 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fd36c00caa0 tx=0x7fd36c00cf60 comp rx=0 tx=0).stop 2024-09-21T07:55:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 shutdown_connections 2024-09-21T07:55:51.083 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fd354079790 0x7fd35407bca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fd374073a40 0x7fd37409c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd3740af630 0x7fd3741f6e10 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 --2- 172.21.15.88:0/1123278159 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fd37406f7b0 0x7fd3741595a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 >> 172.21.15.88:0/1123278159 conn(0x7fd37406d730 msgr2=0x7fd3741522e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 shutdown_connections 2024-09-21T07:55:51.084 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:51.082+0000 7fd378c00640 1 -- 172.21.15.88:0/1123278159 wait complete. 2024-09-21T07:55:52.107 INFO:tasks.ceph:Scrubbing osd.0 2024-09-21T07:55:52.107 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:52.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.341+0000 7f599be00640 1 -- 172.21.15.88:0/1971095699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 msgr2=0x7f5980005150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.341+0000 7f599be00640 1 --2- 172.21.15.88:0/1971095699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 0x7f5980005150 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f5994005fd0 tx=0x7f5980005830 comp rx=0 tx=0).stop 2024-09-21T07:55:52.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.341+0000 7f599b400640 1 -- 172.21.15.88:0/1971095699 reap_dead start 2024-09-21T07:55:52.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.341+0000 7f599be00640 1 -- 172.21.15.88:0/1971095699 shutdown_connections 2024-09-21T07:55:52.343 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.341+0000 7f599be00640 1 -- 172.21.15.88:0/1971095699 >> 172.21.15.88:0/1971095699 conn(0x7f599c0b1dd0 msgr2=0x7f599c0b21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:52.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.342+0000 7f599be00640 1 -- 172.21.15.88:0/1971095699 shutdown_connections 2024-09-21T07:55:52.344 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.342+0000 7f599be00640 1 -- 172.21.15.88:0/1971095699 wait complete. 2024-09-21T07:55:52.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.344+0000 7f599be00640 1 Processor -- start 2024-09-21T07:55:52.345 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.344+0000 7f599be00640 1 -- start start 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.345+0000 7f599be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 0x7f599c09d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.345+0000 7f599be00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1415d0 0x7f599c141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.345+0000 7f599be00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c142250 0x7f599c143d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.345+0000 7f599be00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f599c160640 con 0x7f5980004d40 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.345+0000 7f599be00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f599c1604c0 con 0x7f599c1415d0 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.345+0000 7f599be00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f599c1607c0 con 0x7f599c142250 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 0x7f599c09d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 0x7f599c09d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44620/0 (socket says 172.21.15.88:44620) 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 -- 172.21.15.88:0/2596154382 learned_addr learned my addr 172.21.15.88:0/2596154382 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599a000640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1415d0 0x7f599c141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599b400640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c142250 0x7f599c143d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 -- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c142250 msgr2=0x7f599c143d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f599c142250 0x7f599c143d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 -- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1415d0 msgr2=0x7f599c141d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1415d0 0x7f599c141d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 -- 172.21.15.88:0/2596154382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f599c142660 con 0x7f5980004d40 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599a000640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f599c1415d0 0x7f599c141d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f599aa00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 0x7f599c09d120 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f598c011ae0 tx=0x7f598c009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:52.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.346+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c00fcb0 con 0x7f5980004d40 2024-09-21T07:55:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.347+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f598c00fe50 con 0x7f5980004d40 2024-09-21T07:55:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.347+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f598c004280 con 0x7f5980004d40 2024-09-21T07:55:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.347+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f599c1510d0 con 0x7f5980004d40 2024-09-21T07:55:52.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.347+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f599c09c080 con 0x7f5980004d40 2024-09-21T07:55:52.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.348+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f599c09cd00 con 0x7f5980004d40 2024-09-21T07:55:52.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.349+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f598c0035f0 con 0x7f5980004d40 2024-09-21T07:55:52.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.350+0000 7f5992a00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f597c07db70 0x7f597c080080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.350+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f598c09a410 con 0x7f5980004d40 2024-09-21T07:55:52.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.350+0000 7f599a000640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f597c07db70 0x7f597c080080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.350+0000 7f5992a00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7f597c083a80 0x7f597c085ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.350+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 --> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f598c017ec0 con 0x7f597c083a80 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.350+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f598c09f350 con 0x7f5980004d40 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.351+0000 7f599b400640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7f597c083a80 0x7f597c085ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.351+0000 7f599a000640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f597c07db70 0x7f597c080080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5988001eb0 tx=0x7f5988001e20 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.351+0000 7f599b400640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7f597c083a80 0x7f597c085ef0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:52.353 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.352+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== osd.0 v2:172.21.15.88:6808/2605666815 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f598c017ec0 con 0x7f597c083a80 2024-09-21T07:55:52.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.365+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 --> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f599c0b47c0 con 0x7f597c083a80 2024-09-21T07:55:52.366 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.365+0000 7f5992a00640 1 -- 172.21.15.88:0/2596154382 <== osd.0 v2:172.21.15.88:6808/2605666815 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f599c0b47c0 con 0x7f597c083a80 2024-09-21T07:55:52.366 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:52.367 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:52.367 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.365+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7f597c083a80 msgr2=0x7f597c085ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.365+0000 7f599be00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6808/2605666815,v1:172.21.15.88:6810/2605666815] conn(0x7f597c083a80 0x7f597c085ef0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f597c07db70 msgr2=0x7f597c080080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f597c07db70 0x7f597c080080 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f5988001eb0 tx=0x7f5988001e20 comp rx=0 tx=0).stop 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 msgr2=0x7f599c09d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 --2- 172.21.15.88:0/2596154382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5980004d40 0x7f599c09d120 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7f598c011ae0 tx=0x7f598c009f10 comp rx=0 tx=0).stop 2024-09-21T07:55:52.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599b400640 1 -- 172.21.15.88:0/2596154382 reap_dead start 2024-09-21T07:55:52.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 shutdown_connections 2024-09-21T07:55:52.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 >> 172.21.15.88:0/2596154382 conn(0x7f599c0b1dd0 msgr2=0x7f599c155d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:52.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 shutdown_connections 2024-09-21T07:55:52.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.366+0000 7f599be00640 1 -- 172.21.15.88:0/2596154382 wait complete. 2024-09-21T07:55:52.388 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-09-21T07:55:52.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 -- 172.21.15.88:0/1743527894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba80af170 msgr2=0x7f4ba80b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 --2- 172.21.15.88:0/1743527894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba80af170 0x7f4ba80b2f60 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7f4b90009830 tx=0x7f4b9002a4a0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 -- 172.21.15.88:0/1743527894 shutdown_connections 2024-09-21T07:55:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 --2- 172.21.15.88:0/1743527894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba80af170 0x7f4ba80b2f60 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 -- 172.21.15.88:0/1743527894 >> 172.21.15.88:0/1743527894 conn(0x7f4ba80b1e40 msgr2=0x7f4ba80b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 -- 172.21.15.88:0/1743527894 shutdown_connections 2024-09-21T07:55:52.609 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.607+0000 7f4bad600640 1 -- 172.21.15.88:0/1743527894 wait complete. 2024-09-21T07:55:52.610 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.609+0000 7f4bad600640 1 Processor -- start 2024-09-21T07:55:52.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.609+0000 7f4bad600640 1 -- start start 2024-09-21T07:55:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bad600640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba80ad9c0 0x7f4ba81f51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bad600640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ba80af170 0x7f4ba80c9670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.612 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bad600640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 0x7f4ba8154c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bad600640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba806fb20 con 0x7f4ba815a0f0 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bad600640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f4ba806f9a0 con 0x7f4ba80ad9c0 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bad600640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f4ba815d670 con 0x7f4ba80af170 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bacc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 0x7f4ba8154c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bacc00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 0x7f4ba8154c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44646/0 (socket says 172.21.15.88:44646) 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.611+0000 7f4bacc00640 1 -- 172.21.15.88:0/3415974536 learned_addr learned my addr 172.21.15.88:0/3415974536 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4ba7e00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba80ad9c0 0x7f4ba81f51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.613 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4b9f400640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ba80af170 0x7f4ba80c9670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4bacc00640 1 -- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ba80af170 msgr2=0x7f4ba80c9670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4bacc00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ba80af170 0x7f4ba80c9670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4bacc00640 1 -- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba80ad9c0 msgr2=0x7f4ba81f51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4bacc00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba80ad9c0 0x7f4ba81f51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4bacc00640 1 -- 172.21.15.88:0/3415974536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ba81477a0 con 0x7f4ba815a0f0 2024-09-21T07:55:52.614 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4bacc00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 0x7f4ba8154c00 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c760 tx=0x7f4ba000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:52.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba0003da0 con 0x7f4ba815a0f0 2024-09-21T07:55:52.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.612+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f4ba000ce10 con 0x7f4ba815a0f0 2024-09-21T07:55:52.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.613+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ba8147a80 con 0x7f4ba815a0f0 2024-09-21T07:55:52.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.613+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ba00052d0 con 0x7f4ba815a0f0 2024-09-21T07:55:52.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.613+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ba8148040 con 0x7f4ba815a0f0 2024-09-21T07:55:52.615 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.614+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ba80b2f60 con 0x7f4ba815a0f0 2024-09-21T07:55:52.616 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.615+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f4ba0005470 con 0x7f4ba815a0f0 2024-09-21T07:55:52.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.616+0000 7f4ba4c00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4b84079790 0x7f4b8407bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:52.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.616+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f4ba00994e0 con 0x7f4ba815a0f0 2024-09-21T07:55:52.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.616+0000 7f4ba7e00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4b84079790 0x7f4b8407bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:52.617 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.617+0000 7f4ba7e00640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4b84079790 0x7f4b8407bca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b98000f60 tx=0x7f4b98009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:52.621 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.620+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f4ba0062b90 con 0x7f4ba815a0f0 2024-09-21T07:55:52.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.805+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f4ba80b0b10 con 0x7f4b84079790 2024-09-21T07:55:52.807 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.806+0000 7f4ba4c00640 1 -- 172.21.15.88:0/3415974536 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4ba80b0b10 con 0x7f4b84079790 2024-09-21T07:55:52.807 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 0 to deep-scrub 2024-09-21T07:55:52.809 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4b84079790 msgr2=0x7f4b8407bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4b84079790 0x7f4b8407bca0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f4b98000f60 tx=0x7f4b98009040 comp rx=0 tx=0).stop 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 msgr2=0x7f4ba8154c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 0x7f4ba8154c00 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7f4ba000c760 tx=0x7f4ba000cc20 comp rx=0 tx=0).stop 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 shutdown_connections 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f4b84079790 0x7f4b8407bca0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f4ba80af170 0x7f4ba80c9670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ba815a0f0 0x7f4ba8154c00 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 --2- 172.21.15.88:0/3415974536 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f4ba80ad9c0 0x7f4ba81f51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.808+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 >> 172.21.15.88:0/3415974536 conn(0x7f4ba80b1e40 msgr2=0x7f4ba80b1030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:52.810 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.809+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 shutdown_connections 2024-09-21T07:55:52.811 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:52.809+0000 7f4bad600640 1 -- 172.21.15.88:0/3415974536 wait complete. 2024-09-21T07:55:52.832 INFO:tasks.ceph:Scrubbing osd.1 2024-09-21T07:55:52.832 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:53.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.054+0000 7ff9cb000640 1 -- 172.21.15.88:0/1183255637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9ac005220 msgr2=0x7ff9ac00b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.054+0000 7ff9cb000640 1 --2- 172.21.15.88:0/1183255637 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9ac005220 0x7ff9ac00b9c0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7ff9c40b31d0 tx=0x7ff9ac005830 comp rx=0 tx=0).stop 2024-09-21T07:55:53.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.054+0000 7ff9ca600640 1 -- 172.21.15.88:0/1183255637 reap_dead start 2024-09-21T07:55:53.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.055+0000 7ff9cb000640 1 -- 172.21.15.88:0/1183255637 shutdown_connections 2024-09-21T07:55:53.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.055+0000 7ff9cb000640 1 -- 172.21.15.88:0/1183255637 >> 172.21.15.88:0/1183255637 conn(0x7ff9c408bfe0 msgr2=0x7ff9c40d4520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:53.056 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.055+0000 7ff9cb000640 1 -- 172.21.15.88:0/1183255637 shutdown_connections 2024-09-21T07:55:53.057 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.055+0000 7ff9cb000640 1 -- 172.21.15.88:0/1183255637 wait complete. 2024-09-21T07:55:53.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.057+0000 7ff9cb000640 1 Processor -- start 2024-09-21T07:55:53.058 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.057+0000 7ff9cb000640 1 -- start start 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.058+0000 7ff9cb000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ac005220 0x7ff9c41f5840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.058+0000 7ff9cb000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9c40c98b0 0x7ff9c40c9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.058+0000 7ff9cb000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c41f9110 0x7ff9c41f9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9cb000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40b5040 con 0x7ff9c40c98b0 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9cb000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7ff9c40b4ec0 con 0x7ff9ac005220 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9cb000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7ff9c40d29f0 con 0x7ff9c41f9110 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9c40c98b0 0x7ff9c40c9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9c40c98b0 0x7ff9c40c9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44672/0 (socket says 172.21.15.88:44672) 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 -- 172.21.15.88:0/2316853453 learned_addr learned my addr 172.21.15.88:0/2316853453 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9c00640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ac005220 0x7ff9c41f5840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.061 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9ca600640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c41f9110 0x7ff9c41f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 -- 172.21.15.88:0/2316853453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c41f9110 msgr2=0x7ff9c41f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7ff9c41f9110 0x7ff9c41f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 -- 172.21.15.88:0/2316853453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ac005220 msgr2=0x7ff9c41f5840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7ff9ac005220 0x7ff9c41f5840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.059+0000 7ff9c9200640 1 -- 172.21.15.88:0/2316853453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9c40ca200 con 0x7ff9c40c98b0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.060+0000 7ff9c9200640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9c40c98b0 0x7ff9c40c9cc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000fa00 tx=0x7ff9b000fe00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.060+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b000bd70 con 0x7ff9c40c98b0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.060+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7ff9b0017070 con 0x7ff9c40c98b0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.060+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9b00116a0 con 0x7ff9c40c98b0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.060+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9c40d0350 con 0x7ff9c40c98b0 2024-09-21T07:55:53.062 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.060+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9c41f6090 con 0x7ff9c40c98b0 2024-09-21T07:55:53.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.061+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9c4162cf0 con 0x7ff9c40c98b0 2024-09-21T07:55:53.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.062+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7ff9b0011840 con 0x7ff9c40c98b0 2024-09-21T07:55:53.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.063+0000 7ff9c1600640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9a4079760 0x7ff9a407bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.063+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7ff9b009a5c0 con 0x7ff9c40c98b0 2024-09-21T07:55:53.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.063+0000 7ff9c9c00640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9a4079760 0x7ff9a407bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.063+0000 7ff9c1600640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff9a407f670 0x7ff9a4081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.063+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 --> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9b0024e50 con 0x7ff9a407f670 2024-09-21T07:55:53.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.063+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7ff9b009f590 con 0x7ff9c40c98b0 2024-09-21T07:55:53.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.064+0000 7ff9ca600640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff9a407f670 0x7ff9a4081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.064+0000 7ff9c9c00640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9a4079760 0x7ff9a407bc70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff9c408d450 tx=0x7ff9b8005f50 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.064+0000 7ff9ca600640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff9a407f670 0x7ff9a4081ae0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.065+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== osd.1 v2:172.21.15.88:6800/3750368380 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9b0024e50 con 0x7ff9a407f670 2024-09-21T07:55:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.078+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 --> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7ff9c41f5e40 con 0x7ff9a407f670 2024-09-21T07:55:53.079 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9c1600640 1 -- 172.21.15.88:0/2316853453 <== osd.1 v2:172.21.15.88:6800/3750368380 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7ff9c41f5e40 con 0x7ff9a407f670 2024-09-21T07:55:53.080 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:53.080 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:53.080 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff9a407f670 msgr2=0x7ff9a4081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6800/3750368380,v1:172.21.15.88:6801/3750368380] conn(0x7ff9a407f670 0x7ff9a4081ae0 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9a4079760 msgr2=0x7ff9a407bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7ff9a4079760 0x7ff9a407bc70 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7ff9c408d450 tx=0x7ff9b8005f50 comp rx=0 tx=0).stop 2024-09-21T07:55:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9c40c98b0 msgr2=0x7ff9c40c9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.081 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 --2- 172.21.15.88:0/2316853453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9c40c98b0 0x7ff9c40c9cc0 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7ff9b000fa00 tx=0x7ff9b000fe00 comp rx=0 tx=0).stop 2024-09-21T07:55:53.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9ca600640 1 -- 172.21.15.88:0/2316853453 reap_dead start 2024-09-21T07:55:53.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 shutdown_connections 2024-09-21T07:55:53.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 >> 172.21.15.88:0/2316853453 conn(0x7ff9c408bfe0 msgr2=0x7ff9c406de60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:53.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 shutdown_connections 2024-09-21T07:55:53.082 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.079+0000 7ff9cb000640 1 -- 172.21.15.88:0/2316853453 wait complete. 2024-09-21T07:55:53.101 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-09-21T07:55:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 -- 172.21.15.88:0/1914392186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 msgr2=0x7fc22c15a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 --2- 172.21.15.88:0/1914392186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c15a0f0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7fc22c0aea00 tx=0x7fc21402a4d0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 -- 172.21.15.88:0/1914392186 shutdown_connections 2024-09-21T07:55:53.321 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 --2- 172.21.15.88:0/1914392186 >> v2:172.21.15.175:3300/0 conn(0x7fc22c15d670 0x7fc22c070530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 --2- 172.21.15.88:0/1914392186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c15a0f0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 --2- 172.21.15.88:0/1914392186 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c0ad9c0 0x7fc22c0af170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 -- 172.21.15.88:0/1914392186 >> 172.21.15.88:0/1914392186 conn(0x7fc22c0b1e40 msgr2=0x7fc22c0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:53.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 -- 172.21.15.88:0/1914392186 shutdown_connections 2024-09-21T07:55:53.322 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.320+0000 7fc22b200640 1 -- 172.21.15.88:0/1914392186 wait complete. 2024-09-21T07:55:53.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.322+0000 7fc22b200640 1 Processor -- start 2024-09-21T07:55:53.323 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.322+0000 7fc22b200640 1 -- start start 2024-09-21T07:55:53.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22b200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22c0ad9c0 0x7fc22c1524b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22b200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c1fa570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22b200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c15d670 0x7fc22c09aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.325 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22b200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c06f290 con 0x7fc22c0b2f60 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22b200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fc22c070c80 con 0x7fc22c0ad9c0 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22b200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fc22c06f410 con 0x7fc22c15d670 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c1fa570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c1fa570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44694/0 (socket says 172.21.15.88:44694) 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 -- 172.21.15.88:0/3865303702 learned_addr learned my addr 172.21.15.88:0/3865303702 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22a800640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c15d670 0x7fc22c09aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc229e00640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22c0ad9c0 0x7fc22c1524b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 -- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c15d670 msgr2=0x7fc22c09aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c15d670 0x7fc22c09aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.326 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 -- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22c0ad9c0 msgr2=0x7fc22c1524b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22c0ad9c0 0x7fc22c1524b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc221400640 1 -- 172.21.15.88:0/3865303702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc22c0730b0 con 0x7fc22c0b2f60 2024-09-21T07:55:53.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc22a800640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c15d670 0x7fc22c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:53.327 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.324+0000 7fc229e00640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22c0ad9c0 0x7fc22c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.325+0000 7fc221400640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c1fa570 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc214034040 tx=0x7fc2140039e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.325+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21403e030 con 0x7fc22c0b2f60 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.325+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fc2140047a0 con 0x7fc22c0b2f60 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.325+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc22c073390 con 0x7fc22c0b2f60 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.325+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214035040 con 0x7fc22c0b2f60 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.325+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc22c073950 con 0x7fc22c0b2f60 2024-09-21T07:55:53.328 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.326+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22c06a610 con 0x7fc22c0b2f60 2024-09-21T07:55:53.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.327+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fc214041030 con 0x7fc22c0b2f60 2024-09-21T07:55:53.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.328+0000 7fc222a00640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc208079760 0x7fc20807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.328+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fc2140ba200 con 0x7fc22c0b2f60 2024-09-21T07:55:53.329 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.328+0000 7fc229e00640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc208079760 0x7fc20807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.330 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.329+0000 7fc229e00640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc208079760 0x7fc20807bc70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00ab40 tx=0x7fc21c00a520 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.334 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.333+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fc22c06a610 con 0x7fc22c0b2f60 2024-09-21T07:55:53.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.520+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7fc22c072e60 con 0x7fc208079760 2024-09-21T07:55:53.522 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.521+0000 7fc222a00640 1 -- 172.21.15.88:0/3865303702 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7fc22c072e60 con 0x7fc208079760 2024-09-21T07:55:53.522 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 1 to deep-scrub 2024-09-21T07:55:53.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc208079760 msgr2=0x7fc20807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.524 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc208079760 0x7fc20807bc70 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fc21c00ab40 tx=0x7fc21c00a520 comp rx=0 tx=0).stop 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 msgr2=0x7fc22c1fa570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c1fa570 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7fc214034040 tx=0x7fc2140039e0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 shutdown_connections 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fc208079760 0x7fc20807bc70 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fc22c15d670 0x7fc22c09aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc22c0b2f60 0x7fc22c1fa570 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 --2- 172.21.15.88:0/3865303702 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fc22c0ad9c0 0x7fc22c1524b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 >> 172.21.15.88:0/3865303702 conn(0x7fc22c0b1e40 msgr2=0x7fc22c15c220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 shutdown_connections 2024-09-21T07:55:53.525 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.523+0000 7fc22b200640 1 -- 172.21.15.88:0/3865303702 wait complete. 2024-09-21T07:55:53.546 INFO:tasks.ceph:Scrubbing osd.2 2024-09-21T07:55:53.546 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 -- 172.21.15.88:0/2145463752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc06cda0 msgr2=0x7f98fc06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 --2- 172.21.15.88:0/2145463752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc06cda0 0x7f98fc06d1b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f98fc0d01c0 tx=0x7f98e802a5f0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 -- 172.21.15.88:0/2145463752 shutdown_connections 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 --2- 172.21.15.88:0/2145463752 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc06cda0 0x7f98fc06d1b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 --2- 172.21.15.88:0/2145463752 >> v2:172.21.15.175:3300/0 conn(0x7f98fc0cc830 0x7f98fc0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 -- 172.21.15.88:0/2145463752 >> 172.21.15.88:0/2145463752 conn(0x7f98fc0b2540 msgr2=0x7f98fc0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 -- 172.21.15.88:0/2145463752 shutdown_connections 2024-09-21T07:55:53.771 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.769+0000 7f9902e00640 1 -- 172.21.15.88:0/2145463752 wait complete. 2024-09-21T07:55:53.772 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.772+0000 7f9902e00640 1 Processor -- start 2024-09-21T07:55:53.773 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.772+0000 7f9902e00640 1 -- start start 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9902e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98fc06cda0 0x7f98fc0ca190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9902e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98fc0af100 0x7f98fc156200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9902e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc0cc830 0x7f98fc0cf130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9902e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0b35f0 con 0x7f98fc0cc830 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9902e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f98fc0b3470 con 0x7f98fc06cda0 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9902e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f98fc0b3770 con 0x7f98fc0af100 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9900e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc0cc830 0x7f98fc0cf130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9900e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc0cc830 0x7f98fc0cf130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44710/0 (socket says 172.21.15.88:44710) 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.773+0000 7f9900e00640 1 -- 172.21.15.88:0/2263647336 learned_addr learned my addr 172.21.15.88:0/2263647336 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:53.775 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f98fbe00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98fc06cda0 0x7f98fc0ca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f98fb400640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98fc0af100 0x7f98fc156200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f9900e00640 1 -- 172.21.15.88:0/2263647336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98fc0af100 msgr2=0x7f98fc156200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f9900e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f98fc0af100 0x7f98fc156200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f9900e00640 1 -- 172.21.15.88:0/2263647336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98fc06cda0 msgr2=0x7f98fc0ca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f9900e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f98fc06cda0 0x7f98fc0ca190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.776 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f9900e00640 1 -- 172.21.15.88:0/2263647336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98fc0ca6d0 con 0x7f98fc0cc830 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f9900e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc0cc830 0x7f98fc0cf130 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c730 tx=0x7f98f400cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.774+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4003b70 con 0x7f98fc0cc830 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.775+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f98f4003d10 con 0x7f98fc0cc830 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.775+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98f4005330 con 0x7f98fc0cc830 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.775+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98fc156cc0 con 0x7f98fc0cc830 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.775+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98fc1f9410 con 0x7f98fc0cc830 2024-09-21T07:55:53.777 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.776+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f98fc0b41c0 con 0x7f98fc0cc830 2024-09-21T07:55:53.778 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.777+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f98f40054d0 con 0x7f98fc0cc830 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.778+0000 7f98f3e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f98dc079760 0x7f98dc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.778+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f98f4099f30 con 0x7f98fc0cc830 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.778+0000 7f98fbe00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f98dc079760 0x7f98dc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.778+0000 7f98f3e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f98dc07f670 0x7f98dc081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.778+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 --> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f98f4023e50 con 0x7f98dc07f670 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.778+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f98f409c790 con 0x7f98fc0cc830 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.779+0000 7f98fb400640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f98dc07f670 0x7f98dc081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.779+0000 7f98fbe00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f98dc079760 0x7f98dc07bc70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00bb50 tx=0x7f98ec00b2e0 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.780 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.779+0000 7f98fb400640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f98dc07f670 0x7f98dc081ae0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:53.781 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.780+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== osd.2 v2:172.21.15.88:6809/4047473449 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f98f4023e50 con 0x7f98dc07f670 2024-09-21T07:55:53.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.793+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 --> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f98fc156a70 con 0x7f98dc07f670 2024-09-21T07:55:53.794 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.793+0000 7f98f3e00640 1 -- 172.21.15.88:0/2263647336 <== osd.2 v2:172.21.15.88:6809/4047473449 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f98fc156a70 con 0x7f98dc07f670 2024-09-21T07:55:53.794 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:53.795 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:53.795 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f98dc07f670 msgr2=0x7f98dc081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.795 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6809/4047473449,v1:172.21.15.88:6811/4047473449] conn(0x7f98dc07f670 0x7f98dc081ae0 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f98dc079760 msgr2=0x7f98dc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f98dc079760 0x7f98dc07bc70 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f98ec00bb50 tx=0x7f98ec00b2e0 comp rx=0 tx=0).stop 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc0cc830 msgr2=0x7f98fc0cf130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 --2- 172.21.15.88:0/2263647336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98fc0cc830 0x7f98fc0cf130 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f98f400c730 tx=0x7f98f400cb30 comp rx=0 tx=0).stop 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9900e00640 1 -- 172.21.15.88:0/2263647336 reap_dead start 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 shutdown_connections 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 >> 172.21.15.88:0/2263647336 conn(0x7f98fc0b2540 msgr2=0x7f98fc0b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 shutdown_connections 2024-09-21T07:55:53.796 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:53.794+0000 7f9902e00640 1 -- 172.21.15.88:0/2263647336 wait complete. 2024-09-21T07:55:53.815 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-09-21T07:55:54.036 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 -- 172.21.15.88:0/632783217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c070660 msgr2=0x7f025c0af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 --2- 172.21.15.88:0/632783217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c070660 0x7f025c0af6a0 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f024c005fd0 tx=0x7f024c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 -- 172.21.15.88:0/632783217 shutdown_connections 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 --2- 172.21.15.88:0/632783217 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025c0ce8a0 0x7f025c0ad8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 --2- 172.21.15.88:0/632783217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c070660 0x7f025c0af6a0 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 -- 172.21.15.88:0/632783217 >> 172.21.15.88:0/632783217 conn(0x7f025c06d3d0 msgr2=0x7f025c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 -- 172.21.15.88:0/632783217 shutdown_connections 2024-09-21T07:55:54.037 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.035+0000 7f0262400640 1 -- 172.21.15.88:0/632783217 wait complete. 2024-09-21T07:55:54.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.037+0000 7f0262400640 1 Processor -- start 2024-09-21T07:55:54.038 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.038+0000 7f0262400640 1 -- start start 2024-09-21T07:55:54.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f0262400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025c070660 0x7f025c0c9e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f0262400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f025c0ce490 0x7f025c154e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.041 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f0262400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 0x7f025c156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f0262400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f025c0cb950 con 0x7f025c0ce8a0 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f0262400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f025c0aded0 con 0x7f025c0ce490 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f0262400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f025c0cbad0 con 0x7f025c070660 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f025be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 0x7f025c156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f025be00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 0x7f025c156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44732/0 (socket says 172.21.15.88:44732) 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.039+0000 7f025be00640 1 -- 172.21.15.88:0/106695554 learned_addr learned my addr 172.21.15.88:0/106695554 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f0252a00640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f025c0ce490 0x7f025c154e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025b400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025c070660 0x7f025c0c9e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025be00640 1 -- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025c070660 msgr2=0x7f025c0c9e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025be00640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025c070660 0x7f025c0c9e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025be00640 1 -- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f025c0ce490 msgr2=0x7f025c154e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025be00640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f025c0ce490 0x7f025c154e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.042 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025be00640 1 -- 172.21.15.88:0/106695554 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f025c1f61f0 con 0x7f025c0ce8a0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f025be00640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 0x7f025c156b90 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f025400e990 tx=0x7f025400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254004d20 con 0x7f025c0ce8a0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f025400caa0 con 0x7f025c0ce8a0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f025c1f66f0 con 0x7f025c0ce8a0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0254005020 con 0x7f025c0ce8a0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.040+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f025c157b60 con 0x7f025c0ce8a0 2024-09-21T07:55:54.043 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.041+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f025c0af6a0 con 0x7f025c0ce8a0 2024-09-21T07:55:54.044 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.043+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f02540051c0 con 0x7f025c0ce8a0 2024-09-21T07:55:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.044+0000 7f0253e00640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0238079760 0x7f023807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.044+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f025409a270 con 0x7f025c0ce8a0 2024-09-21T07:55:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.044+0000 7f025b400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0238079760 0x7f023807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.045 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.045+0000 7f025b400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0238079760 0x7f023807bc70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f024c033000 tx=0x7f024c004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.049 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.048+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f0254016030 con 0x7f025c0ce8a0 2024-09-21T07:55:54.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.233+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f025c155360 con 0x7f0238079760 2024-09-21T07:55:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.234+0000 7f0253e00640 1 -- 172.21.15.88:0/106695554 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f024c0055c0 con 0x7f0238079760 2024-09-21T07:55:54.235 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 2 to deep-scrub 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.236+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0238079760 msgr2=0x7f023807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.236+0000 7f0262400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0238079760 0x7f023807bc70 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f024c033000 tx=0x7f024c004730 comp rx=0 tx=0).stop 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.236+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 msgr2=0x7f025c156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.236+0000 7f0262400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 0x7f025c156b90 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f025400e990 tx=0x7f025400ee50 comp rx=0 tx=0).stop 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 shutdown_connections 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0238079760 0x7f023807bc70 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.238 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f025c0ce8a0 0x7f025c156b90 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f025c0ce490 0x7f025c154e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 --2- 172.21.15.88:0/106695554 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f025c070660 0x7f025c0c9e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 >> 172.21.15.88:0/106695554 conn(0x7f025c06d3d0 msgr2=0x7f025c0b37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 shutdown_connections 2024-09-21T07:55:54.239 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.237+0000 7f0262400640 1 -- 172.21.15.88:0/106695554 wait complete. 2024-09-21T07:55:54.263 INFO:tasks.ceph:Scrubbing osd.3 2024-09-21T07:55:54.263 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:54.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.486+0000 7fe4a5400640 1 -- 172.21.15.88:0/43420626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 msgr2=0x7fe484004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.486+0000 7fe4a5400640 1 --2- 172.21.15.88:0/43420626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 0x7fe484004060 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fb0 tx=0x7fe484005830 comp rx=0 tx=0).stop 2024-09-21T07:55:54.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.486+0000 7fe49f200640 1 -- 172.21.15.88:0/43420626 reap_dead start 2024-09-21T07:55:54.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.487+0000 7fe4a5400640 1 -- 172.21.15.88:0/43420626 shutdown_connections 2024-09-21T07:55:54.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.487+0000 7fe4a5400640 1 -- 172.21.15.88:0/43420626 >> 172.21.15.88:0/43420626 conn(0x7fe4a00b2540 msgr2=0x7fe4a00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:54.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.487+0000 7fe4a5400640 1 -- 172.21.15.88:0/43420626 shutdown_connections 2024-09-21T07:55:54.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.487+0000 7fe4a5400640 1 -- 172.21.15.88:0/43420626 wait complete. 2024-09-21T07:55:54.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.489+0000 7fe4a5400640 1 Processor -- start 2024-09-21T07:55:54.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.489+0000 7fe4a5400640 1 -- start start 2024-09-21T07:55:54.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.490+0000 7fe4a5400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 0x7fe4a00cdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.490+0000 7fe4a5400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a00c7f50 0x7fe4a00c8360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.490+0000 7fe4a5400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a00c8eb0 0x7fe4a00c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.492 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.490+0000 7fe4a5400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00b32a0 con 0x7fe484003c10 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.490+0000 7fe4a5400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fe4a00b3120 con 0x7fe4a00c7f50 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.490+0000 7fe4a5400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fe4a00b3420 con 0x7fe4a00c8eb0 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 0x7fe4a00cdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 0x7fe4a00cdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44762/0 (socket says 172.21.15.88:44762) 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 -- 172.21.15.88:0/3408978668 learned_addr learned my addr 172.21.15.88:0/3408978668 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe495e00640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a00c7f50 0x7fe4a00c8360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49f200640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a00c8eb0 0x7fe4a00c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 -- 172.21.15.88:0/3408978668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a00c8eb0 msgr2=0x7fe4a00c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fe4a00c8eb0 0x7fe4a00c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 -- 172.21.15.88:0/3408978668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a00c7f50 msgr2=0x7fe4a00c8360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fe4a00c7f50 0x7fe4a00c8360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 -- 172.21.15.88:0/3408978668 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4a009dda0 con 0x7fe484003c10 2024-09-21T07:55:54.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.491+0000 7fe49e800640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 0x7fe4a00cdfb0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fb0 tx=0x7fe490006c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.492+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490003a00 con 0x7fe484003c10 2024-09-21T07:55:54.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.492+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fe490003c90 con 0x7fe484003c10 2024-09-21T07:55:54.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.492+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe490003e60 con 0x7fe484003c10 2024-09-21T07:55:54.494 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.492+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4a009e080 con 0x7fe484003c10 2024-09-21T07:55:54.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.492+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4a00ce660 con 0x7fe484003c10 2024-09-21T07:55:54.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.493+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe4a00d1030 con 0x7fe484003c10 2024-09-21T07:55:54.495 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.494+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fe490035020 con 0x7fe484003c10 2024-09-21T07:55:54.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.495+0000 7fe497400640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe47c079760 0x7fe47c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.495+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fe4900ba270 con 0x7fe484003c10 2024-09-21T07:55:54.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.495+0000 7fe495e00640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe47c079760 0x7fe47c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.496 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.495+0000 7fe497400640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7fe47c07f670 0x7fe47c081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.496+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 --> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe49002ce50 con 0x7fe47c07f670 2024-09-21T07:55:54.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.496+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7fe4900bcb60 con 0x7fe484003c10 2024-09-21T07:55:54.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.496+0000 7fe49f200640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7fe47c07f670 0x7fe47c081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.496+0000 7fe495e00640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe47c079760 0x7fe47c07bc70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00c9a50 tx=0x7fe488006c10 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.497 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.496+0000 7fe49f200640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7fe47c07f670 0x7fe47c081ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.498 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.497+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== osd.3 v2:172.21.15.88:6824/1227287750 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe49002ce50 con 0x7fe47c07f670 2024-09-21T07:55:54.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.510+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 --> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fe4a009e250 con 0x7fe47c07f670 2024-09-21T07:55:54.511 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.510+0000 7fe497400640 1 -- 172.21.15.88:0/3408978668 <== osd.3 v2:172.21.15.88:6824/1227287750 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fe4a009e250 con 0x7fe47c07f670 2024-09-21T07:55:54.512 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:54.512 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:54.512 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7fe47c07f670 msgr2=0x7fe47c081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6824/1227287750,v1:172.21.15.88:6825/1227287750] conn(0x7fe47c07f670 0x7fe47c081ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe47c079760 msgr2=0x7fe47c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fe47c079760 0x7fe47c07bc70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fe4a00c9a50 tx=0x7fe488006c10 comp rx=0 tx=0).stop 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 msgr2=0x7fe4a00cdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 --2- 172.21.15.88:0/3408978668 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe484003c10 0x7fe4a00cdfb0 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7fe490006fb0 tx=0x7fe490006c40 comp rx=0 tx=0).stop 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe49f200640 1 -- 172.21.15.88:0/3408978668 reap_dead start 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 shutdown_connections 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 >> 172.21.15.88:0/3408978668 conn(0x7fe4a00b2540 msgr2=0x7fe4a00ad0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:54.513 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 shutdown_connections 2024-09-21T07:55:54.514 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.511+0000 7fe4a5400640 1 -- 172.21.15.88:0/3408978668 wait complete. 2024-09-21T07:55:54.537 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-09-21T07:55:54.756 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3923200640 1 -- 172.21.15.88:0/158525823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3900003a30 msgr2=0x7f3900003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3923200640 1 --2- 172.21.15.88:0/158525823 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3900003a30 0x7f3900003ea0 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f390c009b20 tx=0x7f3900005830 comp rx=0 tx=0).stop 2024-09-21T07:55:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3921200640 1 -- 172.21.15.88:0/158525823 reap_dead start 2024-09-21T07:55:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3923200640 1 -- 172.21.15.88:0/158525823 shutdown_connections 2024-09-21T07:55:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3923200640 1 -- 172.21.15.88:0/158525823 >> 172.21.15.88:0/158525823 conn(0x7f391c06d3d0 msgr2=0x7f391c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3923200640 1 -- 172.21.15.88:0/158525823 shutdown_connections 2024-09-21T07:55:54.757 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.755+0000 7f3923200640 1 -- 172.21.15.88:0/158525823 wait complete. 2024-09-21T07:55:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.757+0000 7f3923200640 1 Processor -- start 2024-09-21T07:55:54.758 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.758+0000 7f3923200640 1 -- start start 2024-09-21T07:55:54.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3923200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3900003a30 0x7f391c158340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3923200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 0x7f391c14d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3923200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c0c7df0 0x7f391c0c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3923200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f391c0cccc0 con 0x7f391c1562b0 2024-09-21T07:55:54.761 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3923200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f391c1572a0 con 0x7f391c0c7df0 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3923200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f391c0cce40 con 0x7f3900003a30 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f391b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 0x7f391c14d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f391b400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 0x7f391c14d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44786/0 (socket says 172.21.15.88:44786) 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f391b400640 1 -- 172.21.15.88:0/2674547609 learned_addr learned my addr 172.21.15.88:0/2674547609 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f391be00640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3900003a30 0x7f391c158340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.759+0000 7f3921200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c0c7df0 0x7f391c0c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391b400640 1 -- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3900003a30 msgr2=0x7f391c158340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391b400640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3900003a30 0x7f391c158340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391b400640 1 -- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c0c7df0 msgr2=0x7f391c0c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391b400640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c0c7df0 0x7f391c0c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391b400640 1 -- 172.21.15.88:0/2674547609 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f391c15c9a0 con 0x7f391c1562b0 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391be00640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3900003a30 0x7f391c158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:54.762 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f391b400640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 0x7f391c14d3a0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f3908003f50 tx=0x7f39080016f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908011640 con 0x7f391c1562b0 2024-09-21T07:55:54.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f3908016020 con 0x7f391c1562b0 2024-09-21T07:55:54.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3908011a60 con 0x7f391c1562b0 2024-09-21T07:55:54.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f391c0c8740 con 0x7f391c1562b0 2024-09-21T07:55:54.763 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.760+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f391c157860 con 0x7f391c1562b0 2024-09-21T07:55:54.765 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.761+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f391c15c160 con 0x7f391c1562b0 2024-09-21T07:55:54.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.763+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f3908011c00 con 0x7f391c1562b0 2024-09-21T07:55:54.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.764+0000 7f3913e00640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38fc079760 0x7f38fc07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:54.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.764+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f390809a340 con 0x7f391c1562b0 2024-09-21T07:55:54.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.764+0000 7f391be00640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38fc079760 0x7f38fc07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:54.766 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.765+0000 7f391be00640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38fc079760 0x7f38fc07bc70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f390c035000 tx=0x7f390c02d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:54.769 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.769+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f3908020020 con 0x7f391c1562b0 2024-09-21T07:55:54.959 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.958+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f391c0cd3a0 con 0x7f38fc079760 2024-09-21T07:55:54.960 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.958+0000 7f3913e00640 1 -- 172.21.15.88:0/2674547609 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f391c0cd3a0 con 0x7f38fc079760 2024-09-21T07:55:54.960 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 3 to deep-scrub 2024-09-21T07:55:54.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38fc079760 msgr2=0x7f38fc07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38fc079760 0x7f38fc07bc70 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f390c035000 tx=0x7f390c02d040 comp rx=0 tx=0).stop 2024-09-21T07:55:54.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 msgr2=0x7f391c14d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:54.962 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 0x7f391c14d3a0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f3908003f50 tx=0x7f39080016f0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 shutdown_connections 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f38fc079760 0x7f38fc07bc70 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f391c0c7df0 0x7f391c0c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f391c1562b0 0x7f391c14d3a0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 --2- 172.21.15.88:0/2674547609 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f3900003a30 0x7f391c158340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 >> 172.21.15.88:0/2674547609 conn(0x7f391c06d3d0 msgr2=0x7f391c0b3fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 shutdown_connections 2024-09-21T07:55:54.963 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:54.961+0000 7f3923200640 1 -- 172.21.15.88:0/2674547609 wait complete. 2024-09-21T07:55:54.987 INFO:tasks.ceph:Scrubbing osd.4 2024-09-21T07:55:54.987 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 -- 172.21.15.88:0/3517279507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed015ba50 msgr2=0x7f8ed00b3120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 --2- 172.21.15.88:0/3517279507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed015ba50 0x7f8ed00b3120 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8005fd0 tx=0x7f8eb802a4b0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 -- 172.21.15.88:0/3517279507 shutdown_connections 2024-09-21T07:55:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 --2- 172.21.15.88:0/3517279507 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ed00b36f0 0x7f8ed00cb930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 --2- 172.21.15.88:0/3517279507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed015ba50 0x7f8ed00b3120 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 -- 172.21.15.88:0/3517279507 >> 172.21.15.88:0/3517279507 conn(0x7f8ed00b2540 msgr2=0x7f8ed00d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 -- 172.21.15.88:0/3517279507 shutdown_connections 2024-09-21T07:55:55.212 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.210+0000 7f8ed7a00640 1 -- 172.21.15.88:0/3517279507 wait complete. 2024-09-21T07:55:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.212+0000 7f8ed7a00640 1 Processor -- start 2024-09-21T07:55:55.213 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.212+0000 7f8ed7a00640 1 -- start start 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.213+0000 7f8ed7a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed00b36f0 0x7f8ed01f7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.213+0000 7f8ed7a00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed00cc7d0 0x7f8ed01ed120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed7a00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ed015ba50 0x7f8ed01ed660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed7a00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed006f8a0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed7a00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed006f720 con 0x7f8ed00cc7d0 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed7a00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f8ed006fa20 con 0x7f8ed015ba50 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed00b36f0 0x7f8ed01f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed00b36f0 0x7f8ed01f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44804/0 (socket says 172.21.15.88:44804) 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 -- 172.21.15.88:0/2344794562 learned_addr learned my addr 172.21.15.88:0/2344794562 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ec7e00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed00cc7d0 0x7f8ed01ed120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.216 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ed015ba50 0x7f8ed01ed660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 -- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ed015ba50 msgr2=0x7f8ed01ed660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ed015ba50 0x7f8ed01ed660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 -- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed00cc7d0 msgr2=0x7f8ed01ed120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed00cc7d0 0x7f8ed01ed120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 -- 172.21.15.88:0/2344794562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed01f3a10 con 0x7f8ed00b36f0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ec7e00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f8ed00cc7d0 0x7f8ed01ed120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f8ed015ba50 0x7f8ed01ed660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.214+0000 7f8ed5000640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed00b36f0 0x7f8ed01f7600 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c3d0 tx=0x7f8ec000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.215+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0017070 con 0x7f8ed00b36f0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.215+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f8ec00053c0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.215+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed01f3cf0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.215+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec0003850 con 0x7f8ed00b36f0 2024-09-21T07:55:55.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.215+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed01f42b0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.216+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8ed006fef0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.218 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.217+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f8ec001a070 con 0x7f8ed00b36f0 2024-09-21T07:55:55.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.218+0000 7f8ecd600640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8eac079760 0x7f8eac07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.219 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.218+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f8ec00993c0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.218+0000 7f8ecd600640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7f8eac07f670 0x7f8eac081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.218+0000 7f8ec7e00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8eac079760 0x7f8eac07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.218+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 --> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8ec0009e50 con 0x7f8eac07f670 2024-09-21T07:55:55.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.218+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f8ec009c0c0 con 0x7f8ed00b36f0 2024-09-21T07:55:55.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.219+0000 7f8ed5a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7f8eac07f670 0x7f8eac081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.219+0000 7f8ec7e00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8eac079760 0x7f8eac07bc70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8033000 tx=0x7f8eb8004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.220+0000 7f8ed5a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7f8eac07f670 0x7f8eac081ae0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.220+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== osd.4 v2:172.21.15.175:6810/1473903548 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8ec0009e50 con 0x7f8eac07f670 2024-09-21T07:55:55.234 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.233+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 --> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f8ed00994c0 con 0x7f8eac07f670 2024-09-21T07:55:55.235 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.234+0000 7f8ecd600640 1 -- 172.21.15.88:0/2344794562 <== osd.4 v2:172.21.15.175:6810/1473903548 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f8ed00994c0 con 0x7f8eac07f670 2024-09-21T07:55:55.235 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:55.235 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:55.236 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:55.236 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7f8eac07f670 msgr2=0x7f8eac081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.175:6810/1473903548,v1:172.21.15.175:6811/1473903548] conn(0x7f8eac07f670 0x7f8eac081ae0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8eac079760 msgr2=0x7f8eac07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f8eac079760 0x7f8eac07bc70 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f8eb8033000 tx=0x7f8eb8004730 comp rx=0 tx=0).stop 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed00b36f0 msgr2=0x7f8ed01f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 --2- 172.21.15.88:0/2344794562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed00b36f0 0x7f8ed01f7600 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f8ec000c3d0 tx=0x7f8ec000c890 comp rx=0 tx=0).stop 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed5a00640 1 -- 172.21.15.88:0/2344794562 reap_dead start 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 shutdown_connections 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 >> 172.21.15.88:0/2344794562 conn(0x7f8ed00b2540 msgr2=0x7f8ed00afdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 shutdown_connections 2024-09-21T07:55:55.237 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.235+0000 7f8ed7a00640 1 -- 172.21.15.88:0/2344794562 wait complete. 2024-09-21T07:55:55.261 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-09-21T07:55:55.480 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 -- 172.21.15.88:0/1342708868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e8070cc0 msgr2=0x7f77e8156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 --2- 172.21.15.88:0/1342708868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e8070cc0 0x7f77e8156e40 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f77d4005fd0 tx=0x7f77d402a5a0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 -- 172.21.15.88:0/1342708868 shutdown_connections 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 --2- 172.21.15.88:0/1342708868 >> v2:172.21.15.175:3300/0 conn(0x7f77e815ce30 0x7f77e815d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 --2- 172.21.15.88:0/1342708868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e8070cc0 0x7f77e8156e40 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 -- 172.21.15.88:0/1342708868 >> 172.21.15.88:0/1342708868 conn(0x7f77e806d3d0 msgr2=0x7f77e80b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 -- 172.21.15.88:0/1342708868 shutdown_connections 2024-09-21T07:55:55.481 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.479+0000 7f77ef800640 1 -- 172.21.15.88:0/1342708868 wait complete. 2024-09-21T07:55:55.482 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.481+0000 7f77ef800640 1 Processor -- start 2024-09-21T07:55:55.483 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.482+0000 7f77ef800640 1 -- start start 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ef800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77e80cfc80 0x7f77e81f55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ef800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77e815ce30 0x7f77e81594f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ef800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 0x7f77e81574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ef800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80cd050 con 0x7f77e81f5f40 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ef800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f77e80cced0 con 0x7f77e815ce30 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ef800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f77e80cd1d0 con 0x7f77e80cfc80 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.483+0000 7f77ed800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 0x7f77e81574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.485 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 0x7f77e81574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:44810/0 (socket says 172.21.15.88:44810) 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 -- 172.21.15.88:0/613399525 learned_addr learned my addr 172.21.15.88:0/613399525 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ece00640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77e80cfc80 0x7f77e81f55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77e7e00640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77e815ce30 0x7f77e81594f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 -- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77e80cfc80 msgr2=0x7f77e81f55e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77e80cfc80 0x7f77e81f55e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 -- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77e815ce30 msgr2=0x7f77e81594f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77e815ce30 0x7f77e81594f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 -- 172.21.15.88:0/613399525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77e8159d20 con 0x7f77e81f5f40 2024-09-21T07:55:55.486 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77ed800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 0x7f77e81574f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f77e000e990 tx=0x7f77e000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.484+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0004d20 con 0x7f77e81f5f40 2024-09-21T07:55:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.485+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f77e000caa0 con 0x7f77e81f5f40 2024-09-21T07:55:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.485+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77e815a000 con 0x7f77e81f5f40 2024-09-21T07:55:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.485+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77e0005020 con 0x7f77e81f5f40 2024-09-21T07:55:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.485+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77e80afee0 con 0x7f77e81f5f40 2024-09-21T07:55:55.487 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.486+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77e80afcd0 con 0x7f77e81f5f40 2024-09-21T07:55:55.488 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.487+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f77e00051c0 con 0x7f77e81f5f40 2024-09-21T07:55:55.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.488+0000 7f77e4c00640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77c8079760 0x7f77c807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.488+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f77e009a2e0 con 0x7f77e81f5f40 2024-09-21T07:55:55.489 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.488+0000 7f77ece00640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77c8079760 0x7f77c807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.490 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.489+0000 7f77ece00640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77c8079760 0x7f77c807bc70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f77dc007fd0 tx=0x7f77dc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.493 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.492+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f77e0016030 con 0x7f77e81f5f40 2024-09-21T07:55:55.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.675+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) -- 0x7f77e8159a80 con 0x7f77c8079760 2024-09-21T07:55:55.676 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.675+0000 7f77e4c00640 1 -- 172.21.15.88:0/613399525 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f77e8159a80 con 0x7f77c8079760 2024-09-21T07:55:55.676 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 4 to deep-scrub 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.677+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77c8079760 msgr2=0x7f77c807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.677+0000 7f77ef800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77c8079760 0x7f77c807bc70 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f77dc007fd0 tx=0x7f77dc009040 comp rx=0 tx=0).stop 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.677+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 msgr2=0x7f77e81574f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.677+0000 7f77ef800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 0x7f77e81574f0 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f77e000e990 tx=0x7f77e000ee50 comp rx=0 tx=0).stop 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.677+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 shutdown_connections 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f77c8079760 0x7f77c807bc70 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77e81f5f40 0x7f77e81574f0 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f77e815ce30 0x7f77e81594f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 --2- 172.21.15.88:0/613399525 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f77e80cfc80 0x7f77e81f55e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 >> 172.21.15.88:0/613399525 conn(0x7f77e806d3d0 msgr2=0x7f77e8160d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:55.679 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 shutdown_connections 2024-09-21T07:55:55.680 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.678+0000 7f77ef800640 1 -- 172.21.15.88:0/613399525 wait complete. 2024-09-21T07:55:55.700 INFO:tasks.ceph:Scrubbing osd.5 2024-09-21T07:55:55.701 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.926+0000 7f9935000640 1 -- 172.21.15.88:0/1212131687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9914003c10 msgr2=0x7f9914004060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.926+0000 7f9935000640 1 --2- 172.21.15.88:0/1212131687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9914003c10 0x7f9914004060 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f99300b2e80 tx=0x7f9914005830 comp rx=0 tx=0).stop 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.926+0000 7f992e800640 1 -- 172.21.15.88:0/1212131687 reap_dead start 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.927+0000 7f9935000640 1 -- 172.21.15.88:0/1212131687 shutdown_connections 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.927+0000 7f9935000640 1 -- 172.21.15.88:0/1212131687 >> 172.21.15.88:0/1212131687 conn(0x7f99300b2540 msgr2=0x7f99300d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.927+0000 7f9935000640 1 -- 172.21.15.88:0/1212131687 shutdown_connections 2024-09-21T07:55:55.928 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.927+0000 7f9935000640 1 -- 172.21.15.88:0/1212131687 wait complete. 2024-09-21T07:55:55.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.929+0000 7f9935000640 1 Processor -- start 2024-09-21T07:55:55.930 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.929+0000 7f9935000640 1 -- start start 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.930+0000 7f9935000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9914003c10 0x7f99300cf920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.930+0000 7f9935000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f993015ce00 0x7f993015d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.930+0000 7f9935000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f993015b8b0 0x7f993015bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.930+0000 7f9935000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f99300b32a0 con 0x7f993015b8b0 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.930+0000 7f9935000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f99300b3120 con 0x7f9914003c10 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.930+0000 7f9935000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f99300b3420 con 0x7f993015ce00 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f993015b8b0 0x7f993015bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f993015b8b0 0x7f993015bcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46152/0 (socket says 172.21.15.88:46152) 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 -- 172.21.15.88:0/2729953443 learned_addr learned my addr 172.21.15.88:0/2729953443 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992de00640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9914003c10 0x7f99300cf920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.933 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f9925400640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f993015ce00 0x7f993015d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 -- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f993015ce00 msgr2=0x7f993015d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f993015ce00 0x7f993015d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 -- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9914003c10 msgr2=0x7f99300cf920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f9914003c10 0x7f99300cf920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 -- 172.21.15.88:0/2729953443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993015c200 con 0x7f993015b8b0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.931+0000 7f992e800640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f993015b8b0 0x7f993015bcc0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f99300cb1c0 tx=0x7f9928006720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.932+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992800fa80 con 0x7f993015b8b0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.932+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f9928018070 con 0x7f993015b8b0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.932+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992800fe80 con 0x7f993015b8b0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.932+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99301f5f80 con 0x7f993015b8b0 2024-09-21T07:55:55.934 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.932+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99301f6540 con 0x7f993015b8b0 2024-09-21T07:55:55.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.933+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f99301f5b50 con 0x7f993015b8b0 2024-09-21T07:55:55.935 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.934+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f992801d460 con 0x7f993015b8b0 2024-09-21T07:55:55.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.935+0000 7f9926a00640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f990c079760 0x7f990c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.935+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f992809a140 con 0x7f993015b8b0 2024-09-21T07:55:55.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.935+0000 7f9926a00640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7f990c07f670 0x7f990c081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:55.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.935+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 --> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f992801dc30 con 0x7f990c07f670 2024-09-21T07:55:55.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.935+0000 7f992de00640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f990c079760 0x7f990c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.936 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.935+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f99280a3050 con 0x7f993015b8b0 2024-09-21T07:55:55.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.936+0000 7f9925400640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7f990c07f670 0x7f990c081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:55.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.936+0000 7f992de00640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f990c079760 0x7f990c07bc70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9920035000 tx=0x7f992002d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.937 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.936+0000 7f9925400640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7f990c07f670 0x7f990c081ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:55.938 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.937+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== osd.5 v2:172.21.15.175:6821/4267764075 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f992801dc30 con 0x7f990c07f670 2024-09-21T07:55:55.951 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.950+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 --> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f99301f5d10 con 0x7f990c07f670 2024-09-21T07:55:55.952 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9926a00640 1 -- 172.21.15.88:0/2729953443 <== osd.5 v2:172.21.15.175:6821/4267764075 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f99301f5d10 con 0x7f990c07f670 2024-09-21T07:55:55.952 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:55.952 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:55.952 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7f990c07f670 msgr2=0x7f990c081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.175:6821/4267764075,v1:172.21.15.175:6823/4267764075] conn(0x7f990c07f670 0x7f990c081ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f990c079760 msgr2=0x7f990c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f990c079760 0x7f990c07bc70 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f9920035000 tx=0x7f992002d040 comp rx=0 tx=0).stop 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f993015b8b0 msgr2=0x7f993015bcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 --2- 172.21.15.88:0/2729953443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f993015b8b0 0x7f993015bcc0 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7f99300cb1c0 tx=0x7f9928006720 comp rx=0 tx=0).stop 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f992e800640 1 -- 172.21.15.88:0/2729953443 reap_dead start 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 shutdown_connections 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.951+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 >> 172.21.15.88:0/2729953443 conn(0x7f99300b2540 msgr2=0x7f993006f010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:55.953 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.952+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 shutdown_connections 2024-09-21T07:55:55.954 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:55.952+0000 7f9935000640 1 -- 172.21.15.88:0/2729953443 wait complete. 2024-09-21T07:55:55.978 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d94400640 1 -- 172.21.15.88:0/1922084458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d74003a30 msgr2=0x7f7d74003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d94400640 1 --2- 172.21.15.88:0/1922084458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d74003a30 0x7f7d74003ea0 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c09e510 tx=0x7f7d74005830 comp rx=0 tx=0).stop 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d92400640 1 -- 172.21.15.88:0/1922084458 reap_dead start 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d94400640 1 -- 172.21.15.88:0/1922084458 shutdown_connections 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d94400640 1 -- 172.21.15.88:0/1922084458 >> 172.21.15.88:0/1922084458 conn(0x7f7d8c06d3d0 msgr2=0x7f7d8c0b1ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d94400640 1 -- 172.21.15.88:0/1922084458 shutdown_connections 2024-09-21T07:55:56.199 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.197+0000 7f7d94400640 1 -- 172.21.15.88:0/1922084458 wait complete. 2024-09-21T07:55:56.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.199+0000 7f7d94400640 1 Processor -- start 2024-09-21T07:55:56.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.199+0000 7f7d94400640 1 -- start start 2024-09-21T07:55:56.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d94400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d74003a30 0x7f7d8c155e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.203 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d94400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d8c156770 0x7f7d8c1f6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d94400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 0x7f7d8c1f6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d94400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c1571d0 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d94400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f7d8c157050 con 0x7f7d74003a30 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d94400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f7d8c157350 con 0x7f7d8c156770 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 0x7f7d8c1f6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 0x7f7d8c1f6bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46170/0 (socket says 172.21.15.88:46170) 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 -- 172.21.15.88:0/491256994 learned_addr learned my addr 172.21.15.88:0/491256994 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d91000640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d8c156770 0x7f7d8c1f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 -- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d8c156770 msgr2=0x7f7d8c1f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d8c156770 0x7f7d8c1f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d91a00640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d74003a30 0x7f7d8c155e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 -- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d74003a30 msgr2=0x7f7d8c155e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.201+0000 7f7d92400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d74003a30 0x7f7d8c155e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.204 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d92400640 1 -- 172.21.15.88:0/491256994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d8c0c9e20 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d91000640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d8c156770 0x7f7d8c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d91a00640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d74003a30 0x7f7d8c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d92400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 0x7f7d8c1f6bb0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0afc90 tx=0x7f7d84001630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84011630 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f7d84018070 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d8c0c7950 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d84011a90 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.205 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.202+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d8c0ca0e0 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.203+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d8c0afa70 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.206 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.205+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f7d84011c30 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.205+0000 7f7d89600640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7d6c079760 0x7f7d6c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.206+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f7d84099e60 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.206+0000 7f7d91a00640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7d6c079760 0x7f7d6c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.207 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.206+0000 7f7d91a00640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7d6c079760 0x7f7d6c07bc70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7d80033000 tx=0x7f7d80001d70 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.211 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.210+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f7d84063b20 con 0x7f7d8c1f67a0 2024-09-21T07:55:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.396+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) -- 0x7f7d8c0b3310 con 0x7f7d6c079760 2024-09-21T07:55:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.396+0000 7f7d89600640 1 -- 172.21.15.88:0/491256994 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f7d8c0b3310 con 0x7f7d6c079760 2024-09-21T07:55:56.397 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 5 to deep-scrub 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.398+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7d6c079760 msgr2=0x7f7d6c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.398+0000 7f7d94400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7d6c079760 0x7f7d6c07bc70 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f7d80033000 tx=0x7f7d80001d70 comp rx=0 tx=0).stop 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.398+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 msgr2=0x7f7d8c1f6bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.398+0000 7f7d94400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 0x7f7d8c1f6bb0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c0afc90 tx=0x7f7d84001630 comp rx=0 tx=0).stop 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.398+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 shutdown_connections 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f7d6c079760 0x7f7d6c07bc70 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f7d8c156770 0x7f7d8c1f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.400 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d8c1f67a0 0x7f7d8c1f6bb0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 --2- 172.21.15.88:0/491256994 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f7d74003a30 0x7f7d8c155e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 >> 172.21.15.88:0/491256994 conn(0x7f7d8c06d3d0 msgr2=0x7f7d8c15eb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:56.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 shutdown_connections 2024-09-21T07:55:56.401 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.399+0000 7f7d94400640 1 -- 172.21.15.88:0/491256994 wait complete. 2024-09-21T07:55:56.423 INFO:tasks.ceph:Scrubbing osd.6 2024-09-21T07:55:56.423 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:56.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 -- 172.21.15.88:0/904410958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 msgr2=0x7f055c06d1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 --2- 172.21.15.88:0/904410958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c06d1b0 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7f055c06d8a0 tx=0x7f054402a580 comp rx=0 tx=0).stop 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 -- 172.21.15.88:0/904410958 shutdown_connections 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 --2- 172.21.15.88:0/904410958 >> v2:172.21.15.175:3300/0 conn(0x7f055c0af100 0x7f055c0af510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 --2- 172.21.15.88:0/904410958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c06d1b0 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 --2- 172.21.15.88:0/904410958 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0cc830 0x7f055c0b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 -- 172.21.15.88:0/904410958 >> 172.21.15.88:0/904410958 conn(0x7f055c0b2540 msgr2=0x7f055c0d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 -- 172.21.15.88:0/904410958 shutdown_connections 2024-09-21T07:55:56.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.645+0000 7f0563400640 1 -- 172.21.15.88:0/904410958 wait complete. 2024-09-21T07:55:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.647+0000 7f0563400640 1 Processor -- start 2024-09-21T07:55:56.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.648+0000 7f0563400640 1 -- start start 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0563400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c09ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0563400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0af100 0x7f055c08d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0563400640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0cc830 0x7f055c1e9510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0563400640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b34b0 con 0x7f055c06cda0 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0563400640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f055c0b3330 con 0x7f055c0cc830 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0563400640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f055c0b3630 con 0x7f055c0af100 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0560a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c09ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0560a00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c09ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46182/0 (socket says 172.21.15.88:46182) 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0560a00640 1 -- 172.21.15.88:0/106284772 learned_addr learned my addr 172.21.15.88:0/106284772 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0561400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0cc830 0x7f055c1e9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.649+0000 7f0553e00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0af100 0x7f055c08d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.651 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0560a00640 1 -- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0af100 msgr2=0x7f055c08d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0560a00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0af100 0x7f055c08d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0560a00640 1 -- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0cc830 msgr2=0x7f055c1e9510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0560a00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0cc830 0x7f055c1e9510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0560a00640 1 -- 172.21.15.88:0/106284772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f055c08de00 con 0x7f055c06cda0 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0561400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f055c0cc830 0x7f055c1e9510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0553e00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f055c0af100 0x7f055c08d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:56.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0560a00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c09ddc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f054c00d6d0 tx=0x7f054c00db90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c017070 con 0x7f055c06cda0 2024-09-21T07:55:56.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f054c005330 con 0x7f055c06cda0 2024-09-21T07:55:56.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f055c0ce050 con 0x7f055c06cda0 2024-09-21T07:55:56.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f054c0097c0 con 0x7f055c06cda0 2024-09-21T07:55:56.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.650+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f055c0ce610 con 0x7f055c06cda0 2024-09-21T07:55:56.653 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.651+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f055c0af510 con 0x7f055c06cda0 2024-09-21T07:55:56.654 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.653+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f054c009b70 con 0x7f055c06cda0 2024-09-21T07:55:56.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0559600640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0538079760 0x7f053807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f054c09a1a0 con 0x7f055c06cda0 2024-09-21T07:55:56.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0553e00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0538079760 0x7f053807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0559600640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f053807f670 0x7f0538081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 --> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f054c024e80 con 0x7f053807f670 2024-09-21T07:55:56.655 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f054c09cb10 con 0x7f055c06cda0 2024-09-21T07:55:56.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.654+0000 7f0553e00640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0538079760 0x7f053807bc70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0544033000 tx=0x7f0544004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.655+0000 7f0561400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f053807f670 0x7f0538081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.656 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.655+0000 7f0561400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f053807f670 0x7f0538081ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.657 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.656+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== osd.6 v2:172.21.15.175:6818/443779952 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f054c024e80 con 0x7f053807f670 2024-09-21T07:55:56.670 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.669+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 --> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f055c0d1030 con 0x7f053807f670 2024-09-21T07:55:56.671 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.670+0000 7f0559600640 1 -- 172.21.15.88:0/106284772 <== osd.6 v2:172.21.15.175:6818/443779952 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f055c0d1030 con 0x7f053807f670 2024-09-21T07:55:56.671 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:56.671 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:56.671 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:56.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.670+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f053807f670 msgr2=0x7f0538081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.670+0000 7f0563400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.175:6818/443779952,v1:172.21.15.175:6819/443779952] conn(0x7f053807f670 0x7f0538081ae0 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.672 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0538079760 msgr2=0x7f053807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f0538079760 0x7f053807bc70 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f0544033000 tx=0x7f0544004730 comp rx=0 tx=0).stop 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 msgr2=0x7f055c09ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 --2- 172.21.15.88:0/106284772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f055c06cda0 0x7f055c09ddc0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7f054c00d6d0 tx=0x7f054c00db90 comp rx=0 tx=0).stop 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0561400640 1 -- 172.21.15.88:0/106284772 reap_dead start 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 shutdown_connections 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 >> 172.21.15.88:0/106284772 conn(0x7f055c0b2540 msgr2=0x7f055c0b0450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 shutdown_connections 2024-09-21T07:55:56.673 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.671+0000 7f0563400640 1 -- 172.21.15.88:0/106284772 wait complete. 2024-09-21T07:55:56.693 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-09-21T07:55:56.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.915+0000 7f5c7f200640 1 -- 172.21.15.88:0/1885859117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c60003ff0 msgr2=0x7f5c60004800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.915+0000 7f5c7f200640 1 --2- 172.21.15.88:0/1885859117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c60003ff0 0x7f5c60004800 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009cd0 tx=0x7f5c60005830 comp rx=0 tx=0).stop 2024-09-21T07:55:56.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.915+0000 7f5c7e800640 1 -- 172.21.15.88:0/1885859117 reap_dead start 2024-09-21T07:55:56.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.916+0000 7f5c7f200640 1 -- 172.21.15.88:0/1885859117 shutdown_connections 2024-09-21T07:55:56.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.916+0000 7f5c7f200640 1 -- 172.21.15.88:0/1885859117 >> 172.21.15.88:0/1885859117 conn(0x7f5c7808dc30 msgr2=0x7f5c7809d860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:56.917 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.916+0000 7f5c7f200640 1 -- 172.21.15.88:0/1885859117 shutdown_connections 2024-09-21T07:55:56.918 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.916+0000 7f5c7f200640 1 -- 172.21.15.88:0/1885859117 wait complete. 2024-09-21T07:55:56.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.918+0000 7f5c7f200640 1 Processor -- start 2024-09-21T07:55:56.919 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.918+0000 7f5c7f200640 1 -- start start 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.919+0000 7f5c7f200640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60003ff0 0x7f5c781f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.919+0000 7f5c7f200640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 0x7f5c780cdbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.919+0000 7f5c7f200640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c780b1150 0x7f5c780b1560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.919+0000 7f5c7f200640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c780d2660 con 0x7f5c780c8380 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.919+0000 7f5c7f200640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f5c780d24e0 con 0x7f5c60003ff0 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.919+0000 7f5c7f200640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f5c780d27e0 con 0x7f5c780b1150 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 0x7f5c780cdbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 0x7f5c780cdbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46200/0 (socket says 172.21.15.88:46200) 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 -- 172.21.15.88:0/438119357 learned_addr learned my addr 172.21.15.88:0/438119357 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7de00640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60003ff0 0x7f5c781f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.922 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7e800640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c780b1150 0x7f5c780b1560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 -- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c780b1150 msgr2=0x7f5c780b1560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c780b1150 0x7f5c780b1560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 -- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60003ff0 msgr2=0x7f5c781f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60003ff0 0x7f5c781f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 -- 172.21.15.88:0/438119357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c780b1aa0 con 0x7f5c780c8380 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7e800640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c780b1150 0x7f5c780b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.920+0000 7f5c7d400640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 0x7f5c780cdbb0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009c20 tx=0x7f5c680038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.921+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68038040 con 0x7f5c780c8380 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.921+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5c68041070 con 0x7f5c780c8380 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.921+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c68003c90 con 0x7f5c780c8380 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.921+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c780cac70 con 0x7f5c780c8380 2024-09-21T07:55:56.923 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.921+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c780caf50 con 0x7f5c780c8380 2024-09-21T07:55:56.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.922+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c780b94b0 con 0x7f5c780c8380 2024-09-21T07:55:56.924 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.923+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f5c68046460 con 0x7f5c780c8380 2024-09-21T07:55:56.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.924+0000 7f5c72000640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5c5c079760 0x7f5c5c07bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:56.925 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.924+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f5c680ba780 con 0x7f5c780c8380 2024-09-21T07:55:56.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.924+0000 7f5c7de00640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5c5c079760 0x7f5c5c07bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:56.926 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.925+0000 7f5c7de00640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5c5c079760 0x7f5c5c07bc70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c001350 tx=0x7f5c6c006740 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:56.929 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:56.928+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f5c780b1aa0 con 0x7f5c780c8380 2024-09-21T07:55:57.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.115+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) -- 0x7f5c780caa20 con 0x7f5c5c079760 2024-09-21T07:55:57.117 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.116+0000 7f5c72000640 1 -- 172.21.15.88:0/438119357 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5c780caa20 con 0x7f5c5c079760 2024-09-21T07:55:57.117 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 6 to deep-scrub 2024-09-21T07:55:57.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5c5c079760 msgr2=0x7f5c5c07bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5c5c079760 0x7f5c5c07bc70 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c001350 tx=0x7f5c6c006740 comp rx=0 tx=0).stop 2024-09-21T07:55:57.119 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 msgr2=0x7f5c780cdbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 0x7f5c780cdbb0 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009c20 tx=0x7f5c680038a0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 shutdown_connections 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f5c5c079760 0x7f5c5c07bc70 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f5c780b1150 0x7f5c780b1560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c780c8380 0x7f5c780cdbb0 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 --2- 172.21.15.88:0/438119357 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f5c60003ff0 0x7f5c781f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 >> 172.21.15.88:0/438119357 conn(0x7f5c7808dc30 msgr2=0x7f5c780d0cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 shutdown_connections 2024-09-21T07:55:57.120 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.118+0000 7f5c7f200640 1 -- 172.21.15.88:0/438119357 wait complete. 2024-09-21T07:55:57.141 INFO:tasks.ceph:Scrubbing osd.7 2024-09-21T07:55:57.141 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 -- 172.21.15.88:0/1489828687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52140af100 msgr2=0x7f52140af510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 --2- 172.21.15.88:0/1489828687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52140af100 0x7f52140af510 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7f520c005fd0 tx=0x7f520c02aa90 comp rx=0 tx=0).stop 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 -- 172.21.15.88:0/1489828687 shutdown_connections 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 --2- 172.21.15.88:0/1489828687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52140af100 0x7f52140af510 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 --2- 172.21.15.88:0/1489828687 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140cc830 0x7f52140b4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 -- 172.21.15.88:0/1489828687 >> 172.21.15.88:0/1489828687 conn(0x7f52140b2540 msgr2=0x7f52140d02d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 -- 172.21.15.88:0/1489828687 shutdown_connections 2024-09-21T07:55:57.367 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.365+0000 7f5219000640 1 -- 172.21.15.88:0/1489828687 wait complete. 2024-09-21T07:55:57.368 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.367+0000 7f5219000640 1 Processor -- start 2024-09-21T07:55:57.369 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.368+0000 7f5219000640 1 -- start start 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5219000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f521406cda0 0x7f52141f51c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5219000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140af100 0x7f52141565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5219000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140cc830 0x7f5214156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5219000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52140b3600 con 0x7f521406cda0 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5219000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f52140b3480 con 0x7f52140af100 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5219000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f52140b3780 con 0x7f52140cc830 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5211e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f521406cda0 0x7f52141f51c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5211e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f521406cda0 0x7f52141f51c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46214/0 (socket says 172.21.15.88:46214) 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.369+0000 7f5211e00640 1 -- 172.21.15.88:0/1121859155 learned_addr learned my addr 172.21.15.88:0/1121859155 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5212800640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140cc830 0x7f5214156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.371 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5209400640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140af100 0x7f52141565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5211e00640 1 -- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140cc830 msgr2=0x7f5214156b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5211e00640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f52140cc830 0x7f5214156b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5211e00640 1 -- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140af100 msgr2=0x7f52141565d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5211e00640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f52140af100 0x7f52141565d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.372 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5211e00640 1 -- 172.21.15.88:0/1121859155 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52140c8920 con 0x7f521406cda0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5211e00640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f521406cda0 0x7f52141f51c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f520400d740 tx=0x7f520400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5204017070 con 0x7f521406cda0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f5204009d20 con 0x7f521406cda0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52140c8c00 con 0x7f521406cda0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.370+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52040106c0 con 0x7f521406cda0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.371+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f521409c9e0 con 0x7f521406cda0 2024-09-21T07:55:57.373 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.371+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f521409c7d0 con 0x7f521406cda0 2024-09-21T07:55:57.374 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.373+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f520401a070 con 0x7f521406cda0 2024-09-21T07:55:57.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.374+0000 7f520aa00640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f51f0079760 0x7f51f007bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.374+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f520409a050 con 0x7f521406cda0 2024-09-21T07:55:57.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.374+0000 7f5209400640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f51f0079760 0x7f51f007bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.374+0000 7f520aa00640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f51f007f670 0x7f51f0081ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.375 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.374+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 --> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5204011e50 con 0x7f51f007f670 2024-09-21T07:55:57.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.374+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_get_version_reply(handle=1 version=173) ==== 24+0+0 (secure 0 0 0) 0x7f520409c940 con 0x7f521406cda0 2024-09-21T07:55:57.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.375+0000 7f5212800640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f51f007f670 0x7f51f0081ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.375+0000 7f5209400640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f51f0079760 0x7f51f007bc70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f51fc007fd0 tx=0x7f51fc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:57.376 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.375+0000 7f5212800640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f51f007f670 0x7f51f0081ae0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:57.377 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.377+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== osd.7 v2:172.21.15.175:6802/1199047837 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5204011e50 con 0x7f51f007f670 2024-09-21T07:55:57.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.390+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 --> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f52140b3ca0 con 0x7f51f007f670 2024-09-21T07:55:57.391 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.390+0000 7f520aa00640 1 -- 172.21.15.88:0/1121859155 <== osd.7 v2:172.21.15.175:6802/1199047837 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f52140b3ca0 con 0x7f51f007f670 2024-09-21T07:55:57.392 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-09-21T07:55:57.392 INFO:teuthology.orchestra.run.smithi088.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-09-21T07:55:57.392 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f51f007f670 msgr2=0x7f51f0081ae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.175:6802/1199047837,v1:172.21.15.175:6803/1199047837] conn(0x7f51f007f670 0x7f51f0081ae0 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f51f0079760 msgr2=0x7f51f007bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f51f0079760 0x7f51f007bc70 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f51fc007fd0 tx=0x7f51fc009040 comp rx=0 tx=0).stop 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f521406cda0 msgr2=0x7f52141f51c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 --2- 172.21.15.88:0/1121859155 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f521406cda0 0x7f52141f51c0 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7f520400d740 tx=0x7f520400dc00 comp rx=0 tx=0).stop 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5212800640 1 -- 172.21.15.88:0/1121859155 reap_dead start 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 shutdown_connections 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 >> 172.21.15.88:0/1121859155 conn(0x7f52140b2540 msgr2=0x7f52140ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:57.393 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 shutdown_connections 2024-09-21T07:55:57.394 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.391+0000 7f5219000640 1 -- 172.21.15.88:0/1121859155 wait complete. 2024-09-21T07:55:57.413 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-09-21T07:55:57.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 -- 172.21.15.88:0/2582368874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc0af170 msgr2=0x7f28cc0b2f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 --2- 172.21.15.88:0/2582368874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc0af170 0x7f28cc0b2f60 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7f28b4005fd0 tx=0x7f28b402aa10 comp rx=0 tx=0).stop 2024-09-21T07:55:57.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 -- 172.21.15.88:0/2582368874 shutdown_connections 2024-09-21T07:55:57.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 --2- 172.21.15.88:0/2582368874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc0af170 0x7f28cc0b2f60 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 -- 172.21.15.88:0/2582368874 >> 172.21.15.88:0/2582368874 conn(0x7f28cc0b1e40 msgr2=0x7f28cc0b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:57.639 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 -- 172.21.15.88:0/2582368874 shutdown_connections 2024-09-21T07:55:57.640 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.638+0000 7f28d0e00640 1 -- 172.21.15.88:0/2582368874 wait complete. 2024-09-21T07:55:57.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.640+0000 7f28d0e00640 1 Processor -- start 2024-09-21T07:55:57.641 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.640+0000 7f28d0e00640 1 -- start start 2024-09-21T07:55:57.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28d0e00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc0ad9c0 0x7f28cc158750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28d0e00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc0af170 0x7f28cc0b0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28d0e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 0x7f28cc0b0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28d0e00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc06fb20 con 0x7f28cc15a0f0 2024-09-21T07:55:57.644 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28d0e00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f28cc06f9a0 con 0x7f28cc0af170 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28d0e00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f28cc15d670 con 0x7f28cc0ad9c0 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 0x7f28cc0b0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28cb400640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc0ad9c0 0x7f28cc158750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28cbe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 0x7f28cc0b0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46238/0 (socket says 172.21.15.88:46238) 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28cbe00640 1 -- 172.21.15.88:0/3192708301 learned_addr learned my addr 172.21.15.88:0/3192708301 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.642+0000 7f28c2a00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc0af170 0x7f28cc0b0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28cbe00640 1 -- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc0ad9c0 msgr2=0x7f28cc158750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28cbe00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc0ad9c0 0x7f28cc158750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28cbe00640 1 -- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc0af170 msgr2=0x7f28cc0b0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28cbe00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc0af170 0x7f28cc0b0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28cbe00640 1 -- 172.21.15.88:0/3192708301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28cc1f5b10 con 0x7f28cc15a0f0 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28cbe00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 0x7f28cc0b0bb0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c3c0 tx=0x7f28c400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:57.645 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4017070 con 0x7f28cc15a0f0 2024-09-21T07:55:57.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f28c40035f0 con 0x7f28cc15a0f0 2024-09-21T07:55:57.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c4003a40 con 0x7f28cc15a0f0 2024-09-21T07:55:57.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28cc1f5df0 con 0x7f28cc15a0f0 2024-09-21T07:55:57.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.643+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28cc1f63b0 con 0x7f28cc15a0f0 2024-09-21T07:55:57.646 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.645+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28cc0b2f60 con 0x7f28cc15a0f0 2024-09-21T07:55:57.647 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.646+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f28c401a070 con 0x7f28cc15a0f0 2024-09-21T07:55:57.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.647+0000 7f28c3e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f28a8079760 0x7f28a807bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:57.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.647+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f28c4098a60 con 0x7f28cc15a0f0 2024-09-21T07:55:57.648 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.647+0000 7f28cb400640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f28a8079760 0x7f28a807bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:57.649 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.648+0000 7f28cb400640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f28a8079760 0x7f28a807bc70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000f60 tx=0x7f28bc009040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:57.652 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.651+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f28c4062100 con 0x7f28cc15a0f0 2024-09-21T07:55:57.834 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:57.839 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:55:57.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.847+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) -- 0x7f28cc15bac0 con 0x7f28a8079760 2024-09-21T07:55:57.848 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.847+0000 7f28c3e00640 1 -- 172.21.15.88:0/3192708301 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f28cc15bac0 con 0x7f28a8079760 2024-09-21T07:55:57.849 INFO:teuthology.orchestra.run.smithi088.stderr:instructed osd(s) 7 to deep-scrub 2024-09-21T07:55:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f28a8079760 msgr2=0x7f28a807bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f28a8079760 0x7f28a807bc70 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f28bc000f60 tx=0x7f28bc009040 comp rx=0 tx=0).stop 2024-09-21T07:55:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 msgr2=0x7f28cc0b0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:57.852 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 0x7f28cc0b0bb0 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c3c0 tx=0x7f28c400c880 comp rx=0 tx=0).stop 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 shutdown_connections 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f28a8079760 0x7f28a807bc70 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28cc15a0f0 0x7f28cc0b0bb0 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f28cc0af170 0x7f28cc0b0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 --2- 172.21.15.88:0/3192708301 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f28cc0ad9c0 0x7f28cc158750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 >> 172.21.15.88:0/3192708301 conn(0x7f28cc0b1e40 msgr2=0x7f28cc15a840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 shutdown_connections 2024-09-21T07:55:57.853 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:57.851+0000 7f28d0e00640 1 -- 172.21.15.88:0/3192708301 wait complete. 2024-09-21T07:55:57.879 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T07:55:58.147 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.145+0000 7f6f16000640 1 -- 172.21.15.88:0/756804677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 msgr2=0x7f6f100af170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.145+0000 7f6f16000640 1 --2- 172.21.15.88:0/756804677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 0x7f6f100af170 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f6f04009830 tx=0x7f6f0402a520 comp rx=0 tx=0).stop 2024-09-21T07:55:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.145+0000 7f6f16000640 1 -- 172.21.15.88:0/756804677 shutdown_connections 2024-09-21T07:55:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.145+0000 7f6f16000640 1 --2- 172.21.15.88:0/756804677 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 0x7f6f100af170 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.145+0000 7f6f16000640 1 -- 172.21.15.88:0/756804677 >> 172.21.15.88:0/756804677 conn(0x7f6f100b1e40 msgr2=0x7f6f100b2250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.146+0000 7f6f16000640 1 -- 172.21.15.88:0/756804677 shutdown_connections 2024-09-21T07:55:58.148 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.146+0000 7f6f16000640 1 -- 172.21.15.88:0/756804677 wait complete. 2024-09-21T07:55:58.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.148+0000 7f6f16000640 1 Processor -- start 2024-09-21T07:55:58.149 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.148+0000 7f6f16000640 1 -- start start 2024-09-21T07:55:58.153 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.149+0000 7f6f16000640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 0x7f6f10156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.149+0000 7f6f16000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 0x7f6f100b0880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.149+0000 7f6f16000640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1015a0f0 0x7f6f100b0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.149+0000 7f6f16000640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1006fb20 con 0x7f6f100b2f60 2024-09-21T07:55:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.149+0000 7f6f16000640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7f6f1006f9a0 con 0x7f6f100ad9c0 2024-09-21T07:55:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.149+0000 7f6f16000640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7f6f1015d670 con 0x7f6f1015a0f0 2024-09-21T07:55:58.154 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 0x7f6f100b0880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 0x7f6f100b0880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46254/0 (socket says 172.21.15.88:46254) 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 -- 172.21.15.88:0/54259275 learned_addr learned my addr 172.21.15.88:0/54259275 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f14c00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 0x7f6f10156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f15600640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1015a0f0 0x7f6f100b0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 -- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1015a0f0 msgr2=0x7f6f100b0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1015a0f0 0x7f6f100b0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.155 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 -- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 msgr2=0x7f6f10156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 0x7f6f10156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.150+0000 7f6f0fe00640 1 -- 172.21.15.88:0/54259275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f10156eb0 con 0x7f6f100b2f60 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.151+0000 7f6f0fe00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 0x7f6f100b0880 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c400 tx=0x7f6efc00c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.151+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc011790 con 0x7f6f100b2f60 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.151+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7f6efc017070 con 0x7f6f100b2f60 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.151+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f10157190 con 0x7f6f100b2f60 2024-09-21T07:55:58.156 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.151+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6efc011cc0 con 0x7f6f100b2f60 2024-09-21T07:55:58.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.151+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f10157750 con 0x7f6f100b2f60 2024-09-21T07:55:58.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.152+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f10156c20 con 0x7f6f100b2f60 2024-09-21T07:55:58.157 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.153+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7f6efc01c460 con 0x7f6f100b2f60 2024-09-21T07:55:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.154+0000 7f6f0cc00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ef0079790 0x7f6ef007bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.154+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7f6efc0996f0 con 0x7f6f100b2f60 2024-09-21T07:55:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.155+0000 7f6f14c00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ef0079790 0x7f6ef007bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.158 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.156+0000 7f6f14c00640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ef0079790 0x7f6ef007bca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6f04034000 tx=0x7f6f04004730 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:58.160 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.159+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7f6efc0632a0 con 0x7f6f100b2f60 2024-09-21T07:55:58.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.351+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 --> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6f100af170 con 0x7f6ef0079790 2024-09-21T07:55:58.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.352+0000 7f6f0cc00640 1 -- 172.21.15.88:0/54259275 <== mgr.4116 v2:172.21.15.88:6836/3325523616 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+13370 (secure 0 0 0) 0x7f6f100af170 con 0x7f6ef0079790 2024-09-21T07:55:58.353 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:58.353 INFO:teuthology.orchestra.run.smithi088.stderr:dumped all 2024-09-21T07:55:58.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ef0079790 msgr2=0x7f6ef007bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ef0079790 0x7f6ef007bca0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f6f04034000 tx=0x7f6f04004730 comp rx=0 tx=0).stop 2024-09-21T07:55:58.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 msgr2=0x7f6f100b0880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 0x7f6f100b0880 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7f6efc00c400 tx=0x7f6efc00c8c0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 shutdown_connections 2024-09-21T07:55:58.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7f6ef0079790 0x7f6ef007bca0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7f6f1015a0f0 0x7f6f100b0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f100b2f60 0x7f6f100b0880 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 --2- 172.21.15.88:0/54259275 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7f6f100ad9c0 0x7f6f10156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 >> 172.21.15.88:0/54259275 conn(0x7f6f100b1e40 msgr2=0x7f6f1015a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:58.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 shutdown_connections 2024-09-21T07:55:58.356 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.354+0000 7f6f16000640 1 -- 172.21.15.88:0/54259275 wait complete. 2024-09-21T07:55:58.376 INFO:teuthology.orchestra.run.smithi088.stdout:{"pg_ready":true,"pg_map":{"version":264,"stamp":"2024-09-21T07:55:57.765895+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":276012,"kb_used_data":18184,"kb_used_omap":12,"kb_used_meta":257651,"kb_avail":754698708,"statfs":{"total":773094113280,"available":772811476992,"internally_reserved":0,"allocated":18620416,"data_stored":8884496,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":263835185},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.438653"},"pg_stats":[{"pgid":"1.0","version":"173'81","reported_seq":436,"reported_epoch":173,"state":"active+clean","last_fresh":"2024-09-21T07:55:53.676024+0000","last_change":"2024-09-21T07:55:53.676024+0000","last_active":"2024-09-21T07:55:53.676024+0000","last_peered":"2024-09-21T07:55:53.676024+0000","last_clean":"2024-09-21T07:55:53.676024+0000","last_became_active":"2024-09-21T07:45:57.325335+0000","last_became_peered":"2024-09-21T07:45:57.325335+0000","last_unstale":"2024-09-21T07:55:53.676024+0000","last_undegraded":"2024-09-21T07:55:53.676024+0000","last_fullsized":"2024-09-21T07:55:53.676024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"173'81","last_scrub_stamp":"2024-09-21T07:55:53.675932+0000","last_deep_scrub":"173'81","last_deep_scrub_stamp":"2024-09-21T07:55:53.675932+0000","last_clean_scrub_stamp":"2024-09-21T07:55:53.675932+0000","objects_scrubbed":2,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:15:45.729512+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":14,"seq":60129542268,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":15,"seq":64424509563,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":15,"seq":64424509564,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33892,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94337948,"statfs":{"total":96636764160,"available":96602058752,"internally_reserved":0,"allocated":2772992,"data_stored":1553338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":15,"seq":64424509564,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":37480,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334360,"statfs":{"total":96636764160,"available":96598384640,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":16,"seq":68719476862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33320,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338520,"statfs":{"total":96636764160,"available":96602644480,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":16,"seq":68719476862,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":16,"seq":68719476859,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":38052,"kb_used_data":2708,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94333788,"statfs":{"total":96636764160,"available":96597798912,"internally_reserved":0,"allocated":2772992,"data_stored":1553338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":16,"seq":68719476859,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":33316,"kb_used_data":2128,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":94338524,"statfs":{"total":96636764160,"available":96602648576,"internally_reserved":0,"allocated":2179072,"data_stored":962970,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T07:55:58.379 DEBUG:teuthology.orchestra.run.smithi088:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-09-21T07:55:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 -- 172.21.15.88:0/2014559886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd740b30d0 msgr2=0x7fcd740b34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.598 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2014559886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd740b30d0 0x7fcd740b34e0 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7fcd740b42e0 tx=0x7fcd5c02a4b0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 -- 172.21.15.88:0/2014559886 shutdown_connections 2024-09-21T07:55:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2014559886 >> v2:172.21.15.175:3300/0 conn(0x7fcd740ce7d0 0x7fcd740cebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2014559886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd740b30d0 0x7fcd740b34e0 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 -- 172.21.15.88:0/2014559886 >> 172.21.15.88:0/2014559886 conn(0x7fcd7406d620 msgr2=0x7fcd740b1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 -- 172.21.15.88:0/2014559886 shutdown_connections 2024-09-21T07:55:58.599 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.597+0000 7fcd78c00640 1 -- 172.21.15.88:0/2014559886 wait complete. 2024-09-21T07:55:58.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.599+0000 7fcd78c00640 1 Processor -- start 2024-09-21T07:55:58.600 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.599+0000 7fcd78c00640 1 -- start start 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.600+0000 7fcd78c00640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd740b30d0 0x7fcd740adac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.600+0000 7fcd78c00640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd740ce7d0 0x7fcd74099910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.600+0000 7fcd78c00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 0x7fcd74152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd78c00640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd740cf4b0 con 0x7fcd7415b670 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd78c00640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fcd740cf330 con 0x7fcd740b30d0 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd78c00640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fcd740cf630 con 0x7fcd740ce7d0 2024-09-21T07:55:58.602 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 0x7fcd74152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 0x7fcd74152a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46270/0 (socket says 172.21.15.88:46270) 2024-09-21T07:55:58.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 -- 172.21.15.88:0/2896218023 learned_addr learned my addr 172.21.15.88:0/2896218023 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:58.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73400640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd740b30d0 0x7fcd740adac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.603 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd6be00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd740ce7d0 0x7fcd74099910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 -- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd740ce7d0 msgr2=0x7fcd74099910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd740ce7d0 0x7fcd74099910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 -- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd740b30d0 msgr2=0x7fcd740adac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd740b30d0 0x7fcd740adac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.604 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd73e00640 1 -- 172.21.15.88:0/2896218023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd740ae3c0 con 0x7fcd7415b670 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.601+0000 7fcd6be00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd740ce7d0 0x7fcd74099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.602+0000 7fcd73e00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 0x7fcd74152a00 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9c0 tx=0x7fcd6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.602+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c004d20 con 0x7fcd7415b670 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.602+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(5 keys) ==== 141+0+0 (secure 0 0 0) 0x7fcd6c00cb20 con 0x7fcd7415b670 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.602+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd6c005020 con 0x7fcd7415b670 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.602+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd740ae6a0 con 0x7fcd7415b670 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.602+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd740b03d0 con 0x7fcd7415b670 2024-09-21T07:55:58.605 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.603+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd740b34e0 con 0x7fcd7415b670 2024-09-21T07:55:58.607 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.606+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fcd6c0051c0 con 0x7fcd7415b670 2024-09-21T07:55:58.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.607+0000 7fcd6b400640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd50079790 0x7fcd5007bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:58.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.607+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fcd6c09a5c0 con 0x7fcd7415b670 2024-09-21T07:55:58.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.607+0000 7fcd73400640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd50079790 0x7fcd5007bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:58.608 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.608+0000 7fcd73400640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd50079790 0x7fcd5007bca0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcd64001720 tx=0x7fcd64007e90 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:58.611 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.610+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fcd6c016030 con 0x7fcd7415b670 2024-09-21T07:55:58.803 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.802+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7fcd740ae150 con 0x7fcd7415b670 2024-09-21T07:55:58.808 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.808+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v54)=0 v54) ==== 123+0+0 (secure 0 0 0) 0x7fcd6c063c70 con 0x7fcd7415b670 2024-09-21T07:55:58.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd50079790 msgr2=0x7fcd5007bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.812 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd50079790 0x7fcd5007bca0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fcd64001720 tx=0x7fcd64007e90 comp rx=0 tx=0).stop 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd6b400640 1 -- 172.21.15.88:0/2896218023 <== mon.0 v2:172.21.15.88:3300/0 8 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fcd6c063780 con 0x7fcd7415b670 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 msgr2=0x7fcd74152a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 0x7fcd74152a00 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7fcd6c00e9c0 tx=0x7fcd6c00ee80 comp rx=0 tx=0).stop 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 shutdown_connections 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fcd50079790 0x7fcd5007bca0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fcd740ce7d0 0x7fcd74099910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd7415b670 0x7fcd74152a00 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 --2- 172.21.15.88:0/2896218023 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fcd740b30d0 0x7fcd740adac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 >> 172.21.15.88:0/2896218023 conn(0x7fcd7406d620 msgr2=0x7fcd740d0130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.811+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 shutdown_connections 2024-09-21T07:55:58.813 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:58.812+0000 7fcd78c00640 1 -- 172.21.15.88:0/2896218023 wait complete. 2024-09-21T07:55:58.835 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T07:55:58.835 DEBUG:teuthology.orchestra.run.smithi088:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 -- 172.21.15.88:0/3515671240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f80ad0e0 msgr2=0x7fb6f80ad4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 --2- 172.21.15.88:0/3515671240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f80ad0e0 0x7fb6f80ad4f0 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4005fd0 tx=0x7fb6e402a4b0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 -- 172.21.15.88:0/3515671240 shutdown_connections 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 --2- 172.21.15.88:0/3515671240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f80ad0e0 0x7fb6f80ad4f0 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 --2- 172.21.15.88:0/3515671240 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f815d760 0x7fb6f81535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 -- 172.21.15.88:0/3515671240 >> 172.21.15.88:0/3515671240 conn(0x7fb6f806d730 msgr2=0x7fb6f80b1e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 -- 172.21.15.88:0/3515671240 shutdown_connections 2024-09-21T07:55:59.059 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.057+0000 7fb6fd800640 1 -- 172.21.15.88:0/3515671240 wait complete. 2024-09-21T07:55:59.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.059+0000 7fb6fd800640 1 Processor -- start 2024-09-21T07:55:59.060 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.060+0000 7fb6fd800640 1 -- start start 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ad0e0 0x7fb6f80998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 0x7fb6f81f6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6fd800640 1 --2- >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f8160310 0x7fb6f8151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6fd800640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80b4da0 con 0x7fb6f815d760 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6fd800640 1 -- --> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] -- mon_getmap magic: 0 -- 0x7fb6f80b4c20 con 0x7fb6f8160310 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6fd800640 1 -- --> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] -- mon_getmap magic: 0 -- 0x7fb6f8160e30 con 0x7fb6f80ad0e0 2024-09-21T07:55:59.063 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6f6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 0x7fb6f81f6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6f6000640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 0x7fb6f81f6f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:46282/0 (socket says 172.21.15.88:46282) 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.061+0000 7fb6f6000640 1 -- 172.21.15.88:0/1595096605 learned_addr learned my addr 172.21.15.88:0/1595096605 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f7400640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f8160310 0x7fb6f8151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6a00640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ad0e0 0x7fb6f80998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6000640 1 -- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ad0e0 msgr2=0x7fb6f80998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6000640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ad0e0 0x7fb6f80998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6000640 1 -- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f8160310 msgr2=0x7fb6f8151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6000640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f8160310 0x7fb6f8151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6000640 1 -- 172.21.15.88:0/1595096605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6f81f7bf0 con 0x7fb6f815d760 2024-09-21T07:55:59.064 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.062+0000 7fb6f6000640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 0x7fb6f81f6f70 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4000c00 tx=0x7fb6e4003c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:59.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.063+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e403e030 con 0x7fb6f815d760 2024-09-21T07:55:59.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.063+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 2 ==== config(6 keys) ==== 172+0+0 (secure 0 0 0) 0x7fb6e4004730 con 0x7fb6f815d760 2024-09-21T07:55:59.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.063+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4035040 con 0x7fb6f815d760 2024-09-21T07:55:59.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.063+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6f814f540 con 0x7fb6f815d760 2024-09-21T07:55:59.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.063+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6f814fb00 con 0x7fb6f815d760 2024-09-21T07:55:59.065 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.064+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6f81f7960 con 0x7fb6f815d760 2024-09-21T07:55:59.066 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.065+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 30) ==== 102512+0+0 (secure 0 0 0) 0x7fb6e4041030 con 0x7fb6f815d760 2024-09-21T07:55:59.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.066+0000 7fb6eea00640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb6d8079790 0x7fb6d807bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T07:55:59.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.066+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 5 ==== osd_map(173..173 src has 1..173) ==== 4971+0+0 (secure 0 0 0) 0x7fb6e40ba000 con 0x7fb6f815d760 2024-09-21T07:55:59.067 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.066+0000 7fb6f6a00640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb6d8079790 0x7fb6d807bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T07:55:59.068 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.067+0000 7fb6f6a00640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb6d8079790 0x7fb6d807bca0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800a400 tx=0x7fb6e800d040 comp rx=0 tx=0).ready entity=mgr.4116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T07:55:59.071 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.071+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+197276 (secure 0 0 0) 0x7fb6f80ada30 con 0x7fb6f815d760 2024-09-21T07:55:59.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.291+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6f80ad4f0 con 0x7fb6f815d760 2024-09-21T07:55:59.292 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.292+0000 7fb6eea00640 1 -- 172.21.15.88:0/1595096605 <== mon.0 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 121 v121) ==== 95+0+3913 (secure 0 0 0) 0x7fb6e4083670 con 0x7fb6f815d760 2024-09-21T07:55:59.293 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-09-21T07:55:59.293 INFO:teuthology.orchestra.run.smithi088.stdout:{"epoch":121,"btime":"2024-09-21T07:55:44:311964+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7487,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6835/2761788302","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6801","nonce":2761788302},{"type":"v1","addr":"172.21.15.175:6835","nonce":2761788302}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":8422,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.175:6834/2744940451","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.175:6800","nonce":2744940451},{"type":"v1","addr":"172.21.15.175:6834","nonce":2744940451}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":113},{"gid":9120,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/711519254","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":711519254},{"type":"v1","addr":"172.21.15.88:6833","nonce":711519254}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":112},{"gid":9405,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/60044691","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":60044691},{"type":"v1","addr":"172.21.15.88:6835","nonce":60044691}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":121}],"filesystems":[]} 2024-09-21T07:55:59.293 INFO:teuthology.orchestra.run.smithi088.stderr:dumped fsmap epoch 121 2024-09-21T07:55:59.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb6d8079790 msgr2=0x7fb6d807bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:59.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb6d8079790 0x7fb6d807bca0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fb6e800a400 tx=0x7fb6e800d040 comp rx=0 tx=0).stop 2024-09-21T07:55:59.295 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 msgr2=0x7fb6f81f6f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 0x7fb6f81f6f70 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7fb6e4000c00 tx=0x7fb6e4003c10 comp rx=0 tx=0).stop 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 shutdown_connections 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:6836/3325523616,v1:172.21.15.88:6837/3325523616] conn(0x7fb6d8079790 0x7fb6d807bca0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3300/0,v1:172.21.15.175:6789/0] conn(0x7fb6f8160310 0x7fb6f8151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6f815d760 0x7fb6f81f6f70 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 --2- 172.21.15.88:0/1595096605 >> [v2:172.21.15.175:3301/0,v1:172.21.15.175:6790/0] conn(0x7fb6f80ad0e0 0x7fb6f80998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 >> 172.21.15.88:0/1595096605 conn(0x7fb6f806d730 msgr2=0x7fb6f80b1d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 shutdown_connections 2024-09-21T07:55:59.296 INFO:teuthology.orchestra.run.smithi088.stderr:2024-09-21T07:55:59.294+0000 7fb6fd800640 1 -- 172.21.15.88:0/1595096605 wait complete. 2024-09-21T07:55:59.320 DEBUG:tasks.cephfs.filesystem:already dead... 2024-09-21T07:55:59.320 INFO:teuthology.misc:Shutting down mds daemons... 2024-09-21T07:55:59.320 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2024-09-21T07:55:59.320 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.342 INFO:tasks.ceph.mds.a:Stopped 2024-09-21T07:55:59.342 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2024-09-21T07:55:59.342 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.403 INFO:tasks.ceph.mds.c:Stopped 2024-09-21T07:55:59.404 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2024-09-21T07:55:59.404 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.428 INFO:tasks.ceph.mds.b:Stopped 2024-09-21T07:55:59.428 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2024-09-21T07:55:59.428 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.491 INFO:tasks.ceph.mds.d:Stopped 2024-09-21T07:55:59.491 INFO:teuthology.misc:Shutting down osd daemons... 2024-09-21T07:55:59.491 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-09-21T07:55:59.491 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.540 INFO:tasks.ceph.osd.0:Stopped 2024-09-21T07:55:59.541 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-09-21T07:55:59.541 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.590 INFO:tasks.ceph.osd.1:Stopped 2024-09-21T07:55:59.590 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-09-21T07:55:59.590 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.652 INFO:tasks.ceph.osd.2:Stopped 2024-09-21T07:55:59.652 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-09-21T07:55:59.652 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.712 INFO:tasks.ceph.osd.3:Stopped 2024-09-21T07:55:59.712 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-09-21T07:55:59.712 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.770 INFO:tasks.ceph.osd.4:Stopped 2024-09-21T07:55:59.770 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-09-21T07:55:59.771 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.828 INFO:tasks.ceph.osd.5:Stopped 2024-09-21T07:55:59.828 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-09-21T07:55:59.828 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.886 INFO:tasks.ceph.osd.6:Stopped 2024-09-21T07:55:59.886 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-09-21T07:55:59.887 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:55:59.948 INFO:tasks.ceph.osd.7:Stopped 2024-09-21T07:55:59.948 INFO:teuthology.misc:Shutting down mgr daemons... 2024-09-21T07:55:59.948 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-09-21T07:55:59.949 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:56:00.032 INFO:tasks.ceph.mgr.y:Stopped 2024-09-21T07:56:00.033 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-09-21T07:56:00.033 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:56:00.116 INFO:tasks.ceph.mgr.x:Stopped 2024-09-21T07:56:00.117 INFO:teuthology.misc:Shutting down mon daemons... 2024-09-21T07:56:00.117 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-09-21T07:56:00.117 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:56:00.147 INFO:tasks.ceph.mon.a:Stopped 2024-09-21T07:56:00.147 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-09-21T07:56:00.147 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:56:00.178 INFO:tasks.ceph.mon.b:Stopped 2024-09-21T07:56:00.179 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-09-21T07:56:00.179 INFO:teuthology.orchestra.run:waiting for 300 2024-09-21T07:56:00.247 INFO:tasks.ceph.mon.c:Stopped 2024-09-21T07:56:00.247 INFO:tasks.ceph:Checking cluster log for badness... 2024-09-21T07:56:00.247 DEBUG:teuthology.orchestra.run.smithi088:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'OSD full dropping all updates' | egrep -v 'OSD near full' | egrep -v 'pausewr flag' | egrep -v 'failsafe engaged, dropping updates' | egrep -v 'failsafe disengaged, no longer dropping' | egrep -v 'is full \(reached quota' | egrep -v POOL_FULL | egrep -v POOL_BACKFILLFULL | head -n 1 2024-09-21T07:56:00.332 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:56:00.333 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-09-21T07:56:00.627 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:56:00.627 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-09-21T07:56:00.734 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:56:00.735 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-09-21T07:56:00.831 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:56:00.831 DEBUG:teuthology.orchestra.run.smithi088:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-09-21T07:56:00.938 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:56:00.938 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-09-21T07:56:01.176 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:56:01.176 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-09-21T07:56:01.281 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:56:01.282 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-09-21T07:56:01.367 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:56:01.367 DEBUG:teuthology.orchestra.run.smithi175:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-09-21T07:56:01.471 INFO:tasks.ceph:Archiving mon data... 2024-09-21T07:56:01.472 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/data/mon.a.tgz 2024-09-21T07:56:01.472 DEBUG:teuthology.orchestra.run.smithi088:> mktemp 2024-09-21T07:56:01.508 INFO:teuthology.orchestra.run.smithi088.stdout:/tmp/tmp.ObFvR15vis 2024-09-21T07:56:01.508 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ObFvR15vis 2024-09-21T07:56:01.831 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 0666 /tmp/tmp.ObFvR15vis 2024-09-21T07:56:01.923 DEBUG:teuthology.orchestra.remote:smithi088:/tmp/tmp.ObFvR15vis is 1MB 2024-09-21T07:56:02.022 DEBUG:teuthology.orchestra.run.smithi088:> rm -fr /tmp/tmp.ObFvR15vis 2024-09-21T07:56:02.057 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/data/mon.b.tgz 2024-09-21T07:56:02.057 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-21T07:56:02.094 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.3dXUt4CYy2 2024-09-21T07:56:02.095 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.3dXUt4CYy2 2024-09-21T07:56:02.436 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.3dXUt4CYy2 2024-09-21T07:56:02.529 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.3dXUt4CYy2 is 1MB 2024-09-21T07:56:02.628 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.3dXUt4CYy2 2024-09-21T07:56:02.663 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/data/mon.c.tgz 2024-09-21T07:56:02.663 DEBUG:teuthology.orchestra.run.smithi175:> mktemp 2024-09-21T07:56:02.737 INFO:teuthology.orchestra.run.smithi175.stdout:/tmp/tmp.F9iprQBt5T 2024-09-21T07:56:02.738 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.F9iprQBt5T 2024-09-21T07:56:03.052 DEBUG:teuthology.orchestra.run.smithi175:> sudo chmod 0666 /tmp/tmp.F9iprQBt5T 2024-09-21T07:56:03.145 DEBUG:teuthology.orchestra.remote:smithi175:/tmp/tmp.F9iprQBt5T is 1MB 2024-09-21T07:56:03.248 DEBUG:teuthology.orchestra.run.smithi175:> rm -fr /tmp/tmp.F9iprQBt5T 2024-09-21T07:56:03.283 INFO:tasks.ceph:Cleaning ceph cluster... 2024-09-21T07:56:03.284 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T07:56:03.288 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-09-21T07:56:03.416 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T07:56:03.451 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-09-21T07:56:03.451 INFO:tasks.ceph:Archiving crash dumps... 2024-09-21T07:56:03.453 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/remote/smithi088/crash 2024-09-21T07:56:03.454 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T07:56:03.513 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/lib/ceph/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/remote/smithi175/crash 2024-09-21T07:56:03.513 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-09-21T07:56:03.567 INFO:tasks.ceph:Shutting down logrotate 2024-09-21T07:56:03.568 DEBUG:teuthology.orchestra.run.smithi088:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:56:03.572 DEBUG:teuthology.orchestra.run.smithi175:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2024-09-21T07:56:03.650 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T07:56:03.704 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm /etc/logrotate.d/ceph-test.conf 2024-09-21T07:56:03.760 INFO:tasks.ceph:Compressing logs... 2024-09-21T07:56:03.760 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:56:03.764 DEBUG:teuthology.orchestra.run.smithi175:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T07:56:03.814 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-09-21T07:56:03.815 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46960.log 2024-09-21T07:56:03.816 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55129.log 2024-09-21T07:56:03.817 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46960.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42779.log 2024-09-21T07:56:03.817 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46960.log.gz 2024-09-21T07:56:03.818 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48959.log 2024-09-21T07:56:03.818 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55129.log.gz 2024-09-21T07:56:03.829 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43786.log 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36310.log 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.43786.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43786.log.gz 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36310.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43782.log 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36310.log.gz 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47367.log 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.43782.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43782.log.gz 2024-09-21T07:56:03.830 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36190.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.47367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47367.log.gz 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36390.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36190.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36190.log.gz 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36390.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45367.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36390.log.gz 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44850.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.45367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45367.log.gz 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.44850.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.1.44850.log.gz /var/log/ceph/ceph.audit.log 2024-09-21T07:56:03.831 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-21T07:56:03.832 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45363.log 2024-09-21T07:56:03.832 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42105.log 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.45363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45363.log.gz 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.42105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42105.log.gz 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41518.log 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36350.log 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.3.41518.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41518.log.gz 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43226.log 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T07:56:03.845 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36350.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44846.log 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.36350.log.gz 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.43226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43226.log.gz 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45867.log 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.43230.log/var/log/ceph/ceph-client.1.44846.log: 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44846.log.gz 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.45867.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45867.log.gz 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46346.log 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46350.log 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.43230.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.43230.log.gz 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.46346.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46346.log.gz 2024-09-21T07:56:03.846 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-09-21T07:56:03.847 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.46350.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46350.log.gz 2024-09-21T07:56:03.847 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-09-21T07:56:03.847 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T07:56:03.847 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36230.log 2024-09-21T07:56:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.36230.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36230.log.gz 2024-09-21T07:56:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46846.log 2024-09-21T07:56:03.860 INFO:teuthology.orchestra.run.smithi175.stderr: 92.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T07:56:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.46846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46846.log.gz 2024-09-21T07:56:03.860 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.45863.log 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.47363.log 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.45863.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.45863.log.gz 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.47363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.47363.log.gz 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36550.log 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36550.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36550.log.gz 2024-09-21T07:56:03.861 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-09-21T07:56:03.865 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42109.log 2024-09-21T07:56:03.865 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44334.log 2024-09-21T07:56:03.865 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.42109.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42109.log.gz 2024-09-21T07:56:03.865 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42641.log 2024-09-21T07:56:03.865 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.44334.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44334.log.gz 2024-09-21T07:56:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42779.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42779.log.gz 2024-09-21T07:56:03.869 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74436.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71410.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48959.log.gz 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66817.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.74436.log.gz 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42401.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.71410.log.gz 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66817.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66817.log.gz 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42485.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42401.log.gz 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49408.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69475.log 2024-09-21T07:56:03.870 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42485.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43297.log 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42485.log.gz 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54306.log 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69475.log.gz 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59768.log 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43297.log.gz 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54306.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73546.log 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54306.log.gz 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71328.log 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59768.log.gz 2024-09-21T07:56:03.871 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73546.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.73546.log.gz 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47255.log 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67790.log 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr: 85.0% -- replaced with /var/log/ceph/ceph-client.admin.71328.log.gz 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47255.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62123.log 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47255.log.gz 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67790.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56369.log 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67790.log.gz 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.62123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62123.log.gz 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35858.log 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48011.log 2024-09-21T07:56:03.872 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56369.log.gz 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.35858.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.35858.log.gz 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54506.log 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45125.log 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.48011.log.gz 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54506.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54506.log.gz 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48688.log 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.2.45125.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57211.log 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45125.log.gz 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75799.log 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48688.log.gz 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43302.log 2024-09-21T07:56:03.873 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57211.log.gz 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49965.log 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75799.log.gz 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43302.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43302.log.gz 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71163.log 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57419.log 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49965.log.gz 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45996.log 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.71163.log.gz 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71122.log 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57419.log.gz 2024-09-21T07:56:03.874 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45996.log: gzip -5 87.5% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.45996.log.gz /var/log/ceph/ceph-client.admin.52328.log 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71122.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54098.log 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.71122.log.gz 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45598.log 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52328.log.gz 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.54098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54098.log.gz 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78733.log 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45598.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.45598.log.gz 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54903.log 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78733.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.78733.log.gz 2024-09-21T07:56:03.875 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70297.log 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75498.log 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54903.log.gz 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70297.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70297.log.gz 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53424.log 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75498.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75498.log.gz 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67367.log 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53424.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.53424.log.gz 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57903.log 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67367.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.67367.log.gz 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68994.log 2024-09-21T07:56:03.876 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61823.log 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57903.log.gz 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53256.log 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68994.log.gz 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61823.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61823.log.gz 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56933.log 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53256.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53256.log.gz 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50175.log 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56933.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36146.log 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56933.log.gz 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50175.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50175.log.gz 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55465.log 2024-09-21T07:56:03.877 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36146.log: gzip 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36146.log.gz 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.60690.log 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55465.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55465.log.gz 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80630.log 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60690.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63073.log 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60690.log.gz 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80630.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80630.log.gz 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54264.log 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63073.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63073.log.gz 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67666.log 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54264.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54264.log.gz 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81451.log 2024-09-21T07:56:03.878 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.67666.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67666.log.gz 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42229.log 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81451.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81451.log.gz 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44039.log 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42229.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42229.log.gz 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75125.log 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65753.log 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44039.log.gz 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75125.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75125.log.gz 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60461.log 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65753.log: gzip -5 --verbose -- 87.3% /var/log/ceph/ceph-client.admin.79184.log -- replaced with /var/log/ceph/ceph-client.admin.65753.log.gz 2024-09-21T07:56:03.879 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.46842.log 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.42641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42641.log.gz 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.46842.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.46842.log.gz 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.42637.log 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41123.log 2024-09-21T07:56:03.884 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.42637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.42637.log.gz 2024-09-21T07:56:03.885 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.41123.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41123.log.gz 2024-09-21T07:56:03.885 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.41104.log 2024-09-21T07:56:03.888 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36450.log 2024-09-21T07:56:03.888 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.41104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.41104.log.gz 2024-09-21T07:56:03.888 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36270.log 2024-09-21T07:56:03.888 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36450.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36450.log.gz 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60461.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.60461.log.gz 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78956.log 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79184.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79184.log.gz 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46872.log 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47507.log 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.78956.log.gz 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46872.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.46872.log.gz 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58258.log 2024-09-21T07:56:03.894 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47507.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47507.log.gz 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61781.log 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58258.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.71832.log -- replaced with /var/log/ceph/ceph-client.admin.58258.log.gz 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61781.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61781.log.gz 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45262.log 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71832.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76389.log 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.71832.log.gz 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77674.log 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.45262.log.gz 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76389.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.76389.log.gz 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46620.log 2024-09-21T07:56:03.895 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77674.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77674.log.gz 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69204.log 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56579.log 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46620.log.gz 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69204.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69204.log.gz 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53508.log 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56579.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36016.log 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.56579.log.gz 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53508.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57724.log 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.53508.log.gz 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36016.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36016.log.gz 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58855.log 2024-09-21T07:56:03.896 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57724.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57724.log.gz 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.29825.log 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80989.log 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.58855.log.gz 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.tmp-client.admin.29825.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.29825.log.gz 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67478.log 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80989.log: gzip -5 --verbose 86.8% -- -- replaced with /var/log/ceph/ceph-client.admin.80989.log.gz 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.54651.log 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67478.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67478.log.gz 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73504.log 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54651.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.54651.log.gz 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35878.log 2024-09-21T07:56:03.897 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46535.log 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73504.log.gz 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.35878.log: 76.9% -- replaced with /var/log/ceph/ceph-client.admin.35878.log.gz 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60115.log 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46535.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46535.log.gz 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61612.log 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60115.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63380.log 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60115.log.gz 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61612.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61612.log.gz 2024-09-21T07:56:03.898 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78094.log 2024-09-21T07:56:03.900 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.3.41522.log 2024-09-21T07:56:03.900 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.admin.36270.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36270.log.gz 2024-09-21T07:56:03.900 INFO:teuthology.orchestra.run.smithi175.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44330.log 2024-09-21T07:56:03.901 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.3.41522.log: 0.0% -- replaced with /var/log/ceph/ceph-client.3.41522.log.gz 2024-09-21T07:56:03.903 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63380.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63380.log.gz 2024-09-21T07:56:03.903 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65269.log 2024-09-21T07:56:03.903 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60732.log 2024-09-21T07:56:03.903 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78094.log.gz 2024-09-21T07:56:03.903 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65269.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65269.log.gz 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43300.log 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60732.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60732.log.gz 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49187.log 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43300.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43300.log.gz 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49187.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50831.log 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49187.log.gz 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70128.log 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50831.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48730.log 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50831.log.gz 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70128.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.70128.log.gz 2024-09-21T07:56:03.904 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47465.log 2024-09-21T07:56:03.905 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48730.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47969.log 2024-09-21T07:56:03.905 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48730.log.gz 2024-09-21T07:56:03.906 INFO:teuthology.orchestra.run.smithi175.stderr:/var/log/ceph/ceph-client.1.44330.log: 0.0% -- replaced with /var/log/ceph/ceph-client.1.44330.log.gz 2024-09-21T07:56:03.907 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47465.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.47465.log.gz 2024-09-21T07:56:03.907 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76009.log 2024-09-21T07:56:03.907 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47969.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71574.log 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47969.log.gz 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76009.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76009.log.gz 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46746.log 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58811.log 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.71574.log.gz 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46746.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46746.log.gz 2024-09-21T07:56:03.908 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47423.log 2024-09-21T07:56:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70170.log 2024-09-21T07:56:03.909 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58811.log.gz 2024-09-21T07:56:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47423.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47423.log.gz 2024-09-21T07:56:03.909 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71286.log 2024-09-21T07:56:03.910 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57315.log 2024-09-21T07:56:03.910 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70170.log.gz 2024-09-21T07:56:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71286.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.71286.log.gz 2024-09-21T07:56:03.911 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2024-09-21T07:56:03.912 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57315.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65939.log 2024-09-21T07:56:03.912 INFO:teuthology.orchestra.run.smithi088.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.57315.log.gz 2024-09-21T07:56:03.913 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43674.log: gzip 87.4% -5 -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gz 2024-09-21T07:56:03.913 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.66295.log 2024-09-21T07:56:03.914 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66947.log 2024-09-21T07:56:03.914 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65939.log.gz 2024-09-21T07:56:03.915 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66295.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66295.log.gz 2024-09-21T07:56:03.915 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36343.log 2024-09-21T07:56:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66947.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79142.log 2024-09-21T07:56:03.916 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66947.log.gz 2024-09-21T07:56:03.916 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36343.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.36343.log.gz 2024-09-21T07:56:03.917 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66210.log 2024-09-21T07:56:03.917 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79142.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65491.log 2024-09-21T07:56:03.918 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79142.log.gz 2024-09-21T07:56:03.918 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66210.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66210.log.gz 2024-09-21T07:56:03.918 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79292.log 2024-09-21T07:56:03.919 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62243.log 2024-09-21T07:56:03.919 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65491.log.gz 2024-09-21T07:56:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79292.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.79292.log.gz 2024-09-21T07:56:03.920 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73192.log 2024-09-21T07:56:03.921 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65855.log 2024-09-21T07:56:03.921 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62243.log.gz 2024-09-21T07:56:03.922 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73192.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56060.log 2024-09-21T07:56:03.922 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73192.log.gz 2024-09-21T07:56:03.923 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69696.log 2024-09-21T07:56:03.923 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65855.log.gz 2024-09-21T07:56:03.924 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56060.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56060.log.gz 2024-09-21T07:56:03.924 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80527.log 2024-09-21T07:56:03.925 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44810.log 2024-09-21T07:56:03.925 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69696.log.gz 2024-09-21T07:56:03.926 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80527.log: 87.8%gzip -- replaced with /var/log/ceph/ceph-client.admin.80527.log.gz 2024-09-21T07:56:03.926 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.43871.log 2024-09-21T07:56:03.927 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75167.log 2024-09-21T07:56:03.927 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44810.log.gz 2024-09-21T07:56:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43871.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43871.log.gz 2024-09-21T07:56:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61865.log 2024-09-21T07:56:03.928 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75880.log 2024-09-21T07:56:03.928 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.75167.log.gz 2024-09-21T07:56:03.929 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72227.log 2024-09-21T07:56:03.930 INFO:teuthology.orchestra.run.smithi088.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.61865.log.gz 2024-09-21T07:56:03.930 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75880.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.75880.log.gz 2024-09-21T07:56:03.930 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55931.log 2024-09-21T07:56:03.931 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76179.log 2024-09-21T07:56:03.931 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.72227.log.gz 2024-09-21T07:56:03.932 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55931.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50217.log 2024-09-21T07:56:03.932 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55931.log.gz 2024-09-21T07:56:03.933 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67748.log 2024-09-21T07:56:03.933 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76179.log.gz 2024-09-21T07:56:03.934 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50217.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66432.log 2024-09-21T07:56:03.934 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.50217.log.gz 2024-09-21T07:56:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67748.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.67748.log.gz 2024-09-21T07:56:03.935 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66084.log 2024-09-21T07:56:03.936 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66432.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59684.log 2024-09-21T07:56:03.936 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66432.log.gz 2024-09-21T07:56:03.937 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66084.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66084.log.gz 2024-09-21T07:56:03.937 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51447.log 2024-09-21T07:56:03.938 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45514.log 2024-09-21T07:56:03.938 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59684.log.gz 2024-09-21T07:56:03.939 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51447.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.51447.log.gz --verbose 2024-09-21T07:56:03.939 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.57357.log 2024-09-21T07:56:03.940 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45514.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73714.log 2024-09-21T07:56:03.940 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45514.log.gz 2024-09-21T07:56:03.941 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52431.log 87.0% 2024-09-21T07:56:03.941 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.57357.log.gz 2024-09-21T07:56:03.941 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76581.log 2024-09-21T07:56:03.942 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73714.log.gz 2024-09-21T07:56:03.942 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52431.log: gzip -5 --verbose 87.5% -- -- replaced with /var/log/ceph/ceph-client.admin.52431.log.gz 2024-09-21T07:56:03.942 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.72395.log 2024-09-21T07:56:03.943 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76581.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49103.log 2024-09-21T07:56:03.943 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.76581.log.gz 2024-09-21T07:56:03.944 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72395.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72395.log.gz 2024-09-21T07:56:03.944 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58767.log 2024-09-21T07:56:03.945 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49103.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42569.log 2024-09-21T07:56:03.945 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.49103.log.gz 2024-09-21T07:56:03.946 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58767.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58767.log.gz 2024-09-21T07:56:03.946 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66905.log 2024-09-21T07:56:03.947 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42569.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60606.log 2024-09-21T07:56:03.947 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42569.log.gz 2024-09-21T07:56:03.948 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66905.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66905.log.gz 2024-09-21T07:56:03.948 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52767.log 2024-09-21T07:56:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60606.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77219.log 2024-09-21T07:56:03.949 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.60606.log.gz 2024-09-21T07:56:03.949 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52767.log: gzip -5 87.4% --verbose -- replaced with /var/log/ceph/ceph-client.admin.52767.log.gz -- 2024-09-21T07:56:03.950 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.57987.log 2024-09-21T07:56:03.951 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77219.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67157.log 2024-09-21T07:56:03.951 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.77219.log.gz 2024-09-21T07:56:03.951 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57987.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57987.log.gz 2024-09-21T07:56:03.952 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60335.log 2024-09-21T07:56:03.952 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75965.log 2024-09-21T07:56:03.952 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.67157.log.gz 2024-09-21T07:56:03.953 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60335.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60335.log.gz 2024-09-21T07:56:03.953 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76769.log 2024-09-21T07:56:03.954 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75965.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71492.log 2024-09-21T07:56:03.955 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75965.log.gz 2024-09-21T07:56:03.956 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.76769.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76769.log.gz 2024-09-21T07:56:03.957 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61654.log 2024-09-21T07:56:03.957 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71492.log: 84.3% -- replaced with /var/log/ceph/ceph-client.admin.71492.log.gz 2024-09-21T07:56:03.957 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53382.log 2024-09-21T07:56:03.959 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61654.log: gzip -5 --verbose -- 87.5% /var/log/ceph/ceph-client.admin.52862.log -- replaced with /var/log/ceph/ceph-client.admin.61654.log.gz 2024-09-21T07:56:03.959 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:03.959 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53382.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77716.log 2024-09-21T07:56:03.959 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53382.log.gz 2024-09-21T07:56:03.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52862.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52862.log.gz 2024-09-21T07:56:03.961 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61738.log 2024-09-21T07:56:03.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77716.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68661.log 2024-09-21T07:56:03.961 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.77716.log.gz 2024-09-21T07:56:03.961 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61738.log: gzip 87.0% -- replaced with /var/log/ceph/ceph-client.admin.61738.log.gz 2024-09-21T07:56:03.962 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.74497.log 2024-09-21T07:56:03.964 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68661.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53844.log 2024-09-21T07:56:03.964 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68661.log.gz 2024-09-21T07:56:03.964 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65124.log 2024-09-21T07:56:03.965 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74497.log.gz 2024-09-21T07:56:03.965 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53844.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77968.log 2024-09-21T07:56:03.965 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.53844.log.gz 2024-09-21T07:56:03.970 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65124.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.65124.log.gz 2024-09-21T07:56:03.970 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60816.log 2024-09-21T07:56:03.970 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77968.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70254.log 2024-09-21T07:56:03.970 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.77968.log.gz 2024-09-21T07:56:03.970 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60816.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.60816.log.gz 2024-09-21T07:56:03.970 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53718.log 2024-09-21T07:56:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61301.log 2024-09-21T07:56:03.971 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70254.log.gz 2024-09-21T07:56:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53718.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53718.log.gz 2024-09-21T07:56:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46454.log 2024-09-21T07:56:03.971 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61301.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56155.log 2024-09-21T07:56:03.971 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61301.log.gz 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46454.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46454.log.gz 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45346.log 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63320.log 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56155.log.gz 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45346.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.45346.log.gz 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66989.log 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70998.log 2024-09-21T07:56:03.975 INFO:teuthology.orchestra.run.smithi088.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.63320.log.gz 2024-09-21T07:56:03.976 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66989.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66989.log.gz 2024-09-21T07:56:03.976 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77574.log 2024-09-21T07:56:03.976 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58384.log 2024-09-21T07:56:03.976 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70998.log.gz 2024-09-21T07:56:03.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61440.log 2024-09-21T07:56:03.977 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.77574.log.gz 2024-09-21T07:56:03.977 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54403.log 2024-09-21T07:56:03.977 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58384.log.gz 2024-09-21T07:56:03.978 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61440.log: gzip -5 --verbose -- 87.6% /var/log/ceph/ceph-client.admin.77800.log -- replaced with /var/log/ceph/ceph-client.admin.61440.log.gz 2024-09-21T07:56:03.978 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:03.979 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78603.log 2024-09-21T07:56:03.979 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.54403.log.gz 2024-09-21T07:56:03.980 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77800.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77800.log.gz 2024-09-21T07:56:03.980 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56327.log 2024-09-21T07:56:03.980 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78603.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59036.log 2024-09-21T07:56:03.980 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78603.log.gz 2024-09-21T07:56:03.981 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59306.log 2024-09-21T07:56:03.981 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.56327.log.gz 2024-09-21T07:56:03.982 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49654.log 2024-09-21T07:56:03.982 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.59036.log.gz 2024-09-21T07:56:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59306.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.59306.log.gz 2024-09-21T07:56:03.983 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49570.log 2024-09-21T07:56:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49654.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61570.log 2024-09-21T07:56:03.984 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49654.log.gz 2024-09-21T07:56:03.984 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49570.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.49570.log.gz 2024-09-21T07:56:03.985 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80401.log 2024-09-21T07:56:03.985 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61570.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65627.log 2024-09-21T07:56:03.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61570.log.gz 2024-09-21T07:56:03.986 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80401.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62461.log 2024-09-21T07:56:03.986 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80401.log.gz 2024-09-21T07:56:03.987 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66337.log 2024-09-21T07:56:03.987 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.65627.log.gz 2024-09-21T07:56:03.988 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62461.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62461.log.gz 2024-09-21T07:56:03.988 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50007.log 2024-09-21T07:56:03.989 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66337.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77165.log 2024-09-21T07:56:03.989 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66337.log.gz 2024-09-21T07:56:03.990 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50007.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50007.log.gz 2024-09-21T07:56:03.990 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81814.log 2024-09-21T07:56:03.991 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36482.log 2024-09-21T07:56:03.991 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77165.log.gz 2024-09-21T07:56:03.992 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81814.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.81814.log.gz 2024-09-21T07:56:03.992 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47171.log 2024-09-21T07:56:03.993 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36482.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36482.log.gz 2024-09-21T07:56:03.993 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58725.log 2024-09-21T07:56:03.994 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47171.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47171.log.gz 2024-09-21T07:56:03.994 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48604.log 2024-09-21T07:56:03.995 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58725.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46038.log 2024-09-21T07:56:03.995 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58725.log.gz 2024-09-21T07:56:03.995 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48604.log: 87.3%gzip -- replaced with /var/log/ceph/ceph-client.admin.48604.log.gz 2024-09-21T07:56:03.996 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.50475.log 2024-09-21T07:56:03.996 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46038.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73756.log 2024-09-21T07:56:03.997 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.46038.log.gz 2024-09-21T07:56:03.997 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.50475.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50475.log.gz 2024-09-21T07:56:03.997 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50283.log 2024-09-21T07:56:03.998 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73756.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66722.log 2024-09-21T07:56:03.998 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73756.log.gz 2024-09-21T07:56:03.999 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50283.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50283.log.gz 2024-09-21T07:56:03.999 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48898.log 2024-09-21T07:56:04.000 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66722.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.66722.log.gz 2024-09-21T07:56:04.000 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72127.log 2024-09-21T07:56:04.001 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45556.log 2024-09-21T07:56:04.001 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48898.log.gz 2024-09-21T07:56:04.002 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72127.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63422.log 2024-09-21T07:56:04.003 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72127.log.gz 2024-09-21T07:56:04.003 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45556.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75922.log 2024-09-21T07:56:04.003 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.45556.log.gz 2024-09-21T07:56:04.004 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63422.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79480.log 2024-09-21T07:56:04.004 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63422.log.gz 2024-09-21T07:56:04.005 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75922.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75922.log.gz 2024-09-21T07:56:04.005 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58300.log 2024-09-21T07:56:04.006 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.79480.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79480.log.gz 2024-09-21T07:56:04.006 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45472.log 2024-09-21T07:56:04.007 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58300.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61037.log 2024-09-21T07:56:04.007 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58300.log.gz 2024-09-21T07:56:04.008 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45472.log: gzip -5 87.5% --verbose -- replaced with /var/log/ceph/ceph-client.admin.45472.log.gz -- 2024-09-21T07:56:04.008 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.68517.log 2024-09-21T07:56:04.009 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61037.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53083.log 2024-09-21T07:56:04.009 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.61037.log.gz 2024-09-21T07:56:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68517.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.68517.log.gz 2024-09-21T07:56:04.010 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67073.log 2024-09-21T07:56:04.011 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53083.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53126.log 2024-09-21T07:56:04.011 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53083.log.gz 2024-09-21T07:56:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74857.log 2024-09-21T07:56:04.012 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.67073.log.gz 2024-09-21T07:56:04.012 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53126.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61345.log 2024-09-21T07:56:04.013 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53126.log.gz 2024-09-21T07:56:04.013 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74857.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74857.log.gz 2024-09-21T07:56:04.014 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68910.log 2024-09-21T07:56:04.015 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61345.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57861.log 2024-09-21T07:56:04.015 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61345.log.gz 2024-09-21T07:56:04.015 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68910.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68910.log.gz 2024-09-21T07:56:04.015 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71040.log 2024-09-21T07:56:04.016 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57861.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52557.log 2024-09-21T07:56:04.016 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57861.log.gz 2024-09-21T07:56:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71040.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.71040.log.gz 2024-09-21T07:56:04.017 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58090.log 2024-09-21T07:56:04.018 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52557.log: gzip -5 87.2% --verbose -- replaced with /var/log/ceph/ceph-client.admin.52557.log.gz 2024-09-21T07:56:04.018 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.63194.log 2024-09-21T07:56:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58090.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58090.log.gz 2024-09-21T07:56:04.019 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72624.log 2024-09-21T07:56:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63194.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.63194.log.gz 2024-09-21T07:56:04.020 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49835.log 2024-09-21T07:56:04.021 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72624.log: 87.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.72624.log.gz 2024-09-21T07:56:04.021 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.71451.log 2024-09-21T07:56:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80175.log 2024-09-21T07:56:04.022 INFO:teuthology.orchestra.run.smithi088.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.49835.log.gz 2024-09-21T07:56:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71451.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.71451.log.gz 2024-09-21T07:56:04.022 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73588.log 2024-09-21T07:56:04.023 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81073.log 2024-09-21T07:56:04.024 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80175.log.gz 2024-09-21T07:56:04.024 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73588.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73588.log.gz 2024-09-21T07:56:04.024 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66042.log 2024-09-21T07:56:04.025 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55423.log 2024-09-21T07:56:04.025 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81073.log.gz 2024-09-21T07:56:04.026 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66042.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.66042.log.gz 2024-09-21T07:56:04.026 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77496.log 2024-09-21T07:56:04.027 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55423.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44658.log 2024-09-21T07:56:04.027 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55423.log.gz 2024-09-21T07:56:04.028 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77496.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77496.log.gz 2024-09-21T07:56:04.028 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45430.log 2024-09-21T07:56:04.029 INFO:teuthology.orchestra.run.smithi175.stderr: 92.3% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-09-21T07:56:04.029 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.44658.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44658.log.gz 2024-09-21T07:56:04.030 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49698.log 2024-09-21T07:56:04.030 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45430.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45430.log.gz 2024-09-21T07:56:04.030 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81325.log 2024-09-21T07:56:04.031 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49698.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80714.log 2024-09-21T07:56:04.031 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49698.log.gz 2024-09-21T07:56:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81325.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.81325.log.gz 2024-09-21T07:56:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67241.log 2024-09-21T07:56:04.032 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52641.log 2024-09-21T07:56:04.033 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80714.log.gz 2024-09-21T07:56:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67241.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67241.log.gz 2024-09-21T07:56:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76053.log 2024-09-21T07:56:04.034 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46662.log 2024-09-21T07:56:04.035 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52641.log.gz 2024-09-21T07:56:04.035 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76053.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56411.log 2024-09-21T07:56:04.035 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.76053.log.gz 2024-09-21T07:56:04.036 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46662.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46662.log.gz 2024-09-21T07:56:04.036 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46374.log 2024-09-21T07:56:04.037 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56411.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74915.log 2024-09-21T07:56:04.037 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56411.log.gz 2024-09-21T07:56:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46374.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46374.log.gz 2024-09-21T07:56:04.038 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58994.log 2024-09-21T07:56:04.039 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74915.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43634.log 2024-09-21T07:56:04.039 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74915.log.gz 2024-09-21T07:56:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58994.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58994.log.gz 2024-09-21T07:56:04.040 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76095.log 2024-09-21T07:56:04.041 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43634.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35985.log 2024-09-21T07:56:04.041 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43634.log.gz 2024-09-21T07:56:04.042 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76095.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76095.log.gz 2024-09-21T07:56:04.042 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70872.log 2024-09-21T07:56:04.043 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.35985.log: 81.0% -- replaced with /var/log/ceph/ceph-client.admin.35985.log.gz 2024-09-21T07:56:04.043 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62641.log 2024-09-21T07:56:04.044 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73864.log 2024-09-21T07:56:04.044 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.70872.log.gz 2024-09-21T07:56:04.044 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62641.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62641.log.gz 2024-09-21T07:56:04.044 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76473.log 2024-09-21T07:56:04.045 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73864.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-09-21T07:56:04.045 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73864.log.gz 2024-09-21T07:56:04.046 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76473.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76473.log.gzgzip 2024-09-21T07:56:04.046 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.69349.log 2024-09-21T07:56:04.047 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81115.log 2024-09-21T07:56:04.048 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69349.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69349.log.gz 2024-09-21T07:56:04.048 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42527.log 2024-09-21T07:56:04.049 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81115.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81115.log.gz 2024-09-21T07:56:04.049 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65082.log 2024-09-21T07:56:04.050 INFO:teuthology.orchestra.run.smithi088.stderr: 89.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-09-21T07:56:04.050 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42527.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79226.log 2024-09-21T07:56:04.051 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42527.log.gz 2024-09-21T07:56:04.051 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65082.log: gzip -5 --verbose -- 86.6% /var/log/ceph/ceph-client.admin.73462.log 2024-09-21T07:56:04.052 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65082.log.gz 2024-09-21T07:56:04.052 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79226.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70494.log 2024-09-21T07:56:04.053 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.admin.79226.log.gz 2024-09-21T07:56:04.054 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44165.log 2024-09-21T07:56:04.054 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.73462.log.gz 2024-09-21T07:56:04.054 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70494.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70494.log.gz 2024-09-21T07:56:04.054 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72708.log 2024-09-21T07:56:04.055 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50679.log 2024-09-21T07:56:04.056 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.44165.log.gz 2024-09-21T07:56:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72708.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72708.log.gz 2024-09-21T07:56:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43997.log 2024-09-21T07:56:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50679.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.50679.log.gz 2024-09-21T07:56:04.057 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53592.log 2024-09-21T07:56:04.058 INFO:teuthology.orchestra.run.smithi175.stderr: 93.0% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2024-09-21T07:56:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76431.log 2024-09-21T07:56:04.059 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43997.log.gz 2024-09-21T07:56:04.059 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70365.log 2024-09-21T07:56:04.059 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.53592.log.gz 2024-09-21T07:56:04.060 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36454.log 2024-09-21T07:56:04.060 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.76431.log.gz 2024-09-21T07:56:04.061 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70365.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43554.log 2024-09-21T07:56:04.061 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70365.log.gz 2024-09-21T07:56:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36454.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36454.log.gz 2024-09-21T07:56:04.062 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80002.log 2024-09-21T07:56:04.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75582.log 2024-09-21T07:56:04.063 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43554.log.gz 2024-09-21T07:56:04.063 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67031.log 2024-09-21T07:56:04.064 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80002.log.gz 2024-09-21T07:56:04.064 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75582.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59120.log 2024-09-21T07:56:04.064 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75582.log.gz 2024-09-21T07:56:04.065 INFO:teuthology.orchestra.run.smithi175.stderr: 93.4% -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2024-09-21T07:56:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67031.log: gzip -5 87.7% --verbose -- replaced with /var/log/ceph/ceph-client.admin.67031.log.gz -- 2024-09-21T07:56:04.066 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.47297.log 2024-09-21T07:56:04.066 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59120.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60503.log 2024-09-21T07:56:04.066 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.59120.log.gz 2024-09-21T07:56:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47297.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.47297.log.gz 2024-09-21T07:56:04.067 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46916.log 2024-09-21T07:56:04.068 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60503.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71709.log 2024-09-21T07:56:04.068 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.60503.log.gz 2024-09-21T07:56:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46916.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.46916.log.gz 2024-09-21T07:56:04.069 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72353.log 2024-09-21T07:56:04.070 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71709.log: gzip 84.9% -5 -- replaced with /var/log/ceph/ceph-client.admin.71709.log.gz 2024-09-21T07:56:04.070 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50637.log 2024-09-21T07:56:04.071 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72353.log: gzip -5 --verbose -- 87.4% /var/log/ceph/ceph-client.admin.70031.log 2024-09-21T07:56:04.071 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.72353.log.gz 2024-09-21T07:56:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50637.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69078.log 2024-09-21T07:56:04.072 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50637.log.gz 2024-09-21T07:56:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70031.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70031.log.gz 2024-09-21T07:56:04.072 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78645.log 2024-09-21T07:56:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69078.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69078.log.gz 2024-09-21T07:56:04.073 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43002.log 2024-09-21T07:56:04.074 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78645.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78645.log.gzgzip 2024-09-21T07:56:04.074 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.36448.log 2024-09-21T07:56:04.075 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61482.log 2024-09-21T07:56:04.075 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.43002.log.gz 2024-09-21T07:56:04.076 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36448.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.36448.log.gz 2024-09-21T07:56:04.077 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61528.log 2024-09-21T07:56:04.077 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61482.log: 87.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.61482.log.gz 2024-09-21T07:56:04.077 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53910.log 2024-09-21T07:56:04.078 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61528.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61528.log.gz 2024-09-21T07:56:04.078 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76515.log 2024-09-21T07:56:04.079 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53910.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57461.log 2024-09-21T07:56:04.079 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53910.log.gz 2024-09-21T07:56:04.080 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58174.log 2024-09-21T07:56:04.080 INFO:teuthology.orchestra.run.smithi088.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.76515.log.gz 2024-09-21T07:56:04.081 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57461.log: 87.7%gzip -- replaced with /var/log/ceph/ceph-client.admin.57461.log.gz 2024-09-21T07:56:04.081 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.53170.log 2024-09-21T07:56:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58174.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.58174.log.gz 2024-09-21T07:56:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74558.log 2024-09-21T07:56:04.082 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53170.log: gzip -5 --verbose -- 86.5% /var/log/ceph/ceph-client.admin.57604.log -- replaced with /var/log/ceph/ceph-client.admin.53170.log.gz 2024-09-21T07:56:04.082 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:04.083 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75209.log 2024-09-21T07:56:04.083 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74558.log.gz 2024-09-21T07:56:04.084 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57604.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42991.log 2024-09-21T07:56:04.084 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.57604.log.gz 2024-09-21T07:56:04.085 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81031.log 2024-09-21T07:56:04.085 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75209.log.gz 2024-09-21T07:56:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42991.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42991.log.gz 2024-09-21T07:56:04.086 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36458.log 2024-09-21T07:56:04.087 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81031.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78262.log 2024-09-21T07:56:04.087 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81031.log.gz 2024-09-21T07:56:04.088 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36458.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.36458.log.gz 2024-09-21T07:56:04.088 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75540.log 2024-09-21T07:56:04.089 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78262.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51961.log 2024-09-21T07:56:04.089 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78262.log.gz 2024-09-21T07:56:04.090 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75540.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.75540.log.gz 2024-09-21T07:56:04.090 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35973.log 2024-09-21T07:56:04.091 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74134.log 2024-09-21T07:56:04.091 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51961.log.gz 2024-09-21T07:56:04.092 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.35973.log: 77.1% -- replaced with /var/log/ceph/ceph-client.admin.35973.log.gz 2024-09-21T07:56:04.092 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68371.log 2024-09-21T07:56:04.093 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74134.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51633.log 2024-09-21T07:56:04.093 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.74134.log.gz 2024-09-21T07:56:04.094 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68371.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67410.log 2024-09-21T07:56:04.095 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.68371.log.gz 2024-09-21T07:56:04.095 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69601.log 2024-09-21T07:56:04.096 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.51633.log.gz 2024-09-21T07:56:04.096 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67410.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66766.log 2024-09-21T07:56:04.096 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.67410.log.gz 2024-09-21T07:56:04.097 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61696.log 2024-09-21T07:56:04.097 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.69601.log.gz 2024-09-21T07:56:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66766.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66766.log.gz 2024-09-21T07:56:04.098 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78399.log 2024-09-21T07:56:04.099 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61696.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48546.log 2024-09-21T07:56:04.099 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61696.log.gz 2024-09-21T07:56:04.100 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78399.log: gzip 87.5% -5 -- replaced with /var/log/ceph/ceph-client.admin.78399.log.gz --verbose 2024-09-21T07:56:04.100 INFO:teuthology.orchestra.run.smithi088.stderr: -- /var/log/ceph/ceph-client.admin.69899.log 2024-09-21T07:56:04.101 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48546.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69559.log 2024-09-21T07:56:04.101 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.48546.log.gz 2024-09-21T07:56:04.102 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77884.log 2024-09-21T07:56:04.102 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.69899.log.gz 2024-09-21T07:56:04.103 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77632.log 2024-09-21T07:56:04.103 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69559.log.gz 2024-09-21T07:56:04.104 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58426.log 2024-09-21T07:56:04.104 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77884.log.gz 2024-09-21T07:56:04.105 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77632.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70790.log 2024-09-21T07:56:04.105 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77632.log.gz 2024-09-21T07:56:04.106 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58426.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58426.log.gz 2024-09-21T07:56:04.106 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60900.log 2024-09-21T07:56:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.70790.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70790.log.gz 2024-09-21T07:56:04.107 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46206.log 2024-09-21T07:56:04.108 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60900.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81535.log 2024-09-21T07:56:04.108 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.60900.log.gz 2024-09-21T07:56:04.109 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46206.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51757.log 2024-09-21T07:56:04.109 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46206.log.gz 2024-09-21T07:56:04.110 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81535.log: 87.2%gzip -- replaced with /var/log/ceph/ceph-client.admin.81535.log.gz 2024-09-21T07:56:04.110 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.58948.log 2024-09-21T07:56:04.111 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51757.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55045.log 2024-09-21T07:56:04.111 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51757.log.gz 2024-09-21T07:56:04.112 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58948.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.58948.log.gz 2024-09-21T07:56:04.112 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76977.log 2024-09-21T07:56:04.113 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76263.log 2024-09-21T07:56:04.113 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55045.log.gz 2024-09-21T07:56:04.114 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76977.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76977.log.gz 2024-09-21T07:56:04.114 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56851.log 2024-09-21T07:56:04.115 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76263.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63601.log 2024-09-21T07:56:04.115 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76263.log.gz 2024-09-21T07:56:04.116 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.56851.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56851.log.gz 2024-09-21T07:56:04.116 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59980.log 2024-09-21T07:56:04.117 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63601.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50049.log 2024-09-21T07:56:04.117 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.63601.log.gz 2024-09-21T07:56:04.118 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59980.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.59980.log.gz 2024-09-21T07:56:04.118 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49019.log 2024-09-21T07:56:04.119 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67283.log 2024-09-21T07:56:04.119 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.50049.log.gz 2024-09-21T07:56:04.120 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74624.log 2024-09-21T07:56:04.120 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.49019.log.gz 2024-09-21T07:56:04.121 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67283.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80672.log 2024-09-21T07:56:04.121 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.67283.log.gz 2024-09-21T07:56:04.122 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42997.log 2024-09-21T07:56:04.122 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74624.log.gz 2024-09-21T07:56:04.123 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43334.log 2024-09-21T07:56:04.123 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80672.log.gz 2024-09-21T07:56:04.124 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42997.log: gzip -5 --verbose 87.3% -- -- replaced with /var/log/ceph/ceph-client.admin.42997.log.gz 2024-09-21T07:56:04.124 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.56645.log 2024-09-21T07:56:04.125 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43334.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42063.log 2024-09-21T07:56:04.125 INFO:teuthology.orchestra.run.smithi088.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43334.log.gz 2024-09-21T07:56:04.126 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56645.log: gzip 87.5% -- replaced with /var/log/ceph/ceph-client.admin.56645.log.gz 2024-09-21T07:56:04.126 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.55213.log 2024-09-21T07:56:04.127 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51837.log 2024-09-21T07:56:04.127 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.42063.log.gz 2024-09-21T07:56:04.128 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55213.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55213.log.gz 2024-09-21T07:56:04.128 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74999.log 2024-09-21T07:56:04.129 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51837.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72311.log 2024-09-21T07:56:04.129 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.51837.log.gz 2024-09-21T07:56:04.130 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74999.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.74999.log.gz 2024-09-21T07:56:04.130 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58029.log 2024-09-21T07:56:04.131 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76851.log 2024-09-21T07:56:04.131 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72311.log.gz 2024-09-21T07:56:04.132 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58029.log: gzip -5 87.3% --verbose -- replaced with /var/log/ceph/ceph-client.admin.58029.log.gz -- 2024-09-21T07:56:04.132 INFO:teuthology.orchestra.run.smithi088.stderr: /var/log/ceph/ceph-client.admin.81704.log 2024-09-21T07:56:04.133 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76851.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71914.log 2024-09-21T07:56:04.133 INFO:teuthology.orchestra.run.smithi088.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.76851.log.gz 2024-09-21T07:56:04.134 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81704.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81704.log.gz 2024-09-21T07:56:04.134 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77299.log 2024-09-21T07:56:04.135 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60942.log 2024-09-21T07:56:04.135 INFO:teuthology.orchestra.run.smithi088.stderr: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.71914.log.gz 2024-09-21T07:56:04.136 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77299.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.77299.log.gz 2024-09-21T07:56:04.136 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53676.log 2024-09-21T07:56:04.137 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60942.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67670.log 2024-09-21T07:56:04.138 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60942.log.gz 2024-09-21T07:56:04.139 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73069.log 2024-09-21T07:56:04.139 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53676.log.gz 2024-09-21T07:56:04.139 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.67670.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.67670.log.gz 2024-09-21T07:56:04.140 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55171.log 2024-09-21T07:56:04.140 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73069.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80443.log 2024-09-21T07:56:04.140 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73069.log.gz 2024-09-21T07:56:04.141 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67199.log 2024-09-21T07:56:04.141 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55171.log.gz 2024-09-21T07:56:04.142 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80443.log: gzip 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80443.log.gz 2024-09-21T07:56:04.142 INFO:teuthology.orchestra.run.smithi088.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.78826.log 2024-09-21T07:56:04.143 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67199.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51119.log 2024-09-21T07:56:04.144 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.67199.log.gz 2024-09-21T07:56:04.144 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78826.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78826.log.gz 2024-09-21T07:56:04.144 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50595.log 2024-09-21T07:56:04.145 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51119.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66678.log 2024-09-21T07:56:04.145 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51119.log.gz 2024-09-21T07:56:04.146 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50595.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50595.log.gz 2024-09-21T07:56:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47891.log 2024-09-21T07:56:04.147 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68329.log 2024-09-21T07:56:04.147 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66678.log.gz 2024-09-21T07:56:04.148 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.47891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47891.log.gz 2024-09-21T07:56:04.148 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52244.log 2024-09-21T07:56:04.149 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72666.log 2024-09-21T07:56:04.149 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68329.log.gz 2024-09-21T07:56:04.150 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52244.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.52244.log.gz 2024-09-21T07:56:04.150 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73630.log 2024-09-21T07:56:04.151 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72666.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48313.log 2024-09-21T07:56:04.152 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72666.log.gz 2024-09-21T07:56:04.152 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73630.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.73630.log.gz 2024-09-21T07:56:04.152 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55297.log 2024-09-21T07:56:04.153 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41868.log 2024-09-21T07:56:04.153 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.48313.log.gz 2024-09-21T07:56:04.154 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55297.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73416.log 2024-09-21T07:56:04.154 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.55297.log.gz 2024-09-21T07:56:04.155 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41868.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35899.log 2024-09-21T07:56:04.156 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41868.log.gz 2024-09-21T07:56:04.157 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73416.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73416.log.gz 2024-09-21T07:56:04.157 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42985.log 2024-09-21T07:56:04.159 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.35899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66593.log 2024-09-21T07:56:04.159 INFO:teuthology.orchestra.run.smithi088.stderr: 74.5% -- replaced with /var/log/ceph/ceph-client.admin.35899.log.gz 2024-09-21T07:56:04.159 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42985.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2024-09-21T07:56:04.159 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42985.log.gz 2024-09-21T07:56:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66593.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64978.log 2024-09-21T07:56:04.161 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66593.log.gz 2024-09-21T07:56:04.161 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62119.log 2024-09-21T07:56:04.162 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.64978.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.64978.log.gz 2024-09-21T07:56:04.163 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66252.log 2024-09-21T07:56:04.163 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76773.log 2024-09-21T07:56:04.163 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.62119.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.62119.log.gz 2024-09-21T07:56:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66252.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.66252.log.gz 2024-09-21T07:56:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78998.log 2024-09-21T07:56:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61931.log 2024-09-21T07:56:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.76773.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.76773.log.gz 2024-09-21T07:56:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78998.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78998.log.gz 2024-09-21T07:56:04.166 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36841.log 2024-09-21T07:56:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.61931.log: --verbose -- /var/log/ceph/ceph-client.admin.55728.log 2024-09-21T07:56:04.170 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.61931.log.gz 2024-09-21T07:56:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36841.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.36841.log.gz 2024-09-21T07:56:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57819.log 2024-09-21T07:56:04.170 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76137.log 2024-09-21T07:56:04.171 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55728.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55728.log.gz 2024-09-21T07:56:04.171 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80097.log 2024-09-21T07:56:04.171 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.57819.log.gz 2024-09-21T07:56:04.173 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76137.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62599.log 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.76137.log.gz 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80097.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.80097.log.gz 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65208.log 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78304.log/var/log/ceph/ceph-client.admin.62599.log: 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62599.log.gz 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65208.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65208.log.gz 2024-09-21T07:56:04.174 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58521.log 2024-09-21T07:56:04.176 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67850.log 2024-09-21T07:56:04.176 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78304.log: /var/log/ceph/ceph-client.admin.58521.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78304.log.gz 2024-09-21T07:56:04.176 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.58521.log.gz 2024-09-21T07:56:04.176 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69433.log 2024-09-21T07:56:04.176 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67850.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51039.log 2024-09-21T07:56:04.177 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.67850.log.gz 2024-09-21T07:56:04.178 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69433.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69433.log.gz 2024-09-21T07:56:04.178 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43913.log 2024-09-21T07:56:04.178 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59602.log 2024-09-21T07:56:04.179 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51039.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51039.log.gz 2024-09-21T07:56:04.179 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43913.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43913.log.gz 2024-09-21T07:56:04.180 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43673.log 2024-09-21T07:56:04.181 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.59602.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74218.log 2024-09-21T07:56:04.181 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59602.log.gz 2024-09-21T07:56:04.181 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43673.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43673.log.gz 2024-09-21T07:56:04.182 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63483.log 2024-09-21T07:56:04.183 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79562.log 2024-09-21T07:56:04.183 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74218.log: /var/log/ceph/ceph-client.admin.63483.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.74218.log.gz 2024-09-21T07:56:04.184 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63483.log.gz 2024-09-21T07:56:04.184 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77758.log 2024-09-21T07:56:04.185 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79562.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62285.log 2024-09-21T07:56:04.185 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.79562.log.gz 2024-09-21T07:56:04.186 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77758.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.77758.log.gz 2024-09-21T07:56:04.186 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80275.log 2024-09-21T07:56:04.187 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53802.log 2024-09-21T07:56:04.187 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62285.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.62285.log.gz 2024-09-21T07:56:04.188 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80275.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80275.log.gz 2024-09-21T07:56:04.188 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52160.log 2024-09-21T07:56:04.188 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54464.log 2024-09-21T07:56:04.189 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53802.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.53802.log.gz 2024-09-21T07:56:04.190 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52160.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.52160.log.gz 2024-09-21T07:56:04.190 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73374.log 2024-09-21T07:56:04.191 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71791.log 2024-09-21T07:56:04.191 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54464.log: /var/log/ceph/ceph-client.admin.73374.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.54464.log.gz 2024-09-21T07:56:04.192 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73374.log.gz 2024-09-21T07:56:04.192 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71873.log 2024-09-21T07:56:04.192 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71791.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49488.log 2024-09-21T07:56:04.193 INFO:teuthology.orchestra.run.smithi088.stderr: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.71791.log.gz 2024-09-21T07:56:04.194 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71873.log: 83.4% -- replaced with /var/log/ceph/ceph-client.admin.71873.log.gz 2024-09-21T07:56:04.194 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51181.log 2024-09-21T07:56:04.195 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36484.log 2024-09-21T07:56:04.195 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49488.log: 87.4%/var/log/ceph/ceph-client.admin.51181.log: -- replaced with /var/log/ceph/ceph-client.admin.49488.log.gz 2024-09-21T07:56:04.196 INFO:teuthology.orchestra.run.smithi175.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-09-21T07:56:04.196 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51181.log.gz 2024-09-21T07:56:04.196 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78136.log 2024-09-21T07:56:04.197 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.36484.log: --verbose -- /var/log/ceph/ceph-client.admin.75041.log 2024-09-21T07:56:04.197 INFO:teuthology.orchestra.run.smithi088.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36484.log.gz 2024-09-21T07:56:04.198 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78136.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78136.log.gz 2024-09-21T07:56:04.198 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81283.log 2024-09-21T07:56:04.199 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69776.log 2024-09-21T07:56:04.199 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75041.log: /var/log/ceph/ceph-client.admin.81283.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.75041.log.gz 2024-09-21T07:56:04.200 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.81283.log.gz 2024-09-21T07:56:04.200 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75719.log 2024-09-21T07:56:04.201 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69776.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68245.log 2024-09-21T07:56:04.201 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69776.log.gz 2024-09-21T07:56:04.202 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75719.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75719.log.gz 2024-09-21T07:56:04.202 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46704.log 2024-09-21T07:56:04.203 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74176.log 2024-09-21T07:56:04.203 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68245.log: 86.6%/var/log/ceph/ceph-client.admin.46704.log: -- replaced with /var/log/ceph/ceph-client.admin.68245.log.gz 2024-09-21T07:56:04.204 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46704.log.gz 2024-09-21T07:56:04.204 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69162.log 2024-09-21T07:56:04.204 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.74176.log: /var/log/ceph/ceph-client.admin.81661.log 2024-09-21T07:56:04.205 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.74176.log.gz 2024-09-21T07:56:04.206 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69162.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69162.log.gz 2024-09-21T07:56:04.206 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76305.log 2024-09-21T07:56:04.207 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70536.log 2024-09-21T07:56:04.207 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81661.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.81661.log.gz 2024-09-21T07:56:04.208 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76305.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.76305.log.gz 2024-09-21T07:56:04.208 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76893.log 2024-09-21T07:56:04.209 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71369.log 2024-09-21T07:56:04.210 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70536.log: /var/log/ceph/ceph-client.admin.76893.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.76893.log.gz 2024-09-21T07:56:04.210 INFO:teuthology.orchestra.run.smithi088.stderr: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.70536.log.gz 2024-09-21T07:56:04.210 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56855.log 2024-09-21T07:56:04.211 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46788.log 2024-09-21T07:56:04.212 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71369.log: /var/log/ceph/ceph-client.0.56855.log: 84.4% -- replaced with /var/log/ceph/ceph-client.admin.71369.log.gz 2024-09-21T07:56:04.212 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.56855.log.gz 2024-09-21T07:56:04.212 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55339.log 2024-09-21T07:56:04.213 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46788.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65020.log 2024-09-21T07:56:04.213 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46788.log.gz 2024-09-21T07:56:04.214 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55339.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55339.log.gz 2024-09-21T07:56:04.214 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48646.log 2024-09-21T07:56:04.215 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36109.log 2024-09-21T07:56:04.215 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65020.log: 87.1%/var/log/ceph/ceph-client.admin.48646.log: -- replaced with /var/log/ceph/ceph-client.admin.65020.log.gz 2024-09-21T07:56:04.216 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48646.log.gz 2024-09-21T07:56:04.216 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48095.log 2024-09-21T07:56:04.217 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47699.log 2024-09-21T07:56:04.217 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36109.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36109.log.gz 2024-09-21T07:56:04.218 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48095.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.48095.log.gz 2024-09-21T07:56:04.218 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42145.log 2024-09-21T07:56:04.219 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46290.log 2024-09-21T07:56:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47699.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.47699.log.gz 2024-09-21T07:56:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42145.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42145.log.gz 2024-09-21T07:56:04.220 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42317.log 2024-09-21T07:56:04.221 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75312.log 2024-09-21T07:56:04.221 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46290.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46290.log.gz 2024-09-21T07:56:04.222 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42317.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42317.log.gz 2024-09-21T07:56:04.222 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77926.log 2024-09-21T07:56:04.223 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42443.log 2024-09-21T07:56:04.223 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75312.log: /var/log/ceph/ceph-client.admin.77926.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.75312.log.gz 2024-09-21T07:56:04.224 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.77926.log.gz 2024-09-21T07:56:04.224 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73798.log 2024-09-21T07:56:04.225 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.42443.log: --verbose -- /var/log/ceph/ceph-client.admin.74394.log 2024-09-21T07:56:04.225 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42443.log.gz 2024-09-21T07:56:04.226 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73798.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53041.log 2024-09-21T07:56:04.226 INFO:teuthology.orchestra.run.smithi088.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.admin.73798.log.gz 2024-09-21T07:56:04.227 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53466.log 2024-09-21T07:56:04.228 INFO:teuthology.orchestra.run.smithi088.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.74394.log.gz 2024-09-21T07:56:04.228 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53041.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53041.log.gz 2024-09-21T07:56:04.228 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56104.log 2024-09-21T07:56:04.229 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79803.log 2024-09-21T07:56:04.230 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53466.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53466.log.gz 2024-09-21T07:56:04.230 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56104.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.56104.log.gz 2024-09-21T07:56:04.230 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-09-21T07:56:04.231 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47339.log 2024-09-21T07:56:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79803.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79803.log.gz 2024-09-21T07:56:04.232 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75456.log 2024-09-21T07:56:04.233 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47339.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81241.log 2024-09-21T07:56:04.234 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47339.log.gz 2024-09-21T07:56:04.235 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75456.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68559.log 2024-09-21T07:56:04.235 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.75456.log.gz 2024-09-21T07:56:04.236 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57059.log 2024-09-21T07:56:04.236 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81241.log.gz 2024-09-21T07:56:04.237 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68559.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72269.log 2024-09-21T07:56:04.237 INFO:teuthology.orchestra.run.smithi088.stderr: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.68559.log.gz 2024-09-21T07:56:04.238 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57766.log 2024-09-21T07:56:04.239 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.57059.log.gz 2024-09-21T07:56:04.240 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72269.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43003.log 2024-09-21T07:56:04.240 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72269.log.gz 2024-09-21T07:56:04.240 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57766.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51243.log 2024-09-21T07:56:04.241 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57766.log.gz 2024-09-21T07:56:04.242 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43003.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57945.log 2024-09-21T07:56:04.242 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.43003.log.gz 2024-09-21T07:56:04.243 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51243.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48814.log 2024-09-21T07:56:04.243 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51243.log.gz 2024-09-21T07:56:04.244 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80233.log 2024-09-21T07:56:04.244 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.57945.log.gz 2024-09-21T07:56:04.245 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48814.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65335.log 2024-09-21T07:56:04.245 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48814.log.gz 2024-09-21T07:56:04.247 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80233.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81409.log 2024-09-21T07:56:04.247 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.80233.log.gz 2024-09-21T07:56:04.248 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65335.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54961.log 2024-09-21T07:56:04.248 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65335.log.gz 2024-09-21T07:56:04.249 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62201.log 2024-09-21T07:56:04.249 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.81409.log.gz 2024-09-21T07:56:04.250 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54961.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36425.log 2024-09-21T07:56:04.250 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54961.log.gz 2024-09-21T07:56:04.251 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62201.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72834.log 2024-09-21T07:56:04.251 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.62201.log.gz 2024-09-21T07:56:04.253 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78561.log 2024-09-21T07:56:04.253 INFO:teuthology.orchestra.run.smithi088.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.36425.log.gz 2024-09-21T07:56:04.254 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46164.log 2024-09-21T07:56:04.254 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72834.log.gz 2024-09-21T07:56:04.255 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78561.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65711.log 2024-09-21T07:56:04.255 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78561.log.gz 2024-09-21T07:56:04.256 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46164.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63525.log 2024-09-21T07:56:04.256 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46164.log.gz 2024-09-21T07:56:04.257 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65711.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60251.log 2024-09-21T07:56:04.257 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65711.log.gz 2024-09-21T07:56:04.259 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63525.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43675.log 2024-09-21T07:56:04.259 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.63525.log.gz 2024-09-21T07:56:04.259 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48468.log 2024-09-21T07:56:04.260 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60251.log.gz 2024-09-21T07:56:04.261 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43675.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52683.log 2024-09-21T07:56:04.261 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.43675.log.gz 2024-09-21T07:56:04.262 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48468.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59726.log 2024-09-21T07:56:04.262 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.48468.log.gz 2024-09-21T07:56:04.263 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52683.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43640.log 2024-09-21T07:56:04.263 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52683.log.gz 2024-09-21T07:56:04.264 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59726.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54180.log 2024-09-21T07:56:04.264 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59726.log.gz 2024-09-21T07:56:04.265 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43640.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58906.log 2024-09-21T07:56:04.265 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43640.log.gz 2024-09-21T07:56:04.266 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46332.log 2024-09-21T07:56:04.267 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.54180.log.gz 2024-09-21T07:56:04.268 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58906.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58683.log 2024-09-21T07:56:04.268 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.58906.log.gz 2024-09-21T07:56:04.269 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71668.log 2024-09-21T07:56:04.269 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.46332.log.gz 2024-09-21T07:56:04.270 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58683.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.58683.log.gz 2024-09-21T07:56:04.271 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46248.log 2024-09-21T07:56:04.272 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71668.log: 84.7% -- replaced with /var/log/ceph/ceph-client.admin.71668.log.gz 2024-09-21T07:56:04.272 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42737.log 2024-09-21T07:56:04.273 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80359.log 2024-09-21T07:56:04.273 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.46248.log.gz 2024-09-21T07:56:04.274 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56285.log 2024-09-21T07:56:04.274 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42737.log.gz 2024-09-21T07:56:04.275 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68952.log 2024-09-21T07:56:04.275 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.80359.log.gz 2024-09-21T07:56:04.277 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56285.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42989.log 2024-09-21T07:56:04.277 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.56285.log.gz 2024-09-21T07:56:04.278 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70914.log 2024-09-21T07:56:04.278 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.68952.log.gz 2024-09-21T07:56:04.279 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42989.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44081.log 2024-09-21T07:56:04.279 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42989.log.gz 2024-09-21T07:56:04.280 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70914.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44123.log 2024-09-21T07:56:04.280 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70914.log.gz 2024-09-21T07:56:04.281 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44081.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69391.log 2024-09-21T07:56:04.281 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44081.log.gz 2024-09-21T07:56:04.282 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69036.log 2024-09-21T07:56:04.282 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.44123.log.gz 2024-09-21T07:56:04.284 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69391.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55003.log 2024-09-21T07:56:04.284 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.69391.log.gz 2024-09-21T07:56:04.285 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69265.log 2024-09-21T07:56:04.285 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69036.log.gz 2024-09-21T07:56:04.286 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55003.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-09-21T07:56:04.286 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.55003.log.gz 2024-09-21T07:56:04.287 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42611.log 2024-09-21T07:56:04.287 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69265.log.gz 2024-09-21T07:56:04.288 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53298.log 2024-09-21T07:56:04.290 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37044.log 2024-09-21T07:56:04.290 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.42611.log.gz 2024-09-21T07:56:04.291 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53298.log: gzip -5 87.6% -- replaced with /var/log/ceph/ceph-client.admin.53298.log.gz 2024-09-21T07:56:04.291 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.75372.log 2024-09-21T07:56:04.292 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59810.log 2024-09-21T07:56:04.292 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.37044.log.gz 2024-09-21T07:56:04.293 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75372.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73150.log 2024-09-21T07:56:04.294 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75372.log.gz 2024-09-21T07:56:04.295 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55591.log 2024-09-21T07:56:04.295 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.59810.log.gz 2024-09-21T07:56:04.296 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73150.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81367.log 2024-09-21T07:56:04.296 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.73150.log.gz 2024-09-21T07:56:04.297 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55591.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79604.log 2024-09-21T07:56:04.297 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.55591.log.gz 2024-09-21T07:56:04.299 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81367.log: 86.8%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72792.log 2024-09-21T07:56:04.299 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81367.log.gz 2024-09-21T07:56:04.299 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79604.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.79604.log.gz 2024-09-21T07:56:04.300 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45787.log 2024-09-21T07:56:04.302 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72792.log: gzip -5 87.1% -- replaced with /var/log/ceph/ceph-client.admin.72792.log.gz 2024-09-21T07:56:04.302 INFO:teuthology.orchestra.run.smithi088.stderr: --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2024-09-21T07:56:04.302 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45787.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45787.log.gz 2024-09-21T07:56:04.303 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81157.log 2024-09-21T07:56:04.304 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36046.log 2024-09-21T07:56:04.304 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2024-09-21T07:56:04.305 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73672.log 2024-09-21T07:56:04.306 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81157.log.gz 2024-09-21T07:56:04.307 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75624.log 2024-09-21T07:56:04.307 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36046.log.gz 2024-09-21T07:56:04.308 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73672.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73672.log.gz 2024-09-21T07:56:04.309 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75624.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74052.log 2024-09-21T07:56:04.309 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.75624.log.gz 2024-09-21T07:56:04.310 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43001.log 2024-09-21T07:56:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.74052.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74052.log.gz 2024-09-21T07:56:04.311 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70082.log 2024-09-21T07:56:04.312 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43001.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46830.log 2024-09-21T07:56:04.313 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43001.log.gz 2024-09-21T07:56:04.314 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70082.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36801.log 2024-09-21T07:56:04.314 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.70082.log.gz 2024-09-21T07:56:04.315 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43633.log 2024-09-21T07:56:04.315 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46830.log.gz 2024-09-21T07:56:04.316 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36801.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72750.log 2024-09-21T07:56:04.316 INFO:teuthology.orchestra.run.smithi088.stderr: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36801.log.gz 2024-09-21T07:56:04.317 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71750.log 2024-09-21T07:56:04.317 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43633.log.gz 2024-09-21T07:56:04.319 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68451.log 2024-09-21T07:56:04.319 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.72750.log.gz 2024-09-21T07:56:04.320 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71750.log: 84.6% -- replaced with /var/log/ceph/ceph-client.admin.71750.log.gz 2024-09-21T07:56:04.320 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-09-21T07:56:04.321 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80588.log 2024-09-21T07:56:04.321 INFO:teuthology.orchestra.run.smithi088.stderr: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.68451.log.gz 2024-09-21T07:56:04.323 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72049.log 2024-09-21T07:56:04.323 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-09-21T07:56:04.324 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80588.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46577.log 2024-09-21T07:56:04.324 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80588.log.gz 2024-09-21T07:56:04.325 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72049.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47633.log 2024-09-21T07:56:04.325 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.72049.log.gz 2024-09-21T07:56:04.326 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46577.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58132.log 2024-09-21T07:56:04.327 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46577.log.gz 2024-09-21T07:56:04.328 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47633.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51509.log 2024-09-21T07:56:04.328 INFO:teuthology.orchestra.run.smithi088.stderr: 91.1%/var/log/ceph/ceph-client.admin.58132.log: -- replaced with /var/log/ceph/ceph-client.admin.47633.log.gz 2024-09-21T07:56:04.329 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58132.log.gz 2024-09-21T07:56:04.329 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51899.log 2024-09-21T07:56:04.331 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51509.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65166.log 2024-09-21T07:56:04.331 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.51509.log.gz 2024-09-21T07:56:04.332 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51899.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81857.log 2024-09-21T07:56:04.332 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.51899.log.gz 2024-09-21T07:56:04.333 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65166.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-09-21T07:56:04.333 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.65166.log.gz 2024-09-21T07:56:04.335 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81857.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74779.log 87.5% 2024-09-21T07:56:04.335 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.81857.log.gz 2024-09-21T07:56:04.336 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56975.log 2024-09-21T07:56:04.337 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74779.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74779.log.gz 2024-09-21T07:56:04.338 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56975.log: /var/log/ceph/ceph-client.admin.45877.log 2024-09-21T07:56:04.338 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56975.log.gz 2024-09-21T07:56:04.339 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43264.log 2024-09-21T07:56:04.341 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45877.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45877.log.gz 2024-09-21T07:56:04.342 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67892.log 2024-09-21T07:56:04.343 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43264.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.43264.log.gz 2024-09-21T07:56:04.343 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70212.log 2024-09-21T07:56:04.344 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67892.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.67892.log.gz 2024-09-21T07:56:04.345 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70212.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.70212.log.gz 2024-09-21T07:56:04.346 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52599.log 2024-09-21T07:56:04.347 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66126.log 2024-09-21T07:56:04.348 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52599.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52599.log.gz 2024-09-21T07:56:04.349 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79646.log 2024-09-21T07:56:04.350 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66126.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.66126.log.gz 2024-09-21T07:56:04.351 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78784.log 2024-09-21T07:56:04.352 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79646.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.79646.log.gz 2024-09-21T07:56:04.353 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69856.log 2024-09-21T07:56:04.354 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78784.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.78784.log.gz 2024-09-21T07:56:04.354 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52118.log 2024-09-21T07:56:04.355 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69856.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69856.log.gz 2024-09-21T07:56:04.357 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.52118.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.36492.log 2024-09-21T07:56:04.357 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52118.log.gz 2024-09-21T07:56:04.358 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50935.log 2024-09-21T07:56:04.359 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36492.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.36492.log.gz 2024-09-21T07:56:04.362 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61199.log 2024-09-21T07:56:04.362 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50935.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.50935.log.gz 2024-09-21T07:56:04.362 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59852.log 2024-09-21T07:56:04.368 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61199.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.61199.log.gz 2024-09-21T07:56:04.368 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48053.log 2024-09-21T07:56:04.368 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59852.log: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.59852.log.gz 2024-09-21T07:56:04.368 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-09-21T07:56:04.368 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48053.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.48053.log.gz 2024-09-21T07:56:04.368 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72185.log 2024-09-21T07:56:04.370 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph.log: /var/log/ceph/ceph-client.admin.72185.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56453.log 2024-09-21T07:56:04.372 INFO:teuthology.orchestra.run.smithi088.stderr: 86.5% -- replaced with /var/log/ceph/ceph.log.gz 2024-09-21T07:56:04.372 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72185.log.gz 2024-09-21T07:56:04.372 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79484.log 2024-09-21T07:56:04.373 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56453.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56453.log.gz 2024-09-21T07:56:04.374 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54222.log 2024-09-21T07:56:04.374 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.79484.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.79484.log.gz 2024-09-21T07:56:04.375 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54222.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54222.log.gz 2024-09-21T07:56:04.376 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48210.log 2024-09-21T07:56:04.377 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66512.log 2024-09-21T07:56:04.379 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48210.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.48210.log.gz 2024-09-21T07:56:04.379 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.66512.log: -- /var/log/ceph/ceph-client.admin.50133.log 2024-09-21T07:56:04.380 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66512.log.gz 2024-09-21T07:56:04.381 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72540.log 2024-09-21T07:56:04.382 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50133.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50133.log.gz 2024-09-21T07:56:04.383 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69943.log 2024-09-21T07:56:04.384 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72540.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.72540.log.gz 2024-09-21T07:56:04.385 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42653.log 2024-09-21T07:56:04.386 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69943.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.69943.log.gz 2024-09-21T07:56:04.387 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53634.log 2024-09-21T07:56:04.387 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42653.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.42653.log.gz 2024-09-21T07:56:04.388 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52942.log 2024-09-21T07:56:04.389 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53634.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53634.log.gz 2024-09-21T07:56:04.390 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79688.log 2024-09-21T07:56:04.391 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52942.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52942.log.gz 2024-09-21T07:56:04.392 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54825.log 2024-09-21T07:56:04.394 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79688.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.79688.log.gz 2024-09-21T07:56:04.394 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47075.log 2024-09-21T07:56:04.395 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54825.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.54825.log.gz 2024-09-21T07:56:04.395 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55255.log 2024-09-21T07:56:04.396 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47075.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47075.log.gz 2024-09-21T07:56:04.397 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54102.log 2024-09-21T07:56:04.398 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55255.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55255.log.gz 2024-09-21T07:56:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.54102.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.54102.log.gz 2024-09-21T07:56:04.399 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70602.log 2024-09-21T07:56:04.400 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51305.log 2024-09-21T07:56:04.401 INFO:teuthology.orchestra.run.smithi175.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-09-21T07:56:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70602.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.70602.log.gz 2024-09-21T07:56:04.402 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74056.log 2024-09-21T07:56:04.403 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51305.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51305.log.gz 2024-09-21T07:56:04.404 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68774.log 2024-09-21T07:56:04.405 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.74056.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.74056.log.gz 2024-09-21T07:56:04.406 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68774.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68774.log.gz 2024-09-21T07:56:04.407 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43625.log 2024-09-21T07:56:04.408 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71245.log 2024-09-21T07:56:04.409 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43625.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.43625.log.gz 2024-09-21T07:56:04.410 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49313.log 2024-09-21T07:56:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71245.log: 84.5% -- replaced with /var/log/ceph/ceph-client.admin.71245.log.gz 2024-09-21T07:56:04.412 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55507.log 2024-09-21T07:56:04.413 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49313.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49313.log.gz 2024-09-21T07:56:04.414 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60193.log 2024-09-21T07:56:04.414 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55507.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55507.log.gz 2024-09-21T07:56:04.415 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41967.log 2024-09-21T07:56:04.416 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60193.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60193.log.gz 2024-09-21T07:56:04.418 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42695.log 2024-09-21T07:56:04.418 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.41967.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.41967.log.gz 2024-09-21T07:56:04.419 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80905.log 2024-09-21T07:56:04.420 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42695.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.42695.log.gz 2024-09-21T07:56:04.421 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55889.log/var/log/ceph/ceph-client.admin.80905.log: 2024-09-21T07:56:04.421 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.80905.log.gz 2024-09-21T07:56:04.422 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78220.log 2024-09-21T07:56:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55889.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.55889.log.gz 2024-09-21T07:56:04.424 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78220.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81767.log 2024-09-21T07:56:04.425 INFO:teuthology.orchestra.run.smithi088.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.78220.log.gz 2024-09-21T07:56:04.426 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58342.log 2024-09-21T07:56:04.427 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81767.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.81767.log.gz 2024-09-21T07:56:04.428 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54567.log 2024-09-21T07:56:04.429 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58342.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.58342.log.gz 2024-09-21T07:56:04.430 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80947.log 2024-09-21T07:56:04.430 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54567.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.54567.log.gz 2024-09-21T07:56:04.431 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49271.log 2024-09-21T07:56:04.432 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80947.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.80947.log.gz 2024-09-21T07:56:04.433 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63278.log 2024-09-21T07:56:04.434 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49271.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49271.log.gz 2024-09-21T07:56:04.435 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72989.log 2024-09-21T07:56:04.436 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63278.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63278.log.gz 2024-09-21T07:56:04.436 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47887.log 2024-09-21T07:56:04.437 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72989.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.72989.log.gz 2024-09-21T07:56:04.439 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68287.log 2024-09-21T07:56:04.439 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.47887.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.47887.log.gz 2024-09-21T07:56:04.440 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65813.log 2024-09-21T07:56:04.441 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68287.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.68287.log.gz 2024-09-21T07:56:04.442 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.65813.log: -- /var/log/ceph/ceph-client.admin.72479.log 2024-09-21T07:56:04.442 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65813.log.gz 2024-09-21T07:56:04.443 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75083.log 2024-09-21T07:56:04.445 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72479.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.72479.log.gz 2024-09-21T07:56:04.446 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74340.log 2024-09-21T07:56:04.447 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75083.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.75083.log.gz 2024-09-21T07:56:04.447 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80485.log 2024-09-21T07:56:04.448 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74340.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74340.log.gz 2024-09-21T07:56:04.449 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80485.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.80485.log.gz 2024-09-21T07:56:04.450 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76347.log 2024-09-21T07:56:04.451 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49145.log 2024-09-21T07:56:04.453 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76347.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.76347.log.gz 2024-09-21T07:56:04.453 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75414.log 2024-09-21T07:56:04.454 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49145.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49145.log.gz 2024-09-21T07:56:04.455 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42187.log 2024-09-21T07:56:04.456 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75414.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75414.log.gz 2024-09-21T07:56:04.457 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58216.log 2024-09-21T07:56:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42187.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.42187.log.gz 2024-09-21T07:56:04.458 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45709.log 2024-09-21T07:56:04.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58216.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58216.log.gz 2024-09-21T07:56:04.460 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.45709.log: /var/log/ceph/ceph-client.admin.60020.log 2024-09-21T07:56:04.461 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.45709.log.gz 2024-09-21T07:56:04.462 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66000.log 2024-09-21T07:56:04.463 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60020.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.60020.log.gz 2024-09-21T07:56:04.464 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78178.log 2024-09-21T07:56:04.465 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66000.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.66000.log.gz 2024-09-21T07:56:04.467 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74260.log 2024-09-21T07:56:04.468 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78178.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78178.log.gz 2024-09-21T07:56:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54693.log 2024-09-21T07:56:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74260.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.74260.log.gz 2024-09-21T07:56:04.469 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57253.log 2024-09-21T07:56:04.470 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54693.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54693.log.gz 2024-09-21T07:56:04.472 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60648.log 2024-09-21T07:56:04.472 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57253.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.57253.log.gz 2024-09-21T07:56:04.473 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59264.log 2024-09-21T07:56:04.475 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60648.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60648.log.gz 2024-09-21T07:56:04.475 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49742.log 2024-09-21T07:56:04.476 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59264.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59264.log.gz 2024-09-21T07:56:04.476 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56537.log 2024-09-21T07:56:04.479 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49742.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49742.log.gz 2024-09-21T07:56:04.479 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.56537.log: /var/log/ceph/ceph-client.admin.43339.log 2024-09-21T07:56:04.479 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56537.log.gz 2024-09-21T07:56:04.480 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45304.log 2024-09-21T07:56:04.481 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.43339.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.43339.log.gz 2024-09-21T07:56:04.482 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.45304.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.79082.log 2024-09-21T07:56:04.482 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45304.log.gz 2024-09-21T07:56:04.483 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76221.log 2024-09-21T07:56:04.485 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79082.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.79082.log.gz 2024-09-21T07:56:04.486 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.76221.log: -- /var/log/ceph/ceph-client.admin.47591.log 2024-09-21T07:56:04.486 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76221.log.gz 2024-09-21T07:56:04.487 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52473.log 2024-09-21T07:56:04.489 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47591.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47591.log.gz 2024-09-21T07:56:04.489 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69987.log 2024-09-21T07:56:04.490 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52473.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.52473.log.gz 2024-09-21T07:56:04.491 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79040.log 2024-09-21T07:56:04.492 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69987.log: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.69987.log.gz 2024-09-21T07:56:04.493 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.79040.log: /var/log/ceph/ceph-client.admin.55808.log 2024-09-21T07:56:04.493 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.79040.log.gz 2024-09-21T07:56:04.494 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50873.log 2024-09-21T07:56:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55808.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.55808.log.gz 2024-09-21T07:56:04.496 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62725.log 2024-09-21T07:56:04.497 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50873.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50873.log.gz 2024-09-21T07:56:04.498 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60293.log 2024-09-21T07:56:04.500 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62725.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.62725.log.gz 2024-09-21T07:56:04.500 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.60293.log: --verbose -- /var/log/ceph/ceph-client.admin.46080.log 2024-09-21T07:56:04.501 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60293.log.gz 2024-09-21T07:56:04.502 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42359.log 2024-09-21T07:56:04.503 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46080.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46080.log.gz 2024-09-21T07:56:04.504 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51695.log 2024-09-21T07:56:04.504 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42359.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42359.log.gz 2024-09-21T07:56:04.505 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46122.log 2024-09-21T07:56:04.508 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51695.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51695.log.gz 2024-09-21T07:56:04.508 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49881.log 2024-09-21T07:56:04.509 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.46122.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.46122.log.gz 2024-09-21T07:56:04.510 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63140.log 2024-09-21T07:56:04.511 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49881.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.49881.log.gz 2024-09-21T07:56:04.520 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64935.log 2024-09-21T07:56:04.520 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63140.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.63140.log.gz 2024-09-21T07:56:04.520 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60858.log 2024-09-21T07:56:04.520 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.64935.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.64935.log.gz 2024-09-21T07:56:04.520 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62327.log 2024-09-21T07:56:04.521 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60858.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.60858.log.gz 2024-09-21T07:56:04.521 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77365.log 2024-09-21T07:56:04.521 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62327.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.62327.log.gz 2024-09-21T07:56:04.521 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.77365.log: /var/log/ceph/ceph-client.admin.65897.log 2024-09-21T07:56:04.521 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77365.log.gz 2024-09-21T07:56:04.521 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65378.log 2024-09-21T07:56:04.527 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65897.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.65897.log.gz 2024-09-21T07:56:04.527 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49229.log 2024-09-21T07:56:04.527 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.65378.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.65378.log.gz 2024-09-21T07:56:04.527 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53760.log 2024-09-21T07:56:04.528 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49229.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.49229.log.gz 2024-09-21T07:56:04.528 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36273.log 2024-09-21T07:56:04.528 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53760.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.53760.log.gz 2024-09-21T07:56:04.528 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52286.log 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36273.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36273.log.gz 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73323.log 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52286.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.52286.log.gz 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72881.log 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.73323.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.73323.log.gz 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.72881.log: /var/log/ceph/ceph-client.admin.67115.log 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.72881.log.gz 2024-09-21T07:56:04.534 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.76935.log 2024-09-21T07:56:04.536 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67115.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.67115.log.gz 2024-09-21T07:56:04.536 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51367.log 2024-09-21T07:56:04.536 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.76935.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.76935.log.gz 2024-09-21T07:56:04.538 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45129.log/var/log/ceph/ceph-client.admin.51367.log: 2024-09-21T07:56:04.539 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.51367.log.gz 2024-09-21T07:56:04.540 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71955.log 2024-09-21T07:56:04.541 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.2.45129.log: 0.0% -- replaced with /var/log/ceph/ceph-client.2.45129.log.gz 2024-09-21T07:56:04.541 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56016.log 2024-09-21T07:56:04.542 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71955.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.71955.log.gz 2024-09-21T07:56:04.544 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56016.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36223.log 2024-09-21T07:56:04.544 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.56016.log.gz 2024-09-21T07:56:04.545 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78872.log 2024-09-21T07:56:04.546 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36223.log: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.36223.log.gz 2024-09-21T07:56:04.548 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68619.log 2024-09-21T07:56:04.548 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78872.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.78872.log.gz 2024-09-21T07:56:04.549 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48772.log 2024-09-21T07:56:04.550 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68619.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.68619.log.gz 2024-09-21T07:56:04.551 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.48772.log: --verbose -- /var/log/ceph/ceph-client.admin.42273.log 2024-09-21T07:56:04.551 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.48772.log.gz 2024-09-21T07:56:04.552 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43279.log 2024-09-21T07:56:04.553 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42273.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.42273.log.gz 2024-09-21T07:56:04.554 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69120.log/var/log/ceph/ceph-client.admin.43279.log: 2024-09-21T07:56:04.555 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.43279.log.gz 2024-09-21T07:56:04.556 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70452.log 2024-09-21T07:56:04.557 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69120.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.69120.log.gz 2024-09-21T07:56:04.558 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.70452.log: /var/log/ceph/ceph-client.admin.55633.log 2024-09-21T07:56:04.559 INFO:teuthology.orchestra.run.smithi088.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.70452.log.gz 2024-09-21T07:56:04.559 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81577.log 2024-09-21T07:56:04.561 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55633.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55633.log.gz 2024-09-21T07:56:04.561 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68191.log 2024-09-21T07:56:04.562 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81577.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81577.log.gz 2024-09-21T07:56:04.563 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61389.log 2024-09-21T07:56:04.564 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68191.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.68191.log.gz 2024-09-21T07:56:04.565 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61389.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61389.log.gz 2024-09-21T07:56:04.566 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.80317.log 2024-09-21T07:56:04.567 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59348.log 2024-09-21T07:56:04.569 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.80317.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.80317.log.gz 2024-09-21T07:56:04.569 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43335.log 2024-09-21T07:56:04.572 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59348.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.59348.log.gz 2024-09-21T07:56:04.572 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5/var/log/ceph/ceph-client.admin.43335.log: --verbose -- /var/log/ceph/ceph-client.admin.67325.log 2024-09-21T07:56:04.572 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.43335.log.gz 2024-09-21T07:56:04.573 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47549.log 2024-09-21T07:56:04.575 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.67325.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.67325.log.gz 2024-09-21T07:56:04.575 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62683.log 2024-09-21T07:56:04.576 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47549.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.47549.log.gz 2024-09-21T07:56:04.577 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81619.log 2024-09-21T07:56:04.577 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62683.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.62683.log.gz 2024-09-21T07:56:04.579 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55549.log 2024-09-21T07:56:04.580 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81619.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.81619.log.gz 2024-09-21T07:56:04.580 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79857.log 2024-09-21T07:56:04.582 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55549.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55549.log.gz 2024-09-21T07:56:04.582 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56197.log 2024-09-21T07:56:04.584 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79857.log: 86.6% -- replaced with /var/log/ceph/ceph-client.admin.79857.log.gz 2024-09-21T07:56:04.584 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78052.log 2024-09-21T07:56:04.584 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56197.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.56197.log.gz 2024-09-21T07:56:04.585 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51571.log 2024-09-21T07:56:04.587 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78052.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78052.log.gz 2024-09-21T07:56:04.587 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37135.log 2024-09-21T07:56:04.588 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.51571.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.51571.log.gz 2024-09-21T07:56:04.588 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50977.log 2024-09-21T07:56:04.590 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.37135.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.37135.log.gz 2024-09-21T07:56:04.591 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50753.log 2024-09-21T07:56:04.591 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50977.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.50977.log.gz 2024-09-21T07:56:04.592 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70409.log 2024-09-21T07:56:04.593 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50753.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.50753.log.gz 2024-09-21T07:56:04.594 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57017.log/var/log/ceph/ceph-client.admin.70409.log: 2024-09-21T07:56:04.595 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.70409.log.gz 2024-09-21T07:56:04.596 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73235.log 2024-09-21T07:56:04.597 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57017.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.57017.log.gz 2024-09-21T07:56:04.598 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66635.log/var/log/ceph/ceph-client.admin.73235.log: 2024-09-21T07:56:04.599 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.73235.log.gz 2024-09-21T07:56:04.600 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74957.log 2024-09-21T07:56:04.601 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66635.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66635.log.gz 2024-09-21T07:56:04.602 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52370.log 2024-09-21T07:56:04.602 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.74957.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74957.log.gz 2024-09-21T07:56:04.603 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61241.log 2024-09-21T07:56:04.604 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52370.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52370.log.gz 2024-09-21T07:56:04.606 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78689.log/var/log/ceph/ceph-client.admin.61241.log: 2024-09-21T07:56:04.606 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.61241.log.gz 2024-09-21T07:56:04.607 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56495.log 2024-09-21T07:56:04.608 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78689.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.78689.log.gz 2024-09-21T07:56:04.609 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53550.log/var/log/ceph/ceph-client.admin.56495.log: 2024-09-21T07:56:04.609 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.56495.log.gz 2024-09-21T07:56:04.610 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-09-21T07:56:04.611 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36445.log 2024-09-21T07:56:04.612 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53550.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53550.log.gz 2024-09-21T07:56:04.613 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78914.log 2024-09-21T07:56:04.613 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36445.log: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36445.log.gz 2024-09-21T07:56:04.614 INFO:teuthology.orchestra.run.smithi175.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-09-21T07:56:04.614 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72582.log 2024-09-21T07:56:04.615 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78914.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78914.log.gz 2024-09-21T07:56:04.618 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47128.log 2024-09-21T07:56:04.618 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72582.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.72582.log.gz 2024-09-21T07:56:04.618 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65569.log 2024-09-21T07:56:04.619 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47128.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.47128.log.gz 2024-09-21T07:56:04.620 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.65569.log: /var/log/ceph/ceph-client.admin.59078.log 2024-09-21T07:56:04.620 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.65569.log.gz 2024-09-21T07:56:04.621 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62557.log 2024-09-21T07:56:04.622 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59078.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.59078.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74684.log 2024-09-21T07:56:04.623 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:04.624 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62557.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.62557.log.gz 2024-09-21T07:56:04.625 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-client.admin.74684.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.47381.log 2024-09-21T07:56:04.625 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.74684.log.gz 2024-09-21T07:56:04.627 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59894.log 2024-09-21T07:56:04.628 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47381.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.47381.log.gz 2024-09-21T07:56:04.631 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59414.log 2024-09-21T07:56:04.631 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59894.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.59894.log.gz 2024-09-21T07:56:04.631 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45388.log 2024-09-21T07:56:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59414.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.59414.log.gz 2024-09-21T07:56:04.633 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59606.log 2024-09-21T07:56:04.634 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45388.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45388.log.gz 2024-09-21T07:56:04.641 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54609.log 2024-09-21T07:56:04.641 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.59606.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.59606.log.gz 2024-09-21T07:56:04.641 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48252.log 2024-09-21T07:56:04.641 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.54609.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.54609.log.gz 2024-09-21T07:56:04.642 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50553.log 2024-09-21T07:56:04.642 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48252.log: 86.7% -- replaced with /var/log/ceph/ceph-client.admin.48252.log.gz 2024-09-21T07:56:04.642 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57133.log 2024-09-21T07:56:04.642 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50553.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.50553.log.gz 2024-09-21T07:56:04.646 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.57133.log: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.57133.log.gz 2024-09-21T07:56:04.646 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53214.log 2024-09-21T07:56:04.646 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58601.log 2024-09-21T07:56:04.647 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53214.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.53214.log.gz 2024-09-21T07:56:04.647 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71081.log 2024-09-21T07:56:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.58601.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.58601.log.gz 2024-09-21T07:56:04.661 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62515.log 2024-09-21T07:56:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71081.log: 84.2% -- replaced with /var/log/ceph/ceph-client.admin.71081.log.gz 2024-09-21T07:56:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52515.log 2024-09-21T07:56:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.62515.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.62515.log.gz 2024-09-21T07:56:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50091.log 2024-09-21T07:56:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52515.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.52515.log.gz 2024-09-21T07:56:04.662 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42998.log 2024-09-21T07:56:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.50091.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.50091.log.gz 2024-09-21T07:56:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55087.log 2024-09-21T07:56:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42998.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.42998.log.gz 2024-09-21T07:56:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.77842.log 2024-09-21T07:56:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55087.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.55087.log.gz 2024-09-21T07:56:04.663 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48856.log 2024-09-21T07:56:04.664 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.77842.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.77842.log.gz 2024-09-21T07:56:04.664 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.48856.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.48856.log.gz 2024-09-21T07:56:04.664 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49612.log 2024-09-21T07:56:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36323.log 2024-09-21T07:56:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49612.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.49612.log.gz 2024-09-21T07:56:04.670 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71627.log 2024-09-21T07:56:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.36323.log: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.36323.log.gz 2024-09-21T07:56:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71533.log 2024-09-21T07:56:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71627.log: 93.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 84.6% -- replaced with /var/log/ceph/ceph-client.admin.71627.log.gz 2024-09-21T07:56:04.671 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52202.log 2024-09-21T07:56:04.671 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.71533.log: 84.7%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49923.log 2024-09-21T07:56:04.672 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.71533.log.gz 2024-09-21T07:56:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52202.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.52202.log.gz 2024-09-21T07:56:04.672 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81493.log 2024-09-21T07:56:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49923.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72437.log 2024-09-21T07:56:04.675 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49923.log.gz 2024-09-21T07:56:04.675 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.81493.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59940.log 2024-09-21T07:56:04.676 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.81493.log.gz 2024-09-21T07:56:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.72437.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.72437.log.gz 2024-09-21T07:56:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61117.log 2024-09-21T07:56:04.676 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53340.log 2024-09-21T07:56:04.676 INFO:teuthology.orchestra.run.smithi088.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.59940.log.gz 2024-09-21T07:56:04.678 INFO:teuthology.orchestra.run.smithi175.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-09-21T07:56:04.679 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.61117.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.61117.log.gz 2024-09-21T07:56:04.680 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42821.log 2024-09-21T07:56:04.680 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.53340.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.53340.log.gz 2024-09-21T07:56:04.680 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49061.log 2024-09-21T07:56:04.680 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.42821.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45954.log 2024-09-21T07:56:04.680 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.42821.log.gz/var/log/ceph/ceph-client.admin.49061.log: 2024-09-21T07:56:04.680 INFO:teuthology.orchestra.run.smithi088.stderr: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.49061.log.gz 2024-09-21T07:56:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71204.log 2024-09-21T07:56:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.45954.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.45954.log.gz 2024-09-21T07:56:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68852.log 2024-09-21T07:56:04.683 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.71204.log: -- /var/log/ceph/ceph-client.admin.52725.log 2024-09-21T07:56:04.684 INFO:teuthology.orchestra.run.smithi088.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.71204.log.gz 2024-09-21T07:56:04.684 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.68852.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.68852.log.gz 2024-09-21T07:56:04.684 INFO:teuthology.orchestra.run.smithi175.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-09-21T07:56:04.685 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-09-21T07:56:04.685 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52725.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.52725.log.gz 2024-09-21T07:56:04.685 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60377.log 2024-09-21T07:56:04.685 INFO:teuthology.orchestra.run.smithi175.stderr: 2024-09-21T07:56:04.685 INFO:teuthology.orchestra.run.smithi175.stderr:real 0m0.889s 2024-09-21T07:56:04.686 INFO:teuthology.orchestra.run.smithi175.stderr:user 0m4.207s 2024-09-21T07:56:04.686 INFO:teuthology.orchestra.run.smithi175.stderr:sys 0m0.513s 2024-09-21T07:56:04.688 INFO:teuthology.orchestra.run.smithi088.stderr:gzip/var/log/ceph/ceph-osd.3.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.49793.log 2024-09-21T07:56:04.688 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60377.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.60377.log.gz 2024-09-21T07:56:04.688 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52076.log 2024-09-21T07:56:04.689 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.49793.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.49793.log.gz 2024-09-21T07:56:04.690 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73279.log 2024-09-21T07:56:04.691 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.52076.log: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.52076.log.gz 2024-09-21T07:56:04.692 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59180.log/var/log/ceph/ceph-client.admin.73279.log: 2024-09-21T07:56:04.692 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.73279.log.gz 2024-09-21T07:56:04.693 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2024-09-21T07:56:04.696 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59180.log: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.59180.log.gz 2024-09-21T07:56:04.696 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65669.log 2024-09-21T07:56:04.698 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-mds.c.log: /var/log/ceph/ceph-client.admin.65669.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.65669.log.gz 2024-09-21T07:56:04.699 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50471.log 2024-09-21T07:56:04.701 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.50471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47030.log 2024-09-21T07:56:04.703 INFO:teuthology.orchestra.run.smithi088.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.0.50471.log.gz 2024-09-21T07:56:04.703 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44639.log 2024-09-21T07:56:04.704 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47030.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.47030.log.gz 2024-09-21T07:56:04.706 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.44639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.44639.log.gz 2024-09-21T07:56:04.706 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63236.log 2024-09-21T07:56:04.709 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63236.log: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.63236.log.gz 2024-09-21T07:56:04.710 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66168.log 2024-09-21T07:56:04.712 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69517.log 2024-09-21T07:56:04.713 INFO:teuthology.orchestra.run.smithi088.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.66168.log.gz 2024-09-21T07:56:04.715 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66859.log 2024-09-21T07:56:04.717 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.69517.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.69517.log.gz 2024-09-21T07:56:04.719 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.66859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60545.log 2024-09-21T07:56:04.720 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.66859.log.gz 2024-09-21T07:56:04.721 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60419.log 2024-09-21T07:56:04.726 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60545.log: /var/log/ceph/ceph-client.admin.60419.log: 87.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63031.log 2024-09-21T07:56:04.726 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60545.log.gz 2024-09-21T07:56:04.727 INFO:teuthology.orchestra.run.smithi088.stderr: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.60419.log.gz 2024-09-21T07:56:04.729 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36418.log 2024-09-21T07:56:04.732 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.63031.log: /var/log/ceph/ceph-client.admin.36418.log: 86.9%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56239.log 2024-09-21T07:56:04.733 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63031.log.gz 2024-09-21T07:56:04.734 INFO:teuthology.orchestra.run.smithi088.stderr: 86.4% -- replaced with /var/log/ceph/ceph-client.admin.36418.log.gz 2024-09-21T07:56:04.735 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.81199.log 2024-09-21T07:56:04.739 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.56239.log: /var/log/ceph/ceph-client.admin.81199.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69307.log 2024-09-21T07:56:04.739 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.56239.log.gz 2024-09-21T07:56:04.740 INFO:teuthology.orchestra.run.smithi088.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.81199.log.gz 2024-09-21T07:56:04.742 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.69307.log: -- /var/log/ceph/ceph-client.admin.59222.log 2024-09-21T07:56:04.745 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78010.log 2024-09-21T07:56:04.746 INFO:teuthology.orchestra.run.smithi088.stderr: 87.0% -- replaced with /var/log/ceph/ceph-client.admin.69307.log.gz 2024-09-21T07:56:04.747 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.59222.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.59222.log.gz 2024-09-21T07:56:04.749 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47213.log 2024-09-21T07:56:04.752 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.47213.log: /var/log/ceph/ceph-client.admin.78010.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43955.log 2024-09-21T07:56:04.754 INFO:teuthology.orchestra.run.smithi088.stderr: 87.1% 87.6% -- replaced with /var/log/ceph/ceph-client.admin.78010.log.gz 2024-09-21T07:56:04.754 INFO:teuthology.orchestra.run.smithi088.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47213.log.gz 2024-09-21T07:56:04.755 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44303.log 2024-09-21T07:56:04.758 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.43955.log: /var/log/ceph/ceph-client.0.70794.log 2024-09-21T07:56:04.759 INFO:teuthology.orchestra.run.smithi088.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-09-21T07:56:04.760 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.44303.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.43955.log.gz 2024-09-21T07:56:04.760 INFO:teuthology.orchestra.run.smithi088.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44303.log.gz 2024-09-21T07:56:04.762 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.78499.log 2024-09-21T07:56:04.763 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.0.70794.log: 0.0% -- replaced with /var/log/ceph/ceph-client.0.70794.log.gz 2024-09-21T07:56:04.764 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.78499.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.78499.log.gz 2024-09-21T07:56:04.765 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.79918.log 2024-09-21T07:56:04.766 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70956.log 2024-09-21T07:56:04.767 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.79918.log: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.79918.log.gz 2024-09-21T07:56:04.768 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.70956.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.70956.log.gz 2024-09-21T07:56:04.769 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.75270.log 2024-09-21T07:56:04.771 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60774.log 2024-09-21T07:56:04.772 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.75270.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.75270.log.gz 2024-09-21T07:56:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.60774.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.60774.log.gz 2024-09-21T07:56:04.773 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55974.log 2024-09-21T07:56:04.776 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/ceph-client.admin.55974.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.55974.log.gz 2024-09-21T07:56:04.991 INFO:teuthology.orchestra.run.smithi088.stderr: 93.5% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-09-21T07:56:05.008 INFO:teuthology.orchestra.run.smithi088.stderr: 93.7% -- replaced with /var/log/ceph/ceph-mds.a.log.gz 2024-09-21T07:56:05.101 INFO:teuthology.orchestra.run.smithi088.stderr: 93.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-09-21T07:56:05.174 INFO:teuthology.orchestra.run.smithi088.stderr: 93.3% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2024-09-21T07:56:05.179 INFO:teuthology.orchestra.run.smithi088.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-09-21T07:56:05.460 INFO:teuthology.orchestra.run.smithi088.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-09-21T07:56:05.463 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-09-21T07:56:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m1.671s 2024-09-21T07:56:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m5.897s 2024-09-21T07:56:05.463 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m2.288s 2024-09-21T07:56:05.464 INFO:tasks.ceph:Archiving logs... 2024-09-21T07:56:05.464 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/remote/smithi088/log 2024-09-21T07:56:05.465 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:56:06.753 DEBUG:teuthology.misc:Transferring archived files from smithi175:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/remote/smithi175/log 2024-09-21T07:56:06.754 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T07:56:07.318 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T07:56:07.325 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T07:56:07.325 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:56:07.330 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T07:56:07.445 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T07:56:07.455 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T07:56:07.456 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:56:07.460 DEBUG:teuthology.orchestra.run.smithi175:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T07:56:07.491 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:56:07.494 INFO:teuthology.orchestra.run.smithi175.stderr:bash: line 1: ntpq: command not found 2024-09-21T07:56:07.499 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:56:07.499 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-09-21T07:56:07.499 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv01.front.sepia.ceph.com 3 6 377 2 +43us[ +44us] +/- 48ms 2024-09-21T07:56:07.499 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 1 -153us[ -153us] +/- 58ms 2024-09-21T07:56:07.499 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 4 +150us[ +151us] +/- 78ms 2024-09-21T07:56:07.499 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:56:07.501 INFO:teuthology.orchestra.run.smithi175.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T07:56:07.501 INFO:teuthology.orchestra.run.smithi175.stdout:=============================================================================== 2024-09-21T07:56:07.501 INFO:teuthology.orchestra.run.smithi175.stdout:^* hv01.front.sepia.ceph.com 3 6 377 1 +46us[ +58us] +/- 49ms 2024-09-21T07:56:07.501 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 2 -159us[ -148us] +/- 57ms 2024-09-21T07:56:07.501 INFO:teuthology.orchestra.run.smithi175.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 3 +151us[ +162us] +/- 78ms 2024-09-21T07:56:07.501 INFO:teuthology.orchestra.run.smithi175.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T07:56:07.502 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T07:56:07.510 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T07:56:07.511 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T07:56:07.518 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:56:07.680 DEBUG:teuthology.orchestra.run.smithi175:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T07:56:07.797 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:56:07.858 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:56:07.859 DEBUG:teuthology.orchestra.run.smithi175:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-21T07:56:07.919 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:56:07.919 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T07:56:07.928 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T07:56:07.937 INFO:teuthology.task.internal:Duration was 1069.257028 seconds 2024-09-21T07:56:07.937 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T07:56:07.945 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T07:56:07.945 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:56:07.949 DEBUG:teuthology.orchestra.run.smithi175:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T07:56:08.031 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:56:08.035 INFO:teuthology.orchestra.run.smithi175.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T07:56:08.420 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T07:56:08.420 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-09-21T07:56:08.420 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:56:08.469 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi175.front.sepia.ceph.com 2024-09-21T07:56:08.469 DEBUG:teuthology.orchestra.run.smithi175:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T07:56:08.519 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T07:56:08.520 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:56:08.524 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T07:56:08.582 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T07:56:08.582 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:56:08.587 DEBUG:teuthology.orchestra.run.smithi175:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T07:56:08.757 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T07:56:08.768 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T07:56:08.769 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:56:08.824 DEBUG:teuthology.orchestra.run.smithi175:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T07:56:08.881 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T07:56:08.890 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:56:08.895 DEBUG:teuthology.orchestra.run.smithi175:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:56:08.961 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-09-21T07:56:08.967 INFO:teuthology.orchestra.run.smithi175.stdout:kernel.core_pattern = core 2024-09-21T07:56:08.996 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:56:09.031 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:56:09.031 DEBUG:teuthology.orchestra.run.smithi175:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T07:56:09.064 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T07:56:09.065 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T07:56:09.074 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T07:56:09.075 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/remote/smithi088 2024-09-21T07:56:09.075 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:56:09.140 DEBUG:teuthology.misc:Transferring archived files from smithi175:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913617/remote/smithi175 2024-09-21T07:56:09.140 DEBUG:teuthology.orchestra.run.smithi175:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T07:56:09.203 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T07:56:09.203 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:56:09.207 DEBUG:teuthology.orchestra.run.smithi175:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T07:56:09.242 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T07:56:09.253 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T07:56:09.253 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T07:56:09.264 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T07:56:09.264 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:56:09.282 DEBUG:teuthology.orchestra.run.smithi175:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T07:56:09.312 INFO:teuthology.orchestra.run.smithi088.stdout: 393253 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:56 /home/ubuntu/cephtest 2024-09-21T07:56:09.317 INFO:teuthology.orchestra.run.smithi175.stdout: 393233 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 07:56 /home/ubuntu/cephtest 2024-09-21T07:56:09.319 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T07:56:09.329 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T07:56:09.350 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a client.1-kernel-sha1: 2ca44a2f35ca8f36983aaaa46d49312239ef519a description: fs/volumes/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mgr mon osd} distro/{centos_latest} mount/kclient/{mount-syntax/{v1} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile pg_health} tasks/volumes/{overrides test/misc}} duration: 1069.2570278644562 flavor: default owner: scheduled_pdonnell@teuthology success: true 2024-09-21T07:56:09.351 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T07:56:09.429 INFO:teuthology.run:pass